Claude Opus 5

2026-11-02

RESEARCH · 1 SOURCE · arXiv cs.AI

BABA-Bench paper (arXiv:2610.41076v1) introduces a 47-world benchmark for patient-advocacy LLM agents

BABA-Bench is a benchmark of 47 interactive worlds derived from de-identified real patient advocacy cases that checks both outcomes and procedural correctness (permissions, evidence, prohibitions) at each agent action. The authors evaluated 14 frontier models across 1,974 runs: agents often achieved required outcomes via invalid procedures (of 921 runs that reached all outcomes, 254 — 28% — failed procedural checks); Claude Opus 5 reached all required outcomes in 81.6% of runs but passed all checks in only 52.5% of runs, and providing verified routes dramatically improved pass rates for two models (from 29.8% to 91.5% and from 44.0% to 97.2%).

7.0

2026-10-30

RESEARCH · 1 SOURCE · The Decoder

Claude Sonnet 5 agent autonomously selected and delivered gene therapy in a mouse

Researchers at Exin Therapeutics used Anthropic’s Claude Sonnet 5 to analyze a mouse epilepsy model, choose a gene therapy, pick a brain target and stereotactic coordinates, and control a motorized surgical rig to inject the treatment; a licensed veterinarian approved each irreversible step and the mouse reportedly recovered. Mid-procedure the agent refused to continue after detecting a calibration error, and more capable Claude models (Opus 5/5.5) had earlier declined to perform the live procedure; the work is described in an X thread and a technical report is expected but not yet peer-reviewed.

8.0

2026-10-28

RESEARCH · 1 SOURCE · arXiv cs.AI

MARQ: modular, LLM-agnostic prototype that infuses INCOSE requirement criteria for requirement-quality assessment

The arXiv preprint introduces MARQ, a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack implementing INCOSE requirement-quality criteria. Evaluated on an expert-labeled NASA requirement set with 100 executions each using Claude Opus 5, GPT-5.6 Sol, and Gemma 3 12B, MARQ raises median F1 from off-the-shelf baselines (Claude: 0.55 → 0.80; GPT-5.6 Sol: 0.52 → 0.62; Gemma 3 12B: 0.18 → 0.53), with gains concentrated in Correctness and Conformance and model-dependent trade-offs between misses and false alarms.

7.0

2026-10-26

COMPANIES · 1 SOURCE · InfoQ AI, ML & Data Engineering

Cloudflare launches Auto Router public beta in AI Gateway to rate requests by stakes and other metrics

Cloudflare published Auto Router in public beta via AI Gateway: a two-stage router that builds a candidate model pool then runs a multi-head classifier on Workers AI to assign probabilities across 14 task categories and score requests 1–5 on complexity, ambiguity, stakes and context dependence, using only the newest turns (up to 8 KiB). The router combines those scores with benchmark results (quoted formula: "utility = expected quality - adaptive cost penalty"), exposes routing headers for auditability, supports per-request allowed-models headers, is free in beta, and currently lacks WebSocket support.

7.0

2026-10-25

MODELS · 1 SOURCE · The Decoder

Nvidia open-sources Flora, a Qwen3.5-based robot video labeling engine

Nvidia's GEAR lab open-sourced Flora, a data-engine composed of three fine-tuned Qwen3.5 models (9B for segmentation, 27B for descriptions, and a 27B 'Flora-Rewarder' for scoring) that labels robot videos for action boundaries, step-level rewards, and task success. Nvidia says Flora is already used in training its GR00T models at scale on GB300 GPUs and estimates labeling costs of about $0.72/hour versus ~$88/hour with GPT-6 Astra, though the cost figures are projections and benchmark comparisons include methodological caveats.

7.0

2026-10-22

RESEARCH · 1 SOURCE · Hugging Face

ThinkingBox-Training released on GitHub; RL fine-tune raises Qwen3.8-27B reliability using database-graded rewards

ThinkingBox-Training is now available on GitHub. The toolkit turns executable checks of terminal database state into RL rewards; fine-tuning Qwen3.8-27B with 50 GRPO steps raised average pass@1 from 51.70% to 60.78% and increased tasks solved on all 20 attempts from 38 to 107, using an offline policy-relative difficulty selection of training tasks and evaluation on the 507-task ThinkingBox-Bench.

7.0

2026-10-17

MODELS · 1 SOURCE · The Decoder

Deepseek releases Deepseek‑V4.1‑Flash‑1017, open-sources weights and opens search APIs

Chinese company Deepseek released Deepseek‑V4.1‑Flash‑1017 and published the model weights on Hugging Face and ModelScope. The company says the model—trained with reinforcement learning using self-feedback—runs faster, produces shorter outputs, beats larger rivals in its internal coding and agent benchmarks, and is paired with new Search & Fetch APIs for developers; independent verification of the benchmark claims is not yet available.

6.0

2026-10-09

RESEARCH · 1 SOURCE · arXiv cs.AI

GenUI-Harness: multi-agent data-aware generative UI harness (arXiv:2610.11123v1)

The paper introduces GenUI-Harness, a multi-agent system that pairs a Tool Agent for retrieval and execution with a GUI Coder Agent that detects ambiguities and generates front-end code to produce structured, interactive UIs. It also presents Dynamic UX (a sandboxed interaction/reward collection package), Reward Auditor (a meta-reward mechanism to mitigate reward-hacking), and UI-TAU Bench (Lite: 300 tasks; Full: 1,000 tasks); on UI-TAU Lite GenUI-Harness gains 4.48 percentage points Pass@3 over smolagents, improves a 4B backbone from 9.33% to 58.00% Pass@3 (versus Claude Opus 5 at 46.67%), and a reviewer survey found generated UIs reduced dialogue rounds from 3.4 to 1.2.

7.0

2026-10-03

RESEARCH · 1 SOURCE · Hugging Face

Microsoft’s ThinkingBox benchmark evaluates agents by backend state and is now on Hugging Face

ThinkingBox, a Microsoft benchmark that grades AI agents by the terminal backend state and side effects they leave (not just generated text), runs 507 stateful business workflows 20 times each and is now available through Hugging Face and runnable via OpenEnv. The paper and benchmark report substantial inconsistency: across 121,680 trials on 12 LLMs, 79,853 failed executable checks and many seemingly successful runs still produced wrong or missing database-side effects.

8.0

2026-10-01

RESEARCH · 1 SOURCE · arXiv cs.AI

arXiv:2609.38621v1 — LLM comparisons of scientific findings are shaped by prose and biological priors

The paper probes how language models decide whether two scientific reports conflict by translating an unsatisfiable XOR constraint system into lab-style prose. When constraints are stated formally, GPT-5.6 Sol and Claude Opus 5 recover the better-supported assignment in 90% and 96% of cases respectively; in scientific prose Claude Opus 5 often favors the biologically expected (but less well-supported) assignment, with recovery rising from 27% to 79% after removing that biological preference and reaching 92% when given a formalization prompt and explicit paired-design cue (p<.001); GPT-5.6 Sol showed smaller, statistically insignificant sensitivity to these manipulations.

7.0

2026-09-30

MODELS · 1 SOURCE · AWS Machine Learning

Amazon Bedrock adds Anthropic Claude Opus 5 and Sonnet 5 with in-region inference in Seoul and Singapore

Amazon Bedrock now supports Anthropic Claude Opus 5 and Claude Sonnet 5 with in-region inference on the bedrock-runtime endpoint: both Opus 5 and Sonnet 5 are available in Seoul (ap-northeast-2), and Sonnet 5 is available in Singapore (ap-southeast-1). The release lets customers call the bedrock-runtime endpoint (InvokeModel, Converse, or Anthropic Messages API) so inference requests and data are processed and logged entirely within the specified AWS Region to help meet data residency requirements.

7.0

2026-09-26

MODELS · 1 SOURCE · The Decoder

OpenAI's GPT-6 Astra scores 80% on Epoch AI's furniture assembly benchmark

Epoch AI's Furniture Assembly Benchmark (FAB) tests models on identifying deliberate assembly errors in photos of IKEA furniture. Previously the best model (Claude Opus 4.5) scored 28% in November 2025; ten months later OpenAI's GPT-6 Astra reaches 80% with a processing time of about three minutes per photo, followed by Claude Fable 5.1 at 70% and Claude Opus 5 at 61%, while some Chinese open-weight models like Kimi K3 lag substantially.

8.0

2026-09-25

MODELS · 1 SOURCE · TechCrunch AI

OpenAI’s Astra and Anthropic’s Opus 5 used to crack long-unsolved Enigma messages

Developer Carter Leffen says he instructed OpenAI’s newest model, Astra (referred to as GPT‑6 Astra), to search archival databases and successfully recovered the plaintext of an Enigma message that had been unresolved since 2005, a solution later validated by archivist Frode Weirerud. Separately, cryptanalyst Jack Willis reports using Anthropic’s Claude Opus 5 to break a different unsolved Enigma message; Weirerud says seven Enigma messages remain unbroken.

7.0

2026-09-23

CODING · 1 SOURCE · NVIDIA Developer

SWE-Serve benchmark shows many patches that pass local tests fail live serving for SGLang

SWE-Serve is a new benchmark that turns 83 merged SGLang pull requests into 53 executable inference-serving tasks across six families; 19 tasks start a real server and validate patches through the full serving path. Across those live-serving tasks, the same patches pass 45.9% of the time with the complete verifier versus 69.4% when live-serving checks are excluded (roughly one in three otherwise-passing patches fail live checks); multi-domain tasks show a 21.3 percentage-point lower pass rate, and closed-book model pass@1 scores (11 models, mini-swe-agent) ranged from 34.6%–75.5% with Claude Opus 5 and GPT-5.6 Sol among top performers.

7.0

2026-09-22

MODELS · 4 SOURCES · AWS Machine Learning · TechCrunch AI · OpenAI · The Decoder

OpenAI's GPT-6 Sol and GPT-6 Luna now generally available on Amazon Bedrock

OpenAI's GPT-6 Sol and GPT-6 Luna are now generally available on Amazon Bedrock, running on Bedrock's high-performance inference engine and offered at lower API pricing than their GPT-5.6 predecessors. GPT-6 Sol is positioned for demanding, recurring development and operations tasks with improved coding, reasoning and factuality (OpenAI reports ~50% fewer factual mistakes vs GPT-5.6 Sol), while GPT-6 Luna is optimized for high-volume, repeatable workloads like extraction, classification and summarization; Bedrock supports explicit prompt caching plus IAM-based access control and invocation auditing.

8.0

2026-09-18

MODELS · 4 SOURCES · Anthropic · TechCrunch AI · The Verge AI · The Decoder

Anthropic releases Claude Opus 5 — lower-cost model claiming near‑frontier performance

Anthropic announced Claude Opus 5, a new model positioned as a cost‑efficient successor to Opus 4.8 and the new default on Claude Max (and the strongest on Claude Pro). Anthropic says Opus 5 matches or exceeds prior models on many coding, knowledge‑work and scientific benchmarks (Frontier‑Bench, GDPval‑AA, CursorBench, ARC‑AGI, Zapier AutomationBench, OSWorld) at lower cost per task while remaining behind Mythos 5 on security and biology frontier tasks; the company also reports improved alignment and safety in pre‑deployment audits and links a System Card for more details.

8.0

2026-09-17

MODELS · 1 SOURCE · The Decoder

Vals AI’s GPT-6 Astra completes multiple long-horizon game milestones and posts big ARC-AGI-3 gains

According to Vals AI and related community runs, GPT-6 Astra reached far-end Minecraft goals (built a Nether portal and gathered end resources before a Creeper destroyed its chest), won Pokemon FireRed in about 18 hours (vs. ~96 hours for GPT‑5.6 Sol), launched a Factorio rocket in ~10 hours, and more, while ARC Prize reported Astra scored ~62.7% on the ARC‑AGI‑3 benchmark (vs. ~7.78% for GPT‑5.6 Sol). ARC Prize and Vals AI attribute the jump to Astra’s ability to form compact symbolic descriptions from observations and turn them into reusable plans while operating through general screen/mouse/keyboard interfaces.

8.0