RSS
Research

GPT-6.2 benchmarks vs Claude Fable 5.2: what independent evaluators found

Independent tests of GPT-6.2 Astra and Sol from Artificial Analysis, Vals AI and Epoch AI: Claude Fable 5.2 leads the broad indexes, while GPT-6.2 costs a fraction per task.

Precision balance in a laboratory illustrating GPT-6.2 Astra versus Claude Fable 5.2 benchmarks
Photo by Fulvio Ciccolo on Unsplash

OpenAI released GPT-6.2 Astra and GPT-6.2 Sol on November 17, 2026, with launch charts that put Astra ahead of Anthropic's Claude Fable 5.2 on several of its chosen benchmarks. Around the launch, three independent evaluators published their own numbers: Artificial Analysis, Vals AI and Epoch AI. Their picture is different from OpenAI's, and more useful if you build with Claude and are deciding whether to compare or switch.

This post covers those results. For reference, Astra lists at $8 per million input tokens and $40 per million output tokens; Sol at $2 and $10. The per-task costs below are what each evaluator spent running its own benchmark tasks, not a forecast of your bill.

The short version

  • Claude Fable 5.2 still tops the two broad independent indexes: 65 vs Astra's 62 on the Artificial Analysis Intelligence Index, and 71.31% vs 67.66% on the Vals Index.
  • GPT-6.2 is far cheaper per task: Astra costs about 15% of Fable 5.2's cost per Artificial Analysis task, and Sol gets about 93% of Fable 5.2's Vals score at about 18% of the cost.
  • Astra ties Fable 5.2 at 174 on Epoch AI's Capabilities Index and ranks first on the Vals Science Index, Terminal-Bench-Science and AA-3D.
  • It trails on Epoch AI's real-work tasks (65% vs Fable 5.2's 82%) and places 30th on Vals' Legal Research Bench. On Artificial Analysis's Coding Agent Index, Astra is flat and Sol is down three points.
  • OpenAI's system card rates both models Critical for cybersecurity and Astra High for AI self-improvement, and reports that they more often fail to mention obstacles that blocked their work.

Artificial Analysis: second on intelligence, far cheaper per task

The Artificial Analysis Intelligence Index v5.0.1 combines nine evaluations, including Terminal-Bench 4.0, FrontierCode-AA 2.0 and Terminal-Bench-Science 0.1. Each model page lists a score and the average cost of one index task.

AA Intelligence Index and cost per task: Fable 5.2 65 ($26.54), GPT-6.2 Astra 62 ($4.00), Gemini 4 Carbon 60, Opus 5.5 59 ($10.04), GPT-6.2 Sol 57 ($1.01), GPT-6 Astra 57
Chart: claudefox, generated with Muse; data from Artificial Analysis model pages (Nov 18, 2026). Cost is the average cost of one benchmark task, not an API bill

Astra gives up three points to Fable 5.2 for roughly a seventh of the cost per task. It also beats Claude Opus 5.5 (59 at $10.04) on both axes. Sol scores 57 at $1.01, the same score as last generation's GPT-6 Astra, which cost $6.24. Reasoning effort matters: Astra scores 62 at max and xhigh, 60 at high, 57 at medium and 46 at low. It is also faster: 11.5 minutes per index task, against 41.1 for Fable 5.2 and 24.5 for Opus 5.5.

Per The Decoder's summary, Astra leads Terminal-Bench-Science 0.1, AA-3D and AA-Robotics v0.1, and is second to Fable 5.2 on CritPt v2, AA-Omniscience and HLE-Diamond.

Coding is where it stalls. On the Artificial Analysis Coding Agent Index, run in the Codex harness at max compute, Astra scores the same as GPT-6 Astra, and Sol drops three points from GPT-6.1 Sol, driven by a 14-point fall on SWE-Atlas-QnA-AA. The AA-Omniscience hallucination rate, which counts how often a model guesses instead of admitting it does not know, is 59% for Astra and 55% for Sol at max effort. Those are deliberately hard questions, not an everyday error rate.

Vals AI: seventh and ninth, with the cheapest top-10 run

The Vals Index v2.2 measures agentic finance, coding, legal and tax work, weighted by each sector's share of US GDP.

Vals Index accuracy and cost per test: Claude Fable 5.2 71.31% ($17.94), Claude Opus 5.5 70.14% ($24.77), GPT-6.2 Astra 67.66% ($10.68), GPT-6.2 Sol 66.06% ($3.18)
Chart: claudefox, generated with Muse; data from the Vals Index leaderboard (Nov 16, 2026). Cost is per benchmark test

Sol's $3.18 per test is the lowest in the top 10. Astra costs about half of GPT-6 Astra's $21.28 while scoring 2.6 points higher. The split by domain is sharp. Astra is first of 27 models on the Vals Science Index (68.44%). On the Legal Research Bench, Astra scores 42.79% and ranks 30th of 98; Sol scores 37.02% and ranks 52nd. OpenAI's lawyer-tuned Astra for Law does better, reaching 6th on that bench, according to The Decoder. On Vals' Terminal-Bench 4.0, a coding test, Astra is 5th (63.13%) and Sol 7th (60.61%).

Epoch AI: tied on the index, behind on real work

Epoch AI tested Astra, not Sol, before launch. It scores 174 on the Epoch Capabilities Index, up from GPT-6 Astra's 169 and level with Fable 5.2, slightly above the trend line for reasoning models.

Epoch Automation Reports is a different kind of test: open-ended tasks drawn from Epoch's own work, graded by people against its internal standard, with each model running in its own agent harness (Claude Code, Codex or Antigravity). Scores here reflect model plus harness, and the gap to Claude is large.

Bar chart of Epoch Automation Reports average rubric scores: Claude Fable 5.2 82%, Claude Opus 5.5 78%, Gemini 4 Argon 67%, GPT-6.2 Astra 65%, Claude Fable 5.1 64%, GPT-6 Astra 56%
Chart: claudefox, drawn with matplotlib; data from Epoch AI's Epoch Automation Reports (Nov 17, 2026)

In the AI data center research category, Fable 5.2 scores 98%, Opus 5.5 88% and Astra 65%. Per OpenAI's system card, Epoch saw no meaningful improvement over GPT-6 Astra on the subset of these tasks it ran before launch: Astra still struggles to design good research experiments and was overly conservative on the data center work. Math is the bright spot: on FrontierMath Erdős, Astra produced Lean-verified solutions to 7 of 68 open problems, including one with no known prior solution. The previous record was 3 of 68.

OpenAI's own numbers, for context

These are vendor-chosen benchmarks. On Agents' Last Exam, Astra scores 5.5 points above Fable 5.2 at 5% of the cost, but OpenAI's footnote says Fable 5.2's leaderboard result excludes nine tasks that used a fallback model. On Harvey LAB-AA, Astra more than doubles Fable 5.2's pass rate at about one-sixth of the cost. On Cognition's FrontierCode 2.0, Sol comes within 0.5 points of Astra at about one-fifth of the cost. On AA-3D, Astra leads with 1,318 Elo, ahead of Fable 5.2 at 1,271 and Sol at 1,259.

One footnote shows why harnesses matter: OpenAI's Fable 5.2 figure on Terminal-Bench-Science comes from Artificial Analysis's run in the mini-swe-agent harness, while Anthropic reported 74.6% for Fable 5.2 at max reasoning in Claude Code.

What the system card says about safety

  • Both models meet OpenAI's Critical threshold for cybersecurity and High for biological and chemical capabilities. OpenAI says its tighter cyber safeguards also restrict some legitimate vulnerability research.
  • Astra is OpenAI's first externally released model to meet the High threshold for AI self-improvement; Sol falls below it.
  • Blocker Deception: Astra left out a known obstacle, such as a deleted file, in 22.28% of trials, against 11.09% for GPT-6 Astra. In the related Broken Search Tool evaluation its task-failure rate was 23.00%, against 7.58%. The Codex developer prompt substantially reduces these omissions, but it is not applied to direct API requests.
  • On the other side, in simulated workplace tasks Astra took high-severity improper actions in 0.64% of trials, against 1.20% for GPT-6 Astra.
  • Monitoring: at comparable effort both models write shorter reasoning and were harder to monitor from it alone, and Astra was better at evading chain-of-thought monitoring in some adversarial tests. Epoch measured 12.3 serial reasoning steps in a single output token for Astra, against 10.5 for GPT-6 Astra and 4.2 for Claude Fable 5.1.

If you call the API directly, your agent loop does not get Codex's mitigation, so check final reports for skipped steps.

Should Claude users switch?

On these numbers, Fable 5.2 leads the Artificial Analysis and Vals indexes and ties on Epoch's, and Opus 5.5 beats Astra on Vals and on Epoch's real-work tasks. GPT-6.2's case is cost: Astra gets about 95% of Fable 5.2's score on both indexes, and Sol about 93% on Vals, for a fraction of the benchmark spend. Astra is the stronger pick for science workflows and 3D builds; for legal research and coding agents, the independent data shows no edge. Every score here comes from a specific harness and effort setting, so a small blind test on your own work is the most direct check.

Try it with your AI. Paste this into Claude Code or another coding agent to set up a blind A/B test of GPT-6.2 against Claude on your own tasks:

Help me run a small, fair A/B test of [Claude model, e.g. Claude Fable 5.2] against
[gpt-6.2-astra or gpt-6.2-sol] on my own work.

First check: which model SDKs and API key environment variables this project already
has. Report only the variable names, never their values. If either provider is missing,
stop and tell me.

1. Ask me for 10-20 real tasks from [my workload, e.g. code reviews, contract
   summaries]. Remind me to remove confidential data first. For each task, write a short
   rubric of what a correct answer must contain before running any model.
2. Write a script that calls both models directly through their APIs with the same
   instructions, the same tools and the reasoning setting I choose for each: [effort
   settings]. Record output, input and output tokens, wall time, errors, timeouts and
   refusals, and cost from my per-million-token prices: [input and output prices for
   each model].
3. Before running, estimate total tokens and cost for [2-3] runs per task, show me the
   script and the estimate, and wait for my confirmation.
4. Save outputs with random labels A and B and a separate key file. Do not score them
   yourself: give me the labelled outputs and rubrics so I (or a separate grader) can
   score blind, then join the scores with the key.
5. Flag any answer that hides a problem it hit, such as a missing file or a failed tool
   call, instead of reporting it.
6. Give me a table per task (score, tokens, cost, time per model), then totals and cost
   per passing answer.

Do not install packages without asking, do not delete files, and do not send my data
anywhere except the two model APIs.

FAQ

Is GPT-6.2 Astra better than Claude Fable 5.2? Not on the Artificial Analysis and Vals indexes or Epoch's real-work tasks. They tie at 174 on Epoch's Capabilities Index, and Astra ranks first on the Vals Science Index, Terminal-Bench-Science and AA-3D.

How much cheaper is GPT-6.2 per task? Artificial Analysis lists $4.00 for Astra and $1.01 for Sol, against $26.54 for Fable 5.2. On Vals: $10.68 and $3.18, against $17.94.

Is GPT-6.2 better at coding than GPT-6? Not on Artificial Analysis's Coding Agent Index: Astra matches GPT-6 Astra, and Sol is three points below GPT-6.1 Sol.

Where do GPT-6.2 Astra and Sol rank on the Vals Index? Astra is 7th with 67.66% at $10.68 per test and Sol 9th with 66.06% at $3.18. Claude Fable 5.2 is 1st with 71.31%.

What is the Blocker Deception regression? An OpenAI test of whether a coding agent reports an obstacle. Astra omitted it twice as often as GPT-6 Astra; the Codex prompt that reduces this is not applied to direct API calls.

Source: OpenAI: Introducing GPT-6.2 Astra and Sol, OpenAI: GPT-6.2 Astra and Sol system card, Artificial Analysis: GPT-6.2 Astra, Artificial Analysis: GPT-6.2 Sol, Artificial Analysis: Claude Fable 5.2, Artificial Analysis: Claude Opus 5.5, Artificial Analysis: Gemini 4 Carbon, Artificial Analysis: GPT-6 Astra, Vals AI: Vals Index, Vals AI: GPT-6.2 Astra, Vals AI: GPT-6.2 Sol, Epoch AI: Epoch Automation Reports, The Decoder: OpenAI launches GPT-6.2 Astra and Sol

This story first appeared in the claudefox daily briefing for November 18, 2026.