OpenAI · launched 4 Sept 2026
GPT-6 Astra
OpenAI's flagship GPT-6 model, positioned above GPT-6 Sol and Luna for the most demanding professional work, coding and computer use.
Read the official announcement- Vendor
- OpenAI
- Launch date
- 4 Sept 2026
- Input price
- $10 / 1M tokens
- Output price
- $50 / 1M tokens
- Cached input
- $1.00 / 1M tokens
- Batch discount
- 50%
- Context window
- 1.05M tokens
- Max output
- 128K tokens
- Input types
- Text, Image, File
- OpenRouter ID
- openai/gpt-6-astra
- Terminal-Bench 4.0high effort · $7.21 per task · reported by OpenAI57.9%
- FrontierCode 2.0max effort · $6.68 per task · reported by Anthropic42.8%
- Agents' Last Exam v1max effort · $7.24 per task · reported by OpenAI59.3%
- MMMU-Pro (Vision)max effort · $0.065 per task · reported by OpenAI86.7%
- OSWorld 2.1 (offline set)max effort · $8.17 per task · reported by OpenAI80.9%
- Factual error rate on difficult prompts(lower is better)max effort · $0.66 per task · reported by OpenAI4.2%
- Terminal-Bench 4.0max effort · $10.3 per task · reported by OpenAI58.2%
- Terminal-Bench Science 0.1max effort · $23.8 per task · reported by OpenAI68.1%
- DeepSWE v1.1xhigh effort · $4.43 per task · reported by OpenAI74.12%
- GDP.pdfxhigh effort · $1.91 per task · reported by OpenAI32.2%
- AutomationBench 1.0.6max effort · $1.73 per task · reported by OpenAI41.4%
- OSWorld 2.0 (offline set)max effort · $9.44 per task · reported by OpenAI73.49%
- Terminal-Bench Science 0.1max effort · $23.8 per task · reported by OpenAI68.1%
- Factual error rate on difficult prompts(lower is better)high effort · $0.48 per task · reported by OpenAI3.9%
- AutomationBench 1.0.6max effort · $1.73 per task · reported by OpenAI41.4%
- Agents' Last Exam V1max effort · $6.23 per task · reported by OpenAI59.3%
- Factual error rate(lower is better)high effort · $0.48 per task · reported by OpenAI3.9%
- FrontierCode 1.1max effort · $4.59 per task · reported by OpenAI53.3%
- DeepSWE 1.1xhigh effort · $4.43 per task · reported by OpenAI74.1%
- OSWorld 2.0max effort · $9.07 per task · reported by OpenAI73.5%
- Terminal-Bench 4.0high effort · $7.21 per attempt · reported by Anthropic57.9%
- FrontierCode v1.1max effort · $4.36 per task · reported by Anthropic53.26%
- GDPval-AA v2.1max effort · $4.53 per task · reported by Anthropic1542 Elo
- AutomationBenchmax effort · $1.77 per task · reported by Anthropic41.4%
Price compared with other models
| Model | Launched | Input / 1M | Output / 1M | Cached / 1M | Blended / 1M | vs GPT-6 Astra |
|---|---|---|---|---|---|---|
| Claude Haiku 5.5AnthropicListed prices and comparisons apply to prompts up to 100K tokens. Above 100K, input costs $0.50, output $2.50 and cache reads $0.05 per million tokens. | 7 Oct 2026 | $0.10 | $0.50 | $0.01 | $0.20 | 99% cheaper |
| GPT-6 LunaOpenAI | 22 Sept 2026 | $0.10 | $0.50 | $0.01 | $0.20 | 99% cheaper |
| GPT-5.6 LunaOpenAI | 9 Jul 2026 | $0.20 | $1.20 | $0.02 | $0.45 | 98% cheaper |
| Gemini 3.5 Flash-LiteGoogle DeepMindText, image, video and audio inputs share the listed rate. Cache storage and grounding are billed separately. | 21 Jul 2026 | $0.30 | $2.50 | $0.03 | $0.85 | 96% cheaper |
| Gemini 3.8 FlashGoogle DeepMindIntroductory rates through 31 December 2026. From 1 January 2027, input is $1.50, output $7.50 and cached input $0.15 per million tokens. Cache storage and grounding are billed separately. | 2 Sept 2026 | $0.75 | $3.75 | $0.075 | $1.50 | 93% cheaper |
| Claude Haiku 4.5Anthropic | 15 Oct 2025 | $1.00 | $5.00 | $0.10 | $2.00 | 90% cheaper |
| GPT-6.1 TerraOpenAIListed rates apply up to 272K input tokens. Above 272K, input costs $2, cached input $0.10 and output $7.50 per million tokens for the full request. | 26 Oct 2026 | $1.00 | $5.00 | $0.05 | $2.00 | 90% cheaper |
| Grok 4.7SpaceXAIBase rates apply up to 200K context tokens; higher-context requests use different rates. Batch API is not supported. The separate fast variant costs twice as much. | 21 Sept 2026 | $2.00 | $6.00 | $0.50 | $3.00 | 85% cheaper |
| Claude Sonnet 5Anthropic | 30 Jun 2026 | $2.00 | $10 | $0.20 | $4.00 | 80% cheaper |
| Claude Sonnet 5.5AnthropicCache reads were reduced from $0.20 to $0.10 per million tokens on 7 October 2026. | 28 Sept 2026 | $2.00 | $10 | $0.10 | $4.00 | 80% cheaper |
| Gemini 4 ArgonGoogle DeepMindIntroductory rates through 30 November 2026. From 1 December 2026, input is $4, output $20 and cached input $0.20 per million tokens. Caching is implicit only. Long decode raises the output limit to 1,048,576 tokens. | 30 Sept 2026 | $2.00 | $10 | $0.10 | $4.00 | 80% cheaper |
| GPT-6 SolOpenAIListed rates apply up to 272K input tokens. Above 272K, input costs $4, cached input $0.40 and output $15 per million tokens for the full request. | 22 Sept 2026 | $2.00 | $10 | $0.20 | $4.00 | 80% cheaper |
| GPT-6.1 SolOpenAIListed rates apply up to 272K input tokens. Above 272K, input costs $4, cached input $0.20 and output $15 per million tokens for the full request. | 29 Sept 2026 | $2.00 | $10 | $0.10 | $4.00 | 80% cheaper |
| Gemini 3.1 ProGoogle DeepMind · PreviewRates apply to prompts up to 200K tokens. Above 200K, input is $4, output $18 and cached input $0.40 per million tokens. Cache storage and grounding are billed separately. | 19 Feb 2026 | $2.00 | $12 | $0.20 | $4.50 | 78% cheaper |
| GPT-5.6 TerraOpenAIOpenAI's model documentation supports the Batch API but does not list a batch price. | 9 Jul 2026 | $2.00 | $12 | $0.20 | $4.50 | 78% cheaper |
| Claude Opus 5.5Anthropic | 22 Sept 2026 | $4.00 | $20 | $0.20 | $8.00 | 60% cheaper |
| GPT-5.6 SolOpenAIPrice before the GPT-6 launch, from OpenAI's announcement. OpenRouter now lists $2 / $10. | 9 Jul 2026 | $4.00 | $20 | – | $8.00 | 60% cheaper |
| Claude Opus 5Anthropic | 24 Jul 2026 | $5.00 | $25 | $0.50 | $10 | 50% cheaper |
| Claude Fable 5.2Anthropic | 9 Nov 2026 | $8.00 | $40 | $0.20 | $16 | 20% cheaper |
| Claude Fable 5Anthropic | 9 Jun 2026 | $10 | $50 | $1.00 | $20 | Same price |
| Claude Fable 5.1Anthropic | 1 Sept 2026 | $10 | $50 | $0.25 | $20 | Same price |
| GPT-6 AstraOpenAI | 4 Sept 2026 | $10 | $50 | $1.00 | $20 | Baseline |
GPT-6.1 Sol and GPT-6 Sol rates were checked on 8 October 2026 against OpenAI model documentation. USD per million tokens, checked 2 Oct 2026 via OpenRouter. Comparison uses the blended price (3 input tokens for every output token). Haiku 5.5 and Sonnet 5.5 prices were updated from Anthropic's 7 October 2026 announcement. Gemini and Grok prices were rechecked against official Google and xAI documentation on 27 October 2026. GPT-6.1 Terra prices were checked against OpenAI documentation on 27 October 2026. Claude Fable 5.2 prices are from Anthropic's 9 November 2026 announcement.
Official benchmark results
Reported by Anthropic on 9 Nov 2026 · exact values
| Benchmark | Claude Fable 5.2 | Claude Fable 5.1 | Claude Opus 5.5 | GPT-6 Astra |
|---|---|---|---|---|
| FrontierCode 2.0Agentic coding | 49.1% | 32.4% | 45.5% | 42.8% |
| FrontierSWE v2Agentic coding | 69.9% | 56.3% | 62.3% | 65.5% |
| AutomationBenchBusiness workflows | 51.4% | 31.4% | 42.5% | 41.4% |
| Terminal-Bench-Science 0.1Agentic scientific research | 74.6% | 52.6% | 58.7% | 68.1% |
| WANDRAgentic search · relaxed score | 82.1% | 68.7% | 72.3% | – |
| OSWorld 2.1Computer use · offline set, partial | 89.3% | 83.5% | 87.2% | 79.5% |
| RoboDojo SimRobotics | 64.9% | 27.6% | 39.4% | 29.8% |
| AA-Briefcase v1.1Knowledge work · rubric | 66.1% | 61.5% | 61.2% | 52% |
| AA-Briefcase v1.1 (Elo)Knowledge work | 1785 Elo | 1669 Elo | 1800 Elo | 1569 Elo |
| Remote Labor IndexKnowledge work | 41.3% | 17.9% | 21.3% | 20.8% |
| Humanity's Last ExamMultidisciplinary reasoning · with tools | 74.2% | 65.6% | 67.7% | 57.2% |
| ChartographyVisual chart recognition · no tools | 73.4% | 44.8% | 64.4% | 71% |
- Bold marks the best reported score in each row.
- Scores for the Claude models are with production safeguards on and at each model's highest effort, except FrontierCode 2.0 (Fable 5.2 and 5.1 at xhigh) and Remote Labor Index (Fable 5.2 at high).
- Claude Mythos 5.2, the same underlying model with access limited to vetted organizations, scores 75.3% on Terminal-Bench-Science 0.1. The cybersecurity and protein design rows report Mythos scores, so they are not shown here.
- Anthropic's table also includes Gemini 4 Carbon, which has no published API price, so it is not tracked here. RoboDojo Sim scores were run internally by Anthropic.
- A dash means the lab did not report a result for that model.
Reported by SpaceXAI on 21 Sept 2026 · exact values
Exact published scores · Higher is better. Unreported results are omitted.
| Benchmark | Grok 4.7 | Grok 4.6 | GPT-5.6 Sol | Claude Fable 5.1 | GPT-6 Astra |
|---|---|---|---|---|---|
| CursorBench 4.0Software engineering | 46.3% | 40.4% | 41.7% | 51.8% | – |
| DeepSWE v1.1Software engineering · Grok 4.7 uses high effort; other Grok 4.7 rows use xhigh | 71% | 65.2% | 72.7% | 70% | – |
| EEBenchElectrical engineering | 64% | 53% | 39.4% | 56.4% | – |
| AA-Briefcase v1.1Knowledge work | 1657 Elo | 1546 Elo | 1487 Elo | 1678 Elo | – |
| Terminal-Bench 4.0Agentic terminal work | 37.6% | 20.3% | 37.3% | 57.9% | – |
| Harvey's Legal Agent BenchmarkLegal work | 19.6% | 15.8% | 2.5% | 6.7% | – |
| HealthBench ProfessionalClinical reasoning | 56.7% | 48.5% | 60.5% | 62.1% | – |
| GDPvalProfessional knowledge work · Tabbed chart: Grok 4.7 xhigh, Grok 4.6 high, Fable 5.1 and GPT-6 Astra max | 1695 Elo | 1605 Elo | – | 1735 Elo | 1542 Elo |
| AA BriefcaseMulti-hour office work · Tabbed chart: Grok 4.7 xhigh, Grok 4.6 high, Fable 5.1 and GPT-6 Astra max | 1657 Elo | 1546 Elo | – | 1678 Elo | 1569 Elo |
| EEBenchElectrical engineering · Tabbed chart: Grok 4.7 xhigh, Grok 4.6 high, Fable 5.1 and GPT-6 Astra max | 64% | 53% | – | 56.4% | 69.3% |
- Bold marks the best reported score in each row.
- Exact values from the announcement comparison table and embedded tabbed chart data. Grok 4.7 uses xhigh effort except DeepSWE v1.1, which uses high.
- The comparison table uses GPT-5.6 Sol and Fable 5.1 at max effort, and Grok 4.6 at high effort.
- The table and tabbed charts use different competitor sets; each is preserved with its source qualifiers. Grok 4.6 is a benchmark comparison only.
Reported by Google DeepMind on 30 Sept 2026 · exact values
Exact published scores · Higher is better. Unreported results are omitted.
| Benchmark | Gemini 4 Argon | GPT-6 Astra | Claude Fable 5.1 | Claude Opus 5.5 |
|---|---|---|---|---|
| Vals IndexKnowledge work | 68.9% | 63.1% | 65.8% | 67% |
| AutomationBenchKnowledge work · Score | 51.3% | 41.4% | 31.4% | 42.5% |
| Vals Finance Agent v2Knowledge work | 65.4% | 53.5% | 58.9% | 58.6% |
| Harvey's Legal Agent BenchmarkKnowledge work | 19.6% | 5.4% | 6.7% | 3.8% |
| DeepSWE v1.1Agentic coding | 77.9% | 74.1% | 67.4% | 74.2% |
| FrontierSWE v2Agentic coding | 55% | 65.5% | 56.3% | 62.3% |
| Vibe Code BenchAgentic coding | 91.9% | 89.6% | 90.3% | 90.3% |
| Terminal-bench 4.0Agentic coding | 57.4% | 58.2% | 57.9% | 66.4% |
| PostTrainBenchML engineering | 45.3% | 44.3% | 40.2% | 49.3% |
| Terminal-Bench Science 0.1Science and math | 57.6% | 68.1% | 52.6% | 63.3% |
| LABBench 2Science and math | 88.8% | 85.4% | 68.6% | 73.1% |
| RiemannBenchScience and math | 76% | 72% | 65.6% | 69.6% |
| GraphWalksLong context · Up to 128k, BFS (F1) | 99.7% | 98.7% | 91.4% | 90.6% |
| GraphWalksLong context · 256k to 1M, BFS (F1) | 84.2% | 71.8% | 65% | 66.8% |
| Agent's Last ExamComputer use · Pass rate | 39.5% | 34.2% | – | 38.2% |
| OSWorld-2.0Computer use · Offline subset Partial score | 69.2% | 72.6% | – | – |
| ChartographyMultimodal understanding | 71.6% | 71% | 46.2% | 66.3% |
| LVBenchMultimodal understanding | 91.7% | 87.5% | 79.7% | 83.7% |
| CWE-bench v1Cybersecurity | 68% | 68% | 58% | 67% |
View Google DeepMind's original charts






- Bold marks the best reported score in each row.
- Exact scores from Google DeepMind's performance table. Argon is initially available to trusted testers.
- OSWorld-2.0 uses the offline subset and partial scoring. GraphWalks reports BFS F1, with separate context ranges.
- Scores reflect Google's published evaluation setup. Benchmark versions and harnesses can differ from other labs' launch results. A dash means no result was published.
Reported by OpenAI on 29 Sept 2026 · exact values
| Benchmark | GPT-6.1 Sol | GPT-6 Sol | GPT-6 Astra | GPT-6 Luna |
|---|---|---|---|---|
| Failure to disclose a broken search toolSafety evaluation · Max effort | 2.1% | 4.92% | 1.5% | 28.67% |
| Reviewer bypass attemptsSafety evaluation · Max effort | 0% | 0% | 0% | 0.26% |
| Warning circumventionSafety evaluation · Max effort | 23.48% | 64.39% | 17.42% | 42.37% |
| Computer-use safety stress testSafety evaluation · Xhigh effort | 4.32% | 17.39% | 2.4% | 13.7% |
- Bold marks the best reported score in each row.
- All rates are lower-is-better. These targeted evaluations stress challenging scenarios and do not represent typical usage.
- Exact values from the embedded announcement chart data. No cost values were published for these evaluations.
Reported by OpenAI on 22 Sept 2026 · exact values
| Benchmark | GPT-6 Astra | GPT-6 Sol | GPT-6 Luna | GPT-5.6 Sol | GPT-5.6 Luna |
|---|---|---|---|---|---|
| Coding deception rateAlignment · max effort · lower is better | 0.5% | 1.3% | 2.8% | 10.4% | 9.5% |
- Bold marks the best reported score in each row.
- OpenAI's internal evaluation uses tasks deliberately chosen to provoke dishonesty, so deception is much rarer in typical use.
- The announcement also covers broken search, reviewer bypass, warning circumvention and unauthorized interaction. See OpenAI's system card for those results.
Reported by Anthropic on 22 Sept 2026 · exact values
| Benchmark | Claude Opus 5.5 | Claude Fable 5.1 | Claude Opus 5 | GPT-6 Astra | GPT-5.6 Sol |
|---|---|---|---|---|---|
| Terminal-Bench 4.0Agentic coding | 66.4% | 55.8% | 52.3% | 57.9% | 37.3% |
| FrontierCode v1.1 (main)Agentic coding | 54.4% | 50.3% | 48% | 53.3% | 47.5% |
| CursorBench 4.0Agentic coding | 57.8% | 51.8% | 46.6% | – | 41.7% |
| GDPval-AA v2.1Knowledge work | 1846 Elo | 1735 Elo | 1708 Elo | 1542 Elo | 1588 Elo |
| AutomationBenchBusiness workflows | 40% | 31.4% | 26.9% | 41.4% | 28.8% |
| Humanity's Last ExamMultidisciplinary reasoning · with tools | 67.7% | 65.6% | 63.6% | 57.2% | – |
| Terminal-Bench-Science 0.1Agentic scientific research | 58.7% | 52.6% | 29% | 64.6% | 22.4% |
| OSWorld 2.1Computer use · partial | 81.8% | 80.7% | 74% | – | – |
| ChartographyVisual chart recognition · with tools | 89% | 88.4% | 83.4% | – | – |
- Bold marks the best reported score in each row.
- Claude Opus 5.5 results use adaptive thinking at max effort unless noted.
- Terminal-Bench 4.0 shows each model's highest score: Claude Opus 5.5 at xhigh effort and GPT-6 Astra at high effort. GPT-6 Astra and GPT-5.6 Sol figures there are as reported by OpenAI.
- A dash means the lab did not report a result for that model.
Accuracy against API cost · reported by OpenAI on 9 Sept 2026 · exact values
Tests agents on complex terminal-based tasks, including software engineering, system configuration and data analysis.
- GPT-6 Astra
- GPT-5.6 Sol
- Claude Fable 5.1
- Claude Fable 5
- Claude Opus 5
Show data table
| Model | Effort | Accuracy | Cost per task |
|---|---|---|---|
| GPT-6 Astra | low | 49.7% | $4.95 |
| GPT-6 Astra | medium | 53.9% | $6.15 |
| GPT-6 Astra | high | 57.9% | $7.21 |
| GPT-6 Astra | xhigh | 57.6% | $7.48 |
| GPT-6 Astra | max | 56.7% | $10.3 |
| GPT-5.6 Sol | low | 7.9% | $1.46 |
| GPT-5.6 Sol | medium | 20.9% | $2.69 |
| GPT-5.6 Sol | high | 26.1% | $4.12 |
| GPT-5.6 Sol | xhigh | 28.5% | $5.39 |
| GPT-5.6 Sol | max | 37.3% | $7.89 |
| Claude Fable 5.1 | low | 40.2% | $5.70 |
| Claude Fable 5.1 | medium | 43.4% | $7.80 |
| Claude Fable 5.1 | high | 49.4% | $10.5 |
| Claude Fable 5.1 | xhigh | 51.3% | $15.8 |
| Claude Fable 5.1 | max | 55.8% | $19.5 |
| Claude Fable 5 | max | 44.5% | $22.2 |
| Claude Opus 5 | max | 52.6% | $18.4 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- Claude Fable 5 and Claude Opus 5 are shown at max effort only, as published.
Accuracy against cost · reported by Anthropic on 9 Nov 2026 · exact values
Whether a model's code changes would be merged by the maintainers of open-source repositories.
- Claude Fable 5.2
- Claude Opus 5.5
- Claude Fable 5.1
- GPT-6 Astra
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| Claude Fable 5.2 | low | 34.2% | $2.10 |
| Claude Fable 5.2 | medium | 37.6% | $2.48 |
| Claude Fable 5.2 | high | 45.3% | $6.72 |
| Claude Fable 5.2 | xhigh | 49.1% | $14.2 |
| Claude Fable 5.2 | max | 45.6% | $44.7 |
| Claude Opus 5.5 | low | 14.7% | $0.75 |
| Claude Opus 5.5 | medium | 30.1% | $1.70 |
| Claude Opus 5.5 | high | 34.5% | $2.42 |
| Claude Opus 5.5 | xhigh | 43.4% | $5.18 |
| Claude Opus 5.5 | max | 45.5% | $12.8 |
| Claude Fable 5.1 | low | 20.6% | $3.01 |
| Claude Fable 5.1 | medium | 21.5% | $3.89 |
| Claude Fable 5.1 | high | 25.7% | $5.39 |
| Claude Fable 5.1 | xhigh | 32.4% | $8.68 |
| Claude Fable 5.1 | max | 31.7% | $11.5 |
| GPT-6 Astra | low | 31.1% | $2.44 |
| GPT-6 Astra | medium | 36.8% | $3.49 |
| GPT-6 Astra | high | 38.3% | $4.28 |
| GPT-6 Astra | xhigh | 41.4% | $4.54 |
| GPT-6 Astra | max | 42.8% | $6.68 |
- Exact score and cost values from the chart data published with Anthropic's announcement.
- Anthropic's chart also shows Gemini 4 Carbon, which has no published API price, so it is not tracked here.
Score against cost · reported by OpenAI on 26 Oct 2026 · exact values
Professional computer-use tasks, such as turning architectural drawings into a 3D model or logging an animated film shot by shot.
- GPT-6.1 Terra
- GPT-5.6 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- GPT-6 Luna
- Claude Opus 5.5
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 55.9% | $0.16 |
| GPT-6.1 Terra | medium | 57.6% | $0.29 |
| GPT-6.1 Terra | high | 57.2% | $0.39 |
| GPT-6.1 Terra | xhigh | 58.5% | $0.66 |
| GPT-6.1 Terra | max | 60% | $1.01 |
| GPT-5.6 Terra | low | 40.2% | $0.59 |
| GPT-5.6 Terra | medium | 42.6% | $0.84 |
| GPT-5.6 Terra | high | 46.6% | $1.72 |
| GPT-5.6 Terra | xhigh | 48.5% | $2.50 |
| GPT-5.6 Terra | max | 50.7% | $3.58 |
| GPT-6.1 Sol | low | 52.4% | $0.70 |
| GPT-6.1 Sol | medium | 53.8% | $1.31 |
| GPT-6.1 Sol | high | 55.3% | $1.45 |
| GPT-6.1 Sol | xhigh | 56.5% | $1.45 |
| GPT-6.1 Sol | max | 58.6% | $2.00 |
| GPT-6 Astra | low | 53.4% | $3.55 |
| GPT-6 Astra | medium | 57.6% | $4.82 |
| GPT-6 Astra | high | 57.8% | $5.15 |
| GPT-6 Astra | xhigh | 58.3% | $5.91 |
| GPT-6 Astra | max | 59.3% | $7.24 |
| GPT-6 Luna | low | 36.3% | $0.022 |
| GPT-6 Luna | medium | 46.8% | $0.093 |
| GPT-6 Luna | high | 43.6% | $0.087 |
| GPT-6 Luna | xhigh | 47.9% | $0.093 |
| GPT-6 Luna | max | 50.9% | $0.14 |
| Claude Opus 5.5 | max | 63.2% | $8.82 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
Score against cost · reported by OpenAI on 26 Oct 2026 · exact values
Visual reasoning across expert-level, multi-discipline questions.
- GPT-6.1 Terra
- GPT-5.6 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- GPT-6 Luna
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 83.7% | $0.0038 |
| GPT-6.1 Terra | medium | 84.6% | $0.0041 |
| GPT-6.1 Terra | high | 85% | $0.0045 |
| GPT-6.1 Terra | xhigh | 85.4% | $0.0055 |
| GPT-6.1 Terra | max | 85.5% | $0.012 |
| GPT-5.6 Terra | low | 74.9% | $0.011 |
| GPT-5.6 Terra | medium | 76.4% | $0.012 |
| GPT-5.6 Terra | high | 77.5% | $0.014 |
| GPT-5.6 Terra | xhigh | 78.7% | $0.02 |
| GPT-5.6 Terra | max | 81.7% | $0.079 |
| GPT-6.1 Sol | low | 84% | $0.0076 |
| GPT-6.1 Sol | medium | 84.7% | $0.0079 |
| GPT-6.1 Sol | high | 85.8% | $0.0088 |
| GPT-6.1 Sol | xhigh | 85.8% | $0.01 |
| GPT-6.1 Sol | max | 86.3% | $0.013 |
| GPT-6 Astra | low | 85.3% | $0.038 |
| GPT-6 Astra | medium | 85.3% | $0.039 |
| GPT-6 Astra | high | 86.4% | $0.044 |
| GPT-6 Astra | xhigh | 86.5% | $0.051 |
| GPT-6 Astra | max | 86.7% | $0.065 |
| GPT-6 Luna | low | 73.5% | $0.00049 |
| GPT-6 Luna | medium | 75.1% | $0.00061 |
| GPT-6 Luna | high | 76.5% | $0.00074 |
| GPT-6 Luna | xhigh | 77.9% | $0.00095 |
| GPT-6 Luna | max | 79.9% | $0.0021 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
Score against cost · reported by OpenAI on 26 Oct 2026 · exact values
Multistep tasks completed by agents using desktop apps, such as building a 3D model from a technical drawing.
- GPT-6.1 Terra
- GPT-5.6 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- GPT-6 Luna
- Claude Sonnet 5.5
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 53.8% | $0.36 |
| GPT-6.1 Terra | medium | 66.7% | $0.50 |
| GPT-6.1 Terra | high | 68.7% | $0.51 |
| GPT-6.1 Terra | xhigh | 71.3% | $0.62 |
| GPT-6.1 Terra | max | 74.2% | $1.19 |
| GPT-5.6 Terra | low | 24.1% | $0.44 |
| GPT-5.6 Terra | medium | 27.4% | $0.59 |
| GPT-5.6 Terra | high | 37.7% | $1.19 |
| GPT-5.6 Terra | xhigh | 48.4% | $2.25 |
| GPT-5.6 Terra | max | 57.1% | $4.76 |
| GPT-6.1 Sol | low | 60.9% | $0.38 |
| GPT-6.1 Sol | medium | 73.5% | $0.65 |
| GPT-6.1 Sol | high | 75.5% | $0.90 |
| GPT-6.1 Sol | xhigh | 75.9% | $0.95 |
| GPT-6.1 Sol | max | 78% | $1.15 |
| GPT-6 Astra | low | 63.3% | $2.32 |
| GPT-6 Astra | medium | 75.4% | $4.37 |
| GPT-6 Astra | high | 78.3% | $5.76 |
| GPT-6 Astra | xhigh | 79% | $6.60 |
| GPT-6 Astra | max | 80.9% | $8.17 |
| GPT-6 Luna | low | 7.5% | $0.02 |
| GPT-6 Luna | medium | 26.3% | $0.079 |
| GPT-6 Luna | high | 39.7% | $0.14 |
| GPT-6 Luna | xhigh | 44.6% | $0.19 |
| GPT-6 Luna | max | 55.5% | $0.30 |
| Claude Sonnet 5.5 | low | 57.8% | $0.68 |
| Claude Sonnet 5.5 | medium | 65.9% | $0.93 |
| Claude Sonnet 5.5 | high | 73.3% | $1.39 |
| Claude Sonnet 5.5 | xhigh | 81% | $2.22 |
| Claude Sonnet 5.5 | max | 83.9% | $5.74 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
Error rate against cost · reported by OpenAI on 26 Oct 2026 · exact values
Answers containing at least one factual error on deliberately difficult prompts. Lower is better.
- GPT-6.1 Terra
- GPT-5.6 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- GPT-6 Luna
Show data table
| Model | Effort | Answers with any factual error (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 16.7% | $0.024 |
| GPT-6.1 Terra | medium | 10.1% | $0.029 |
| GPT-6.1 Terra | high | 7.7% | $0.036 |
| GPT-6.1 Terra | xhigh | 5% | $0.05 |
| GPT-6.1 Terra | max | 4.3% | $0.12 |
| GPT-5.6 Terra | low | 53.6% | $0.055 |
| GPT-5.6 Terra | medium | 51.5% | $0.061 |
| GPT-5.6 Terra | high | 41.1% | $0.078 |
| GPT-5.6 Terra | xhigh | 33.9% | $0.11 |
| GPT-5.6 Terra | max | 22.3% | $0.25 |
| GPT-6.1 Sol | low | 14.7% | $0.048 |
| GPT-6.1 Sol | medium | 12% | $0.056 |
| GPT-6.1 Sol | high | 6.6% | $0.079 |
| GPT-6.1 Sol | xhigh | 5.5% | $0.099 |
| GPT-6.1 Sol | max | 4.4% | $0.12 |
| GPT-6 Astra | low | 13.3% | $0.26 |
| GPT-6 Astra | medium | 9.7% | $0.29 |
| GPT-6 Astra | high | 5.9% | $0.43 |
| GPT-6 Astra | xhigh | 4.6% | $0.55 |
| GPT-6 Astra | max | 4.2% | $0.66 |
| GPT-6 Luna | low | 44.3% | $0.0025 |
| GPT-6 Luna | medium | 31.2% | $0.0035 |
| GPT-6 Luna | high | 27.6% | $0.0047 |
| GPT-6 Luna | xhigh | 24.5% | $0.0061 |
| GPT-6 Luna | max | 18.4% | $0.013 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Prompts are de-identified conversations where users flagged an earlier model error. They are not representative of typical usage.
Score against cost · reported by OpenAI on 26 Oct 2026 · exact values
Coding agents solving problems from the command line, such as fixing a webpage whose layout shifts as it loads.
- GPT-6.1 Terra
- GPT-5.6 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- Claude Sonnet 5.5
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 24.8% | $0.24 |
| GPT-6.1 Terra | medium | 35.8% | $0.46 |
| GPT-6.1 Terra | high | 41.8% | $0.65 |
| GPT-6.1 Terra | xhigh | 46.4% | $0.99 |
| GPT-6.1 Terra | max | 50.3% | $1.84 |
| GPT-5.6 Terra | max | 21.5% | $5.25 |
| GPT-6.1 Sol | low | 47.1% | $0.76 |
| GPT-6.1 Sol | medium | 52.7% | $1.02 |
| GPT-6.1 Sol | high | 53% | $1.28 |
| GPT-6.1 Sol | xhigh | 53.4% | $1.28 |
| GPT-6.1 Sol | max | 55.9% | $1.98 |
| GPT-6 Astra | low | 50.6% | $4.93 |
| GPT-6 Astra | medium | 54.4% | $6.06 |
| GPT-6 Astra | high | 58.1% | $7.18 |
| GPT-6 Astra | xhigh | 57.9% | $7.45 |
| GPT-6 Astra | max | 58.2% | $10.3 |
| Claude Sonnet 5.5 | low | 20% | $0.62 |
| Claude Sonnet 5.5 | medium | 28.8% | $0.68 |
| Claude Sonnet 5.5 | high | 43% | $1.46 |
| Claude Sonnet 5.5 | xhigh | 61.5% | $4.34 |
| Claude Sonnet 5.5 | max | 70.6% | $10.4 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- GPT-5.6 Terra is shown at its published max-effort score only.
Score against cost · reported by OpenAI on 26 Oct 2026 · exact values
Scientific research tasks completed through a terminal, such as tracking cell divisions in a time-lapse video.
- GPT-6.1 Terra
- GPT-6.1 Sol
- GPT-6 Astra
- Claude Opus 5.5 with fallbacks
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Terra | low | 24.3% | $0.47 |
| GPT-6.1 Terra | medium | 34.3% | $0.95 |
| GPT-6.1 Terra | high | 39.7% | $1.24 |
| GPT-6.1 Terra | xhigh | 43.4% | $1.72 |
| GPT-6.1 Terra | max | 58.3% | $4.15 |
| GPT-6.1 Sol | low | 43.7% | $1.79 |
| GPT-6.1 Sol | medium | 47.6% | $2.34 |
| GPT-6.1 Sol | high | 51.1% | $2.76 |
| GPT-6.1 Sol | xhigh | 53.7% | $2.89 |
| GPT-6.1 Sol | max | 57% | $5.47 |
| GPT-6 Astra | low | 55.4% | $11.4 |
| GPT-6 Astra | medium | 57.4% | $12.3 |
| GPT-6 Astra | high | 62% | $14.9 |
| GPT-6 Astra | xhigh | 60.9% | $15.8 |
| GPT-6 Astra | max | 68.1% | $23.8 |
| Claude Opus 5.5 with fallbacks | max | 63.3% | $23.2 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Opus 5.5 results use fallback models as reported by OpenAI; these are not standalone Opus 5.5 results.
Score against cost · reported by OpenAI on 29 Sept 2026 · exact values
Agentic software engineering performance across reasoning effort settings.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 64.38% | $0.17 |
| GPT-6.1 Sol | medium | 73.01% | $0.42 |
| GPT-6.1 Sol | high | 75.22% | $0.65 |
| GPT-6.1 Sol | xhigh | 71.9% | $0.79 |
| GPT-6.1 Sol | max | 71.9% | $1.57 |
| GPT-6 Sol | low | 37.17% | $0.16 |
| GPT-6 Sol | medium | 56.64% | $0.38 |
| GPT-6 Sol | high | 65.27% | $0.64 |
| GPT-6 Sol | xhigh | 66.59% | $1.00 |
| GPT-6 Sol | max | 68.81% | $2.74 |
| GPT-6 Astra | low | 67.04% | $1.60 |
| GPT-6 Astra | medium | 72.79% | $3.08 |
| GPT-6 Astra | high | 73.23% | $3.92 |
| GPT-6 Astra | xhigh | 74.12% | $4.43 |
| GPT-6 Astra | max | 73.23% | $7.50 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
Score against cost · reported by OpenAI on 29 Sept 2026 · exact values
Accuracy on professional questions using complex PDF documents.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
- Claude Opus 5.5 with fallbacks
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 27% | $0.33 |
| GPT-6.1 Sol | medium | 30% | $0.34 |
| GPT-6.1 Sol | high | 32% | $0.35 |
| GPT-6.1 Sol | xhigh | 31.8% | $0.37 |
| GPT-6.1 Sol | max | 31% | $0.42 |
| GPT-6 Sol | low | 21.8% | $0.33 |
| GPT-6 Sol | medium | 25.4% | $0.34 |
| GPT-6 Sol | high | 28% | $0.35 |
| GPT-6 Sol | xhigh | 23.8% | $0.37 |
| GPT-6 Sol | max | 24.8% | $0.43 |
| GPT-6 Astra | low | 30.4% | $1.70 |
| GPT-6 Astra | medium | 30.4% | $1.72 |
| GPT-6 Astra | high | 31% | $1.79 |
| GPT-6 Astra | xhigh | 32.2% | $1.91 |
| GPT-6 Astra | max | 31% | $2.08 |
| Claude Opus 5.5 with fallbacks | low | 25.6% | $0.76 |
| Claude Opus 5.5 with fallbacks | medium | 25.6% | $0.80 |
| Claude Opus 5.5 with fallbacks | high | 28.8% | $0.83 |
| Claude Opus 5.5 with fallbacks | xhigh | 26.6% | $0.96 |
| Claude Opus 5.5 with fallbacks | max | 26.2% | $1.55 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Opus 5.5 results use fallback models as reported by OpenAI; these are not standalone Opus 5.5 results.
Score against cost · reported by OpenAI on 29 Sept 2026 · exact values
End-to-end business workflows using 47 tools across sales, marketing, operations, support, finance and HR.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
- Claude Opus 5.5 with fallbacks
- Claude Fable 5.1 with Opus 5 fallback
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 24.7% | $0.16 |
| GPT-6.1 Sol | medium | 31.7% | $0.19 |
| GPT-6.1 Sol | high | 33.2% | $0.23 |
| GPT-6.1 Sol | xhigh | 35.5% | $0.25 |
| GPT-6.1 Sol | max | 36.1% | $0.30 |
| GPT-6 Sol | low | 21.16% | $0.19 |
| GPT-6 Sol | medium | 26.94% | $0.21 |
| GPT-6 Sol | high | 31.2% | $0.24 |
| GPT-6 Sol | xhigh | 33.18% | $0.27 |
| GPT-6 Sol | max | 31.96% | $0.34 |
| GPT-6 Astra | low | 30.3% | $1.08 |
| GPT-6 Astra | medium | 34.1% | $1.27 |
| GPT-6 Astra | high | 37.1% | $1.44 |
| GPT-6 Astra | xhigh | 39% | $1.50 |
| GPT-6 Astra | max | 41.4% | $1.73 |
| Claude Opus 5.5 with fallbacks | low | 24.2% | $0.51 |
| Claude Opus 5.5 with fallbacks | medium | 29.53% | $0.65 |
| Claude Opus 5.5 with fallbacks | high | 33.03% | $0.71 |
| Claude Opus 5.5 with fallbacks | xhigh | 35.77% | $0.89 |
| Claude Opus 5.5 with fallbacks | max | 42.47% | $1.44 |
| Claude Fable 5.1 with Opus 5 fallback | max | 31.4% | $2.45 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Opus 5.5 results use fallback models as reported by OpenAI; these are not standalone Opus 5.5 results.
- Fable 5.1 is reported at max effort only. Its cost omits fallbacks, which occurred on approximately 40% of tasks, so its actual cost is higher.
Score against cost · reported by OpenAI on 29 Sept 2026 · exact values
Partial reward on long-horizon computer-use workflows, using the offline set from release v2026.08.08.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 58.96% | $0.42 |
| GPT-6.1 Sol | medium | 66.84% | $0.77 |
| GPT-6.1 Sol | high | 69.56% | $0.96 |
| GPT-6.1 Sol | xhigh | 69.38% | $1.05 |
| GPT-6.1 Sol | max | 71.42% | $1.27 |
| GPT-6 Sol | low | 43.9% | $1.01 |
| GPT-6 Sol | medium | 54% | $1.38 |
| GPT-6 Sol | high | 58.29% | $1.71 |
| GPT-6 Sol | xhigh | 60.54% | $2.30 |
| GPT-6 Sol | max | 64.43% | $3.37 |
| GPT-6 Astra | low | 62.17% | $2.72 |
| GPT-6 Astra | medium | 69.25% | $5.36 |
| GPT-6 Astra | high | 70.02% | $6.91 |
| GPT-6 Astra | xhigh | 71.27% | $7.49 |
| GPT-6 Astra | max | 73.49% | $9.44 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
Score against cost · reported by OpenAI on 29 Sept 2026 · exact values
Scientific research tasks completed through a terminal.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
- Claude Opus 5.5 with fallbacks
Show data table
| Model | Effort | Score (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 43.71% | $1.79 |
| GPT-6.1 Sol | medium | 47.56% | $2.34 |
| GPT-6.1 Sol | high | 51.14% | $2.76 |
| GPT-6.1 Sol | xhigh | 53.71% | $2.89 |
| GPT-6.1 Sol | max | 57.02% | $5.47 |
| GPT-6 Sol | low | 9.17% | $3.00 |
| GPT-6 Sol | medium | 14.49% | $4.41 |
| GPT-6 Sol | high | 14.61% | $4.63 |
| GPT-6 Sol | xhigh | 25.29% | $6.77 |
| GPT-6 Sol | max | 27.59% | $12.2 |
| GPT-6 Astra | low | 55.43% | $11.4 |
| GPT-6 Astra | medium | 57.43% | $12.3 |
| GPT-6 Astra | high | 62% | $15 |
| GPT-6 Astra | xhigh | 60.86% | $15.8 |
| GPT-6 Astra | max | 68.1% | $23.8 |
| Claude Opus 5.5 with fallbacks | max | 63.33% | $23.2 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Opus 5.5 results use fallback models as reported by OpenAI; these are not standalone Opus 5.5 results.
Error rate against cost · reported by OpenAI on 29 Sept 2026 · exact values
Answers containing at least one factual error on deliberately difficult prompts. Lower is better.
- GPT-6.1 Sol
- GPT-6 Sol
- GPT-6 Astra
Show data table
| Model | Effort | Answers with any factual error (%) | Cost per task |
|---|---|---|---|
| GPT-6.1 Sol | low | 7.72% | $0.045 |
| GPT-6.1 Sol | medium | 6.29% | $0.056 |
| GPT-6.1 Sol | high | 4.52% | $0.082 |
| GPT-6.1 Sol | xhigh | 4.12% | $0.10 |
| GPT-6.1 Sol | max | 4.61% | $0.13 |
| GPT-6 Sol | low | 11.42% | $0.05 |
| GPT-6 Sol | medium | 6.86% | $0.069 |
| GPT-6 Sol | high | 5.14% | $0.099 |
| GPT-6 Sol | xhigh | 4.52% | $0.13 |
| GPT-6 Sol | max | 4.57% | $0.18 |
| GPT-6 Astra | low | 6.26% | $0.24 |
| GPT-6 Astra | medium | 4.41% | $0.31 |
| GPT-6 Astra | high | 3.9% | $0.48 |
| GPT-6 Astra | xhigh | 3.99% | $0.60 |
| GPT-6 Astra | max | 3.91% | $0.79 |
- Exact score and cost values from the chart data embedded in OpenAI's announcement.
- Prompts are de-identified conversations where users flagged an earlier model error. They are not representative of typical usage.
Score against cost · reported by OpenAI on 22 Sept 2026 · exact values
AI agents are tested on end-to-end workflows using 47 tools across sales, marketing, operations, support, finance and HR.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
- Claude Opus 5
- Claude Fable 5.1
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 1.2% | $0.006 |
| GPT-6 Luna | medium | 9.4% | $0.016 |
| GPT-6 Luna | high | 14.5% | $0.021 |
| GPT-6 Luna | xhigh | 12.6% | $0.025 |
| GPT-6 Luna | max | 20.7% | $0.037 |
| GPT-5.6 Luna | low | 1.8% | $0.01 |
| GPT-5.6 Luna | medium | 4.3% | $0.02 |
| GPT-5.6 Luna | high | 9.1% | $0.05 |
| GPT-5.6 Luna | xhigh | 12.9% | $0.06 |
| GPT-5.6 Luna | max | 17% | $0.07 |
| GPT-6 Sol | low | 21.2% | $0.19 |
| GPT-6 Sol | medium | 26.9% | $0.21 |
| GPT-6 Sol | high | 31.2% | $0.24 |
| GPT-6 Sol | xhigh | 33.2% | $0.27 |
| GPT-6 Sol | max | 32% | $0.34 |
| GPT-5.6 Sol | low | 11.7% | $0.31 |
| GPT-5.6 Sol | medium | 19.6% | $0.42 |
| GPT-5.6 Sol | high | 24.8% | $0.47 |
| GPT-5.6 Sol | xhigh | 26.3% | $0.54 |
| GPT-5.6 Sol | max | 28.8% | $0.67 |
| GPT-6 Astra | low | 30.3% | $1.08 |
| GPT-6 Astra | medium | 34.1% | $1.27 |
| GPT-6 Astra | high | 37.1% | $1.44 |
| GPT-6 Astra | xhigh | 39% | $1.50 |
| GPT-6 Astra | max | 41.4% | $1.73 |
| Claude Opus 5 | low | 20.4% | $1.64 |
| Claude Opus 5 | medium | 23.9% | $2.22 |
| Claude Opus 5 | high | 20.5% | $2.27 |
| Claude Opus 5 | xhigh | 25.3% | $2.71 |
| Claude Opus 5 | max | 26.9% | $3.05 |
| Claude Fable 5.1 | max | 31.4% | $2.45 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- Claude Fable 5.1 ran with Claude Opus 5 as a fallback. OpenAI notes its cost omits the fallback runs (about 40% of tasks), so its real cost is higher.
Score against cost · reported by OpenAI on 22 Sept 2026 · exact values
AI agents are evaluated on long-horizon, economically valuable tasks spanning 55 sub-industries of professional computer work.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
- Claude Opus 5
- Claude Fable 5
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 36.3% | $0.025 |
| GPT-6 Luna | medium | 46.8% | $0.11 |
| GPT-6 Luna | high | 43.6% | $0.11 |
| GPT-6 Luna | xhigh | 47.9% | $0.11 |
| GPT-6 Luna | max | 50.9% | $0.15 |
| GPT-5.6 Luna | low | 31% | $0.18 |
| GPT-5.6 Luna | medium | 38.5% | $0.38 |
| GPT-5.6 Luna | high | 46.1% | $0.84 |
| GPT-5.6 Luna | xhigh | 49.4% | $1.55 |
| GPT-5.6 Luna | max | 50.4% | $2.57 |
| GPT-6 Sol | low | 48.7% | $0.86 |
| GPT-6 Sol | medium | 53.1% | $1.27 |
| GPT-6 Sol | high | 52.6% | $1.53 |
| GPT-6 Sol | xhigh | 55.4% | $1.67 |
| GPT-6 Sol | max | 56.4% | $2.93 |
| GPT-5.6 Sol | low | 45.1% | $1.63 |
| GPT-5.6 Sol | medium | 52.1% | $3.41 |
| GPT-5.6 Sol | high | 52.4% | $3.79 |
| GPT-5.6 Sol | xhigh | 53.6% | $5.08 |
| GPT-5.6 Sol | max | 52.8% | $7.13 |
| GPT-6 Astra | low | 53.4% | $3.07 |
| GPT-6 Astra | medium | 57.6% | $4.10 |
| GPT-6 Astra | high | 57.8% | $4.64 |
| GPT-6 Astra | xhigh | 58.3% | $5.40 |
| GPT-6 Astra | max | 59.3% | $6.23 |
| Claude Opus 5 | low | 51.9% | $4.06 |
| Claude Opus 5 | medium | 53% | $5.29 |
| Claude Opus 5 | high | 55.9% | $7.29 |
| Claude Opus 5 | xhigh | 55.5% | $10 |
| Claude Opus 5 | max | 52.7% | $9.76 |
| Claude Fable 5 | xhigh | 48.7% | $28.6 |
| Claude Fable 5 | adaptive | 41.3% | $15.2 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- Claude Fable 5 ran with Claude Opus 4.8 as a fallback.
Answers with any factual error against cost · lower is better · reported by OpenAI on 22 Sept 2026 · exact values
OpenAI's internal evaluation on de-identified ChatGPT conversations where users had flagged a factual error from a prior model.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
Show data table
| Model | Effort | Error rate | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 27.7% | $0.0024 |
| GPT-6 Luna | medium | 17.5% | $0.0033 |
| GPT-6 Luna | high | 12.5% | $0.0045 |
| GPT-6 Luna | xhigh | 10.2% | $0.0062 |
| GPT-6 Luna | max | 7.6% | $0.012 |
| GPT-5.6 Luna | low | 36.8% | $0.0045 |
| GPT-5.6 Luna | medium | 24.7% | $0.0062 |
| GPT-5.6 Luna | high | 17.3% | $0.011 |
| GPT-5.6 Luna | xhigh | 12.2% | $0.015 |
| GPT-5.6 Luna | max | 12% | $0.022 |
| GPT-6 Sol | low | 11.4% | $0.05 |
| GPT-6 Sol | medium | 6.9% | $0.069 |
| GPT-6 Sol | high | 5.1% | $0.099 |
| GPT-6 Sol | xhigh | 4.5% | $0.13 |
| GPT-6 Sol | max | 4.6% | $0.18 |
| GPT-5.6 Sol | low | 19.4% | $0.10 |
| GPT-5.6 Sol | medium | 15% | $0.15 |
| GPT-5.6 Sol | high | 10.8% | $0.23 |
| GPT-5.6 Sol | xhigh | 8.4% | $0.39 |
| GPT-5.6 Sol | max | 8.5% | $0.87 |
| GPT-6 Astra | low | 6.3% | $0.24 |
| GPT-6 Astra | medium | 4.4% | $0.31 |
| GPT-6 Astra | high | 3.9% | $0.48 |
| GPT-6 Astra | xhigh | 4% | $0.60 |
| GPT-6 Astra | max | 3.9% | $0.79 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- These prompts were chosen because they caused errors before, so error rates are far higher than in typical use.
Score against cost · reported by OpenAI on 22 Sept 2026 · exact values
AI agents write code graded on correctness and mergeability: test quality, scope discipline, code style and codebase standards.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
- Claude Opus 5
- Claude Fable 5.1
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 25.7% | $0.021 |
| GPT-6 Luna | medium | 35.5% | $0.053 |
| GPT-6 Luna | high | 37.3% | $0.067 |
| GPT-6 Luna | xhigh | 37.1% | $0.073 |
| GPT-6 Luna | max | 42.4% | $0.11 |
| GPT-5.6 Luna | low | 15.4% | $0.06 |
| GPT-5.6 Luna | medium | 25.7% | $0.13 |
| GPT-5.6 Luna | high | 35.9% | $0.23 |
| GPT-5.6 Luna | xhigh | 38.9% | $0.31 |
| GPT-5.6 Luna | max | 39.8% | $0.37 |
| GPT-6 Sol | low | 37.3% | $0.45 |
| GPT-6 Sol | medium | 45.9% | $0.80 |
| GPT-6 Sol | high | 47.7% | $1.08 |
| GPT-6 Sol | xhigh | 48.4% | $1.37 |
| GPT-6 Sol | max | 49.3% | $2.14 |
| GPT-5.6 Sol | low | 35.4% | $1.89 |
| GPT-5.6 Sol | medium | 39.9% | $2.69 |
| GPT-5.6 Sol | high | 45.1% | $3.48 |
| GPT-5.6 Sol | xhigh | 46.8% | $4.15 |
| GPT-5.6 Sol | max | 47.5% | $5.19 |
| GPT-6 Astra | low | 45.3% | $1.70 |
| GPT-6 Astra | medium | 48.8% | $2.43 |
| GPT-6 Astra | high | 50.9% | $3.01 |
| GPT-6 Astra | xhigh | 50.6% | $3.28 |
| GPT-6 Astra | max | 53.3% | $4.59 |
| Claude Opus 5 | low | 41.9% | $2.68 |
| Claude Opus 5 | medium | 53.4% | $4.31 |
| Claude Opus 5 | high | 48% | $7.24 |
| Claude Opus 5 | xhigh | 43.6% | $9.14 |
| Claude Opus 5 | max | 48% | $11.4 |
| Claude Fable 5.1 | low | 49.8% | $2.38 |
| Claude Fable 5.1 | medium | 50.9% | $3.28 |
| Claude Fable 5.1 | high | 50.3% | $5.27 |
| Claude Fable 5.1 | xhigh | 48.7% | $9.27 |
| Claude Fable 5.1 | max | 50.3% | $12.8 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
Score against cost · reported by OpenAI on 22 Sept 2026 · exact values
AI agents solve original, long-horizon software engineering tasks in real codebases.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
- Claude Opus 5
- Claude Fable 5
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 2.4% | $0.0057 |
| GPT-6 Luna | medium | 44.5% | $0.052 |
| GPT-6 Luna | high | 59.3% | $0.084 |
| GPT-6 Luna | xhigh | 61.3% | $0.11 |
| GPT-6 Luna | max | 66.6% | $0.22 |
| GPT-5.6 Luna | low | 1.2% | $0.011 |
| GPT-5.6 Luna | medium | 9.3% | $0.031 |
| GPT-5.6 Luna | high | 42.4% | $0.13 |
| GPT-5.6 Luna | xhigh | 56.2% | $0.27 |
| GPT-5.6 Luna | max | 62.2% | $0.53 |
| GPT-6 Sol | low | 37.2% | $0.16 |
| GPT-6 Sol | medium | 56.6% | $0.38 |
| GPT-6 Sol | high | 65.3% | $0.64 |
| GPT-6 Sol | xhigh | 66.6% | $1.00 |
| GPT-6 Sol | max | 68.8% | $2.74 |
| GPT-5.6 Sol | low | 45.4% | $0.82 |
| GPT-5.6 Sol | medium | 61.1% | $1.42 |
| GPT-5.6 Sol | high | 69.4% | $2.66 |
| GPT-5.6 Sol | xhigh | 70.7% | $3.60 |
| GPT-5.6 Sol | max | 72.7% | $6.46 |
| GPT-6 Astra | low | 67% | $1.60 |
| GPT-6 Astra | medium | 72.8% | $3.08 |
| GPT-6 Astra | high | 73.2% | $3.92 |
| GPT-6 Astra | xhigh | 74.1% | $4.43 |
| GPT-6 Astra | max | 73.2% | $7.50 |
| Claude Opus 5 | low | 58.1% | $1.66 |
| Claude Opus 5 | medium | 68.9% | $3.29 |
| Claude Opus 5 | high | 72.8% | $6.08 |
| Claude Opus 5 | xhigh | 73.2% | $9.07 |
| Claude Opus 5 | max | 73.7% | $11.8 |
| Claude Fable 5 | low | 59.6% | $3.76 |
| Claude Fable 5 | medium | 65.4% | $6.09 |
| Claude Fable 5 | high | 68.6% | $9.18 |
| Claude Fable 5 | xhigh | 69.9% | $13.4 |
| Claude Fable 5 | max | 69.7% | $21.6 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- OpenAI used Claude Fable 5 scores because Fable 5.1 scores were unavailable.
Partial reward against cost · release v2026.08.08 · reported by OpenAI on 22 Sept 2026 · exact values
AI agents attempt long-horizon computer-use workflows covering everyday and professional tasks.
- GPT-6 Luna
- GPT-5.6 Luna
- GPT-6 Sol
- GPT-5.6 Sol
- GPT-6 Astra
- Claude Opus 5
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| GPT-6 Luna | low | 8.3% | $0.03 |
| GPT-6 Luna | medium | 31.5% | $0.062 |
| GPT-6 Luna | high | 41.4% | $0.12 |
| GPT-6 Luna | xhigh | 46.7% | $0.16 |
| GPT-6 Luna | max | 52.7% | $0.27 |
| GPT-5.6 Luna | low | 12% | $0.019 |
| GPT-5.6 Luna | medium | 22.7% | $0.052 |
| GPT-5.6 Luna | high | 35.9% | $0.16 |
| GPT-5.6 Luna | xhigh | 48% | $0.36 |
| GPT-5.6 Luna | max | 52.7% | $0.49 |
| GPT-6 Sol | low | 43.9% | $0.97 |
| GPT-6 Sol | medium | 54% | $1.32 |
| GPT-6 Sol | high | 58.3% | $1.64 |
| GPT-6 Sol | xhigh | 60.5% | $2.21 |
| GPT-6 Sol | max | 64.4% | $3.25 |
| GPT-5.6 Sol | low | 29.8% | $0.91 |
| GPT-5.6 Sol | medium | 49.7% | $2.73 |
| GPT-5.6 Sol | high | 56.5% | $4.46 |
| GPT-5.6 Sol | xhigh | 60.9% | $5.93 |
| GPT-5.6 Sol | max | 66.2% | $7.71 |
| GPT-6 Astra | low | 62.2% | $2.55 |
| GPT-6 Astra | medium | 69.3% | $5.10 |
| GPT-6 Astra | high | 70% | $6.60 |
| GPT-6 Astra | xhigh | 71.3% | $7.17 |
| GPT-6 Astra | max | 73.5% | $9.07 |
| Claude Opus 5 | low | 55.2% | $9.88 |
| Claude Opus 5 | medium | 60.3% | $12.7 |
| Claude Opus 5 | high | 65.9% | $15.9 |
| Claude Opus 5 | xhigh | 70.1% | $23.9 |
| Claude Opus 5 | max | 70.2% | $24.1 |
- Exact values from the data labels on OpenAI's charts. OpenAI took competitor scores from publicly available reports.
- Claude Opus 5 values match the official OSWorld 2.0 leaderboard (osworld-v2.xlang.ai).
Accuracy against cost · reported by Anthropic on 22 Sept 2026 · exact values
Measures how well a model completes complex, multi-step professional tasks within a command line interface.
- Claude Opus 5.5
- Claude Fable 5.1
- Claude Opus 5
- GPT-6 Astra
- GPT-5.6 Sol
Show data table
| Model | Effort | Score | Cost per attempt |
|---|---|---|---|
| Claude Opus 5.5 | low | 38.5% | $1.29 |
| Claude Opus 5.5 | medium | 57.6% | $2.94 |
| Claude Opus 5.5 | high | 64.2% | $3.88 |
| Claude Opus 5.5 | xhigh | 66.4% | $7.35 |
| Claude Opus 5.5 | max | 64.8% | $11.2 |
| Claude Fable 5.1 | low | 40.2% | $5.70 |
| Claude Fable 5.1 | medium | 43.4% | $7.80 |
| Claude Fable 5.1 | high | 49.4% | $10.5 |
| Claude Fable 5.1 | xhigh | 51.3% | $15.8 |
| Claude Fable 5.1 | max | 55.8% | $19.5 |
| Claude Opus 5 | low | 28.5% | $4.25 |
| Claude Opus 5 | medium | 41.2% | $7.00 |
| Claude Opus 5 | high | 47% | $10.6 |
| Claude Opus 5 | xhigh | 50.6% | $13.5 |
| Claude Opus 5 | max | 52.3% | $15.8 |
| GPT-6 Astra | low | 49.7% | $4.95 |
| GPT-6 Astra | medium | 53.9% | $6.15 |
| GPT-6 Astra | high | 57.9% | $7.21 |
| GPT-6 Astra | xhigh | 57.6% | $7.48 |
| GPT-6 Astra | max | 56.7% | $10.3 |
| GPT-5.6 Sol | low | 7.9% | $1.46 |
| GPT-5.6 Sol | medium | 20.9% | $2.69 |
| GPT-5.6 Sol | high | 26.1% | $4.12 |
| GPT-5.6 Sol | xhigh | 28.5% | $5.39 |
| GPT-5.6 Sol | max | 37.3% | $7.89 |
- Exact values from the data published with Anthropic's announcement.
- GPT-6 Astra and GPT-5.6 Sol figures are as reported by OpenAI.
Accuracy against cost · reported by Anthropic on 22 Sept 2026 · exact values
Measures whether an agent's code changes would be merged.
- Claude Opus 5.5
- Claude Fable 5.1
- Claude Opus 5
- GPT-6 Astra
- GPT-5.6 Sol
Show data table
| Model | Effort | Score | Cost per task |
|---|---|---|---|
| Claude Opus 5.5 | low | 47.3% | $0.40 |
| Claude Opus 5.5 | medium | 54.64% | $0.80 |
| Claude Opus 5.5 | high | 53.99% | $1.09 |
| Claude Opus 5.5 | xhigh | 51.42% | $2.25 |
| Claude Opus 5.5 | max | 54.43% | $6.19 |
| Claude Fable 5.1 | low | 52.8% | $2.47 |
| Claude Fable 5.1 | medium | 50.91% | $3.28 |
| Claude Fable 5.1 | high | 50.34% | $5.27 |
| Claude Fable 5.1 | xhigh | 48.73% | $9.27 |
| Claude Fable 5.1 | max | 50.28% | $12.8 |
| Claude Opus 5 | low | 41.95% | $2.64 |
| Claude Opus 5 | medium | 53.38% | $4.61 |
| Claude Opus 5 | high | 47.99% | $7.62 |
| Claude Opus 5 | xhigh | 43.65% | $8.99 |
| Claude Opus 5 | max | 48.04% | $12.3 |
| GPT-6 Astra | low | 45.27% | $1.59 |
| GPT-6 Astra | medium | 48.83% | $2.28 |
| GPT-6 Astra | high | 50.94% | $2.85 |
| GPT-6 Astra | xhigh | 50.62% | $3.10 |
| GPT-6 Astra | max | 53.26% | $4.36 |
| GPT-5.6 Sol | low | 35.44% | $1.75 |
| GPT-5.6 Sol | medium | 39.93% | $2.50 |
| GPT-5.6 Sol | high | 45.06% | $3.25 |
| GPT-5.6 Sol | xhigh | 46.84% | $3.88 |
| GPT-5.6 Sol | max | 47.49% | $4.85 |
- Exact values from the data published with Anthropic's announcement.
Elo against cost · reported by Anthropic on 22 Sept 2026 · exact values
Artificial Analysis's evaluation of agents on real-world professional work across 44 occupations.
- Claude Opus 5.5
- Claude Fable 5.1
- Claude Opus 5
- GPT-6 Astra
- GPT-5.6 Sol
Show data table
| Model | Effort | Elo | Cost per task |
|---|---|---|---|
| Claude Opus 5.5 | low | 1224 Elo | $0.21 |
| Claude Opus 5.5 | medium | 1576 Elo | $0.86 |
| Claude Opus 5.5 | high | 1692 Elo | $1.54 |
| Claude Opus 5.5 | xhigh | 1820 Elo | $4.21 |
| Claude Opus 5.5 | max | 1846 Elo | $8.92 |
| Claude Fable 5.1 | low | 1450 Elo | $1.41 |
| Claude Fable 5.1 | medium | 1536 Elo | $2.17 |
| Claude Fable 5.1 | high | 1617 Elo | $3.43 |
| Claude Fable 5.1 | xhigh | 1721 Elo | $7.09 |
| Claude Fable 5.1 | max | 1735 Elo | $9.59 |
| Claude Opus 5 | low | 1294 Elo | $0.58 |
| Claude Opus 5 | medium | 1476 Elo | $1.36 |
| Claude Opus 5 | high | 1581 Elo | $3.03 |
| Claude Opus 5 | xhigh | 1676 Elo | $4.96 |
| Claude Opus 5 | max | 1708 Elo | $6.76 |
| GPT-6 Astra | low | 1366 Elo | $0.85 |
| GPT-6 Astra | medium | 1468 Elo | $1.82 |
| GPT-6 Astra | high | 1485 Elo | $2.43 |
| GPT-6 Astra | xhigh | 1516 Elo | $3.04 |
| GPT-6 Astra | max | 1542 Elo | $4.53 |
| GPT-5.6 Sol | low | 1289 Elo | $0.27 |
| GPT-5.6 Sol | medium | 1403 Elo | $0.60 |
| GPT-5.6 Sol | high | 1480 Elo | $1.11 |
| GPT-5.6 Sol | xhigh | 1548 Elo | $1.70 |
| GPT-5.6 Sol | max | 1588 Elo | $2.81 |
- Exact values from the data published with Anthropic's announcement.
Pass rate against cost · reported by Anthropic on 22 Sept 2026 · exact values
Built by Zapier, tests whether an agent can carry out real business workflows across many connected apps.
- Claude Opus 5.5
- Claude Opus 5
- GPT-6 Astra
- GPT-5.6 Sol
Show data table
| Model | Effort | Pass rate | Cost per task |
|---|---|---|---|
| Claude Opus 5.5 | low | 23.3% | $0.50 |
| Claude Opus 5.5 | medium | 28.6% | $0.64 |
| Claude Opus 5.5 | high | 32% | $0.70 |
| Claude Opus 5.5 | xhigh | 34.4% | $0.86 |
| Claude Opus 5.5 | max | 40% | $1.37 |
| Claude Opus 5 | low | 20.4% | $0.75 |
| Claude Opus 5 | medium | 23.9% | $0.89 |
| Claude Opus 5 | high | 20.6% | $1.03 |
| Claude Opus 5 | xhigh | 25.3% | $1.15 |
| Claude Opus 5 | max | 26.9% | $1.27 |
| GPT-6 Astra | low | 30.3% | $1.08 |
| GPT-6 Astra | medium | 34.1% | $1.28 |
| GPT-6 Astra | high | 37.1% | $1.45 |
| GPT-6 Astra | xhigh | 39% | $1.53 |
| GPT-6 Astra | max | 41.4% | $1.77 |
| GPT-5.6 Sol | low | 11.7% | $0.42 |
| GPT-5.6 Sol | medium | 19.6% | $0.57 |
| GPT-5.6 Sol | high | 24.8% | $0.64 |
| GPT-5.6 Sol | xhigh | 26.3% | $0.74 |
| GPT-5.6 Sol | max | 28.8% | $0.91 |
- Exact values from the data published with Anthropic's announcement.
- Runs were performed by Zapier without fallback models, so safeguard interventions counted as failures.