SWE-bench leaderboard
SWE-bench evaluates language models on their ability to resolve real GitHub issues from popular Python repositories. The model is given an issue description and the repository state, and must produce a patch that resolves the issue and passes the project's existing test suite. SWE-bench is the benchmark that most closely tracks "useful for autonomous coding agents" because the tasks are not toy problems, the success criteria is the project's actual tests, and the input footprint forces the model to reason over real-world code at scale.
Last refreshed 2026-09-06. 29 models scored on this benchmark.
Full leaderboard
| # | Model | Provider | Score | Released |
|---|---|---|---|---|
| 1 | Claude Opus 5 | Anthropic | 96%vendor | 2026-07 |
| 2 | GLM-5.3 | Z.ai | 95.4%independent | 2026-08 |
| 3 | Claude Fable 5 | Anthropic | 95% | 2026-06 |
| 4 | Kimi K3 | Moonshot AI | 93.4%independent | 2026-07 |
| 5 | Claude Opus 4.8 | Anthropic | 88.6% | 2026-05 |
| 6 | Claude Opus 4.7 | Anthropic | 87.6% | 2026-04 |
| 7 | Grok 4.5 | xAI | 86.6%independent | 2026-07 |
| 8 | Qwen3.8-Max | Alibaba | 85.6%independent | 2026-08 |
| 9 | Claude Sonnet 5 | Anthropic | 85.2%vendor | 2026-06 |
| 10 | GLM-5.2 | Z.ai | 82.8%independent | 2026-06 |
| 11 | GPT-5.5 | OpenAI | 82.6% | 2026-04 |
| 12 | Claude Opus 4.6 | Anthropic | 80.8% | 2026-03 |
| 13 | DeepSeek V4 Pro | DeepSeek | 80.6% | 2026-04 |
| 14 | MiniMax M3 | MiniMax | 80.5%vendor | 2026-06 |
| 15 | Claude Sonnet 4.6 | Anthropic | 79.6% | 2026-02 |
| 16 | DeepSeek V4 Flash | DeepSeek | 79% | 2026-04 |
| 17 | Inkling | Thinking Machines | 77.6%vendor | 2026-07 |
| 18 | Mistral Medium 3.5 | Mistral | 77.6% | 2026-05 |
| 19 | Muse Glimmer 30B | Meta | 76%vendor | 2026-08 |
| 20 | Claude Haiku 4.5 | Anthropic | 73.3% | 2026-01 |
| 21 | Gemini 2.5 Pro | 63.8% | 2026-01 | |
| 22 | Nemotron 3.5 Lightning | NVIDIA | 51.56%vendor | 2026-08 |
| 23 | o3-mini | OpenAI | 49.3% | 2025-01 |
| 24 | o1 | OpenAI | 48.9% | 2024-12 |
| 25 | Mistral Large | Mistral | 47.2% | 2025-11 |
| 26 | DeepSeek V3 | DeepSeek | 42% | 2025-12 |
| 27 | GPT-4.5 | OpenAI | 38% | 2025-12 |
| 28 | GPT-4o | OpenAI | 33.2% | 2024-05 |
| 29 | Llama 4 Maverick | Meta | 24% | 2025-04 |
Score interpretation
Scores are reported as resolution rate (% of issues correctly patched). The headline number on TensorFeed is the SWE-bench Verified subset, the human-validated tasks where the test suite has been confirmed to be a fair signal. Mind the gap between sources: the official swebench.com board tops out around 79% and has taken no new submission since early 2026, while vendor launch materials and independent evaluators now report figures in the low to mid nineties on their own harnesses. Both are real measurements of different setups, and the spread between them is wider than the spread between most models. Anything above 60% is a genuinely useful coding agent; above 90% on a vendor harness, check which harness before you compare.
Why this matters for AI agents
If you are building a coding agent, this is the benchmark that matters most. Models with high SWE-bench scores produce patches that compile, pass tests, and respect existing patterns in the codebase. Models with low SWE-bench scores produce code that looks plausible but breaks the build.
Other benchmarks
Premium API: time-series for SWE-bench
The leaderboard above is a snapshot. Want to see how a model's SWE-bench score has moved over the last 30-90 days, or set a webhook that fires when a score crosses a threshold? The premium API has both:
/api/premium/history/benchmarks/series?model=&benchmark=swe_bench: daily score evolution for one model on this benchmark, 1 credit per call