Which AI benchmarks should you stop trusting for model selection?

The TF Verdict, as of 29 May 2026: stop ranking frontier models on MMLU, the original GSM8K, and HumanEval. MMLU-Pro is next on the chopping block. Select on contamination-resistant and held-out evals instead.

Here is the math. MMLU is saturated, with top models near 92.4 (GPT-5.5) against a human-expert baseline of about 89.8. When the field clusters past the humans, the gaps you see are noise, not signal. MMLU-Pro was the patch, and it already reads around 90 (Gemini 3 Pro at roughly 90.1). The patch is wearing out.

HumanEval belongs on the same pile. Frontier models clear pass@1 above 95 percent on it, so it stopped separating anyone years ago. Treat a high HumanEval score as table stakes, not a ranking.

Contamination is the uglier problem. Scale's GSM1k study rebuilt grade-school math from scratch and watched Phi and Mistral families drop up to 13 points. That is memorization wearing a reasoning costume. Frontier labs barely flinched there, which is exactly why GSM1k still has signal and the original GSM8K does not.

GPQA Diamond is the trap people still fall for. Its top four span about half a point, one question on a 198-item test. That is a tiebreaker, not a leaderboard.

The clearest tell: Claude Opus 4.5 scores 80.9 on SWE-bench Verified and 45.9 on SWE-bench Pro. A 35-point gap, which is why labs now cite Pro right next to Verified.

What we trust: SWE-bench Pro for agentic coding, and Humanity's Last Exam, where the top still spreads across real gaps (45.7, 44.7, 44.3). LiveCodeBench earns its spot for a different reason. Its leader is near 92 too, so it is not the low scores; it only ever scores post-cutoff problems, so contamination cannot pile up over time.

Caveat: saturated does not mean useless. A model that flunks MMLU is still disqualified. Use the dead ones to screen out, never to crown a winner. Run your own task-specific eval before you commit budget. The leaderboard is marketing; your workload is the truth.