DeepSeek's benchmark chart is doing work its own harness can't back up

Here's the trick DeepSeek pulled in August: ship a model with one genuinely strong, independently checked result, bury the price hike behind four days of silence, and let a self-reported benchmark chart do the talking before anyone checks whose chart it actually is.

On 12 August 2026, DeepSeek swapped a single version string on its own pricing page. No launch post. No changelog. No fanfare for a model that had been cooking as a preview since 24 April.

Companies confident in a launch tend to want credit for it. DeepSeek instead let the model graduate in the background and only spoke up four days later — not to celebrate the release, but to explain why it was about to cost a great deal more.

Credit where it's due: Vals.ai's neutral SWE-bench Verified pipeline placed V4-Pro-0813 second of 82 models at 96.40%, behind only Claude Opus 5, at roughly a fraction of the cost per test. DeepSeek didn't even claim that number itself — which is exactly why it's the one figure in this whole launch worth taking at face value.

Now the chart DeepSeek did publish. DeepSWE jumps from 12.8 to 62.7 — a gain of 49.9 points, the single biggest number in the entire release. CyberGym goes from 52.7 to 83.3. Terminal Bench 2.1 goes from 72.1 to 87.9. All three are DeepSeek grading its own homework, on a harness nobody else has access to.

Every model loses points moving to the stricter Terminus 2 reference harness — that's normal, harnesses vary. What isn't normal is losing 33 points when the typical hit across the field is 7 to 13. That's not measurement noise. That's a model tuned hard for the test DeepSeek controls.

Zoom out from the cherry-picked wins to a broader, independently run test and the picture flips entirely.

Last. Of seven. On the exact category — agentic coding — that the entire 0813 marketing push is built around. That is not a rounding error in a benchmark suite; it is the headline claim failing its own broadest independent test.

It gets worse on a metric that matters more in production than any coding score: knowing what you don't know.

That's AA-Omniscience, Artificial Analysis's test of whether a model recognises the edge of its own knowledge instead of confidently guessing. A score near zero means the model doesn't reliably know when it's wrong — precisely the failure mode you cannot afford in an autonomous coding agent making a hundred unsupervised decisions.

Strip away the benchmark chart and what actually happened is: DeepSeek raised prices by up to 1,100%, called the cheaper half of the new schedule a '50% discount', and let that framing do the work of not saying 'we just made this more expensive at every hour of the day.' Off-peak input at $0.66 is still 52% above the $0.435 flat rate that held until 16 August. There is no hour where V4-Pro costs what it cost in July.

None of this makes V4-Pro a bad model — the SWE-bench result alone is worth building against for narrow, well-specified coding tasks at scale. It makes it a model whose marketing has outrun its own third-party evidence. Trust the number nobody at DeepSeek gets to grade. Be a lot more sceptical of the ones only DeepSeek can check.