Sign in
Benchmarks
Coding
Dated

There is no trustworthy current leaderboard, so the most recent reliable figure is already months old.

HumanEval

HumanEval checks whether a model can write a short, correct Python function from a plain-English description, graded by whether the code passes a set of unit tests.

What it measures

From 2021 it was the field's default coding test, until models got so good at it that it stopped telling anyone apart.

Example task

The model gets a function signature and a docstring (for example, 'return the list of prime numbers below n') and must write the body so it passes the hidden unit tests. The 164 problems are mostly short, self-contained, and easy by today's standards.

Why you should care

HumanEval is worth knowing precisely because it is over: it was the benchmark that defined AI coding progress for years, and watching it die of success is the clearest example of benchmark saturation. A model scoring below 70 percent here is a red flag, but above 95 percent the number tells you almost nothing, because nearly every model clears it and the test was thoroughly memorized.

How scoring works

Pass@1, the share of problems where the model's first attempt passes all unit tests. OpenAI created it but maintains no leaderboard; the closest independent tracker is EvalPlus, which checks the stricter HumanEval+ variant.

How to read the numbers

The original 2021 baseline (OpenAI's Codex) was 28.8 percent. By 2025 nearly every frontier model clears 90 percent and many reach 95 to 97 percent, so a one or two point difference is statistical noise, not skill.

What to watch

There is no trustworthy current leaderboard, which is why this panel shows a dated anchor. HumanEval is saturated and effectively retired: its 164 problems have been public since 2021 and appear in essentially every code-training corpus (the median problem shows up over 9,000 times on GitHub), so high scores partly reflect memorization. The test is also narrow, mostly easy single-function problems with few unit tests, so it misses real engineering. The 2026 numbers floating around aggregators are vendor self-reported; evaluators have moved to SWE-bench and LiveCodeBench.

To go deeper

The benchmark creates the need to know. The catalog explains the ideas behind it:

Leaders

No reliable current leaderboard. The most cited figure is a dated anchor:

EvalPlus board (last updated DEC 2024, independently evaluated): o1 Preview and o1 Mini tied at the top with 96.3% HumanEval Pass@1 (89.0% on the stricter HumanEval+). No 2025 or 2026 model appears on the board; circulating newer figures are all vendor self-reported.

StatusDated
View the board