"AI makes developers 19% slower" is still everywhere.
The catch? Its own authors call it out of date
The METR study behind "AI makes developers 19% slower" is real. Its authors now call that result out of date, and their 2026 redo has caveats of its own.
- Claim
- AI makes devs 19% slower?
- Catch
- NEITHER NUMBER IS SETTLED
- Verdict
- A snapshot, not a verdict
"AI makes experienced developers 19% slower" has been shared since July 2025, and it still turns up in every argument about AI coding tools. The number comes from a real, carefully designed study. But the people who ran it have since published an update, and they now label the original result out of date. Here's what both reports actually say.
The study
In early 2025, from February to June, the nonprofit research group METR ran a randomized controlled trial. 16 experienced open-source developers worked on 246 real issues in their own large repositories: projects averaging more than 22,000 GitHub stars and over a million lines of code. Each issue was randomly assigned to "AI allowed" or "AI disallowed." When AI was allowed, developers mostly used Cursor Pro with Claude 3.5 or 3.7 Sonnet. Tasks took about two hours on average, and developers were paid $150 an hour.
The result
| Developers | Speed with AI |
|---|---|
| Expected beforehand | 24% faster |
| Felt afterward | 20% faster |
| Measured | 19% slower |
With AI allowed, tasks took 19% longer, with a confidence interval running from 2% to 39% longer. The more striking finding is the perception gap: developers expected a 24% speedup, and even after being measured slower they still believed AI had made them 20% faster. METR was careful to list what the study does not show, for example that AI doesn't speed up most developers.
The update
In February 2026, METR reported a second experiment with late-2025 tools: 57 developers (10 returning from the first study and 47 new), 143 repositories and more than 800 tasks, at $50 an hour.
| Late-2025 redo | Speed with AI | Interval (change in task time) |
|---|---|---|
| 10 returning developers | about 18% faster | −38% to +9% |
| 47 new developers | about 4% faster | −15% to +9% |
Both intervals cross zero, so the data are also consistent with no speedup at all.
The catch
Neither number is settled. The original study was 16 developers using early-2025 tools, in big repositories they already knew well. The redo has its own problem, and METR flags it itself: 30% to 50% of developers said they held back tasks they didn't want to do without AI, some people declined to take part at all, and the pay dropped from $150 to $50 an hour. METR calls the new data an unreliable signal that likely underestimates the speedup. And it now marks the original 19% results as out of date.
Verdict
A snapshot of early-2025 AI, not a verdict on AI. The honest summary: experienced developers working in familiar code were slower with early-2025 tools and felt faster anyway, and newer tools may help, by an amount nobody has pinned down yet.
If you're deciding how much to rely on AI at work, measure it on your own tasks and talk it through with your team.
SOURCES
- Joel Becker, Nate Rush, Elizabeth Barnes, David Rein. "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity." METR, July 10, 2025. arXiv:2507.09089, doi:10.48550/arXiv.2507.09089 (metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study).
- Joel Becker, Nate Rush, Tom Cunningham, David Rein, Khalid Mahamud. "We are Changing our Developer Productivity Experiment Design." METR, February 24, 2026 (metr.org/blog/2026-02-24-uplift-update).
Not affiliated with METR. Educational content. If you're deciding how much to rely on AI at work, measure it on your own tasks and talk it through with your team.
