Ask a developer if AI makes them faster and you get a confident yes. Measure it and the story gets complicated. That gap, between how fast people feel and how fast they actually are, is the most useful thing we have learned about coding assistants, and almost nobody talks about it because it flatters no one.
The study that ruined the vibe
In July 2025, METR ran the kind of trial this field badly needed. They took 16 experienced open-source developers, gave them 246 real tasks from mature repositories they already knew well, and randomly assigned each task to allow AI tools or forbid them. Not toy problems. Not a coding contest. Real issues in real codebases these people maintain.
The developers expected AI to make them about 24 percent faster. Afterward they believed it had made them about 20 percent faster. The stopwatch said they were 19 percent slower with the tools than without. They were slower and they walked away certain they had been quicker. Read that twice, because the perception gap is the real finding, not the headline number.
Why slower? On code you know cold, typing was never the bottleneck. The time went into reading the model's suggestions, judging them, correcting the ones that were subtly wrong, and steering it back onto a codebase with conventions it did not fully grasp. All of that felt productive. Reviewing generated code feels like work getting done. The clock disagreed.
I want to be fair to the tools here, because it is easy to swing too hard the other way. METR itself now flags the result as historical and says it does not necessarily describe today's models or workflows. The study used early-2025 tooling, mostly Cursor with Claude 3.5 and 3.7. It measured 16 senior people on their own turf, which is close to the worst case for an assistant. It is one careful study, not a law of nature. But it is real evidence, it was randomized, and it points the opposite direction from every vendor deck.
Where the surveys point
The big developer surveys tell a two-sided story that fits. Adoption is enormous and still climbing, most professional developers now use these tools regularly. But trust has been sliding. Stack Overflow's data has shown the share of developers who actually trust the accuracy of AI output falling even as usage rises, and a large chunk report that fixing almost-correct AI code is a real and recurring drain. People keep using it and keep getting burned by it. Both things are true at once.
Surveys measure feelings, and feelings are exactly what the METR study showed we cannot trust on this question. So hold them loosely. The signal worth keeping is the shape: heavy adoption, real enthusiasm, and a persistent undertow of "it looked right and was not."
Where it genuinely helps, and where it drags
Put the trial and the surveys next to your own week and a pattern shows up. The tools win when you know least and the work is most routine.
- Unfamiliar territory: a new language, a library you have never touched, boilerplate you would otherwise copy off the internet. Here the assistant is faster than you because your baseline is slow.
- Throwaway and first drafts: a script, a test scaffold, a regex, a quick prototype where being roughly right is enough and nobody has to maintain it.
It drags in the mirror image of those. Deep in a mature codebase you know well, where the constraints are subtle and the cost of a plausible-but-wrong line is high, the review tax outweighs the typing you saved. That is precisely the METR setup, and precisely where it lost.
The honest verdict
AI coding tools are real and they are useful, and they are not the raw speed multiplier the marketing implies, at least not the way most experienced people use them on code they own. The most reliable effect is not that they make you faster. It is that they make the work feel easier, lower friction, less staring at a blank file. That has genuine value. It is also exactly the feeling that fools you into thinking you sped up when you did not.
So use them where your baseline is weak and stay skeptical where it is strong. And when you feel fast, treat that feeling as a hunch, not a measurement. The one group that knows whether these tools are paying off is the group that actually timed it, and when they timed it, the answer surprised them. Until you time your own, you are guessing too.