Coding

AI coding tools are genuinely great and genuinely dangerous, often in the same suggestion. They will write in thirty seconds something that would have taken you twenty minutes, and it will contain a subtle bug you would never have written yourself. Here is the workflow I actually use to get the speed without shipping the mistakes.

"Agent" is the most abused word in AI right now. Vendors slap it on anything that calls an API twice. So let us be plain: an AI agent is a model that can take actions in a loop, decide what to do next based on the result, and keep going until a goal is met or it gives up. That is it. The interesting question is not what they are. It is when you should let one loose.

Coding assistant comparisons usually happen on toy problems with clean answers, which is exactly why they are useless. So I did the opposite. I took one genuinely annoying real bug, the kind that spans a few files and does not announce itself, and pointed three assistants at it: Claude, GPT, and Microsoft new MAI coding model in Copilot. Here is how they actually did.

You wire a language model into your app, ask it to "respond in JSON," and it does. It works the first ten times. Then, on a Tuesday, it wraps the JSON in a markdown code fence, or opens with "Sure, here is the data," or trails off with a comment, and your parser throws, and the pager goes off. "Just ask nicely" is not a strategy. It is a demo that has not failed yet.

The reason it fails is structural. A base language model predicts likely text, and there are a thousand plausible ways to wrap a JSON object in friendly words. A tiny fraction of your calls will land on one of those, and a tiny fraction of a large number is a lot of broken responses. You need something stronger than a polite request.

Say what shape you want, exactly

The first move is to stop describing the output in prose and start defining it as a schema. A JSON Schema, or a typed object in your language that compiles to one, spells out the fields, their types, which are required, and what values are allowed. This does two things. It tells the model precisely what to produce, and it gives you something to validate against afterward, which matters more than people expect.

A schema also kills a whole class of quiet bugs. Without one, the model decides on a Tuesday to call the field "phone_number" instead of "phone," or returns the price as the string "nineteen dollars" instead of the number 19. With a strict schema those drifts either cannot happen or get caught immediately instead of three steps downstream.

Use the mode built for this

Most serious providers now offer a real structured-output feature, and you should use it instead of hoping. There are two flavors worth knowing.

• Structured outputs. You pass a schema and the model is constrained so its output conforms. OpenAI's version, launched in August 2024 with a strict flag, reported 100 percent adherence on their internal schema evals, up from under 40 percent for an older model that was merely asked. That gap is the whole argument.
• Function or tool calling. You describe a function with typed arguments, and the model returns a call to it with the arguments filled in. Same underlying idea, framed as calling code, and the right fit when the JSON is meant to trigger an action.

When the constrained mode is available, the model is not writing free text and then hoping it parses. The valid tokens are restricted as it generates, so malformed JSON becomes close to impossible. That is a completely different reliability story from prompt-and-pray, and it is why the failure rate on the good implementations sits at a fraction of a percent rather than a few percent.

Validate anyway, and retry on purpose

Constrained output handles the shape. It does not handle refusals, truncation from hitting a token limit, or values that are the right type but wrong in context, like a date of February 30. So the last piece is a loop you build yourself.

Parse the response. Validate it against the same schema you sent. If it passes, you are done. If it fails, do not just retry blind: feed the error back to the model, say what broke, and ask it to fix that specific thing. Cap the retries at two or three so a stubborn input cannot spin forever, and log the failures, because a cluster of them usually means your schema is asking for something ambiguous rather than the model misbehaving.

One detail that saves real pain: watch for the refusal path. The good structured-output APIs return a separate refusal field when the model declines, precisely so you do not try to parse an apology as data. Check it before you reach for the parser.

The version of this that holds up in production is boring, and boring is the compliment. A schema the model is constrained to follow, a validation step you never skip, and a short retry loop that tells the model what went wrong. Do that and JSON stops being the flaky part of your pipeline. Skip it, and you will meet the markdown code fence at the worst possible hour.

Kimi K2 belongs to my favorite category of model: the one that makes you recheck the pricing page because the numbers look like a typo. Near-frontier reasoning, notably cheaper than the big names, and it gets to the answer using fewer tokens than its predecessor. Mostly, it holds up. Mostly.

Microsoft did something quietly significant: it dropped its own MAI-Code-1-Flash model into GitHub Copilot, the coding assistant a lot of us use every day. This is a review of how it actually feels to code with, not how it looks in a keynote, because those are very different things.

The demo worked because you were the only person hitting it. Then ten users showed up, a provider had a slow afternoon, and your app started returning blank screens and half-written answers. None of that is an AI problem. It is a plumbing problem, and the plumbing for AI apps is a little different because the calls are slow, expensive, and occasionally rejected on purpose.

Here is the honest version of what it takes to make a model-backed feature that does not fall over the first time the network has a bad day.

Rate limits are a promise, not an insult

Every provider caps how many requests and tokens you can send per minute. When you cross the line you get an HTTP 429. People treat that response like a failure, but it is closer to a traffic light. The provider is telling you exactly how to behave, and most of the time the answer is written in the headers. Anthropic and OpenAI both send a Retry-After value on many 429s. If it is there, wait that long. Do not invent your own number when the server already told you the truth.

The mistake I see most often is retrying immediately, in a tight loop, across every worker at once. That does not recover from a rate limit. It deepens it. You have now turned one rejected request into fifty, all landing at the same instant, guaranteeing the next fifty get rejected too. This is the thundering herd, and it is self-inflicted.

Retries: back off, and add jitter

The fix is exponential backoff. Wait one second, then two, then four, then eight, doubling each time up to a ceiling. That spaces your attempts out so the system has room to recover. But backoff alone is not enough, because if a hundred clients all failed at the same moment, they will all wait exactly one second and then all retry at exactly the same moment. You have synchronized your herd instead of scattering it.

So you add jitter: a random amount of slop on each wait. Instead of sleeping exactly two seconds, sleep somewhere between zero and two. This one trick, randomizing the delay, does more for stability than almost anything else you can do, and it is two lines of code.

A few rules that keep retries sane:

• Retry on 429 and 5xx and network timeouts. Do not retry a 400 or a 401, because a malformed request or a bad key will fail identically every time.
• Cap the total. Three to five attempts, then give up and surface an error. Infinite retries just hide the outage from you while your users feel every second of it.

Timeouts, and the streaming trap

Model calls are slow and the tail is long. A response that usually takes three seconds will sometimes take forty. If you have no timeout, a single stuck request can hold a connection open until something upstream kills it, and you find out when your whole pool is exhausted. Set an explicit timeout. Pick a number based on your real latency distribution, not on the median, because the median is not what hurts you.

Streaming changes the shape of this. When you stream tokens, the connection can be alive and healthy while the model has quietly stalled and stopped sending. A total request timeout will not catch that cleanly. What you want is an idle timeout: if no token has arrived in some number of seconds, treat the stream as dead and fall back. Watch the gaps between chunks, not just the clock on the whole call.

Idempotency and degrading on purpose

Here is the part people skip until it bites them. You send a request, the model does the work, and then the response gets lost on the way back to you. Your retry logic kicks in and sends it again. Now you have paid for the same completion twice, and if that call also wrote a row or charged a card, you have a duplicate in your data.

Send an idempotency key with anything that has a side effect. It is a unique string per logical operation. If the provider or your own service sees the same key twice, it returns the first result instead of doing the work again. Providers support this for exactly this reason. Use it, and a lost response becomes a non-event instead of a double charge.

Sometimes the model is just down, or slow enough that waiting is worse than not answering. Decide ahead of time what happens then. Maybe you drop to a smaller, cheaper model. Maybe you return a cached answer that is a little stale. Maybe you show the raw search results without the AI summary on top. The worst option is a spinner that never resolves, because that teaches people your product is broken.

None of this is glamorous and none of it shows up in a launch video. But it is the difference between a feature that survives contact with real traffic and one that only ever worked on your laptop. The models get the headlines. The retry loop with jitter is what keeps the thing standing.

Moonshot released Kimi K2.7 Code, a trillion-parameter coding model, and buried in the announcement was the number that actually matters: it reaches its answers using roughly 30 percent fewer reasoning tokens than its predecessor. In a world obsessed with capability scores, efficiency is the upgrade people underrate, and it is quietly the more useful one.

Here is an uncomfortable result that deserves more airtime than it gets. In 2025 the research group METR ran a randomized trial with experienced open-source developers on real issues from repositories they already knew well. With AI tools allowed, they finished tasks 19 percent slower. Afterward, the same developers estimated that AI had made them about 20 percent faster. They were slower and felt faster, by nearly the same margin. That gap is the whole story, and it is worth sitting with.

Why fast can feel faster than it is

The researchers pointed at reduced cognitive effort. AI-assisted work felt easier, and we quietly file easier under faster even when the clock disagrees. Watching a model produce a wall of plausible code feels like progress in a way that staring at a blank editor does not, and the feeling is real even when the output is not saving you anything.

The catch is what the feeling hides. You did not write the code, so now you have to read it, and reading someone else's code closely enough to trust it is not free. If it is wrong in a subtle way, you pay twice: once to spot the problem and again to fix it, often after you have already convinced yourself it was fine. The generation was fast. The verification was not, and verification is the part that does not show up in the demo.

Before I am accused of doom: I should note METR themselves later flagged that their study design had problems, partly because the developers who benefit most from AI would not join a no-AI condition even at 50 dollars an hour. So do not read 19 percent as a law of nature. Read it as a real, measured case where the tool that everyone assumed was a speedup was not, and the users could not tell. That is the part that generalizes.

Where the time actually leaks

The productivity leaks are boring and specific, which is exactly why they are easy to miss.

• Reviewing confident nonsense. The failure mode that costs the most is a wrong answer delivered with total assurance: a made-up function that looks real, a plausible statute that does not exist, a config flag that was never a flag. Confident and wrong is more expensive than obviously broken, because obviously broken you catch in a second and confident-wrong you ship.
• Context-switching. Every trip out to the tool and back is a small tax on your attention, and enough small taxes add up to a workday where you were busy and moved nothing.
• The almost-right rabbit hole. The model gets you 80 percent of the way, and you spend longer chasing the last 20 percent through its logic than you would have spent writing the thing yourself. Nudging a nearly-correct output into a correct one can cost more than starting clean.

None of these feel like waste in the moment. They feel like work. That is the trap.

It still works, when you aim it right

I am not telling you to put the tools down. I use them daily and would be slower without them, which is precisely why I take the paradox seriously instead of waving it off. The tools genuinely help, but the win is conditional, and the conditions are learnable.

They shine when verification is cheap or the stakes are low. Boilerplate you can eyeball in seconds. A first draft you were going to heavily rewrite anyway. A language or API you half-remember, where the model jogs your memory faster than the docs. Throwaway scripts. Anything where being roughly right is good enough and checking is quick. In those spots the speedup is real and often large.

They quietly cost you when verification is expensive and correctness is non-negotiable. Subtle logic in unfamiliar code. Anything security-sensitive. Domains where you cannot easily tell right from wrong-but-plausible, which is exactly where the model's confidence is most dangerous, because you have no cheap way to check it.

The skill, and it is a skill, is noticing which situation you are in before you reach for the tool, not after. The developers in that study were not fools. They were experienced people who genuinely could not feel the slowdown while it was happening. That is the real lesson. The tool is not the problem and neither are you. The problem is that speed and the feeling of speed have come apart, and the only fix is to occasionally check the clock instead of trusting the vibe.

xAI reportedly acquired the company behind Cursor, the AI coding editor, for $60 billion in all stock. You can argue about the number all day. What you cannot argue with is the message it sends: the smart money has decided the value in AI is no longer in the model. It is in the thing wrapped around it.