AI Opinion & Analysis
The first time I used an AI assistant to fix a gnarly bug, I felt two things at once: relief that it was solved in four minutes, and a small, nagging sense that I had been robbed. Not by the tool. By myself. Two weeks later the same class of bug showed up in a different file, and I stared at it like a stranger. The AI had solved the problem. I had learned nothing.
This is the part of the AI conversation that gets flattened into slogans. One camp says friction is waste and anything that removes it is pure gain. The other says we are raising a generation that cannot think. Both are too neat. The honest position is that some struggle is where understanding gets built, and some struggle is just pain with no payoff, and the whole skill now is telling them apart.
Not all friction is the same
There is a concept from learning research called desirable difficulty. The idea is that certain kinds of effort, retrieving something from memory instead of rereading it, working a problem before seeing the solution, make the knowledge stick harder precisely because they were hard. The difficulty is not a tax on learning. It is the mechanism of learning. When you struggle to recall how a data structure works and then get it right, you are not being inefficient. You are laying down the thing that lets you recall it next time without help.
Now compare that to the friction of fighting a badly documented API for an afternoon, or hand-formatting a table, or writing the same boilerplate for the tenth time. That friction teaches you nothing you did not already know. It is the pain of a task, not the effort of learning. Handing it to a machine is an unambiguous win, and anyone romanticizing it has confused suffering with growth.
The trouble is that these two kinds of friction feel identical in the moment. Both are annoying. Both make you want the answer now. And the AI is equally happy to dissolve either one. So the tool that saves you from pointless boilerplate is the same tool, with the same one-keystroke ease, that saves you from the productive struggle that would have made you better. It does not know the difference, and if you are not paying attention, neither will you.
The muscle you do not know you are losing
Here is what worries me, and I say this as someone who loves these tools and uses them all day. The loss is invisible while it happens. You do not get a warning that says your debugging intuition is atrophying. You just notice, months later, that you reach for the assistant a beat earlier each time, that the threshold of difficulty at which you give up and ask has quietly dropped. The struggle you skip does not announce itself. It shows up as an absence, later, when the tool is not there or is confidently wrong and you have lost the ability to check it.
There is a real professional risk in this. The senior engineer who is valuable is valuable because of ten thousand small struggles that built judgment. If the next cohort skips those struggles, they arrive at senior titles with junior instincts, and they will not know it, because the code shipped and the tests passed the whole way up.
A rule I actually use
I have started asking one question before I hand something off: will I need to understand this again? If the answer is yes, if this is a domain I work in, a system I own, a skill I want, I do the struggle first and let the AI check my work or fill the gaps. If the answer is no, if this is a one-time chore in a corner of the world I will never revisit, I hand it over without a shred of guilt and move on with my afternoon.
This is not a call to suffer for its own sake. Suffering for its own sake is a religion, not an engineering practice. It is a call to be deliberate about which difficulties you outsource, because the machine will take all of them if you let it, and a few of them were the point.
I still fixed that bug in four minutes. I would do it again. But now, on the problems I actually care about getting good at, I make myself sit in the not-knowing a little longer before I ask. It is slower. It is also the only part of the work that is still making me better, and I am not ready to give that up for a faster Tuesday.
Every few months a bigger model lands and the timeline decides intelligence just went up a notch. Meanwhile the people actually shipping features are not sitting around waiting for more parameters. They are fighting a different battle entirely, one that no benchmark score fixes: getting the right information in front of the model at the right moment. That, not size, is where most AI features live or die now.
The model is smart. It just does not know your stuff.
A frontier model has read a staggering slice of the public internet. What it has not read is your customer's last three support tickets, your internal pricing rules, the state of the order the user is asking about, or the document sitting in the tab next to your app. On everything that matters to your actual product, the smartest model on earth starts out ignorant, and no amount of extra parameters changes that. The knowledge it needs lives in your systems, and the job is delivery.
Watch where real failures come from. When an assistant confidently invents a policy, it is usually not because the model was too dumb to reason. It is because nobody put the real policy in front of it, so it filled the gap with something plausible. Swap in a bigger model and you get a more articulate wrong answer. Give a smaller model the right paragraph and it answers correctly. The bottleneck was never the brain. It was the briefing.
More context is not the same as better context
The obvious counter is that context windows are exploding, some models now take a million tokens or more, so just throw everything in and let the model sort it out. This does not work, and it is worth understanding why, because it kills the laziest version of the idea.
Long-context models do not attend evenly across everything you give them. The well-documented "lost in the middle" effect shows models reliably use what sits at the start and end of a long input while quietly glossing over the stuff buried in the middle. Stuff a huge context full and you can watch quality sag, not climb, as the signal gets diluted by noise the model has to wade through. There is a reason people now talk about "context rot," the way a model's grip loosens as the window fills with marginally relevant material. Dumping is not the answer. Curation is.
So the skill is not fetching more. It is fetching less, better. Retrieval that surfaces the three passages that matter instead of the thirty that might. A memory layer that remembers the two facts about this user that change the answer, and forgets the noise. Tool calls that pull a live value at the moment it is needed instead of a stale snapshot baked into a prompt an hour ago. Every one of those is a context problem wearing a different hat.
Why this is where the frontier actually is
Look at where serious effort is going and the shift is obvious. RAG, agents, tool use, memory systems, the whole apparatus is engineering to assemble the right context on the fly. None of it makes the base model smarter. All of it makes the model better-informed at the instant it answers, and that turns out to matter far more for whether your feature works.
This is also why two teams using the identical model ship wildly different products. The model is a commodity they both rent from the same API. The difference is entirely in what each team feeds it: how they chunk and rank their documents, when they call which tool, what they choose to remember and what they let go. That plumbing is the actual product. The model is the easy part, because you can buy it off a menu.
I am not claiming bigger models are pointless. A stronger model does more with a messy briefing and is more forgiving of a sloppy retrieval step, and that is real. But the returns from a better model are shrinking while the returns from better context are wide open, because most teams have barely started on the context side. If your AI feature is underperforming, the honest first question is almost never "do I need a bigger model." It is "did I actually give it what it needed to answer." Nine times out of ten, you did not, and that is a problem you can fix this week without waiting for anyone's next release.
You run a prompt, get a good answer, run the exact same prompt again, and get something different. Sometimes it is a rephrasing. Sometimes it is a different conclusion. If you come from normal software, where the same input gives the same output forever, this feels like the thing is broken. It is not broken. It is doing exactly what it was built to do, and understanding why will change how you use these tools.
The model does not pick a word, it picks from a distribution
At every step, a language model does not decide on the next word. It produces a probability for every possible next token. The word "blue" might get 60 percent, "green" 20 percent, "warm" 5 percent, and a long tail of everything else splitting the rest. Then a sampler reaches into that distribution and draws one.
Temperature is the knob that reshapes those odds before the draw. Turn it up and the flat, unlikely options get more weight, so the output wanders and surprises you. Turn it down and probability piles onto the front-runners, so the model plays it safe. This is a feature. You want a brainstorming assistant to roam and a data-extraction call to stay boring. The variation you are seeing at normal temperature is the model exploring, on purpose.
So the obvious move is to set temperature to 0. Now it always takes the single most likely token, greedy decoding, no dice roll. Same prompt, same answer, forever. Right?
Temperature 0 is not the guarantee you think it is
In practice, temperature 0 gets you close but not all the way. People assumed for years that the leftover wobble was just floating-point noise, some vague hand-wave about GPUs being messy. That answer was never satisfying, and in 2025 researchers at Thinking Machines laid out the actual culprit, and it is more interesting than randomness.
The real cause is batch invariance, or rather the lack of it. When you send a request to a hosted model, you are not alone. The serving system bundles your request together with whatever other requests arrived at the same moment and runs them through the GPU as one batch. That batch is different every time, because it depends on who else is calling the API at that instant, something you have zero control over.
Here is why that matters. Floating-point addition is not associative. Adding a set of numbers in one order can give a very slightly different result than adding them in another order, down in the last bits. When the batch size changes, the GPU kernels split and sum their work in a different order, so the model's internal numbers drift by a hair. Almost always that hair is invisible. But every so often two candidate tokens are nearly tied, the drift nudges one above the other, and the model picks a different word. From there the two answers diverge, because each new token conditions on the last one.
Sit with that for a second. The nondeterminism you see is not the model being random. It is your request sharing a GPU with strangers. The cause is numerical, not magical, and it is fixable. Batch-invariant kernels that pin the reduction order now ship in serving engines like vLLM and SGLang, and with them you can get genuinely reproducible output. Most hosted APIs do not turn that on by default, because it costs a bit of throughput, so the wobble stays.
How to actually think about it
Stop treating a single model output as a fact and start treating it as a sample. One run tells you what the model tends to say. It does not tell you what it always says. That reframe fixes a lot of bad habits.
If you need the same answer every time, do not lean on the model's internals to give it to you. Cache the result keyed on the input, and serve the cached copy. That is deterministic by construction and it is cheaper. If you are extracting structured data, run the call, but validate the shape and constrain the output with a schema so a reworded answer cannot break your parser. If you are evaluating a prompt, never judge it on one run. Sample it ten times and look at the spread, because the spread is the real behavior and a single lucky run is a trap that will embarrass you in production.
And when a demo shows you one perfect output, remember what you are looking at. You are seeing one draw from a distribution, chosen and probably cherry-picked. The honest question is not whether it can produce that answer. It is how often, and what the other draws look like. Ask for a few more and you will learn more in thirty seconds than the polished screenshot will ever tell you.
I write with AI most days and I am not going to pretend I do not. It drafts, it rephrases, it gets me past the blank page. So take what follows as a note from someone who likes the tool, not someone who wants it banned. There is a cost to handing it all your writing, and it is quiet enough that you can rack up a lot of it before you notice.
Writing is thinking, and you can skip the thinking
The uncomfortable truth about writing is that most of the work is not the words. It is the figuring out. You think you understand an idea until you try to put it in a sentence and discover the hole in the middle of it. The struggle to phrase something is the struggle to actually know it. That is not a flaw in writing, it is the entire point.
When you let a model produce the sentence, you get the artifact without the process. The paragraph looks like understanding. It reads like you thought it through. But the mental work writing usually forces, the part that turns a vague sense into a real position, quietly did not happen. You skipped the gym and kept the mirror.
This is not a hunch. A 2025 study out of MIT wired people up while they wrote essays with an LLM, a search engine, or nothing, and the LLM group showed the lowest engagement, what the researchers called cognitive debt. A separate study of 319 knowledge workers found that the more people trusted the AI, the less critical thinking they reported doing. The effect is measurable, and it points the way you would fear.
Everyone starts to sound the same
There is a second cost, harder to measure and easy to feel. These models write in a house style: smooth, balanced, agreeable, faintly corporate. Lean on it and your writing drifts toward that average. The odd phrasing that was actually yours gets sanded off. Multiply that across everyone using the same handful of models and you get a strange flattening, a web where a lot of prose has the same tidy cadence and no fingerprints.
Voice is not decoration. It is the trace of a specific person having a specific thought. When you outsource the sentence, you outsource the fingerprint, and the reader feels the absence even when they cannot name it. Half of why anyone reads a particular writer is to hear how that person, and no one else, would put it.
Where I actually draw the line
I am not arguing for writing everything by hand out of principle. That would be its own kind of pose. The line I try to hold is about what the writing is for.
If the goal is to move information from A to B, a status update, a boilerplate email, a summary nobody will reread, let the model do it and get your afternoon back. The thinking there is not worth protecting. But if the writing is where you work out what you believe, an argument, a design you are still unsure of, anything you will have to defend later, write the first pass yourself. Struggle through the bad draft. That draft is you learning the subject, and the model cannot do that part for you. It can only hide that you skipped it.
The skill you stop using is the skill you lose. Not dramatically, not all at once, just a slow softening you do not clock until the day you sit down to write something that matters and find the muscle is not there. Keep writing the things worth thinking about. Let the machine have the rest.
Ask a language model a question it has no business answering and watch what happens. It answers. Fluently, in complete sentences, with the same even tone it uses for the things it actually knows. The confidence does not move. That gap, between how sure the model sounds and how sure it should be, is the single most dangerous thing about the current generation of these tools.
People call this hallucination and treat it like a bug that a bigger model will fix. It is not a bug. It is baked into how the models are trained.
We taught them to guess
Think about how a model is graded. On most benchmarks a correct answer scores a point, a wrong answer scores zero, and "I do not know" also scores zero. Line those incentives up and the math is brutal: if abstaining pays the same as being wrong, you should always guess. A model that guesses on every uncertain question will beat an honest one that admits its gaps.
So that is the model we built. OpenAI researchers have made the point directly: under binary grading, systems are rewarded for guessing and penalized for saying they are unsure. We optimized for good test-takers, and good test-takers do not leave answers blank. The confident wrong answer is not the model failing. It is the model doing exactly what we rewarded.
Confidence is not knowledge
Here is the part that trips up smart people. We are wired to read fluency as competence. Someone who speaks in clear, structured, well-organized sentences usually knows their subject, because for humans that fluency was expensive to fake. Models make it free. A model produces the same polished prose whether it is reciting a fact or inventing one, and our instinct to trust the smooth talker fires anyway.
That is why "always check the output" is weaker advice than it sounds. Checking is exactly the effort the fluent answer discourages. The better the prose, the less you feel the urge to verify, and the more the one wrong sentence in twenty slips through.
Designing for a witness who never doubts
I do not think the answer is to distrust these tools. I use them every day and they are genuinely good. The answer is to stop treating a model like a source and start treating it like a very fast, very well-read assistant who is constitutionally incapable of saying "I am not sure." Once you accept that, the design choices get clearer.
Put the model where being wrong is cheap and visible: drafting, brainstorming, summarizing text you can see, writing code you are about to run and test. Keep it away from places where a confident fabrication is expensive and hard to catch, like a medical dose, a legal citation, a number that flows straight into a decision with no human in the loop. Retrieval helps, because grounding an answer in a document you can inspect turns "trust me" into "here is where it came from." And when you can, prefer systems that surface uncertainty at all, even a rough confidence score, over ones that render everything in the same calm voice.
There is real research now on training models to abstain, to earn credit for admitting a gap instead of being punished for it. I hope it works. Until it ships, the burden sits with us, and the honest move is to build workflows that assume the machine will never tell you when it is out of its depth. It will keep talking. The question worth sitting with is why we find that so much more comforting than a system that occasionally, usefully, went quiet.