Ethics

The AI safety conversation is obsessed with the ceiling: the smartest model, the frontier, the hypothetical superintelligence. I think we are watching the wrong number. The change that will actually reshape the world this decade is not that the best model got smarter. It is that a good-enough model got almost free.

Ask a language model a question it has no business answering and watch what happens. It answers. Fluently, in complete sentences, with the same even tone it uses for the things it actually knows. The confidence does not move. That gap, between how sure the model sounds and how sure it should be, is the single most dangerous thing about the current generation of these tools.

People call this hallucination and treat it like a bug that a bigger model will fix. It is not a bug. It is baked into how the models are trained.

We taught them to guess

Think about how a model is graded. On most benchmarks a correct answer scores a point, a wrong answer scores zero, and "I do not know" also scores zero. Line those incentives up and the math is brutal: if abstaining pays the same as being wrong, you should always guess. A model that guesses on every uncertain question will beat an honest one that admits its gaps.

So that is the model we built. OpenAI researchers have made the point directly: under binary grading, systems are rewarded for guessing and penalized for saying they are unsure. We optimized for good test-takers, and good test-takers do not leave answers blank. The confident wrong answer is not the model failing. It is the model doing exactly what we rewarded.

Confidence is not knowledge

Here is the part that trips up smart people. We are wired to read fluency as competence. Someone who speaks in clear, structured, well-organized sentences usually knows their subject, because for humans that fluency was expensive to fake. Models make it free. A model produces the same polished prose whether it is reciting a fact or inventing one, and our instinct to trust the smooth talker fires anyway.

That is why "always check the output" is weaker advice than it sounds. Checking is exactly the effort the fluent answer discourages. The better the prose, the less you feel the urge to verify, and the more the one wrong sentence in twenty slips through.

Designing for a witness who never doubts

I do not think the answer is to distrust these tools. I use them every day and they are genuinely good. The answer is to stop treating a model like a source and start treating it like a very fast, very well-read assistant who is constitutionally incapable of saying "I am not sure." Once you accept that, the design choices get clearer.

Put the model where being wrong is cheap and visible: drafting, brainstorming, summarizing text you can see, writing code you are about to run and test. Keep it away from places where a confident fabrication is expensive and hard to catch, like a medical dose, a legal citation, a number that flows straight into a decision with no human in the loop. Retrieval helps, because grounding an answer in a document you can inspect turns "trust me" into "here is where it came from." And when you can, prefer systems that surface uncertainty at all, even a rough confidence score, over ones that render everything in the same calm voice.

There is real research now on training models to abstain, to earn credit for admitting a gap instead of being punished for it. I hope it works. Until it ships, the burden sits with us, and the honest move is to build workflows that assume the machine will never tell you when it is out of its depth. It will keep talking. The question worth sitting with is why we find that so much more comforting than a system that occasionally, usefully, went quiet.

If your work involves anything sensitive, client data, health records, unreleased products, legal material, the idea of piping it through someone else API should make you at least a little uncomfortable. The good news is that in 2026 you do not have to. A genuinely capable, fully private AI stack is now within reach for a normal team. Here is how to think about building one.

I am instinctively allergic to tech regulation written by people who have never shipped anything. A lot of the EU AI Act fits that description. It is late, convoluted, and parts of it will age badly. And yet, reading through what actually takes effect in 2026, I keep landing somewhere uncomfortable: most of it is the kind of thing the industry should have done on its own and did not.

I write with AI most days and I am not going to pretend I do not. It drafts, it rephrases, it gets me past the blank page. So take what follows as a note from someone who likes the tool, not someone who wants it banned. There is a cost to handing it all your writing, and it is quiet enough that you can rack up a lot of it before you notice.

Writing is thinking, and you can skip the thinking

The uncomfortable truth about writing is that most of the work is not the words. It is the figuring out. You think you understand an idea until you try to put it in a sentence and discover the hole in the middle of it. The struggle to phrase something is the struggle to actually know it. That is not a flaw in writing, it is the entire point.

When you let a model produce the sentence, you get the artifact without the process. The paragraph looks like understanding. It reads like you thought it through. But the mental work writing usually forces, the part that turns a vague sense into a real position, quietly did not happen. You skipped the gym and kept the mirror.

This is not a hunch. A 2025 study out of MIT wired people up while they wrote essays with an LLM, a search engine, or nothing, and the LLM group showed the lowest engagement, what the researchers called cognitive debt. A separate study of 319 knowledge workers found that the more people trusted the AI, the less critical thinking they reported doing. The effect is measurable, and it points the way you would fear.

Everyone starts to sound the same

There is a second cost, harder to measure and easy to feel. These models write in a house style: smooth, balanced, agreeable, faintly corporate. Lean on it and your writing drifts toward that average. The odd phrasing that was actually yours gets sanded off. Multiply that across everyone using the same handful of models and you get a strange flattening, a web where a lot of prose has the same tidy cadence and no fingerprints.

Voice is not decoration. It is the trace of a specific person having a specific thought. When you outsource the sentence, you outsource the fingerprint, and the reader feels the absence even when they cannot name it. Half of why anyone reads a particular writer is to hear how that person, and no one else, would put it.

Where I actually draw the line

I am not arguing for writing everything by hand out of principle. That would be its own kind of pose. The line I try to hold is about what the writing is for.

If the goal is to move information from A to B, a status update, a boilerplate email, a summary nobody will reread, let the model do it and get your afternoon back. The thinking there is not worth protecting. But if the writing is where you work out what you believe, an argument, a design you are still unsure of, anything you will have to defend later, write the first pass yourself. Struggle through the bad draft. That draft is you learning the subject, and the model cannot do that part for you. It can only hide that you skipped it.

The skill you stop using is the skill you lose. Not dramatically, not all at once, just a slow softening you do not clock until the day you sit down to write something that matters and find the muscle is not there. Keep writing the things worth thinking about. Let the machine have the rest.

The most important AI story of the next year will not be a model release. It will be a court ruling. The lawsuits over what these models were trained on, the New York Times against OpenAI, Getty against Stability, are entering decisive phases, and the question they answer will quietly reshape the entire industry. I have a side, and I want to explain it honestly.

Every model launches with a splashy chart and a breathless thread. Almost nobody reads the boring document that ships alongside it, the model card, with its dull sections on training data, limitations, and known failure modes. I read them, every time, and I think it is one of the highest-value habits you can build in this field. Here is why.

There are enough AI blogs. Most of them read like a press release with the serial numbers filed off. This one is trying to be the thing I actually wanted to read: written by someone who uses these tools every day, likes them more than is probably healthy, and still reads the fine print.