Someone in every AI project eventually says it: "We are spending too much on the API. Let us just host our own model." It sounds obvious. You are renting when you could own. But the spreadsheet almost never says what people expect, and the version both sides show you leaves out the line items that decide the whole thing.
The number that looks damning
Per-token API pricing feels expensive when you watch it add up. A busy product can run a real monthly bill, and the temptation is to compare that against a GPU rental and declare victory. A single H100 rents for somewhere between $1.50 and $10 an hour depending on provider and commitment, call it $1,100 to $7,300 a month running steadily. Next to a fat API invoice, renting the GPU can look cheaper.
It usually is not, because the GPU is the part of self-hosting you can see. The rest hides.
The costs that do not show up on the GPU bill
Running your own inference is not renting a graphics card. It is standing up a service. That means someone who knows how to serve models, batch requests, handle load, patch the stack, and get paged at 3am when it falls over. MLOps engineers are not cheap, and their time is the real cost. Reasonable estimates put the fully loaded cost of self-hosting at three to five times the raw GPU price once you count the people and the plumbing.
Then there is utilization, the quiet killer. The API charges you per token, so you pay for exactly what you use. Your rented GPU charges you per hour whether it is busy or idle. Traffic is spiky. If your GPU sits at 15 percent utilization most of the day, you are paying full price for an empty machine, and your effective cost per token climbs above what the API charged.
Where the break-even actually sits
Concrete numbers help. Against premium APIs, self-hosting a comparable open model tends to break even somewhere around 5 to 10 million tokens a month, and only if you already have the ops capacity. Against the cheap tier, the mini and small models that cost almost nothing per token, the line moves way out: you might need hundreds of millions of tokens a month before owning beats renting.
One worked example from a 2026 analysis: 50 million tokens a day through a small hosted model came to about $2,250 a month on the API, versus roughly $5,175 self-hosted on four mid-range GPUs. More than double, to run the exact same workload yourself.
So the honest brackets look like this:
- Under about 50M tokens a month: the API wins almost every time. Do not self-host to save money, you will not.
- 50M to 500M against frontier-priced APIs, with real MLOps staff on hand: now it is a genuine decision.
- Sustained heavy volume against comparable open models: self-hosting can cut cost several fold, and at that point you probably already know who you are.
There are reasons to self-host that have nothing to do with cost. Data that legally cannot leave your walls. Latency you cannot get from a shared endpoint. A model you have fine-tuned and want to own outright. Those are real, and they can justify the bill on their own. Just do not dress them up as savings, because for most teams below serious scale, self-hosting is the more expensive choice wearing a thrifty costume. If you are moving to it, move for control or privacy, name that out loud, and go in knowing the GPU line was the cheap part.