Real Costs AI On-Prem: Case Study Hetzner CCX23 + Ollama 12 Months

The sum almost nobody does: where the break-even sits between your own server and an API

"AI of your own costs less than APIs" is a sentence that depends entirely on four numbers: what the server costs, how much work it requires, how many tokens you consume and what your pay-per-use alternative costs. This article builds the calculation with a program you can rerun, states which numbers are real and which are assumptions, and reaches a conclusion that may surprise: on a server without a GPU the saving, almost always, is not there. The reasons for keeping models in-house are other ones, and it is worth knowing that before you buy.

A note on the title: it speaks of a twelve-month business case, but there is no real measured case here. What you find is a calculation model with explicit assumptions, which is more useful than an invented case because you can redo it with your own data.

What goes into the cost, and what I left out

An honest comparison puts costs of different natures on the same footing. The server fee is a certain, dated expense; the time of whoever administers it is a real expense even when it does not appear on an invoice; the API price is a price list that changes. Here is the list of what the program counts and what it excludes.

• Counted, verified data: the monthly fee of the server, taken from the official page (CCX23 in Nuremberg, 86.49 euros per month excluding VAT, consulted on 2026-11-13).
• Counted, assumption: 24 hours of setup amortized over 12 months and a number of administration hours per month, at an internal hourly rate of 50 euros.
• Counted, assumption: generation speed and the share of time in which the server really works, which fix how many tokens per month it can produce.
• Excluded: storage of backups, monitoring, energy and cooling (included in the fee of a VPS, not in a server of your own), insurance, tax incentives and the cost of unavailability.

The exclusion of tax incentives is deliberate: tax credits and depreciation depend on the year and on your case, I have not verified them here against an official source, and a wrong relief in a company calculation is worse than none. If you think they apply to your case, the accountant adds them as a separate line. VAT is out too: the server prices are net, and for a small or medium company that deducts VAT the calculation does not change.

Server capacity is the constraint that price lists hide

A pay-per-use API scales almost without limit: if you double the requests, you pay double. A server does not: it has a ceiling of tokens per month, fixed by its speed and by how long it works. The ceiling is the first thing to calculate, because it decides whether the break-even is reachable. With 8 tokens per second, a plausible value as the ceiling of a machine with about 40 GB/s of memory bandwidth (as worked out in the article on the server in production), and a utilization of 25%, the server produces about 5 million generated tokens per month.

The number must be read with caution in two directions. Downwards, because it counts only generated tokens: requests with attached documents also spend a lot of time reading the input text, and reading time reduces the effective capacity. Upwards, because with more requests processed together the machine can increase overall throughput, at the cost of more memory. For this reason the program shows a grid of speeds and utilizations instead of a single value.

"""Break-even point between a server of your own and a pay-per-use API.

REAL DATA: the monthly price of the CCX23 server (Hetzner General Purpose page,
NBG1 location, excluding VAT, consulted on 2026-11-13).
Everything else is an ASSUMPTION to be replaced with your own numbers.
"""
SERVER_EUR_MONTH = 86.49
HOURLY_RATE = 50.0          # ASSUMPTION: internal cost of one hour of work
SETUP_HOURS = 24            # ASSUMPTION: one-off, amortized over 12 months
SECONDS_MONTH = 30 * 24 * 3600
API_PRICES = [0.5, 2.0, 10.0, 50.0]  # ASSUMPTION: euros per million generated tokens, not a price list


def monthly_cost(admin_hours):
    return SERVER_EUR_MONTH + admin_hours * HOURLY_RATE + SETUP_HOURS * HOURLY_RATE / 12


def capacity_mtok(tokens_s, utilization):
    return tokens_s * utilization * SECONDS_MONTH / 1e6


print("capacity in millions of generated tokens per month (tokens/s x utilization)")
for tokens_s in (4, 8, 16):
    rows = "  ".join(f"{u:.0%}: {capacity_mtok(tokens_s, u):5.1f}" for u in (0.10, 0.25, 0.50))
    print(f"  {tokens_s:2d} tokens/s   {rows}")

cap = capacity_mtok(8, 0.25)
print(f"\nreference scenario: 8 tokens/s at 25% -> {cap:.1f} Mtok/month")
for hours in (0, 4):
    c = monthly_cost(hours)
    print(f"\nadministration {hours} h/month: monthly cost {c:6.2f} euros, "
          f"{c / cap:6.2f} euros per million tokens")
    for p in API_PRICES:
        breakeven = c / p
        verdict = "reachable" if breakeven <= cap else "BEYOND capacity"
        print(f"  API at {p:4.1f} euros/Mtok -> break-even at {breakeven:6.1f} Mtok/month ({verdict})")

# Output:
# capacity in millions of generated tokens per month (tokens/s x utilization)
#    4 tokens/s   10%:   1.0  25%:   2.6  50%:   5.2
#    8 tokens/s   10%:   2.1  25%:   5.2  50%:  10.4
#   16 tokens/s   10%:   4.1  25%:  10.4  50%:  20.7
#
# reference scenario: 8 tokens/s at 25% -> 5.2 Mtok/month
#
# administration 0 h/month: monthly cost 186.49 euros,  35.97 euros per million tokens
#   API at  0.5 euros/Mtok -> break-even at  373.0 Mtok/month (BEYOND capacity)
#   API at  2.0 euros/Mtok -> break-even at   93.2 Mtok/month (BEYOND capacity)
#   API at 10.0 euros/Mtok -> break-even at   18.6 Mtok/month (BEYOND capacity)
#   API at 50.0 euros/Mtok -> break-even at    3.7 Mtok/month (reachable)
#
# administration 4 h/month: monthly cost 386.49 euros,  74.55 euros per million tokens
#   API at  0.5 euros/Mtok -> break-even at  773.0 Mtok/month (BEYOND capacity)
#   API at  2.0 euros/Mtok -> break-even at  193.2 Mtok/month (BEYOND capacity)
#   API at 10.0 euros/Mtok -> break-even at   38.6 Mtok/month (BEYOND capacity)
#   API at 50.0 euros/Mtok -> break-even at    7.7 Mtok/month (BEYOND capacity)

The first table is the capacity: from about 1 to over 20 million tokens per month, depending on speed and utilization. The next block compares the cost with four hypothetical API prices (from 0.5 to 50 euros per million generated tokens: they are scenarios, not price lists of any provider). With 4 hours of administration per month the server costs 386 euros, that is about 75 euros per million tokens, and the break-even with an API at 10 euros per million tokens falls at 38.6 million tokens per month, more than seven times beyond the capacity of the server. Even with zero hours of administration (the amortized setup remains) the break-even is reachable only if the API costs as much as 50 euros per million tokens.

What the result really says

The result does not prove that a server of your own is always a bad deal. It says that, with the stated assumptions, the server fee weighs little and the work of whoever manages it weighs a lot, and that a CPU server does not produce enough tokens to pay for itself against a cheap API. The assumptions that overturn it are three: an API much more expensive than expected, a volume close to the capacity of the server, or an administration cost close to zero because the expertise is already in the company and the time is marginal.

Three practical indications follow. If the reason is cost, do the sum first and you will probably find that APIs win. If the reason is keeping data under your control, the cost becomes the price of that choice and must be accepted with open eyes (the article on privacy by design shows what that control protects and what it does not). And if the volume grows, the question becomes whether a GPU changes the numbers: I have not calculated that case, because it needs a price and a speed measured on your load, and none of my sources provides them.

Costs that appear later, and one that falls by law

On the cloud side there is a cost that is thought to be binding and that the law is reducing: the cost of switching to another provider. Regulation (EU) 2023/2854, the Data Act, provides in Article 29 that from 12 January 2027 providers of data processing services shall not impose switching charges on the customer; until that date they may impose reduced charges, which do not exceed the costs directly linked to the switching process. Article 50 establishes that the regulation applies from 12 September 2025. It is a rule on data processing services in general: whether and how it applies to a specific artificial intelligence service must be assessed by your adviser, because it depends on how the service is classified.

On the side of doing it yourself the hidden costs are operational: updating the system, rotating keys, restoring a backup, responding to a failure at night. The program sums them up in hours per month, and the sensitivity is simple: every additional hour of administration adds 50 euros per month at the assumed rate, so the cost grows faster with the work than with the fee. A note on method: record the hours actually spent for three months and replace the assumption with the data. In the article on operational continuity you find what a restore involves.

How to redo the calculation with your numbers

The procedure is mechanical and takes half an hour. Open the provider's price page and copy the fee of the instance you need, with the location and the date. Measure the speed of your machine with a request to Ollama's API, dividing the generated tokens by the generation time. Estimate the monthly volume by looking at the logs of what you already use, in tokens and not in requests. Take from the API provider's price list the price per million generated tokens and input tokens, and calculate the weighted average according to the typical proportion of your prompts. Replace the values in the constants at the top of the program and rerun it.

If the break-even falls beyond the capacity, the verdict is "only for reasons other than cost". If it falls within, add a margin for the unexpected and compare the monthly saving with the hours you will have to devote: a saving of a few tens of euros per month does not offset one more hour of work as much as you think. The topic of the cases in which the European cloud is the better choice is developed in when self-hosting is not worth it, and the way to decide the size of the model without overdoing it is in small or large model, while the memory and license constraints of the individual families are in which model to choose.

Frequently asked questions

What you can do now

You can calculate the monthly capacity of a server from its generation rate, you can build the total cost with the hours of work inside it, and you can find the break-even volume for a given API price. Above all you can tell real data, like the fee with its date, from assumptions to be replaced. The rest of the series works on the same axis: the privacy that justifies the cost, the models that fit in the server, the document retrieval that makes them useful.

The same series also includes hybrid retrieval over company documents, sizing for more users and the recap with the decision sheet.