When NOT self-hosting: 7 scenarios where cloud EU wins over on-prem AI 2026

Before buying a server: seven questions that may make you choose the cloud

Keeping an AI model in-house is a sensible choice in many cases, and the series has shown how to do it. But it is not a universal choice, and an article that argues a thesis must also say when it does not hold. This article takes seven situations in which a managed, European, pay-per-use service can be the best choice for an SME, and for each one gives the test you can run with your own numbers. You will not find provider prices or company cases with figures: price lists change, and an invented case proves nothing. You will find programs, run, that let you calculate your own case.

The most useful result to anticipate concerns cost: a server of your own costs a fixed amount, so the cost of each token depends on how hard you make it work, and the time of whoever runs it weighs as much as the machine. With declared and variable assumptions, the first program shows it. The rest of the article is the application of this idea to the seven cases, plus the European regulatory framework that you need when the provider is not European. For the costs of the in-house case, on the other side, see the analysis of 12 months of costs and the sizing in Ollama from 1 to 50 users.

The cost per token depends on how hard the server works

A server costs the same whether it answers a thousand requests or none. A pay-per-use API costs only what you use. The honest comparison between the two is therefore the cost per million tokens as a function of utilisation, that is of the fraction of time in which the machine really works, plus the cost of the work of whoever keeps it running. The program computes the table with test values: 200 euro per month for the machine, 50 euro per hour for whoever runs it, 25 tokens per second (the speed must be measured on your machine) and API prices from 1 to 50 euro per million tokens. None of these numbers is the price of a provider: they are assumptions to replace with your own price list and measurements.

"""Cost per million tokens of your own server against a pay-per-use API, as a function of utilisation and management time."""
SECONDI_MESE = 30 * 24 * 3600

# ALL TEST VALUES, not the prices of any provider: replace them with your own price list and measurements
SERVER_EURO_MESE = 200.0       # monthly cost of the machine
TARIFFA_ORARIA = 50.0          # cost of an hour of work by whoever runs the service
TOKEN_AL_SECONDO = 25.0        # MEASURED speed (see the article on sizing)
API_EURO_PER_MILIONE = 1.0     # pay-per-use price per million tokens (input and output averaged)


def costo_proprio(utilizzo, ore_gestione_mese):
    """Euro per million tokens generated, if the server works for the fraction 'utilizzo' of the time."""
    token_mese = TOKEN_AL_SECONDO * SECONDI_MESE * utilizzo
    return (SERVER_EURO_MESE + ore_gestione_mese * TARIFFA_ORARIA) / (token_mese / 1e6)


print(f"capacity at 100% utilisation: {TOKEN_AL_SECONDO * SECONDI_MESE / 1e6:.1f} million tokens per month")
print("euro per million tokens of your own server (reference API: 1.00)")
print("utilisation   0 h admin   8 h admin   20 h admin")
for u in (0.05, 0.20, 0.50, 0.80):
    print(f"{u:11.0%}   {costo_proprio(u, 0):9.2f}   {costo_proprio(u, 8):9.2f}   {costo_proprio(u, 20):10.2f}")

print()
print("minimum utilisation for your own server to cost less than the API, by API price (euro per million tokens)")
print("API price      0 h admin     8 h admin    20 h admin")
for prezzo in (1.0, 5.0, 20.0, 50.0):
    righe = []
    for ore in (0, 8, 20):
        u_min = (SERVER_EURO_MESE + ore * TARIFFA_ORARIA) / (TOKEN_AL_SECONDO * SECONDI_MESE / 1e6 * prezzo)
        righe.append("never (>100%)" if u_min > 1 else f"{u_min:.0%}")
    print(f"{prezzo:9.2f}   {righe[0]:>13s}   {righe[1]:>13s}   {righe[2]:>13s}")

# Output:
# capacity at 100% utilisation: 64.8 million tokens per month
# euro per million tokens of your own server (reference API: 1.00)
# utilisation   0 h admin   8 h admin   20 h admin
#          5%       61.73      185.19       370.37
#         20%       15.43       46.30        92.59
#         50%        6.17       18.52        37.04
#         80%        3.86       11.57        23.15
#
# minimum utilisation for your own server to cost less than the API, by API price (euro per million tokens)
# API price      0 h admin     8 h admin    20 h admin
#      1.00   never (>100%)   never (>100%)   never (>100%)
#      5.00             62%   never (>100%)   never (>100%)
#     20.00             15%             46%             93%
#     50.00              6%             19%             37%

The two tables are read together. The first shows that with those assumptions your own server costs from 61.73 euro per million tokens at 5% utilisation down to 3.86 euro at 80% utilisation, and that 8 hours of management per month bring the cost at low utilisation above 180 euro per million. The second answers the inverse question: from what utilisation your own server beats the API. With an API at 1 euro per million, never; at 5 euro you need 62% utilisation and no management hours; at 20 euro you need 15% without management, 46% with 8 hours and 93% with 20. The point is the structure, not the figures: the break-even almost always falls at high utilisations, and management time moves it more than the machine does. That is why the seven cases that follow have the same cause in common: a utilisation that will never be high, or a management cost that you cannot absorb.

Scenario 1: the load comes in peaks

If requests concentrate in a few hours, you have to size the server for the peak and pay for it even when it is idle. The test is the peak factor, the ratio between the busiest hour and the hourly average. The program computes it on a constructed day of an internal assistant, with a peak at 11, and then asks what happens if you buy less capacity than the peak.

"""Peak factor: what it costs to size for the worst moment instead of for the average."""
# Requests per hour in a day of an internal assistant, CONSTRUCTED SERIES: office hours and a peak at 11
orarie = [0, 0, 0, 0, 0, 0, 2, 10, 40, 80, 120, 300, 90, 50, 70, 100, 60, 30, 10, 4, 2, 0, 0, 0]
media = sum(orarie) / len(orarie)
picco = max(orarie)
print(f"requests per day: {sum(orarie)}, hourly average {media:.1f}, hourly peak {picco}")
print(f"peak factor: {picco / media:.1f}")
print(f"average utilisation if you size for the peak: {media / picco:.0%}")

# With capacity equal to 60% of the peak, how many requests exceed it in the worst hour?
capacita = 0.6 * picco
eccesso = sum(max(0, r - capacita) for r in orarie)
print(f"capacity at 60% of the peak ({capacita:.0f} requests/hour): {eccesso:.0f} requests out of {sum(orarie)} exceed it ({eccesso / sum(orarie):.1%})")

# Output:
# requests per day: 968, hourly average 40.3, hourly peak 300
# peak factor: 7.4
# average utilisation if you size for the peak: 13%
# capacity at 60% of the peak (180 requests/hour): 120 requests out of 968 exceed it (12.4%)

With 968 requests per day the average is 40.3 per hour and the peak is 300: factor 7.4. Sizing for the peak, the server works on average 13% of the time, and with that utilisation value the previous table says that the cost per token is high. Sizing at 60% of the peak, 12.4% of requests exceed capacity: if the excess can wait (queue) or go elsewhere, the smaller machine is fine; if not, a peaky load is a good reason for a pay-per-use service, or for the hybrid model in the last section. First ask yourself whether the peak is really unpredictable or just concentrated at a known time: in the second case a well-sized queue, like the one in the article on sizing, may be enough.

Scenario 2: the model you need does not fit in your machine

Large models ask for memory in proportion to their parameters. The program computes it from the fields published in the config.json of two models, one of about 7 billion and one of about 72 (examples, not recommendations): for each, the weights at 16, 8 and 4 bits and the cache per token. The note at the end of the output says that the calculation ignores Qwen2.5's biases, and that four bits is a theoretical minimum.

"""How much memory a model asks for, from the public parameters of its config.json (consulted on 2026-11-13)."""
GIB = 1024 ** 3

MODELLI = {
    # Mistral-7B-Instruct-v0.3 (mistralai) and Qwen2.5-72B-Instruct (Qwen), config.json fields
    "7B  (Mistral-7B-Instruct-v0.3)": dict(h=4096, i=14336, L=32, a=32, kv=8, v=32768),
    "72B (Qwen2.5-72B-Instruct)": dict(h=8192, i=29568, L=80, a=64, kv=8, v=152064),
}


def parametri(c):
    h, i, L = c["h"], c["i"], c["L"]
    kv_dim = c["kv"] * (h // c["a"])
    strato = h * h + 2 * h * kv_dim + h * h + 3 * h * i + 2 * h   # attention + MLP + norms (no bias)
    return L * strato + 2 * c["v"] * h + h


print("model                            parameters  16-bit weights  8-bit weights  4-bit weights   KV per token")
for nome, c in MODELLI.items():
    p = parametri(c)
    kv = 2 * c["L"] * c["kv"] * (c["h"] // c["a"]) * 2
    print(f"{nome:32s} {p / 1e9:7.2f} G   {p * 2 / GIB:8.1f} GiB  {p / GIB:9.1f} GiB  {p / 2 / GIB:9.1f} GiB  {kv / 1024:6.0f} KiB")
print("Note: Qwen2.5 uses bias in the attention layers, which the calculation ignores (negligible difference); 4 bit = theoretical minimum.")

# Output:
# model                            parameters  16-bit weights  8-bit weights  4-bit weights   KV per token
# 7B  (Mistral-7B-Instruct-v0.3)      7.25 G       13.5 GiB        6.8 GiB        3.4 GiB     128 KiB
# 72B (Qwen2.5-72B-Instruct)         72.71 G      135.4 GiB       67.7 GiB       33.9 GiB     320 KiB
# Note: Qwen2.5 uses bias in the attention layers, which the calculation ignores (negligible difference); 4 bit = theoretical minimum.

The number that decides is the 4-bit column of the large model: 33.9 GiB, theoretical minimum, for the weights alone, before the cache (320 KiB per token, so 1.25 GiB for a 4096 context) and the rest of the system. At 16 bits you need 135 GiB. If your machine has less memory than the model plus the cache, the comparison is no longer between costs: the model does not run, or runs so degraded as to be of no use. If instead you need the model only for a few difficult requests, a pay-per-use API for those requests alone, with the small model in-house for the rest, is an architecture to consider. Remember that the quality of a quantised model must be measured with your own golden dataset, as in the article on evaluation, not presumed.

Scenario 3: nobody can take care of the service

This is the case that the cost table puts into numbers: the management hours per month are a cost item, and if nobody has those hours, the item is not zero, it is a risk. The service stops at night, the disk fills up, an update changes the behaviour. The test is simple and needs no program: list the activities of the article on disaster recovery (archiving, backup verification, restore drill), the sizing monitoring and the security updates, and ask yourself who they are assigned to and within what hours. If the answer is "nobody in particular", a managed service moves that work to whoever does it for a living, and you pay for it. The work of checking the contract stays yours, which is the theme of the next scenario.

Scenario 4: you need contractual guarantees or certifications you cannot give yourself

Some activities require demonstrating to customers or to an authority that the service is managed according to precise requirements. HIPAA, often cited in this context, is a United States law on healthcare and is not the framework of a European SME: for you the GDPR and sector rules count. From the GDPR comes a principle that holds in both directions: when you entrust processing to a provider, Article 28(1) says that the controller "shall use only processors providing sufficient guarantees to implement appropriate technical and organisational measures", and paragraph 2 that "the processor shall not engage another processor without prior specific or general written authorisation of the controller". For the entities within the scope of the NIS2 Directive, Article 21(2) lists among the measures also "supply chain security, including security-related aspects concerning the relationships between each entity and its direct suppliers or service providers".

The corollary is that the cloud does not remove responsibility, it moves it into a contract to be read: where the data are, who the sub-processors are, what happens in case of an incident, what you can demonstrate to a customer. The European cloud certification scheme (EUCS) is a reference that I would have liked to verify: the official page of the competent agency was not reachable while I was writing, so I do not use it as a criterion and I do not declare its status. For a concrete certification requirement, ask the provider for the attestation and read what it covers.

When the provider is not European

If the service is offered by a non-EU company or moves data outside the Union, Chapter V of the GDPR comes into play. Article 44 says that any transfer of personal data to a third country "shall take place only if" the conditions laid down in the chapter are complied with by the controller and processor; Article 45 allows the transfer where the Commission has decided that the third country ensures an adequate level of protection. For the United States, on 10 July 2023 the Commission adopted the adequacy decision on the EU-US Data Privacy Framework, Commission Implementing Decision (EU) 2023/1795: the Commission's official page explains that "personal data can flow freely from the EU to companies in the United States that participate in the Data Privacy Framework".

This framework has been challenged in court. The General Court of the European Union, by judgment of 3 September 2025 in Case T-553/23 (Latombe v Commission), dismissed the action. An appeal against this judgment was brought before the Court of Justice on 31 October 2025 (Case C-703/25 P, notice in the Official Journal C/2025/6610 of 22 December 2025). In the sources I consulted on 2026-11-13 I did not find the outcome of the appeal: check its status before basing a contract on this decision. The practical consequence for an SME is that relying on a United States provider means depending on a decision still sub judice, whereas a provider and processing entirely within the Union do not raise the transfer problem. This is not legal advice, and it is not a recommendation of providers.

Scenario 5: the users are far away

If the users are on several continents, physical distance sets a limit on latency that no optimisation overcomes. The program computes, with the sphere distance formula and assuming the signal travels in fibre at about two thirds of the speed of light in vacuum, the theoretical minimum round trip from Frankfurt. It is a lower bound: the real value is always greater, because of non-straight routing and processing in the routers.

"""Physical limit of latency: round trip in fibre along the great circle, with no routers and no processing."""
from math import asin, cos, radians, sin, sqrt

R_TERRA_KM = 6371.0
KM_S_FIBRA = 200_000.0     # about two thirds of the speed of light in vacuum: order of magnitude for fibre

CITTA = {"Frankfurt": (50.11, 8.68), "Milan": (45.46, 9.19), "New York": (40.71, -74.01),
         "Sao Paulo": (-23.55, -46.63), "Singapore": (1.35, 103.82), "Sydney": (-33.87, 151.21)}


def distanza_km(a, b):
    (la1, lo1), (la2, lo2) = (tuple(map(radians, a)), tuple(map(radians, b)))
    h = sin((la2 - la1) / 2) ** 2 + cos(la1) * cos(la2) * sin((lo2 - lo1) / 2) ** 2
    return 2 * R_TERRA_KM * asin(sqrt(h))


origine = CITTA["Frankfurt"]
print("from Frankfurt: distance and theoretical minimum round trip (the real value is always greater)")
for nome, pos in CITTA.items():
    d = distanza_km(origine, pos)
    print(f"{nome:12s} {d:8.0f} km   {2 * d / KM_S_FIBRA * 1000:6.1f} ms")

# Output:
# from Frankfurt: distance and theoretical minimum round trip (the real value is always greater)
# Frankfurt           0 km      0.0 ms
# Milan             518 km      5.2 ms
# New York         6203 km     62.0 ms
# Sao Paulo        9829 km     98.3 ms
# Singapore       10258 km    102.6 ms
# Sydney          16483 km    164.8 ms

From Frankfurt to New York the minimum is 62 milliseconds, to Singapore 103, to Sydney 165. For a streamed chat, where the first token arrives after the model's processing and after the network trip, 165 ms more is a small fraction of the seconds that an answer takes on a processor without a GPU: distance really matters only if the experience depends on many short requests in sequence. If instead your app makes ten calls one after the other for a page, the bill from Sydney becomes 1.6 seconds of physics alone. Before replicating the infrastructure in several regions, measure how many sequential calls your application makes.

Scenario 6: you use the system for only part of the year

The seasonality case is scenario 1 writ large: a low annual average utilisation, because for months the server is idle. From the break-even table you get the criterion: compute your average utilisation over the year, not in the peak month, and use it in the table. A service switched on only for the period that is needed, or a pay-per-use API, weighs for the hours used; a fixed server weighs for twelve months. Do check, however, the contractual conditions of any hourly machine (minimum commitment, guaranteed availability), because capacity that is not available at the moment of the peak is of no use to you. On this point I have no figures: you read the price list yourself, on the date you decide.

Scenario 7: you are testing an idea

For a prototype that will last a few weeks, the time you spend installing and securing a server is a fixed cost that is not recovered if the idea is discarded. A pay-per-use service lets you answer the question "does it serve anybody?" before deciding where to host the system. Two precautions: use non-personal data, because a prototype often skips the contracts chapter, and from the start write the access to the model behind an interface of your own, so that moving to an in-house model is a configuration change and not a rewrite. If the prototype succeeds, the break-even program says when it pays to move it.

The hybrid model: rules, not optimism

Between the two extremes there is the architecture that keeps a model in-house for the ordinary load and for personal data, and sends to the external service only what the local one cannot serve. It must be written as rules, not as an intention. The program implements those of the constructed case: if the context exceeds that of the local model the request goes to the remote one (if the policy allows it); if the local one has capacity the request stays in-house; if it is saturated and the text contains recognisable personal data it stays in-house and waits; if it is saturated and the text contains no recognised data it goes to the remote one. It then shows the replacement of emails and tax codes (the Italian codice fiscale) with stable placeholders, which can be used when the policy allows sending a cleaned text.

"""Rule-based routing between local model and remote service: personal data does not leave, excess load does."""
import re

EMAIL = re.compile(r"[\w.+-]+@[\w-]+\.[\w.-]+")
CODICE_FISCALE = re.compile(r"\b[A-Z]{6}\d{2}[A-Z]\d{2}[A-Z]\d{3}[A-Z]\b")   # Italian tax code
CODA_MASSIMA_LOCALE = 3          # waiting requests beyond which the local model is considered saturated
TOKEN_MASSIMI_LOCALE = 4000      # maximum context of the local model (example)


def contiene_dati_personali(testo):
    return bool(EMAIL.search(testo) or CODICE_FISCALE.search(testo))


def pseudonimizza(testo):
    """Replaces emails and tax codes with stable placeholders: the remote service receives the text, not the identities."""
    mappa, contatore = {}, 0

    def sostituisci(m):
        nonlocal contatore
        if m.group(0) not in mappa:
            contatore += 1
            mappa[m.group(0)] = f"[PERSON_{contatore}]"
        return mappa[m.group(0)]

    pulito = CODICE_FISCALE.sub(sostituisci, EMAIL.sub(sostituisci, testo))
    return pulito, mappa


def instrada(richiesta, coda_locale, remoto_consentito=True):
    testo, token = richiesta["testo"], richiesta["token"]
    if token > TOKEN_MASSIMI_LOCALE:
        return ("remote", "context longer than the local model's") if remoto_consentito else ("reject", "too long and remote not allowed")
    if coda_locale <= CODA_MASSIMA_LOCALE:
        return "local", "capacity available"
    if contiene_dati_personali(testo):
        # Personal data: they stay local even at the cost of waiting (or you pseudonymise, if the policy allows it)
        return "local", "local saturated but the text contains personal data: wait"
    return ("remote", "local saturated, no personal data recognised") if remoto_consentito else ("local", "remote not allowed")


casi = [
    ("A", "Summarise the expense reimbursement procedure", 300, 0),
    ("B", "Summarise the expense reimbursement procedure", 300, 7),
    ("C", "Write to mario.rossi@example.com about the reimbursement", 300, 7),
    ("D", "Analyse this long 30-page contract", 9000, 0),
    ("E", "Analyse this long 30-page contract", 9000, 0),
]
for nome, testo, token, coda in casi:
    ok = nome != "E"                      # in case E the policy forbids the remote service
    print(nome, instrada({"testo": testo, "token": token}, coda, ok))

print()
esempio = "Customer RSSMRA80A01H501U, write to mario.rossi@example.com and to anna.verdi@example.com, then call back mario.rossi@example.com"
print(pseudonimizza(esempio))

# Output:
# A ('local', 'capacity available')
# B ('remote', 'local saturated, no personal data recognised')
# C ('local', 'local saturated but the text contains personal data: wait')
# D ('remote', "context longer than the local model's")
# E ('reject', 'too long and remote not allowed')
#
# ('Customer [PERSON_3], write to [PERSON_1] and to [PERSON_2], then call back [PERSON_1]', {'mario.rossi@example.com': '[PERSON_1]', 'anna.verdi@example.com': '[PERSON_2]', 'RSSMRA80A01H501U': '[PERSON_3]'})

In the output case A stays local, B (saturated, no personal data) goes to the remote service, C (saturated, with an email address) stays local and waits, D goes to the remote service because of the context length, and E, with the same length but the remote service forbidden, is rejected with a clear message instead of trying anyway. In the pseudonymisation, every identity has the same placeholder in its repetitions, and the map that lets you restore the names stays in-house. The limit not to forget: a regular expression recognises emails and tax codes, not a name in free text or a clinical case described in words. The decision to send a text outside must not be entrusted to a format recogniser without review, and it must be recorded with the fingerprint and without the content, as in the article on the audit log.

Lock-in can be reduced, and there is a rule

Whoever fears being held prisoner by a service has a new tool: the Data Act, Regulation (EU) 2023/2854, which applies from 12 September 2025, governs switching between providers of data processing services. Article 29 provides that from 12 January 2027 providers shall not impose any switching charges on the customer, and that from 11 January 2024 to that date they may impose reduced switching charges that do not exceed the costs directly linked to the switching process. I have not verified how this rule applies to a model inference service as against a generic computing service: read the definition of "data processing service" in the regulation and your contract before counting on it. The practical advice of scenario 7 still holds: keep the access to the model behind an interface of your own.

Frequently asked questions

Is the cloud better if my utilisation is low?

As a rule yes, because the fixed cost of a server is spread over few tokens. The break-even program shows it with declared assumptions: at 5% utilisation your own server costs tens of times the price of an API at 1 euro per million tokens. Replace the assumptions with your own data.

Does a European cloud solve the data transfer problem?

For transfers to third countries under Chapter V of the GDPR, processing that stays in the Union does not raise the question of Article 44. The other obligations remain, starting with the contract with the processor under Article 28 and the choice of sub-processors.

Is the EU-US adequacy decision still valid?

It appears adopted by Implementing Decision (EU) 2023/1795 and the Latombe action was dismissed by the General Court on 3 September 2025. An appeal against the judgment is pending (C-703/25 P): I did not find an outcome at the consultation date, so check the updated status.

Can I use a remote service only for the peaks?

Yes, with explicit rules on what can leave and what cannot. Recognisable personal data stay in-house, and the decision to send a text outside must be recorded and checked on a sample.

What you can do now

You can calculate, with your own measurements, from what utilisation a server of your own pays off against a pay-per-use API, and how much the time of whoever runs it weighs. You can recognise the seven cases in which the managed service pays off: peaks, models that are too large, nobody to run it, contractual requirements, distant users, seasonality, prototypes. You know that the cloud moves responsibility into a contract to be read, where transfers to third countries follow Chapter V of the GDPR and the EU-US decision still has an appeal pending. You can write the routing between local and remote as rules. The last article in the series gathers the path: the recap and what to take away.

The same series also includes the Ollama server on a European VPS, choosing between a small and a large model and privacy and the GDPR with an in-house model.