Q3 2026 takeaways + ebook teaser Sovereign AI PMI EU: recap 12 articles

What really decides whether to keep AI in-house: five numbers and three constraints

Twelve articles on "sovereign" AI for European SMEs can be summed up in one uncomfortable sentence: the series does not answer "yes, keep everything in-house" or "no, go to the cloud", because the answer depends on numbers that only you can measure. There are five of them: the tokens per second your machine produces, the memory that the model and the parallel requests occupy, the utilisation you will really have, the cost of the time of whoever runs the service, and the time you need to get back up after a failure. On top of these come three constraints that come from the rules: where personal data goes, what you must be able to demonstrate with a log, and what the European regulation on artificial intelligence requires and from when.

This article closes the path in three moves. It puts the twelve articles back in order by saying what each one leaves in your hands, it builds an executable decision sheet that brings the checks together, and it says frankly what the series could not verify. There is no summary of figures for you to take away: the figures of one machine, one price list or one provider do not hold for yours, and every figure that appears here comes from a program that was run or a rule that was read.

The path, article by article

The series has an order, and the order matters: it starts from serving a model and understanding how much memory it asks for, moves on to choosing the model and to costs, builds the retrieval of documents, and only then arrives at evaluation, load, disaster, the log and the final question about when the cloud is better. A reader who already has a working system can enter at the point they need, but the first six are there to help you read the last ones properly.

1. Serving Ollama in production on a European VPS: how much memory is needed, what speed to expect and how to protect the endpoint.
2. SLM versus LLM: how to choose between a small and a large model, with memory, licences read from the texts and the number of questions needed to compare two models.
3. Real costs: when a server of your own costs less than an API, with declared assumptions and a break-even point.
4. Ollama, RAG and RediSearch: hybrid retrieval for company documents, with chunking, BM25, vectors and fusion.
5. Privacy by design: where the copies of personal data end up and what the GDPR really asks for.
6. Which model: Llama, Mistral, Qwen and Gemma compared on architecture, cache, context and licences.
7. Evaluating RAG: the four metrics, the golden dataset, a local judge and the paired A/B test.
8. From 1 to 50 users: tokens per second, memory per request, a bounded queue and a token bucket.
9. Disaster recovery: RPO and RTO per component, WAL archiving, backup verification and the restore drill.
10. Audit log: what Article 12 of the AI Act says, minimal records, HMAC and a chain of fingerprints.
11. When NOT to self-host: seven cases in which the managed service pays off, with the break-even calculation.

The list has a reading order, not an order of importance: the cost in the third article cannot be understood without the memory in the first, and the log in the tenth presupposes the pseudonymisation in the fifth. If you have to skip something, skip the comparison between models, which ages in months, and not the calculations of memory, capacity and recovery, which stay true until your machine changes. The article just before this one, on when self-hosting does not pay, is the corrective for the whole series: after reading it, a path that always ended in "keep everything in-house" would no longer be credible.

The decision sheet: the checks in one program

The series' numbers can be put together in a single check that tells you where your case holds and where it does not. The program takes a constructed case (twelve users, a model of about 7 billion parameters at 4 bits, two parallel requests, 16 GiB of memory) and applies five checks: memory, with weights plus cache for the parallel requests plus a margin for the system; capacity, which uses the queueing model at utilisation 0.5; cost, that is your own server against an API at the utilisation you will really have; recovery, the sum of the steps measured in the drill against the target; and the log, which separates the Article 12 obligation from a choice of prudence. All the values are test values: replace them with your own measurements.

"""Decision sheet: the series' checks on a constructed case, with YOUR measurements in place of the test ones."""
SECONDI_MESE = 30 * 24 * 3600
GIB = 1024 ** 3


def controlla(c):
    esiti = []

    # 1. Memory: weights + cache for the parallel requests + margin for the system and other services
    cache = c["parallele"] * c["contesto"] * c["kv_kib_per_token"] * 1024 / GIB
    serve = c["pesi_gib"] + cache + c["margine_gib"]
    esiti.append(("memory", serve <= c["ram_gib"], f"need {serve:.1f} GiB out of {c['ram_gib']} available"))

    # 2. Capacity: sustainable users at utilisation 0.5 (M/M/1 model from the sizing article)
    mu = c["token_al_secondo"] / c["token_risposta"]
    utenti_max = int(0.5 * mu * c["pausa_s"])
    esiti.append(("capacity", utenti_max >= c["utenti"], f"at utilisation 0.5 you sustain {utenti_max} users, you have {c['utenti']}"))

    # 3. Cost: your own server against an API, at the utilisation you will really have
    token_mese = c["utenti"] / c["pausa_s"] * c["token_risposta"] * c["ore_attive_mese"] * 3600
    capacita_mese = c["token_al_secondo"] * SECONDI_MESE
    utilizzo = token_mese / capacita_mese
    proprio = (c["server_mese"] + c["ore_gestione"] * c["tariffa"]) / (token_mese / 1e6)
    esiti.append(("cost", proprio <= c["api_per_milione"],
                  f"utilisation {utilizzo:.1%}: {proprio:.2f} euro per million tokens against {c['api_per_milione']:.2f}"))

    # 4. Recovery: the sum of the steps measured in the drill against the target
    rto = sum(c["passi_ripristino_min"])
    esiti.append(("recovery", rto <= c["rto_obiettivo_min"], f"recovery drill {rto} min, target {c['rto_obiettivo_min']}"))

    # 5. Log: an obligation only if high-risk under Annex III (from 2 December 2027), otherwise a prudence choice
    nota = "obligation from 2 December 2027 (Annex III)" if c["alto_rischio_allegato_III"] else "no Article 12 obligation; minimal log for accountability"
    esiti.append(("log", True, nota))
    return esiti


CASO = dict(pesi_gib=3.4, kv_kib_per_token=128, contesto=4096, parallele=2, margine_gib=6.0, ram_gib=16,
            token_al_secondo=25.0, token_risposta=300, pausa_s=120, utenti=12, ore_attive_mese=160,
            server_mese=200.0, ore_gestione=8, tariffa=50.0, api_per_milione=5.0,
            passi_ripristino_min=[10, 15, 20, 10], rto_obiettivo_min=60, alto_rischio_allegato_III=False)


def stampa(titolo, caso):
    print(titolo)
    for nome, ok, dettaglio in controlla(caso):
        print(f"  {'OK ' if ok else 'NO '} {nome:10s} {dettaglio}")


stampa("constructed case (assumptions: replace with your own measurements)", CASO)
stampa("same case, no admin hours and an API at 20 euro per million", {**CASO, "ore_gestione": 0, "api_per_milione": 20.0})

# Output:
# constructed case (assumptions: replace with your own measurements)
#   OK  memory     need 10.4 GiB out of 16 available
#   NO  capacity   at utilisation 0.5 you sustain 5 users, you have 12
#   NO  cost       utilisation 26.7%: 34.72 euro per million tokens against 5.00
#   OK  recovery   recovery drill 55 min, target 60
#   OK  log        no Article 12 obligation; minimal log for accountability
# same case, no admin hours and an API at 20 euro per million
#   OK  memory     need 10.4 GiB out of 16 available
#   NO  capacity   at utilisation 0.5 you sustain 5 users, you have 12
#   OK  cost       utilisation 26.7%: 11.57 euro per million tokens against 20.00
#   OK  recovery   recovery drill 55 min, target 60
#   OK  log        no Article 12 obligation; minimal log for accountability

The first case is instructive because almost everything goes well and two checks do not. Memory and recovery hold; capacity does not, because 25 tokens per second with 300-token answers sustain five users at utilisation 0.5, not twelve; cost does not, because with that load the monthly utilisation is 26.7% and the eight hours of administration bring your own server to 34.72 euro per million tokens against 5 for the test API. The second case changes two assumptions, no administration hours and a more expensive API, and the cost check turns positive while capacity stays insufficient: a result counts only if you also read the other checks. The log line is an indication, not a calculated obligation: it depends on your system and on the list in Annex III.

A note on the program: the controlla function returns a list of triples (name, outcome, detail) and not a single verdict, on purpose. An overall score hides which constraint is blocking you, and the constraint that blocks you is the information you need: if it is memory you buy RAM or change model, if it is capacity you add machines or reduce the context, if it is cost you evaluate the managed service. The thresholds and formulas are those of the sizing and cost articles, copied here in the form of code so that you can touch them.

What the series could not verify

A series that proposes calculations must list what it did not measure, otherwise the reader takes it for measured. Below you find the things left out, either because there was no primary source within reach or because it required an execution that was not possible.

• The prices of machines, GPUs and APIs: they change, and you read them from the official price list on the day you decide; the articles use declared test values.
• Speed in tokens per second on a specific piece of hardware: no model was run for these articles, and the value that counts is the one you measure yourself with the fields that Ollama returns.
• The state of the appeal against the General Court judgment on the EU-US adequacy decision, and the state of the European cloud certification scheme.
• Whether the legislative decrees foreseen by Law 132/2025 have been issued, and with what penalties.
• The way a Redis backup treats a vector search index: it has to be verified in your own restore drill.

Each of these items has a remedy that goes through you: reading a price list, running a measurement, checking a legal text, repeating a drill. The series has given you the method for doing it and the sources to start from, and the consultation date written next to every source is there to tell you from when to check again. The rules that matter most for this series have precise dates: Regulation 2024/1689 applies to the high-risk systems of Annex III from 2 December 2027, and the obligation to keep logs for at least six months, unless otherwise provided, is in Articles 19 and 26 of the consolidated text.

Frequently asked questions

For an SME, is in-house AI or a managed service better?

It depends on four numbers that you can measure: the utilisation you will have, the cost of management time, the memory the model needs and the recovery time you can guarantee. With low utilisation and nobody to run the service, the managed service is usually better; with high utilisation and personal data that must not leave, your own server.

Where do I start?

From the measurement: your machine's tokens per second, with the fields that Ollama returns, and the memory the model occupies. With those two numbers the decision sheet tells you straight away whether the in-house road is practicable.

Do I have to keep a log of my assistant's events?

Article 12 of the AI Act provides for it for high-risk systems, from 2 December 2027 for those in Annex III. For the others the minimal log remains a choice of prudence linked to the responsibility of the data controller, and it must be designed without copying personal data.

What you can do now

You can decide where to keep an AI model with your own numbers instead of other people's figures: measure the speed, calculate the memory from the parameters, estimate the break-even with the administration hours included, size with a bounded queue, test recovery with a stopwatch, log events without copying personal data and choose the cloud when the numbers say so. You also know where your information expires, because every source carries its date. If one of the checks in the sheet tells you no, the corresponding article of the series is the place to start again from.