SLM vs LLM: how to choose the right model for your European SME

Small or large: the right question is what fits and what you need

Choosing between a small model (SLM) and a large one (LLM) is not a choice between two categories: it is a choice along a scale of parameters, where every step costs memory, time and licensing, and returns something in capability. For a small or medium company the practical threshold is not a number of billions of parameters, but the machine you can afford and the minimum quality your task requires. The right way to decide is to measure on your own documents, and the article shows you how to do it with a program that runs on your own computer.

An everyday example: an assistant that answers questions about the internal rules (holidays, business trips, expense reports) works on short texts and on facts, and a small model with the document in front of it may be enough; an assistant that has to summarize a hundred-page contract and compare it with another works on a long context and on multi-step reasoning, where the difference in capability is felt more. Neither case is decided in advance: it is decided with a test.

What changes as you go up in size

Before comparing models it helps to know which quantities move when the number of parameters grows. The consequences are predictable, unlike quality, which depends on the task.

• Weight memory: grows in proportion to the parameters and to the bits per weight; doubling the parameters doubles the file.
• Generation speed: on a CPU it is limited by memory bandwidth, so it falls roughly in inverse proportion to the file size.
• Server cost: jumps in steps, because RAM is bought in fixed sizes.
• Capability on your task: generally increases, but by an amount that only a test on your own data can tell you.

The first three items are calculated; the last one is measured. This is the asymmetry to keep in mind: model cards and public benchmarks describe general tasks, while your question concerns your documents in Italian, with your jargon. That is why the article reports no rankings: I could not verify them against a primary source that measures your case, and an MMLU number does not tell you whether the model understands your rules.

Size, memory and license of some open models

The program below lists eight open models with the exact number of parameters and the license declared in their Hugging Face card (consulted on 2026-11-13), and calculates how much memory the weights alone take at 16 bits, at 8 bits and at about 4.9 bits, the value implied in the quantized files of the Ollama library as shown in the article on the server. The formula is parameters times bits divided by eight.

"""Size and license of some open models.

Parameters: safetensors.total field of the Hugging Face API (2026-11-13).
License: license field of the model card (same date).
Weight memory is parameters x bits / 8; 4.9 bits is the value implied
by the Ollama files, calculated in the previous article of the series.
"""
GIB = 1024 ** 3
MODELS = [
    # name, exact parameters, license declared by the model card
    ("Qwen2.5-3B-Instruct", 3_085_938_688, "qwen-research (commercial use: ask for a license)"),
    ("Phi-3.5-mini-instruct", 3_821_079_552, "MIT"),
    ("Mistral-7B-Instruct-v0.3", 7_248_023_552, "Apache-2.0"),
    ("Qwen2.5-7B-Instruct", 7_615_616_512, "Apache-2.0"),
    ("Llama-3.1-8B-Instruct", 8_030_261_248, "llama3.1 (Meta license)"),
    ("gemma-2-9b-it", 9_241_705_984, "gemma (Google license)"),
    ("Mistral-Nemo-Instruct-2407", 12_247_782_400, "Apache-2.0"),
    ("Qwen2.5-14B-Instruct", 14_770_033_664, "Apache-2.0"),
]


def gib(params, bits):
    return params * bits / 8 / GIB


print(f"{'model':28s} {'Bn':>6s} {'16 bit':>8s} {'8 bit':>7s} {'4.9 bit':>8s}  license")
for name, par, lic in MODELS:
    print(f"{name:28s} {par / 1e9:6.2f} {gib(par, 16):7.1f}G {gib(par, 8):6.1f}G {gib(par, 4.9):7.1f}G  {lic}")

# Output:
# model                            Bn   16 bit   8 bit  4.9 bit  license
# Qwen2.5-3B-Instruct            3.09     5.7G    2.9G     1.8G  qwen-research (commercial use: ask for a license)
# Phi-3.5-mini-instruct          3.82     7.1G    3.6G     2.2G  MIT
# Mistral-7B-Instruct-v0.3       7.25    13.5G    6.8G     4.1G  Apache-2.0
# Qwen2.5-7B-Instruct            7.62    14.2G    7.1G     4.3G  Apache-2.0
# Llama-3.1-8B-Instruct          8.03    15.0G    7.5G     4.6G  llama3.1 (Meta license)
# gemma-2-9b-it                  9.24    17.2G    8.6G     5.3G  gemma (Google license)
# Mistral-Nemo-Instruct-2407    12.25    22.8G   11.4G     7.0G  Apache-2.0
# Qwen2.5-14B-Instruct          14.77    27.5G   13.8G     8.4G  Apache-2.0

The 16-bit column shows why a model of 8 billion parameters does not fit in an 8 GB machine without quantization: it needs about 15 GiB for the weights alone. Quantization to a little under 5 bits brings them down to 4-5 GiB, at the cost of a loss of precision that is not quantified in this table and that has to be measured with the method in the next section. The two models of 12 and 15 billion require about 7 and 8 GiB of weights, so a 16 GB machine is the realistic minimum, not counting the cache.

The license column is the most neglected and the riskiest. Models under the Apache-2.0 or MIT license allow commercial use without special conditions. The 3-billion Qwen2.5 is released under a research license: the text defines "non-commercial" as use for research or evaluation, and says that whoever uses the materials commercially must ask Alibaba Cloud for a license. The same family name, Qwen2.5, therefore has different licenses depending on the size: the 7 and 14 billion sizes are listed as Apache-2.0, the 3 billion one is not.

The Llama 3.1 license is Meta's own license, not a standard open source license. From the official text, consulted on 2026-11-13, three concrete obligations emerge: if the products of you and your affiliates exceed 700 million monthly active users you must ask Meta for a license; if you distribute the materials or a product containing them you must attach the license and display "Built with Llama"; and the use must comply with the acceptable use policy. For a small or medium company the first limit is far away, the other two are not. The Gemma license is also Google's own and I have not read it in full here: read it before using it in a product.

Quantization: fewer bits, less memory, quality to be checked

Quantizing means representing each weight with fewer bits than the original model uses. The result is a smaller file that is faster to read from memory, with a loss of precision. How much that loss weighs on your task nobody knows in advance: there are tasks in which a quantization to 4-5 bits is indistinguishable and tasks, such as calculations or extracting numbers, in which errors surface sooner.

The operating rule is to compare the same model in two quantizations on the same set of questions, and to choose the smallest one that does not worsen the result beyond your threshold. If the table says that going from 8 to about 5 bits saves 3 GiB, that saving is only worth it if quality stays within the limits: the method to find out is the next one.

How to compare two models on your documents

The method is simple: prepare a set of real questions with the expected answer expressed as words that must appear, have the two models answer, and count the correct answers. The problem is statistical, not technical: with few questions the score is too uncertain to tell two models apart. The program below shows it with eight questions from an imaginary company rulebook and two "models" that are functions with hand-written answers (no real model is called): it serves to see the calculation, not to judge a product.

"""How many questions you need to compare two models on YOUR documents.

The two 'models' are functions that return hand-written answers: they serve
to show the method, not to measure a real model. No model is called.
"""
from math import sqrt

Z95 = 1.959964   # normal quantile for the 95% interval
Z80 = 0.841621   # normal quantile for 80% power

# Control set: question -> words the answer MUST contain.
QUESTIONS = {
    "How many days of notice for holidays?": ["10", "days"],
    "Who approves a business trip?": ["manager"],
    "By when must the expense report be submitted?": ["30", "days"],
    "Where do you report an IT incident?": ["helpdesk"],
    "What is the spending limit without approval?": ["500"],
    "Who keeps the keys to the archive?": ["office"],
    "When is the safety training renewed?": ["year"],
    "How do you request time off?": ["portal"],
}

ANSWERS_A = [  # model 'A' (made up): gets the incident question wrong
    "10 days", "the manager", "within 30 days", "write to a colleague",
    "500 euros", "the front office", "every year", "from the portal",
]
ANSWERS_B = [  # model 'B' (made up): gets the time-off question wrong
    "10 days", "the manager", "within 30 days", "to the helpdesk",
    "500 euros", "the front office", "every year", "ask your boss",
]


def outcomes(answers):
    return [all(w in a for w in expected) for a, expected in zip(answers, QUESTIONS.values())]


def wilson(successes, n, z=Z95):
    """Wilson interval for a proportion: holds up even with small n."""
    p = successes / n
    center = (p + z * z / (2 * n)) / (1 + z * z / n)
    half = z * sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
    return center - half, center + half


for name, a in [("A", ANSWERS_A), ("B", ANSWERS_B)]:
    s = sum(outcomes(a))
    lo, hi = wilson(s, len(QUESTIONS))
    print(f"model {name}: {s}/{len(QUESTIONS)} correct, 95% interval = [{lo:.2f}, {hi:.2f}]")


def n_per_group(p1, p2):
    """Questions per model needed to tell two proportions apart
    (two-tailed test, alpha 5%, power 80%, normal approximation)."""
    var = p1 * (1 - p1) + p2 * (1 - p2)
    return (Z95 + Z80) ** 2 * var / (p1 - p2) ** 2


print("\nquestions needed for each model")
for p1, p2 in [(0.70, 0.80), (0.70, 0.90), (0.50, 0.90)]:
    print(f"  {p1:.0%} versus {p2:.0%}: about {n_per_group(p1, p2):.0f}")

# Output:
# model A: 7/8 correct, 95% interval = [0.53, 0.98]
# model B: 7/8 correct, 95% interval = [0.53, 0.98]
#
# questions needed for each model
#   70% versus 80%: about 290
#   70% versus 90%: about 59
#   50% versus 90%: about 17

The two models get the same 7 out of 8, while getting different questions wrong, and the 95% Wilson confidence interval runs from 0.53 to 0.98: with eight questions we do not know whether the true accuracy is 53% or 98%, nor whether one model is better than the other. The second part of the output answers the practical question: to tell 70% from 80% you need about 290 questions for each model, to tell 70% from 90% about 59, to tell 50% from 90% about 17. The calculation uses the normal approximation with a 5% threshold and 80% power.

Two consequences follow. Small differences between similar models are almost impossible to demonstrate for a small or medium company, so it is better to choose with other criteria (license, memory, speed) and to reserve the test for large differences. And the control set is worth more than the model: building it well (real questions, answers checked by someone who knows the rules) is the work that stays useful even when the model changes. One limit of the method: looking for expected words penalizes correct answers that are phrased differently, and rewards those that contain the words by chance; for non-factual tasks you need a human evaluation.

When to move to a larger model

The signals for going up in size are observable, not vague. I list them as things to check on your own set of questions, not as universal rules: I have no source that quantifies them.

• The score on your control set stays below the threshold even after improving the prompt and the retrieval of documents.
• The errors are reasoning errors (comparisons, calculations, nested conditions), not missing information.
• The task requires a longer context than the small model handles well.
• The language or jargon of your sector is poorly covered and the model invents terms.

The first signal is the most important, because it separates a model problem from a data problem. Giving the model the right document (it is called retrieval, and it is the topic of the article on retrieval with vector search) remedies what the model does not know, but not what it cannot do. If even with the right document in front of it the model gets it wrong, the problem is capability and the answer is a larger model, of your own or in the cloud: the cost comparison is in the article on costs, and the cases in which the cloud is the better choice are in when it is not worth doing it yourself.

Do not tie the code to a model name

Whatever the choice today, in six months a better model will come out. The defense is trivial and costs little: the name of the model sits in a configuration file, not in the code; the calls go through a single function; your control set is rerun with one command. That way changing model means changing one line and rerunning the test, not rewriting the application. For the current model families, with their architecture and the differences in cache memory, see the comparison in which model to choose.

Frequently asked questions

What you can do now

You can estimate the weight memory of a model from the number of parameters and the bits, you can read the license before choosing, and you know the three most relevant conditions of the Llama 3.1 one. You also know why eight questions are not enough to compare two models, and you can calculate how many you need for the difference you care about. The sources below show where I took each piece of data; model scores are not reported because I have not verified them on your task.

The same series also includes the recap with the decision sheet.