Why a model trained in English wastes Italian
If in 2023 you tried to make a model born on English work in Italian, the most concrete flaw was not the grammatical mistake but the bill: every Italian word was split into more fragments than the equivalent English word, so the same text took up more "seats" in the context window and needed more generation steps. The Italian models born between 2023 and 2024 (Camoscio, Fauno, Cerbero, LLaMAntino, Minerva) answer this problem in different ways, and understanding which way they chose is worth more than any ranking. This article is a snapshot of that moment; for what came after there is the overview of Italian models in 2025.
The measure that separates the two worlds is called the tokenizer's fertility: the average number of fragments (tokens) per word. A tokenizer with high fertility on a language "minces" it: the Minerva card reports, on a CulturaX sample, 1.87 tokens per Italian word with the Mistral-7B-v0.1 tokenizer against 1.32 for English, while the Minerva-7B tokenizer drops to 1.32 on Italian. A little over half a token per word, but on a long document it means a context that fills up ahead of time.
Fertility, measured on a toy tokenizer
I cannot download a real tokenizer without downloading a model, and here I do not. I can, however, show the mechanism with a BPE (byte-pair encoding) tokenizer written in a few lines of Python: it starts from single characters and merges, one step at a time, the most frequent pair of symbols in the training corpus. I train two identical ones, with 150 merges each: one on twenty-seven Italian sentences written by me, the other on twenty-seven English sentences with the same content. Then I measure how many fragments per word they produce on the same Italian text, never seen in training. The Italian sentences are the data of the experiment and stay in Italian (the English corpus is their rendering, sentence by sentence): translating them would change what the tokenizer sees.
Three lines matter. In addestra (train) the line with min(coppie, key=...) picks the most frequent pair and, on a tie, the lexicographically smaller one: without this tie-break the outcome would depend on the order of a dictionary and two runs could give different vocabularies. segmenta (segment) applies the merges in the order in which they were learned, as a real BPE does. fertilita (fertility) divides the total number of fragments by the number of words. The </w> marker closes every word and counts as one symbol: for this reason the absolute values are not comparable with those of a real tokenizer, only the comparison between the two tokenizers is meaningful.
import re
from collections import Counter
CORPUS_IT = """
La mattina l'autista dell'autobus controlla che le porte si chiudano e che nessuno sia rimasto sul marciapiede.
Nella biblioteca dell'università gli studenti consultano i volumi di storia, di filosofia e di geografia.
L'insegnante ha spiegato perché la città è cresciuta intorno al fiume e quali strade collegavano i mercati.
Quando piove, la gente cammina più velocemente e le macchine rallentano davanti agli attraversamenti pedonali.
Abbiamo discusso a lungo della proposta, ma alla fine nessuno ha trovato un'obiezione davvero convincente.
Il ristorante all'angolo prepara ogni giorno la pasta fresca e il pane con farine locali.
Gli abitanti del quartiere chiedono da anni che la piazza venga illuminata meglio durante l'inverno.
Il medico ha raccomandato di camminare ogni sera e di mangiare più verdura, frutta e legumi.
Nell'archivio comunale si conservano documenti, mappe e lettere scritte dai cittadini nell'Ottocento.
Le aziende italiane esportano macchinari, tessuti, olio e vino verso molti paesi europei.
Il professore ha assegnato un compito difficile, e gli studenti hanno lavorato insieme fino a tarda sera.
La nonna racconta sempre le stesse storie, e ogni volta i bambini ascoltano con la stessa attenzione.
Domani la squadra giocherà la partita decisiva e l'allenatore ha già scelto i giocatori.
Il treno per Lecce parte dal binario tre, ma oggi è in ritardo a causa di un guasto alla linea.
Quest'anno la primavera è arrivata presto, e nei giardini sono già fioriti i mandorli.
Gli ingegneri stanno verificando che il ponte resista al vento, al peso dei camion e alle vibrazioni.
La sindaca ha annunciato che le scuole resteranno chiuse finché non sarà risolto il problema del riscaldamento.
Ho comprato un biglietto per il concerto, ma purtroppo non potrò partecipare perché lavoro fino alle nove.
Nelle campagne pugliesi gli ulivi secolari sono stati danneggiati dal gelo, e i contadini temono per il raccolto.
Lo scrittore ha pubblicato un romanzo sulle famiglie del sud, sulla emigrazione e sul desiderio di tornare.
Una buona traduzione non segue le parole una per una, ma cerca di mantenere il tono e il significato.
Il laboratorio dell'ospedale analizza ogni giorno centinaia di campioni e comunica rapidamente i risultati.
Le tradizioni popolari cambiano lentamente, però le feste patronali richiamano ancora moltissime persone.
Il tecnico ha sostituito il cavo difettoso e adesso la connessione funziona regolarmente.
Accanto alla stazione c'è un parcheggio gratuito, aperto tutti i giorni dalle sei alle ventidue.
Le conclusioni dell'indagine sono state presentate ai cittadini durante un'assemblea pubblica molto partecipata.
Spesso la gentilezza di un estraneo rende più sopportabile una giornata faticosa e piena di imprevisti.
""".strip()
CORPUS_EN = """
In the morning the bus driver checks that the doors close and that nobody is left standing on the pavement.
In the university library students read books about history, philosophy and geography.
The teacher explained why the city grew around the river and which roads connected the markets.
When it rains, people walk faster and cars slow down in front of the pedestrian crossings.
We talked about the proposal for a long time, but in the end nobody found a really convincing objection.
The restaurant on the corner makes fresh pasta every day and bread with local flour.
The residents of the neighbourhood have been asking for years for the square to be better lit in winter.
The doctor recommended walking every evening and eating more vegetables, fruit and beans.
The town archive keeps documents, maps and letters written by citizens in the nineteenth century.
Italian companies export machinery, fabrics, oil and wine to many European countries.
The professor set a difficult assignment, and the students worked together until late in the evening.
My grandmother always tells the same stories, and every time the children listen with the same attention.
Tomorrow the team will play the deciding match and the coach has already chosen the players.
The train to Lecce leaves from platform three, but today it is late because of a fault on the line.
This year spring arrived early, and the almond trees are already blooming in the gardens.
The engineers are checking that the bridge can withstand the wind, the weight of the trucks and the vibrations.
The mayor announced that the schools will stay closed until the heating problem is solved.
I bought a ticket for the concert, but unfortunately I cannot attend because I work until nine.
In the countryside of Puglia the ancient olive trees were damaged by the frost, and the farmers fear for the harvest.
The writer published a novel about southern families, about emigration and about the wish to come back.
A good translation does not follow the words one by one, but tries to keep the tone and the meaning.
The hospital laboratory analyses hundreds of samples every day and reports the results quickly.
Popular traditions change slowly, yet the patron saint festivals still attract a great many people.
The technician replaced the faulty cable and now the connection works properly.
Next to the station there is a free car park, open every day from six in the morning to ten at night.
The conclusions of the investigation were presented to the citizens during a very well attended public meeting.
Often the kindness of a stranger makes a tiring day full of surprises a little easier to bear.
""".strip()
TEST_IT = """
La biblioteca comunale organizza incontri sulla storia dell'agricoltura e sulle tradizioni del territorio.
Gli studenti hanno presentato un progetto per rendere più accessibili i servizi della città.
L'assemblea ha approvato la proposta, nonostante le obiezioni di alcuni cittadini del quartiere.
""".strip()
MERGES = 150
def parole(testo):
return re.findall(r"\w+", testo.lower())
def addestra(testo, n_merge):
"""Character-level BPE; the </w> marker closes every word."""
vocab = Counter()
for p in parole(testo):
vocab[tuple(p) + ("</w>",)] += 1
regole = []
for _ in range(n_merge):
coppie = Counter()
for simboli, f in vocab.items():
for a, b in zip(simboli, simboli[1:]):
coppie[(a, b)] += f
if not coppie:
break
# on equal frequency the lexicographically smaller pair wins: deterministic outcome
migliore = min(coppie, key=lambda c: (-coppie[c], c))
regole.append(migliore)
nuovo = Counter()
for simboli, f in vocab.items():
out, i = [], 0
while i < len(simboli):
if i + 1 < len(simboli) and (simboli[i], simboli[i + 1]) == migliore:
out.append(simboli[i] + simboli[i + 1]); i += 2
else:
out.append(simboli[i]); i += 1
nuovo[tuple(out)] += f
vocab = nuovo
return regole
def segmenta(parola, regole):
simboli = list(parola) + ["</w>"]
for a, b in regole:
i = 0
while i < len(simboli) - 1:
if simboli[i] == a and simboli[i + 1] == b:
simboli[i:i + 2] = [a + b]
else:
i += 1
return simboli
def fertilita(testo, regole):
ps = parole(testo)
return sum(len(segmenta(p, regole)) for p in ps) / len(ps)
regole_it = addestra(CORPUS_IT, MERGES)
regole_en = addestra(CORPUS_EN, MERGES)
print("words in the test text:", len(parole(TEST_IT)))
print("fertility with BPE trained on Italian:", round(fertilita(TEST_IT, regole_it), 2))
print("fertility with BPE trained on English: ", round(fertilita(TEST_IT, regole_en), 2))
for p in ["agricoltura", "obiezioni", "accessibili"]:
print(p, "->", segmenta(p, regole_it), "|", segmenta(p, regole_en))
# the byte cost of accented letters, which matters in byte-level tokenizers
for s in ["citta", "città", "perché", "l'università"]:
print(repr(s), len(s), "characters,", len(s.encode("utf-8")), "bytes")
# Actual output (python3 -I fertilita.py, 2026-11-12, identical in two runs):
# words in the test text: 42
# fertility with BPE trained on Italian: 3.29
# fertility with BPE trained on English: 4.6
# agricoltura -> ['a', 'g', 'r', 'ic', 'olt', 'u', 'r', 'a</w>'] | ['a', 'g', 'ri', 'co', 'l', 't', 'u', 'r', 'a</w>']
# obiezioni -> ['o', 'bi', 'e', 'zion', 'i</w>'] | ['o', 'b', 'i', 'e', 'z', 'i', 'on', 'i</w>']
# accessibili -> ['a', 'c', 'c', 'ess', 'i', 'bi', 'l', 'i</w>'] | ['a', 'c', 'c', 'es', 'si', 'b', 'i', 'li', '</w>']
# 'citta' 5 characters, 5 bytes
# 'città' 5 characters, 6 bytes
# 'perché' 6 characters, 7 bytes
# "l'università" 12 characters, 13 bytes
The result is the expected one: on the same Italian text the tokenizer trained on Italian produces 3.29 symbols per word and the one trained on English 4.60. Looking at the three printed words shows why: "agricoltura" stops at ic, olt in the first and shatters into single letters in the second, because the Italian sequences zion or ess had never appeared often enough. It is a tiny experiment (about 440 words per language) and must be read as an illustration: it does not prove a numeric factor valid for other tokenizers. The last four lines show a second cost, that of byte-level tokenizers: "città" has five characters but six bytes in UTF-8, because the "à" takes two. With Italian elisions and accents the gap between characters and bytes is not negligible.
Five models, three different roads
From the models' cards and papers (consulted on 2026-11-12) three ways emerge of reaching a model "that knows Italian": training it from scratch, continuing the training of a foreign model, fine-tuning it with little compute. Here is where the five names of the era fall.
- Minerva (Sapienza NLP with FAIR and CINECA): a family "pretrained from scratch on Italian" with sizes from 350 million to 7 billion parameters; the 7B is trained on about 2.48 trillion tokens (1.14 Italian, 1.14 English, 0.2 code); Apache 2.0 licence.
- LLaMAntino: linguistic adaptation of the LLaMA 2 family to Italian (paper of December 2023); the 2024 ANITA variant starts from Llama 3 8B Instruct, uses QLoRA for supervised fine-tuning and DPO for alignment.
- Camoscio: LLaMA 7B fine-tuned with LoRA on Alpaca instructions translated into Italian with ChatGPT; the repository declares research-only use.
- Fauno (RSTLess group, Sapienza): trained on synthetic conversational data (medical, Stack Overflow, Quora, Alpaca) translated into Italian.
- Cerbero: built on Mistral 7B with an Italian chat corpus generated with the self-chat technique, Apache 2.0 licence declared by the authors also for commercial use.
The criterion that holds the entries together is how much of the model is truly Italian. Only Minerva is born from scratch with half of the data in Italian; the other four inherit weights, tokenizer and licence from a foreign model (LLaMA, Llama 3, Mistral) and add a handful of Italian on top. For whoever chooses, the inherited licence matters more than the skill: Camoscio, because of its base model, is not a candidate for a product, whereas Minerva and Cerbero are. The case-by-case, with a programmable check of the cards, is in the article on choosing the Italian LLM starting from the licence.
What the scores of the time say (and do not say)
The Minerva card says it evaluates with ITA-Bench, a collection of 18 Italian benchmarks on scientific knowledge, common-sense reasoning and mathematical problems, but it presents the results as images, not as a text table; the Cerbero paper proposes a quality metric for its own corpus; the cards of the other models use different tests. There is therefore no single, honest and comparable ranking of those five models, and for this reason I do not report one. Deciding with your own test bench remains the only reliable road: a hundred real questions from your users, with the expected answer, as it is built in Fine-tuning an LLM for the Italian language.
How much memory the weights need
A practical question the cards do not settle is whether a model fits in the machine you have. The starting number is the parameter count, which the Hub's public API reports for Minerva-7B: 7,399,018,496. Multiplied by the bytes per weight it gives the memory of the weights alone, without the attention cache and without working space. For the GGUF format Q4_K_M I do not use a nominal "4 bits": the llama.cpp README measures 4.8944 bits per weight on Llama-3.1-8B (4.58 GiB against 14.96 GiB in float16), because quantisation keeps scales and the more sensitive tensors at higher precision. I apply the same ratio to our model as an estimate.
# Parameters declared by the Minerva-7B-base-v1.0 model card (safetensors field of the Hub API)
PARAMETRI = 7_399_018_496
GIB = 2 ** 30
# bytes per weight. For Q4_K_M I use the 4.8944 bits per weight measured by the llama.cpp README on Llama-3.1-8B
formati = [
("float16", 16.0),
("8 bit (nominal)", 8.0),
("4 bit (nominal)", 4.0),
("GGUF Q4_K_M (4.8944 bit/weight)", 4.8944),
]
for nome, bit in formati:
print(f"{nome:32s} {PARAMETRI * bit / 8 / GIB:6.2f} GiB")
# Actual output (python3 -I pesi.py, 2026-11-12, identical in two runs):
# float16 13.78 GiB
# 8 bit (nominal) 6.89 GiB
# 4 bit (nominal) 3.45 GiB
# GGUF Q4_K_M (4.8944 bit/weight) 4.22 GiBThe program prints the four formats: about 13.8 GiB in float16, 6.9 in 8 bit, 3.4 in nominal 4 bit and 4.2 in Q4_K_M. The difference between the last two lines is the reason why an estimate made in your head with "4 bits" is always too optimistic. It remains an estimate: the real sizes depend on the file, and runtime memory grows with the length of the context (for Minerva-7B the maximum is 4,096 tokens, from the model's config). As for the format, the Cerbero card points to a ready-made GGUF version, which avoids the conversion.
How these models are tried, and what I did not run
The route documented by the Minerva card is the transformers library: you load the model with AutoModelForCausalLM and generate with a text-generation pipeline. This step requires PyTorch and tens of GiB of memory, so here it is a sketch to adapt, not a listing I ran. Two details of the card are worth more than the code: the "base" version is a foundation model without alignment, so it can produce incorrect, repetitive or unsuitable text, and for a chat the instruct variant of the same family is better; the 7B's context window is 4,096 tokens, no more.
To use Camoscio or the other models with shared weights as adapters the sequence is different: you load the base model and apply the LoRA weights to it with PeftModel.from_pretrained, as the Camoscio README shows. It is also the reason why the model's repository contains few files but requires you to obtain the base model separately, with its licence. How to train an adapter of your own is described in Fine-tuning an LLM for the Italian language, and the limits of document retrieval in RAG in Italian.
The limits that held back then
- Short context: 4,096 tokens for Minerva-7B; the smaller sizes declare 16,384 with a sliding window of 2,048.
- Base models without alignment: the Minerva card lists the risks (stereotypes, offensive content, wrong information presented as fact).
- Translated synthetic data: Camoscio and Fauno come from translated ChatGPT answers, with the translation artefacts that the authors themselves admit in the README.
- Inherited licences: the base model sets the limits of use, as for Camoscio, whose README points out the non-commercial licence of LLaMA.
The four entries have a common root: a model is a result of the data and the licence it starts from. Short context and lack of alignment are overcome by changing model; translation artefacts and the inherited licence are not, because they are written in the model's history. For this reason, before choosing, it is worth reading the card from the end (risks, data, licence) and not from the beginning.
Frequently asked questions
Can Minerva be used in a product?
The licence declared on the card is Apache 2.0, which allows commercial use. The "base" 7B, however, is a foundation model without alignment: for an assistant you need the instruct version of the same family or a fine-tuning of your own, and you need to try it on your data.
Which Italian model do I choose for commercial use?
Start from the licence: Minerva and Cerbero declare Apache 2.0, Camoscio is limited to research, LLaMAntino-3 inherits the Llama 3 licence. Among the remaining candidates, the test with a hundred examples of your own decides, not a published score.
Why does the tokenizer matter so much?
Because every extra token takes up context and requires one more generation step. In the Minerva card, 1.87 against 1.32 tokens per Italian word means about 42 per cent more fragments (1.87 divided by 1.32) on the same text. To see tokenization and lemmas on real Italian sentences, there is Italian NLP with spaCy and Transformers: A Practical Pipeline.
What you can do now
You can tell apart the three ways in which a model becomes "Italian" and what each one inherits, you can explain fertility with an experiment you can rerun, and you can estimate the weight memory of a 7-billion-parameter model in the most common formats. Above all you know why a 2024 ranking is not enough to choose, and what to do in its place.
Sources
- sapienzanlp/Minerva-7B-base-v1.0 model card on Hugging Face - sizes, tokens per language, Apache 2.0 licence, context, fertility table, risk warnings, consulted on 2026-11-12
- Hub API for Minerva-7B-base-v1.0 - total parameter count (7,399,018,496), consulted on 2026-11-12
- LLaMAntino: LLaMA 2 Models for Effective Text Generation in Italian Language (arXiv 2312.09993) - linguistic adaptation of LLaMA 2, consulted on 2026-11-12
- LLaMAntino-3-ANITA (arXiv 2405.07101) - Llama 3 8B Instruct, SFT with QLoRA and DPO, consulted on 2026-11-12
- Camoscio: an Italian Instruction-tuned LLaMA (arXiv 2307.16456) - LoRA on LLaMA 7B with instructions translated via ChatGPT, consulted on 2026-11-12
- Camoscio repository - research-only use, licence of the base model, loading the adapter, consulted on 2026-11-12
- Fauno repository - synthetic conversational datasets, consulted on 2026-11-12
- Cerbero-7B (arXiv 2311.15698) - chat corpus generated with self-chat, consulted on 2026-11-12
- galatolo/cerbero-7b model card - Mistral 7B base, Apache 2.0 licence, pointer to the GGUF version, consulted on 2026-11-12
- llama-quantize README (llama.cpp) - bits per weight of Q4_K_M and sizes on Llama-3.1-8B, consulted on 2026-11-12
- Measurements in the repository:
python3 -I fertilita.pyandpython3 -I pesi.py, outputs reported above, run on 2026-11-12 twice with identical results (English listings)
The sketch with transformers was not run (no PyTorch on the working machine). The BPE tokenizer is a teaching model trained on texts written for the occasion: it demonstrates the mechanism, it does not measure the tokenizers mentioned.
Commenti
Caricamento commenti...
Sign in to comment