Italian NLP with spaCy and Transformers: A Practical Pipeline

What spaCy and transformers really do, and when to pick one or the other

spaCy is a pipeline library: it takes a string and passes it through components in sequence, each of which adds an annotation (token boundaries, part of speech, lemma, syntactic relations, entities). spaCy's Italian models are small networks trained for these tasks and run on CPU. A transformer such as BERT, on the other hand, produces a numerical representation of each token or sentence, on which you build a classifier, a similarity engine or an entity extractor yourself. In the first case you buy a ready-made annotation; in the second you buy a good starting point that you must complete with your own data.

The practical rule is to start from the simplest pipeline that solves the problem. To extract names of people and places from news articles, spaCy is enough. To classify customer messages into ten categories with a few hundred examples, a linear classifier on character n-grams (see below) is a harder opponent to beat than people believe. The transformer comes in when you need to tell apart shades of meaning that the words alone do not carry, and when you have labelled data to check it.

Tokenization: the apostrophe and clitics

Tokenization is the step that decides what "a word" is, and in Italian split() fails in two predictable ways. The first is elision: in "L'auto" (the car) or "all'università" (at the university) the apostrophe joins the article or preposition to the next word, and a count by spaces treats them as a single word, which you will never find in the dictionary. The second is the periods and punctuation marks attached to words, which turn "rotta." and "rotta" (broken) into two different entries.

The program below solves both with a regular expression and adds a second level, the splitting of clitics. The expression has its alternatives in priority order: first the exception "po'" (which is a whole word, "a bit"), then the sequence "letters + apostrophe followed by a letter" that isolates the elision, then the word, the number and the single symbol. The period stays attached only to the abbreviations in the ABBREVIAZIONI list, otherwise the sentence-final period merges with the last word.

import re

TESTI = [
    "L'auto di Giuseppe è rotta.",
    "Il prof. Bianchi insegna all'università di Roma, un po' tardi.",
    "Vediamoli domani; portamelo subito e mandaglielo.",
]

ABBREVIAZIONI = {"prof", "ing", "dott", "sig"}   # the period stays attached only to these
LETTERE = "A-Za-zÀ-ÿ"
# Order of the alternatives = order of priority: "po'" whole, then elision (l', dell'),
# then word, then number, then a single symbol.
TOKEN = re.compile(rf"po'|[{LETTERE}]+'(?=[{LETTERE}])|[{LETTERE}]+|\d+|[^\s{LETTERE}\d]")

def tokenizza(testo):
    out, pos = [], 0
    for m in TOKEN.finditer(testo):
        tok = m.group(0)
        fine = m.end()
        if tok.lower() in ABBREVIAZIONI and testo[fine:fine + 1] == ".":
            tok += "."
        elif tok == "." and out and out[-1].endswith("."):
            continue
        out.append(tok)
    return out

# Clitics, from the longest to the shortest: "glie" must beat "gli" and "li".
CLITICI = ["glie", "gli", "mi", "ti", "ci", "vi", "si", "me", "te", "ce", "ve", "se",
           "lo", "la", "li", "le", "ne"]
# Without a verb lexicon you cannot tell "portamelo" from any other word.
BASI = {"vediamo", "porta", "manda"}

def scomponi_clitici(parola):
    minuscola = parola.lower()
    for base in sorted(BASI, key=len, reverse=True):
        if minuscola.startswith(base) and len(minuscola) > len(base):
            resto, pezzi = minuscola[len(base):], []
            while resto:
                for c in sorted(CLITICI, key=len, reverse=True):
                    if resto.startswith(c):
                        pezzi.append(c)
                        resto = resto[len(c):]
                        break
                else:
                    return [parola]     # the rest is not made of clitics: word left intact
            return [base] + pezzi
    return [parola]

for t in TESTI:
    nostro = tokenizza(t)
    print("input   :", t)
    print("split() :", t.split())
    print("regex   :", nostro)
    print("clitics :", [p for w in nostro for p in scomponi_clitici(w)])
    print()

# Output:
# input   : L'auto di Giuseppe è rotta.
# split() : ["L'auto", 'di', 'Giuseppe', 'è', 'rotta.']
# regex   : ["L'", 'auto', 'di', 'Giuseppe', 'è', 'rotta', '.']
# clitics : ["L'", 'auto', 'di', 'Giuseppe', 'è', 'rotta', '.']
#
# input   : Il prof. Bianchi insegna all'università di Roma, un po' tardi.
# split() : ['Il', 'prof.', 'Bianchi', 'insegna', "all'università", 'di', 'Roma,', 'un', "po'", 'tardi.']
# regex   : ['Il', 'prof.', 'Bianchi', 'insegna', "all'", 'università', 'di', 'Roma', ',', 'un', "po'", 'tardi', '.']
# clitics : ['Il', 'prof.', 'Bianchi', 'insegna', "all'", 'università', 'di', 'Roma', ',', 'un', "po'", 'tardi', '.']
#
# input   : Vediamoli domani; portamelo subito e mandaglielo.
# split() : ['Vediamoli', 'domani;', 'portamelo', 'subito', 'e', 'mandaglielo.']
# regex   : ['Vediamoli', 'domani', ';', 'portamelo', 'subito', 'e', 'mandaglielo', '.']
# clitics : ['vediamo', 'li', 'domani', ';', 'porta', 'me', 'lo', 'subito', 'e', 'manda', 'glie', 'lo', '.']

In the third sentence ("Let's see them tomorrow; bring it to me right away and send it to him") the clitic splitting separates "portamelo" into porta + me + lo and "mandaglielo" into manda + glie + lo, while "Vediamoli" becomes vediamo + li thanks to the lowercase comparison. The condition that holds everything up is the BASI list: without a lexicon of verb forms you cannot tell "portamelo" from a word that happens to end in "lo", and the function returns the word intact if the rest is not made of clitics. The hard case, "dammelo" (give it to me), is not in the program because the monosyllabic imperative doubles the consonant (da' + mi becomes "dammi"): you need one more rule, or a trained model. This is why in production you use a library's tokenizer and spot-check the cases of your own domain.

Parts of speech and lemmas: why the lemma depends on the category

A lemmatizer reduced to a "form → lemma" dictionary gets it wrong every time a form has two categories. "Porta" is either the noun for the door of a house (lemma "porta") or the third person of the verb "portare" (to carry); "sale" is the seasoning (salt) or the verb "salire" (to go up); "la" is an article (lemma "il") or a pronoun (lemma "la"). The right lemma is chosen after the category is chosen, and this is how pipelines work: the tagger first, the lemmatizer after.

To see how the tagger resolves the ambiguity, the program below implements the simplest model that does so: a hidden Markov model. It learns two tables by counting on a mini corpus annotated by hand: the probability that a category emits a given word, and the probability that one category follows another. The Viterbi algorithm then picks the most probable sequence of categories for the whole sentence. This is why "la porta è chiusa" (the door is closed: article + noun) and "Maria la porta in casa" (Maria takes her home: pronoun + verb) get different labels for the same word "porta".

import math
from collections import Counter, defaultdict

# Mini corpus annotated by hand (Universal Dependencies-style tags). It is not a real corpus:
# it shows the mechanism, it does not measure accuracy.
CORPUS = [
    "la/DET porta/NOUN è/AUX aperta/ADJ",
    "la/DET porta/NOUN della/ADP casa/NOUN è/AUX chiusa/ADJ",
    "la/PRON porta/VERB in/ADP casa/NOUN",
    "la/PRON vedo/VERB ogni/DET mattina/NOUN",
    "Maria/PROPN la/PRON porta/VERB a/ADP scuola/NOUN",
    "il/DET sale/NOUN è/AUX bianco/ADJ",
    "il/DET sale/NOUN costa/VERB poco/ADV",
    "il/DET gatto/NOUN sale/VERB sul/ADP tetto/NOUN",
    "il/DET gatto/NOUN nero/ADJ sale/VERB sul/ADP muro/NOUN",
    "le/DET ragazze/NOUN correvano/VERB verso/ADP la/DET scuola/NOUN",
    "la/DET scuola/NOUN è/AUX vecchia/ADJ",
]
LEMMI = {  # (form, tag) -> lemma: the tag is what disambiguates
    ("porta", "NOUN"): "porta", ("porta", "VERB"): "portare",
    ("sale", "NOUN"): "sale", ("sale", "VERB"): "salire",
    ("la", "DET"): "il", ("la", "PRON"): "la", ("correvano", "VERB"): "correre",
    ("è", "AUX"): "essere", ("le", "DET"): "il", ("ragazze", "NOUN"): "ragazza",
}

emiss, trans, tags = defaultdict(Counter), defaultdict(Counter), set()
for frase in CORPUS:
    prec = "<s>"
    for coppia in frase.split():
        parola, tag = coppia.rsplit("/", 1)
        emiss[tag][parola.lower()] += 1
        trans[prec][tag] += 1
        tags.add(tag); prec = tag
    trans[prec]["</s>"] += 1

V = len({p for t in emiss.values() for p in t}) + 1   # +1 for unknown words
def lp_em(tag, parola):   # add-one: a word never seen does not zero out the probability
    return math.log((emiss[tag][parola] + 1) / (sum(emiss[tag].values()) + V))
def lp_tr(a, b):
    return math.log((trans[a][b] + 1) / (sum(trans[a].values()) + len(tags) + 1))

def viterbi(parole):
    parole = [p.lower() for p in parole]
    best = [{t: (lp_tr("<s>", t) + lp_em(t, parole[0]), None) for t in sorted(tags)}]
    for p in parole[1:]:
        riga = {}
        for t in sorted(tags):
            punteggio, da = max((best[-1][a][0] + lp_tr(a, t), a) for a in sorted(tags))
            riga[t] = (punteggio + lp_em(t, p), da)
        best.append(riga)
    t = max(sorted(tags), key=lambda x: best[-1][x][0] + lp_tr(x, "</s>"))
    out = [t]
    for riga in reversed(best[1:]):
        t = riga[t][1]; out.append(t)
    return list(reversed(out))

for frase in ["La porta è chiusa", "Maria la porta in casa", "Il gatto sale sul muro", "Il sale costa poco"]:
    parole = frase.split()
    tg = viterbi(parole)
    righe = [(p, t, LEMMI.get((p.lower(), t), "-")) for p, t in zip(parole, tg)]
    print(frase)
    for p, t, l in righe:
        print(f"   {p:8} {t:6} {l}")

# Output:
# La porta è chiusa
#    La       DET    il
#    porta    NOUN   porta
#    è        AUX    essere
#    chiusa   ADJ    -
# Maria la porta in casa
#    Maria    PROPN  -
#    la       PRON   la
#    porta    VERB   portare
#    in       ADP    -
#    casa     NOUN   -
# Il gatto sale sul muro
#    Il       DET    -
#    gatto    NOUN   -
#    sale     VERB   salire
#    sul      ADP    -
#    muro     NOUN   -
# Il sale costa poco
#    Il       DET    -
#    sale     NOUN   sale
#    costa    AUX    -
#    poco     ADJ    -

The first three sentences are labelled as expected and the lemmas come from the LEMMI lexicon keyed by (form, category): "porta" gives "porta" as a noun and "portare" as a verb. The last sentence shows the honest limit of the method. "Costa" (costs) is labelled as an auxiliary and "poco" (little) as an adjective, although in the corpus they appear as a verb and an adverb: with eleven sentences, the transitions (after a noun, the auxiliary is frequent) weigh more than the single observation of the word. It is the same mistake a real model makes in front of a domain it has not seen, with the difference that the real model has read hundreds of thousands of sentences.

Named entities: how to measure whether they work

A named entity is a fragment of text that indicates a person, a place, an organization. The simplest way to recognize them is a list of known names, and it is a good way to understand how a recognizer is evaluated, which is what matters when you have to choose between one model and another. The standard rule is strict: a predicted entity is correct only if start, end and type all match. A fragment that takes only "Giuseppe" instead of "Giuseppe Verdi" is not worth half a point: it counts as one false positive and one missed entity.

# Minimal NER (name lists) and entity-level evaluation.
# An entity is correct only if start, end AND type all match: this is the rule
# used to compute precision and recall of real models.
NOMI_PROPRI = {"Giuseppe": "PER", "Maria": "PER", "Bianchi": "PER", "Rossi": "PER",
               "Roma": "LOC", "Firenze": "LOC", "Parigi": "LOC", "Italia": "LOC",
               "Fiat": "ORG"}

def predici(parole):
    """Spans (start, end_exclusive, type): consecutive known names of the same type are merged."""
    span, i = [], 0
    while i < len(parole):
        if parole[i] not in NOMI_PROPRI:
            i += 1
            continue
        tipo, j = NOMI_PROPRI[parole[i]], i + 1
        while j < len(parole) and NOMI_PROPRI.get(parole[j]) == tipo:
            j += 1                       # "Maria Rossi" = a single person
        span.append((i, j, tipo))
        i = j
    return span

# (sentence, true entities written by hand)
GOLD = [
    ("Maria Rossi lavora a Firenze".split(), {(0, 2, "PER"), (4, 5, "LOC")}),
    ("Giuseppe Verdi nacque vicino a Parma".split(), {(0, 2, "PER"), (5, 6, "LOC")}),
    ("La Fiat ha chiuso lo stabilimento di Roma".split(), {(1, 2, "ORG"), (7, 8, "LOC")}),
    ("Il Consiglio di Stato ha deciso".split(), {(1, 4, "ORG")}),
    ("Il prof. Bianchi insegna a Parigi".split(), {(2, 3, "PER"), (5, 6, "LOC")}),
]

vp = fp = fn = 0
for parole, gold in GOLD:
    pred = set(predici(parole))
    vp += len(pred & gold); fp += len(pred - gold); fn += len(gold - pred)
    print(" ".join(parole))
    print("   missed  :", sorted(gold - pred))
    print("   spurious:", sorted(pred - gold))
p, r = vp / (vp + fp), vp / (vp + fn)
f1 = 2 * p * r / (p + r)
print(f"\nvp={vp} fp={fp} fn={fn}  precision={p:.2f} recall={r:.2f} F1={f1:.2f}")

# Output:
# Maria Rossi lavora a Firenze
#    missed  : []
#    spurious: []
# Giuseppe Verdi nacque vicino a Parma
#    missed  : [(0, 2, 'PER'), (5, 6, 'LOC')]
#    spurious: [(0, 1, 'PER')]
# La Fiat ha chiuso lo stabilimento di Roma
#    missed  : []
#    spurious: []
# Il Consiglio di Stato ha deciso
#    missed  : [(1, 4, 'ORG')]
#    spurious: []
# Il prof. Bianchi insegna a Parigi
#    missed  : []
#    spurious: []
#
# vp=6 fp=1 fn=3  precision=0.86 recall=0.67 F1=0.75

The program merges consecutive names of the same type (Maria Rossi becomes a single span), and the output gives the three numbers with which recognizers are compared: precision 0.86 (how many predicted entities are right), recall 0.67 (how many true entities were found) and F1 0.75, which summarizes them. The errors are of two different natures. "Parma" is simply outside the list (missed), while "Giuseppe Verdi" produces a partial span that counts twice, and "Consiglio di Stato" (Council of State) is not recognized because none of its words is in the list. These are exactly the cases on which a statistical model gains over lists, and the cases to look at when you evaluate one on your own texts.

Syntactic dependencies: from a tree to a triple

Dependency parsing assigns to each word a head and a relation (subject, object, modifier) according to the Universal Dependencies scheme. The result is a tree and it serves to answer questions such as "who does what to whom?". The program below starts from a tree written by hand in the format a parser returns (index, text, head index, relation) and extracts the subject-verb-object triple, adding the determiners to the head of the phrase.

# From the dependency tree to subject-verb-object triples.
# The tree is written by hand in the format a Universal Dependencies parser returns:
# for each word (index, text, head index, relation). The root has head = itself.
FRASE = [
    (0, "Il", 1, "det"), (1, "gatto", 2, "nsubj"), (2, "mangia", 2, "root"),
    (3, "il", 4, "det"), (4, "pesce", 2, "obj"), (5, "in", 7, "case"),
    (6, "una", 7, "det"), (7, "cucina", 2, "obl"),
]

def sintagma(parole, i):
    """The head plus its direct determiners and modifiers, in sentence order."""
    idx = sorted([i] + [t[0] for t in parole if t[2] == i and t[0] != i and t[3] in ("det", "amod", "case")])
    return " ".join(parole[k][1] for k in idx)

def triple(parole):
    out = []
    for i, testo, testa, rel in parole:
        if rel != "root":
            continue
        soggetti = [t[0] for t in parole if t[2] == i and t[3] == "nsubj"]
        oggetti = [t[0] for t in parole if t[2] == i and t[3] == "obj"]
        out += [(sintagma(parole, s), testo, sintagma(parole, o)) for s in soggetti for o in oggetti]
    return out

print(triple(FRASE))
# The complement "in una cucina" has relation obl, not obj: it does not enter the triple.
print([sintagma(FRASE, t[0]) for t in FRASE if t[3] == "obl"])

# Output:
# [('Il gatto', 'mangia', 'il pesce')]
# ['in una cucina']

The tree is written by hand because no parser runs here, but the format is the same that spaCy exposes with token.i, token.head.i and token.dep_. The triple function looks for the root, collects the children with relation nsubj and obj and ignores the complement "in una cucina" (in a kitchen), which has relation obl and is not a direct object. When you use a real parser, the engineering work is exactly this: deciding which relations matter for your problem and what to do with passive sentences or sentences with an implied subject, which are very frequent in Italian.

Classifying texts: words versus character n-grams

Here the problem is the opposite of parsing: you do not need to understand the sentence, you need to assign it to a category. Italian makes the "one column per word" model difficult, because the same idea appears in many forms: "vincere", "vinse", "vincevano", "vincitori" (to win, won, were winning, winners). With whole words as features, "calciatori" and "calcio" (footballers, football) are different columns; with character n-grams they share "calci". The program below compares the two representations on the same linear classifier (LinearSVC, with a fixed seed), trained on 30 sentences written by hand and tested on 9 sentences in which the keywords are inflected differently from those seen. The class labels are sport, politics and technology.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import make_pipeline
from sklearn.svm import LinearSVC

# Corpus written by hand: 3 classes, 10 training sentences per class.
TRAIN = {
    "sport": [
        "La squadra ha vinto la partita con un gol nel secondo tempo",
        "L'allenatore ha schierato un attaccante veloce contro la difesa avversaria",
        "Il campionato di calcio riparte con il derby della capitale",
        "La tennista ha battuto la rivale in tre set",
        "Il portiere ha parato un rigore decisivo",
        "I tifosi hanno riempito lo stadio per la finale",
        "Il ciclista ha vinto la tappa in salita",
        "La nazionale ha convocato venti giocatori per il torneo",
        "Il pilota ha conquistato la pole position al gran premio",
        "La squadra di pallavolo ha perso la semifinale",
    ],
    "politics": [
        "Il governo ha presentato una nuova legge in Parlamento",
        "Il ministro ha annunciato la riforma del lavoro",
        "L'opposizione ha votato contro la manovra finanziaria",
        "Il presidente del Consiglio ha incontrato i sindacati",
        "Le elezioni regionali si terranno in autunno",
        "La Camera ha approvato il decreto con la fiducia",
        "Il senatore ha presentato un emendamento alla riforma",
        "I partiti della maggioranza discutono la candidatura del sindaco",
        "Il ministro dell'interno ha firmato una circolare",
        "La coalizione ha perso il voto sulla legge elettorale",
    ],
    "technology": [
        "L'azienda ha lanciato un nuovo processore per smartphone",
        "Gli sviluppatori hanno rilasciato un aggiornamento del sistema operativo",
        "Il nuovo software usa l'intelligenza artificiale per tradurre i testi",
        "La startup ha progettato un chip a basso consumo",
        "Il laptop ha una batteria che dura venti ore",
        "I ricercatori hanno pubblicato un algoritmo per riconoscere le immagini",
        "L'applicazione per telefono permette di pagare con il cellulare",
        "Il produttore di schede video presenta una nuova generazione di processori",
        "La rete cinque gi è arrivata in altre cento città",
        "Il programmatore ha corretto un errore nel codice del server",
    ],
}
# Test: words inflected DIFFERENTLY from those seen (vincere/vinse, giocare, votare, ...).
TEST = [
    ("Gli atleti vincevano le gare e le squadre pareggiavano", "sport"),
    ("I calciatori segnavano e i portieri parando i rigori", "sport"),
    ("I tennisti giocheranno i tornei sui campi in terra", "sport"),
    ("I ministri votarono le leggi proposte dai parlamentari", "politics"),
    ("Gli elettori sceglieranno i sindaci con le elezioni", "politics"),
    ("I governi presentavano decreti e i partiti discutevano", "politics"),
    ("Gli algoritmi riconoscono le immagini dei telefoni", "technology"),
    ("Le applicazioni usano processori e batterie diverse", "technology"),
    ("I programmatori correggevano gli errori dei server", "technology"),
]
X = [f for frasi in TRAIN.values() for f in frasi]
y = [c for c, frasi in TRAIN.items() for _ in frasi]
Xt, yt = [f for f, _ in TEST], [c for _, c in TEST]

CONFIGURAZIONI = {
    "words":           TfidfVectorizer(lowercase=True),
    "character n-grams (3-5)": TfidfVectorizer(lowercase=True, analyzer="char_wb", ngram_range=(3, 5)),
}
for nome, vect in CONFIGURAZIONI.items():
    modello = make_pipeline(vect, LinearSVC(random_state=0))
    modello.fit(X, y)
    pred = modello.predict(Xt)
    giuste = sum(p == t for p, t in zip(pred, yt))
    print(f"{nome:30} {giuste}/{len(yt)} correct; errors:", [(Xt[i][:30], str(pred[i])) for i in range(len(yt)) if pred[i] != yt[i]])

# Output:
# words                          7/9 correct; errors: [('Gli atleti vincevano le gare e', 'technology'), ('I calciatori segnavano e i por', 'technology')]
# character n-grams (3-5)        9/9 correct; errors: []

The word representation gets two sentences out of nine wrong, the character n-gram one none: the sentences about the "squadre" (teams) and the "calciatori" were attributed to technology because the exact forms were not in the training set. Read the result with caution, because nine sentences are an experiment, not a benchmark: the difference of two sentences can change with another choice of examples. What the test demonstrates is the mechanism, and it is a good reason to try character n-grams, or to lemmatize first, when the data are few and the language is rich in inflections. The same program, with other labels, works for the sentiment of reviews: but you need a set of labelled sentences in Italian, which no generic model replaces.

What the real models declare

The numbers of spaCy's models for Italian can be read in the metadata files published in the explosion/spacy-models repository, which I opened on 2026-11-12. For version 3.8.0 the three sizes differ in size, not in components:

• it_core_news_sm: 12 MB, entity F1 0.856, LAS 0.859 on dependencies, lemma accuracy 0.972;
• it_core_news_md: 40 MB, entity F1 0.874, LAS 0.874, lemmas 0.973;
• it_core_news_lg: 541 MB, entity F1 0.885, LAS 0.876, UAS 0.911, lemmas 0.975, part of speech 0.974.

These numbers disprove two common ideas. The first is that the "lg" model is much more accurate: the difference on entities between small and large is about 3 points of F1 and on dependencies less than 2 points, for a size 45 times larger (the metadata indicate that the large model includes 500000 word vectors of 300 dimensions, the medium 20000 and the small none). The second concerns entities: the labelling provides four types (PER, LOC, ORG, MISC), not ten. Dates, amounts and products have no type: to extract them you need rules or a model of your own. Then there is a legal constraint: the metadata declare the CC BY-NC-SA 3.0 licence, which excludes commercial use; if your project is commercial, check the licence with someone who knows how before adopting the model.

To use them, the installation is this (not run on this machine, where no models were downloaded):

pip install spacy
python -m spacy download it_core_news_lg

The second command downloads the model package from the project's repository; the model version must be compatible with the installed spaCy version (for model 3.8.0 the metadata indicate spaCy from 3.8.0 up to but excluding 3.9.0), otherwise loading warns or fails. After the download, usage is that of the following block: a schema to adapt, not run here.

import spacy

nlp = spacy.load("it_core_news_lg")
doc = nlp("Maria Rossi lavora a Firenze da dieci anni.")

for ent in doc.ents:                      # entities: text and type (PER, LOC, ORG, MISC)
    print(ent.text, ent.label_)
for t in doc:                             # one token per line: category, lemma, relation, head
    print(t.text, t.pos_, t.lemma_, t.dep_, t.head.text)

The block uses the same four pieces of information that the programs above built by hand: doc.ents for entities, t.pos_ and t.lemma_ for category and lemma, t.dep_ and t.head for the tree. To find out which components are active in a loaded model look at nlp.pipe_names: if you remove one you do not need (for example the parser, when you only want entities) the pipeline runs faster. I give you no timings, because they depend on the machine and the text: measure them on your own documents. The example sentence is Italian ("Maria Rossi has worked in Florence for ten years").

Transformers for Italian: representations, not annotations

The most cited Italian models are dbmdz's BERT and UmBERTo. The card of dbmdz/bert-base-italian-xxl-cased declares an MIT licence and a training corpus of 81 GB, 13.1 billion tokens (Wikipedia, OPUS and the Italian part of OSCAR), and warns that the vocabulary size in config.json does not match the real one (31102). UmBERTo, which the card describes as a RoBERTa-based model with SentencePiece tokenization (32 thousand-entry vocabulary), is trained on the deduplicated Italian part of OSCAR (70 GB), and the card reports F1 92.5 on WikiNER-ITA entity recognition and accuracy 99.0 on the parts of speech of UD Italian ISDT. These are the authors' numbers, on specific datasets, not a promise for your texts.

import torch
from transformers import AutoTokenizer, AutoModel

nome = "dbmdz/bert-base-italian-xxl-cased"
tok = AutoTokenizer.from_pretrained(nome)
modello = AutoModel.from_pretrained(nome).eval()

frasi = ["La qualità del servizio è eccellente.", "Pessimo servizio, non torno più."]
batch = tok(frasi, padding=True, truncation=True, return_tensors="pt")
with torch.no_grad():
    h = modello(**batch).last_hidden_state          # (sentences, tokens, hidden dimension)
maschera = batch["attention_mask"].unsqueeze(-1)   # 1 on real tokens, 0 on padding
embedding = (h * maschera).sum(1) / maschera.sum(1)  # mean over the real tokens only

A schema to adapt, not run here: there is no PyTorch on this machine and no model was downloaded. The two sentences are Italian review data ("The quality of the service is excellent." and "Terrible service, I'm not coming back."). The line that matters is the last one, the mean weighted by the mask: computing the mean over all tokens, padding included, mixes into the vector the length of the batch in which the sentence happens to fall. Taking only the vector of the first token (the "CLS") is the tutorials' habit, but on a BERT not trained for similarity it generally gives mediocre representations; to search for similar sentences you use models trained for the purpose, such as those in RAG in Italian: Multilingual Embeddings and Semantic Retrieval. If instead you want to adapt the transformer to your task, the path is in Fine-tuning an LLM for Italian: LoRA, Datasets, and Evaluation.

Frequently asked questions

Which Italian spaCy model should I pick?

Start from it_core_news_sm if CPU or memory matter, and move to the large one only if a measurement on your own texts justifies 541 MB: the metadata declare an entity F1 of 0.856 against 0.885. Check the licence first (CC BY-NC-SA 3.0), because it excludes commercial use.

How do I handle out-of-vocabulary words?

Transformers split rare words into known word pieces, so they never meet an unknown word, but they represent it worse. A classifier with character n-grams has the same advantage in a simple version, as the comparison above shows. For domain names (product codes, acronyms) an explicit rule is preferable.

How reliable are spaCy's lemmas for Italian?

The metadata of the large model declare a lemma accuracy of 0.975 on the test set of its own corpus. The rest depends on the domain: on legal or medical texts, read a sample by hand before trusting the lemmas for a search index.

Where do I find labelled Italian data?

For dependencies and parts of speech the reference is the Universal Dependencies treebanks for Italian (ISDT, ParTUT and others); the entity data spaCy used for its model is WikiNER. For your own tasks (complaints, tickets, contracts) the best data are those you label yourself, even if only a few hundred.

What you can do now

You can explain why a count by spaces fails in Italian and where a tokenizer's work ends, you know why the lemma depends on the category, you can measure an entity recognizer with precision, recall and F1 at span level, and you know when a character n-gram pipeline beats whole words. You can also read the real numbers of the models (sizes, F1, licence) instead of the round figures of tutorials. The next step is to take fifty sentences from your domain, label them and run the programs above on them. For the level of the generative models that sit on top of these foundations, start from The Italian LLM Landscape in 2025: Models, Ecosystem, and Practical Choices and the overview of the year before, Italian LLMs in 2024: Minerva, LLaMAntino, and the First Native Models.