05 - Retrieval quality deep-dive: NDCG, MRR, HitRate, when the retriever fails

The retriever must be measured on its own, because the generator covers for it

The retriever delivers ten documents and the first is off-topic; the right one is in seventh place. The generator reads everything, ignores the noise, fills the gaps with what it knows and produces a plausible answer. The metrics computed on the final answer stay high, and you conclude that the system is fine. The retriever's defect still exists, but another component hides it. As long as the domain is known to the model and the corpus is small, the cover holds; when the corpus grows and the questions become specific, the cover ends and you have no data to understand why. That is why retrieval is measured separately, with metrics that require only two things: the ordered list of the documents returned and which of them are right.

This article defines four retrieval metrics (hit rate, recall, MRR, nDCG), implements them from the definition, checks them against scikit-learn, applies them to four retrievers on an example corpus and shows how a failure is read. It presupposes the golden dataset of episode 4, because every question must have its right documents, and the vocabulary of episode 1. The corpus is invented (28 sentences about an imaginary software product, 14 questions) and the comparison between retrievers is a teaching model: no embedding model was run, and no vector database was started. The values do not say which retriever is better in general.

The corpus and two retrievers without models

The first listing is the material the others work on: save it as corpus_rag.py. Each question has a graded relevance: 2 for the document that contains the answer, 1 for one that helps but is partial. The grading is needed by nDCG, which otherwise would have nothing to weigh. The two retrievers are BM25, written from the formula with the Lucene default parameters (k1 = 1.2 and b = 0.75, and the same definition of idf), and LSA, that is, TF-IDF reduced to six dimensions with a singular value decomposition. LSA is the deterministic substitute for a "dense" retriever: it brings documents and questions into a space of few dimensions, but it does so starting from the same words, so it is not a semantic embedding. The function rank_docs discards null scores, because a score of zero is not a result.

# corpus_rag.py - invented corpus (28 sentences, 8 topics), 14 questions with graded relevance and two retrievers
import math, re
import numpy as np
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer

DOCS = {
    "pricing1":  "The Pro plan costs 49 euros a month and includes 10 users.",
    "pricing2":  "The Enterprise plan costs more because it adds SSO and dedicated support.",
    "pricing3":  "Paying a year in advance, the subscription costs less than twelve monthly payments.",
    "refund1":   "A refund is possible within 14 days of purchase, by contacting support.",
    "refund2":   "To recover the funds you spent write to support stating your order number.",
    "refund3":   "After 14 days the amount paid is not returned, except for service defects.",
    "export1":   "Data export produces a CSV file from the Settings page.",
    "export2":   "You can download all your notes in CSV or JSON format from the Settings area.",
    "export3":   "The exported file contains one row for each document and the modification dates.",
    "sso1":      "SSO login is available only on the Enterprise plan through SAML.",
    "sso2":      "Company-wide single sign-on is configured with the company's identity provider.",
    "sso3":      "Without SSO each user signs in with a personal email and password.",
    "backup1":   "Automatic backups run every night and are kept for 30 days.",
    "backup2":   "Safety copies are daily and remain available for a month.",
    "backup3":   "To restore a backup open a ticket stating the desired date.",
    "api1":      "The API accepts at most 100 requests per minute for each key.",
    "api2":      "When the request limit is exceeded the server answers with error 429 and a wait time.",
    "api3":      "API keys are created and revoked from the Developers panel.",
    "invoice1":  "Invoices can be downloaded as PDF from the Billing section of the account.",
    "invoice2":  "For a receipt with the VAT number update the company details before payment.",
    "invoice3":  "Every month you receive the invoice by email at the administrative contact's address.",
    "pwd1":      "To reset your password use the link sent to your registration email.",
    "pwd2":      "If you have forgotten your access credentials you can request a new link.",
    "pwd3":      "The password must have at least twelve characters and cannot repeat the last five.",
    "misc1":     "Integrations with Slack and Teams are enabled from the Integrations panel.",
    "misc2":     "Each project accepts up to 500 documents uploaded in PDF format.",
    "misc3":     "The phone app can be downloaded from the stores and syncs your notes in a few seconds.",
    "misc4":     "Support answers from Monday to Friday, from nine to six.",
}
# (id, question, {document: relevance}) - 2 = contains the answer, 1 = useful but partial
QUERIES = [
    ("q01", "how much does the Pro plan cost",                  {"pricing1": 2, "pricing2": 1}),
    ("q02", "can I get my money back",                          {"refund1": 2, "refund2": 2, "refund3": 1}),
    ("q03", "I want to take all my notes with me",              {"export2": 2, "export1": 1}),
    ("q04", "single sign-on for the company with SAML",         {"sso1": 2, "sso2": 2}),
    ("q05", "how often are safety copies made",                 {"backup1": 2, "backup2": 2}),
    ("q06", "how many requests per minute does the API accept", {"api1": 2, "api2": 1}),
    ("q07", "where do I find the receipt for my payment",       {"invoice1": 2, "invoice2": 1, "invoice3": 1}),
    ("q08", "I lost my credentials",                            {"pwd2": 2, "pwd1": 1}),
    ("q09", "how do I connect Slack",                           {"misc1": 2}),
    ("q10", "I want my data from yesterday back",               {"backup3": 2, "backup2": 1}),
    ("q11", "is there a discount if I pay for a year",          {"pricing3": 2}),
    ("q12", "what time does support answer",                    {"misc4": 2}),
    ("q13", "limit of documents per project",                   {"misc2": 2}),
    ("q14", "is the Enterprise plan needed for single sign-on", {"sso1": 2, "sso2": 1, "pricing2": 1}),
]
STOP = set("the a an of to in on for and or but that this is are was be can i my me you your do does did how what "
           "where when which much many it its with at by from as if there not no s".split())

def token(text: str) -> list:
    return [t for t in re.findall(r"[a-z0-9]+", text.lower()) if t not in STOP]

IDS = list(DOCS)
TOK = [token(DOCS[i]) for i in IDS]

# --- BM25 Okapi, from the formula (k1=1.2, b=0.75, like the Lucene defaults) ---
N = len(IDS)
DF = {}
for doc in TOK:
    for t in set(doc):
        DF[t] = DF.get(t, 0) + 1
AVG_LEN = sum(len(d) for d in TOK) / N

def bm25(question: str, k1: float = 1.2, b: float = 0.75) -> np.ndarray:
    scores = np.zeros(N)
    for t in set(token(question)):
        if t not in DF:
            continue
        idf = math.log((N - DF[t] + 0.5) / (DF[t] + 0.5) + 1)
        for j, doc in enumerate(TOK):
            f = doc.count(t)
            if f:
                scores[j] += idf * f * (k1 + 1) / (f + k1 * (1 - b + b * len(doc) / AVG_LEN))
    return scores

# --- LSA: TF-IDF reduced to 6 dimensions with SVD (a deterministic "dense", no models) ---
TF = TfidfVectorizer(tokenizer=token, lowercase=False, token_pattern=None)
X = TF.fit_transform(DOCS.values())
SVD = TruncatedSVD(n_components=6, random_state=0, algorithm="arpack")
Z = SVD.fit_transform(X)
Zn = Z / np.maximum(np.linalg.norm(Z, axis=1, keepdims=True), 1e-12)

def lsa(question: str) -> np.ndarray:
    q = SVD.transform(TF.transform([question]))
    n = np.linalg.norm(q)
    return Zn @ (q[0] / n) if n > 1e-12 else np.zeros(N)

def rank_docs(scores: np.ndarray, threshold: float = 1e-9) -> list:
    """Document ids in decreasing score order; zero scores are not results."""
    order = sorted(range(N), key=lambda j: (-scores[j], IDS[j]))
    return [IDS[j] for j in order if scores[j] > threshold]

if __name__ == "__main__":
    print(f"documents={N}  questions={len(QUERIES)}  vocabulary={len(DF)} terms")
    for qid, question, _ in QUERIES[:2]:
        print(qid, question)
        print("  BM25:", rank_docs(bm25(question))[:3])
        print("  LSA: ", rank_docs(lsa(question))[:3])

# Output:
# documents=28  questions=14  vocabulary=177 terms
# q01 how much does the Pro plan cost
#   BM25: ['pricing1', 'sso1', 'pricing2']
#   LSA:  ['sso1', 'pricing1', 'pricing2']
# q02 can I get my money back
#   BM25: []
#   LSA:  []

The run already shows one fact: for "can I get my money back" both retrievers return an empty list. No word of the question, after the removal of stop words, appears in the corpus vocabulary: the refund document talks of "recover the funds" and "the amount paid". It is the vocabulary failure, and at this point it is not a matter of ranking but of the absence of candidates.

The metrics, from the definition

Four functions, each with a different question. Hit rate (hit_rate) asks: is there at least one right document in the top k? It is the most indulgent metric and the most useful as an admissibility threshold, because it says whether the generator has at least one chance. Recall (recall) asks how many of the right documents you brought: it matters when the answer requires several documents. MRR (rr, whose mean over the questions is the MRR) looks only at the position of the first right document, as the reciprocal of the rank: 1, 1/2, 1/3. nDCG (ndcg) is the only one that uses the grading and that rewards the order of all the right documents: it sums the gains discounted by 1/log₂(position + 1) and divides by the sum obtainable with the ideal ordering, so it is 1 only if the most relevant documents are at the top, in the right order.

# metrics.py - HitRate@k, Recall@k, MRR and NDCG@k written from the definition, checked against scikit-learn
import math
import numpy as np
from sklearn.metrics import ndcg_score

def hit_rate(ranking: list, relevant: dict, k: int) -> float:
    """1 if at least one relevant document is in the top k, otherwise 0."""
    return float(any(d in relevant for d in ranking[:k]))

def recall(ranking: list, relevant: dict, k: int) -> float:
    """Share of the relevant documents that appear in the top k."""
    return sum(d in relevant for d in ranking[:k]) / len(relevant)

def rr(ranking: list, relevant: dict) -> float:
    """Reciprocal of the rank of the first relevant document (0 if it does not appear). The mean over questions is the MRR."""
    for i, d in enumerate(ranking, 1):
        if d in relevant:
            return 1 / i
    return 0.0

def dcg(gains: list) -> float:
    """DCG with linear gain and discount 1/log2(position+1)."""
    return sum(g / math.log2(i + 1) for i, g in enumerate(gains, 1))

def ndcg(ranking: list, relevant: dict, k: int) -> float:
    """DCG@k divided by the DCG@k of the best possible ordering of the relevant documents."""
    actual = dcg([relevant.get(d, 0) for d in ranking[:k]])
    ideal = dcg(sorted(relevant.values(), reverse=True)[:k])
    return actual / ideal if ideal else 0.0

if __name__ == "__main__":
    relevant = {"a": 2, "b": 1, "c": 1}                  # a answers, b and c help
    for name, ranking in (("a in first position ", ["a", "x", "y", "b", "z"]),
                          ("a in third position ", ["x", "y", "a", "b", "z"]),
                          ("no relevant document", ["x", "y", "z", "w", "v"])):
        print(f"{name}: hit@3={hit_rate(ranking, relevant, 3):.0f}  recall@5={recall(ranking, relevant, 5):.2f}  "
              f"RR={rr(ranking, relevant):.2f}  nDCG@5={ndcg(ranking, relevant, 5):.3f}")

    # check against scikit-learn. Careful: ndcg_score derives the ideal DCG only from the documents you pass it,
    # so ALL the documents of the universe must be passed, even the ones not retrieved (score 0)
    ranking = ["x", "y", "a", "b", "z"]
    universe = ["a", "b", "c", "x", "y", "z", "w", "v"]
    truth = np.array([[relevant.get(d, 0) for d in universe]])
    scores = np.array([[5 - ranking.index(d) if d in ranking else 0 for d in universe]])
    mine, theirs = ndcg(ranking, relevant, 5), ndcg_score(truth, scores, k=5)
    print(f"nDCG@5 mine={mine:.6f}  scikit-learn={theirs:.6f}  equal={math.isclose(mine, theirs)}")
    only_retrieved = np.array([[relevant.get(d, 0) for d in ranking]])
    print(f"scikit-learn passing only the 5 retrieved: {ndcg_score(only_retrieved, np.array([[5, 4, 3, 2, 1]]), k=5):.6f}  (wrong: document c is missing)")

# Output:
# a in first position : hit@3=1  recall@5=0.67  RR=1.00  nDCG@5=0.776
# a in third position : hit@3=1  recall@5=0.67  RR=0.33  nDCG@5=0.457
# no relevant document: hit@3=0  recall@5=0.00  RR=0.00  nDCG@5=0.000
# nDCG@5 mine=0.456949  scikit-learn=0.456949  equal=True
# scikit-learn passing only the 5 retrieved: 0.543791  (wrong: document c is missing)

Look at how the four metrics read the same ranking. With document "a" in first position, hit rate and RR are 1, recall at 5 is 0.67 (two right out of three found) and nDCG is 0.776, not 1, because "c" is missing. With "a" in third position the hit rate stays 1, the RR drops to 0.33 and the nDCG to 0.457: the hit rate does not see the difference, the others do. The last part is a check and contains a real trap: scikit-learn's ndcg_score computes the ideal DCG only from the documents you pass it. If you pass the five retrieved documents and forget the relevant ones that were not retrieved, it gets 0.544 instead of 0.457, an inflated nDCG that looks correct. Always pass the whole universe of documents, with score 0 for the ones not retrieved. The definition used here is the one with linear gain; there is a variant with exponential gain (2^rel − 1) that gives more weight to very relevant documents, so declare which one you use.

One last distinction avoids a frequent misunderstanding with Ragas. Its identifier-based context metrics (IDBasedContextPrecision and IDBasedContextRecall) are computed on the sets of identifiers and do not look at the order: on an example tried in the test environment, with retrieved d3, d1, d9 and right d1, d2, they give precision 0.333 and recall 0.5 for any order of the three. The order is seen by the judge-verdict ContextPrecision, described in episode 2, and by the metrics of this article.

Four retrievers on the same fourteen questions

The listing compares BM25, LSA and two ways of fusing them: the weighted sum of the scores normalised with min-max, and Reciprocal Rank Fusion, which sums 1/(k + rank) over each list. RRF uses k = 60, which is the default value of rank_constant in the Elasticsearch documentation. The construction of a hybrid retriever is covered in the article on hybrid retrieval; here the fusion serves only as an object to measure. After the table of means, the listing prints the rank of the first right document for each question and a bootstrap on the nDCG difference between BM25 and LSA.

# evaluate.py - BM25, LSA and two fusions on the same 14 questions: the mean and where each one goes wrong
import numpy as np
from corpus_rag import IDS, QUERIES, bm25, lsa, rank_docs
from metrics import hit_rate, ndcg, recall, rr

def minmax(x: np.ndarray) -> np.ndarray:
    return (x - x.min()) / (x.max() - x.min()) if x.max() > x.min() else np.zeros_like(x)

def weighted(q: str, alpha: float = 0.5) -> list:
    """Weighted sum of the normalised scores: alpha*BM25 + (1-alpha)*LSA."""
    b, l = bm25(q), lsa(q)
    if b.max() == 0 and l.max() <= 0:
        return []
    return rank_docs(alpha * minmax(b) + (1 - alpha) * minmax(l), threshold=-1)[:10]

def rrf(q: str, k: int = 60) -> list:
    """Reciprocal Rank Fusion: sum of 1/(k+rank) over the two lists."""
    pts = {}
    for lst in (rank_docs(bm25(q))[:10], rank_docs(lsa(q))[:10]):
        for position, d in enumerate(lst, 1):
            pts[d] = pts.get(d, 0) + 1 / (k + position)
    return sorted(pts, key=lambda d: (-pts[d], d))

SYSTEMS = {
    "BM25": lambda q: rank_docs(bm25(q)),
    "LSA": lambda q: rank_docs(lsa(q)),
    "weighted": weighted,
    "RRF": rrf,
}
results = {n: [f(text) for _, text, _ in QUERIES] for n, f in SYSTEMS.items()}

print(f"{'system':9s} hit@3  recall@5   MRR  nDCG@5")
for name, ranks in results.items():
    h = np.mean([hit_rate(r, q[2], 3) for r, q in zip(ranks, QUERIES)])
    c = np.mean([recall(r, q[2], 5) for r, q in zip(ranks, QUERIES)])
    m = np.mean([rr(r, q[2]) for r, q in zip(ranks, QUERIES)])
    n = np.mean([ndcg(r, q[2], 5) for r, q in zip(ranks, QUERIES)])
    print(f"{name:9s} {h:5.2f}  {c:8.2f}  {m:5.2f}  {n:6.3f}")

print("\nrank of the first relevant document (-- = not found):")
for i, (qid, text, rel) in enumerate(QUERIES):
    cells = []
    for name, ranks in results.items():
        pos = next((j for j, d in enumerate(ranks[i], 1) if d in rel), None)
        cells.append(f"{name}={pos if pos else '--'}")
    print(f"  {qid} {text[:38]:38s} " + "  ".join(cells))

# BM25 beats LSA by a nDCG@5 gap: is it a difference or noise? Bootstrap over the questions, fixed seed
d = np.array([ndcg(results["BM25"][i], q[2], 5) - ndcg(results["LSA"][i], q[2], 5) for i, q in enumerate(QUERIES)])
rng = np.random.default_rng(0)
means = np.sort([rng.choice(d, size=len(d)).mean() for _ in range(5000)])
print(f"\nmean nDCG@5 difference (BM25 - LSA) = {d.mean():+.3f}  95% bootstrap interval = [{means[125]:+.3f}, {means[4874]:+.3f}]")

# Output:
# system    hit@3  recall@5   MRR  nDCG@5
# BM25       0.86      0.67   0.82   0.691
# LSA        0.86      0.68   0.83   0.679
# weighted   0.86      0.74   0.86   0.734
# RRF        0.86      0.74   0.86   0.737
#
# rank of the first relevant document (-- = not found):
#   q01 how much does the Pro plan cost        BM25=1  LSA=2  weighted=1  RRF=1
#   q02 can I get my money back                BM25=--  LSA=--  weighted=--  RRF=--
#   q03 I want to take all my notes with me    BM25=1  LSA=1  weighted=1  RRF=1
#   q04 single sign-on for the company with SA BM25=1  LSA=1  weighted=1  RRF=1
#   q05 how often are safety copies made       BM25=1  LSA=1  weighted=1  RRF=1
#   q06 how many requests per minute does the  BM25=1  LSA=1  weighted=1  RRF=1
#   q07 where do I find the receipt for my pay BM25=1  LSA=1  weighted=1  RRF=1
#   q08 I lost my credentials                  BM25=1  LSA=1  weighted=1  RRF=1
#   q09 how do I connect Slack                 BM25=1  LSA=1  weighted=1  RRF=1
#   q10 I want my data from yesterday back     BM25=--  LSA=13  weighted=--  RRF=--
#   q11 is there a discount if I pay for a yea BM25=1  LSA=1  weighted=1  RRF=1
#   q12 what time does support answer          BM25=2  LSA=1  weighted=1  RRF=1
#   q13 limit of documents per project         BM25=1  LSA=1  weighted=1  RRF=1
#   q14 is the Enterprise plan needed for sing BM25=1  LSA=1  weighted=1  RRF=1
#
# mean nDCG@5 difference (BM25 - LSA) = +0.012  95% bootstrap interval = [-0.081, +0.107]

The means are close: all four have a hit rate of 0.86 (twelve questions out of fourteen), the MRR goes from 0.82 for BM25 to 0.86 for the weighted sum and for RRF, nDCG at 5 from 0.679 to 0.737. Ordering the systems on these means would be a mistake, and the last line says why: BM25 exceeds LSA by 0.012 of nDCG, but the 95% bootstrap interval goes from −0.081 to +0.107 and contains zero. With fourteen questions this difference cannot be told apart from noise. The bootstrap reshuffles the questions, not the scores, which is the right choice because the questions are the independent unit. More useful than the mean is the table of ranks: twelve questions out of fourteen are solved by at least one system, and the two that fail (q02 and q10) fail for almost all of them; LSA recovers q10 only at rank 13, which for hit@3 and nDCG@5 is as good as a miss.

The fusions score a little higher than the two components, and the table says where the gain comes from: BM25 puts q12 second, LSA puts q01 second, and the fusions bring both to first place because the two retrievers err at different points on those two questions. It is a gain of two questions moving up one place, on fourteen: I did not test it with an interval, so do not read it as an established difference. The fusion cannot do anything for the questions on which both retrievers rest on the same missing vocabulary.

Why a question fails: the diagnosis

A low mean does not say what to repair. For each question in which the first right document is not in the top three, the listing prints the rank, the first result returned and the words that the question shares with the right documents. The lexical comparison is the first check because it is the one that costs least and that immediately separates the two most common failures.

# diagnosis.py - why a question fails: which words does it share with the documents it should have found?
from corpus_rag import DOCS, QUERIES, bm25, rank_docs, token

for qid, text, relevant in QUERIES:
    ranking = rank_docs(bm25(text))
    pos = next((j for j, d in enumerate(ranking, 1) if d in relevant), None)
    if pos is not None and pos <= 3:
        continue                                               # found well: nothing to diagnose
    in_common = {d: sorted(set(token(text)) & set(token(DOCS[d]))) for d in relevant}
    print(f"{qid} '{text}' -> rank {pos if pos else 'not found'}, first result: {ranking[0] if ranking else 'none'}")
    for d, words in in_common.items():
        print(f"    {d} (relevance {relevant[d]}): words in common = {words if words else 'none'}")

# Output:
# q02 'can I get my money back' -> rank not found, first result: none
#     refund1 (relevance 2): words in common = none
#     refund2 (relevance 2): words in common = none
#     refund3 (relevance 1): words in common = none
# q10 'I want my data from yesterday back' -> rank not found, first result: export1
#     backup3 (relevance 2): words in common = none
#     backup2 (relevance 1): words in common = none

The two cases are different failures. For "can I get my money back" the retriever returns nothing, and the words in common with the three right documents are zero: it is a vocabulary gap. For "I want my data from yesterday back" it returns a wrong document (export1) because it shares the word "data" with the export sentence: it is a surface homonymy, and the retriever is worse than one that does not answer, because it produces a convincing and wrong context. The countermeasures are different. The first case is tackled with a retriever that does not depend on the same words, such as an embedding model, or with a rewriting of the question; the second with a reordering of the candidates that looks at the whole question, as in a re-ranking with a cross-encoder. These are reasonable hypotheses, and in this article I did not measure them: try them with the same metrics on your dataset before adopting them.

A third failure is the late rank: the right document is there but beyond the threshold. It is recognised in the table of ranks when a question has a value between 4 and 10 (or, like LSA on q10, beyond 10), and it is read with MRR and nDCG, not with the hit rate at 10. The fourth is the duplication of versions of the same document, seen in episode 1. One note of common sense on the reference benchmark: in the BEIR work, which compares retrieval systems on eighteen datasets, the reported result is that BM25 is a robust baseline; do not take it as an invitation not to try anything else, but as the reason why in your comparison a BM25 must always be there.

What you can do now

You can compute hit rate, recall, MRR and nDCG from the definition, you know which to use for which decision and you have seen the ndcg_score trap that inflates the result when the universe of documents is missing. You can compare several retrievers on the same questions, recognise with the bootstrap when a difference is noise and read the rank per question to understand whether the failure is one of vocabulary, of homonymy or of order. The next step is to connect these measures to a regression gate like the one in episode 3, and to bring them into service with the monitoring described in Evaluating RAG: metrics, benchmarking and production monitoring.

The same series also includes the faithfulness metric and optimization driven by evaluation.