Complete Setup Ollama + RAG + RediSearch production: open-source stack 2026

Retrieval decides the quality of the answer more than the model does

An assistant that answers questions about your company's documents works in two steps: first it finds the relevant passages, then it asks the model to answer using them. The second step is the one people look at; the first is the one that decides. If the right passage does not reach the model, no model size saves it, and if it does, even a small model is often enough. This scheme is called RAG (retrieval-augmented generation), and this article builds the part that matters, retrieval, with a runnable program and a schema to move it to Redis.

An everyday example: an employee asks "how do I get the refund?". The procedure they need is titled "expense claim" and does not contain the word "refund". A search for exact words does not find it; a search by meaning does. Conversely, if they ask for "SOP-014", the code of a procedure, the search by meaning may get it wrong while the search for exact words is infallible. That is why both are used and the results are fused.

The pieces of a hybrid retrieval

A retrieval system has five components, each with a decision that belongs to it. Listing them helps to see where quality is lost, because errors concentrate in the first two.

• Splitting (chunking): documents are cut into passages of limited size, often with a small overlap between neighboring passages.
• Embedding: each passage becomes a vector of numbers; passages with similar meaning have nearby vectors.
• Index: a structure that quickly finds the vectors closest to the one of the question.
• Word search (BM25): a second way of ranking the passages, based on the terms shared with the question.
• Fusion: a method that combines the two rankings into one.

Splitting is the most underestimated decision. Passages that are too small lose context, and the question finds a sentence without the rest of the reasoning; passages that are too large dilute the meaning into a single vector, and the model receives a lot of useless text. There is no size that is right for everyone: you choose by trying on your own set of questions, as the article on evaluation on your own data shows.

A complete pipeline that runs without models

The program below implements all five pieces in standard Python, without libraries and without downloading anything, on an invented corpus of six company procedures. One essential warning: the embedding here is not a model. It is a deterministic surrogate that groups by hand some synonym words (holidays and leave, refund and expense claim) and projects them into a vector with a stable hash. It serves to run the whole pipeline and to see how fusion and word search behave; the results, by construction, say nothing about the quality of a real model.

There are four functions to look at. chunk cuts a text into windows of words with overlap; the values 12 and 3 are didactic, in production you reason in hundreds of tokens. embed replaces each word with its group of synonyms before computing the vector, and this line is what gives the surrogate a toy "semantics". The Bm25 class computes the classic per-term score, with greater weight for rare words and a cap on frequency. rrf adds up, for each passage, the inverse of its position in each ranking, with the constant 60 proposed by the authors of the method.

"""Hybrid retrieval on a small corpus: BM25 + vectors + RRF fusion.

WARNING: the "embedding" below is NOT a model. It is a deterministic
surrogate: it groups some synonym words by hand and projects them with
a stable hash (sha256). It serves to run the whole pipeline without downloading
any model; the scores say nothing about the quality of a real model.
"""
import hashlib
import math
import re
from collections import Counter

DOCUMENTS = {
    "SOP-001": "Holidays are requested on the portal at least ten days in advance. The manager approves or rejects within three days.",
    "SOP-007": "The expense claim must be submitted within thirty days of the trip, with the original receipts attached.",
    "SOP-011": "An IT incident is reported immediately to the helpdesk, stating the time, the computer and what happened.",
    "SOP-014": "Purchases over five hundred euros require the approval of the administrative director before the order.",
    "SOP-019": "The archive keys are kept by the front office and handed over only with a signature on the register.",
    "SOP-023": "Safety training is repeated every year; the refund of external courses follows the expenses procedure.",
}
CORPUS = {d: f"{d}. {text}" for d, text in DOCUMENTS.items()}  # the code is part of the indexed text
SYNONYMS = [  # ASSUMPTION: hand-written groups, the surrogate for 'semantics'
    {"holidays", "leave", "absence"},
    {"refund", "expense", "expenses", "claim", "reimbursement"},
    {"incident", "failure", "problem", "breach"},
    {"purchases", "purchase", "order", "supplier"},
]
DIM = 128
STOP = {"the", "a", "an", "of", "to", "in", "on", "at", "for", "with", "and", "or", "that", "is", "are", "be", "by", "from", "how", "do", "i", "get", "who", "what", "must", "every", "only", "over", "within", "before"}


def tokens(text):
    return [w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOP]


def chunk(text, words=12, overlap=3):
    w = text.split()
    step = words - overlap
    return [" ".join(w[i:i + words]) for i in range(0, max(1, len(w) - overlap), step)]


def slot(word):
    return int(hashlib.sha256(word.encode()).hexdigest(), 16) % DIM


def embed(text):
    v = [0.0] * DIM
    for t in tokens(text):
        group = next((g for g in SYNONYMS if t in g), {t})
        for word in group:
            v[slot(word)] += 1.0 / len(group)
    n = math.sqrt(sum(x * x for x in v)) or 1.0
    return [x / n for x in v]


def cosine(a, b):
    return sum(x * y for x, y in zip(a, b))


class Bm25:
    def __init__(self, docs, k1=1.5, b=0.75):
        self.k1, self.b = k1, b
        self.tf = {d: Counter(tokens(t)) for d, t in docs.items()}
        self.len = {d: sum(c.values()) for d, c in self.tf.items()}
        self.avg = sum(self.len.values()) / len(docs)
        self.df = Counter(t for c in self.tf.values() for t in c)
        self.n = len(docs)

    def score(self, query, d):
        s = 0.0
        for t in tokens(query):
            if t not in self.tf[d]:
                continue
            idf = math.log(1 + (self.n - self.df[t] + 0.5) / (self.df[t] + 0.5))
            f = self.tf[d][t]
            s += idf * f * (self.k1 + 1) / (f + self.k1 * (1 - self.b + self.b * self.len[d] / self.avg))
        return s


def ranking(scores):
    ordered = sorted(scores.items(), key=lambda kv: (-kv[1], kv[0]))
    return [d for d, s in ordered if s > 0]


def rrf(*lists, k=60):
    """Reciprocal Rank Fusion: sums 1/(k + position) over every ranking."""
    tot = Counter()
    for lst in lists:
        for pos, d in enumerate(lst, start=1):
            tot[d] += 1.0 / (k + pos)
    return [d for d, _ in sorted(tot.items(), key=lambda kv: (-kv[1], kv[0]))]


bm = Bm25(CORPUS)
vect = {d: embed(t) for d, t in CORPUS.items()}
for q in ["SOP-014", "how do I get the refund", "who keeps the keys"]:
    a = ranking({d: bm.score(q, d) for d in CORPUS})
    qv = embed(q)
    b = ranking({d: cosine(qv, vect[d]) for d in CORPUS})
    print(f"query: {q!r}")
    print("  BM25 :", a[:3])
    print("  vect.:", b[:3])
    print("  RRF  :", rrf(a, b)[:3])

print("\nchunks of SOP-001 with a window of 12 words and overlap 3:")
for c in chunk(DOCUMENTS["SOP-001"]):
    print("  -", c)

# Output:
# query: 'SOP-014'
#   BM25 : ['SOP-014', 'SOP-019', 'SOP-007']
#   vect.: ['SOP-014', 'SOP-007', 'SOP-011']
#   RRF  : ['SOP-014', 'SOP-007', 'SOP-019']
# query: 'how do I get the refund'
#   BM25 : ['SOP-023']
#   vect.: ['SOP-007', 'SOP-023']
#   RRF  : ['SOP-023', 'SOP-007']
# query: 'who keeps the keys'
#   BM25 : ['SOP-019']
#   vect.: ['SOP-019', 'SOP-001']
#   RRF  : ['SOP-019', 'SOP-001']
#
# chunks of SOP-001 with a window of 12 words and overlap 3:
#   - Holidays are requested on the portal at least ten days in advance.
#   - days in advance. The manager approves or rejects within three days.

The second question shows the reason for fusion. The word search finds only SOP-023, which contains the word "refund" but is about external courses; the vector surrogate puts SOP-007, the expense claim, which is the right one, in first place, and keeps SOP-023 second. Fusion brings both into the top two positions, and that is the result you want: the model receives both passages and can answer. The first question, the exact code, is solved by both searches. An effect to know about, which the program hides by discarding null scores: a KNN search always returns the k nearest neighbors, even when none is relevant, while the word search returns an empty list if no word matches. In production you need a distance threshold, to be tuned on your own set of questions.

The fusion formula is simple and its merit is that it does not require comparing scores of different natures: a BM25 score and a cosine distance are not commensurable, while positions are. The method has a weakness: it uses only the order, so two candidates almost identical in score and one very distant occupy consecutive positions. If more finesse is needed, a second pass is added that reorders the top candidates with a dedicated model; it is an optimization to evaluate after measuring the simple retrieval.

Moving retrieval to Redis

Redis offers vector search through its search engine: you create an index with FT.CREATE on the fields of your hashes, one of which is of type VECTOR, and you query it with FT.SEARCH using a KNN clause. The index type is a choice with a criterion stated in the Redis documentation: FLAT for small datasets (under one million vectors) or when perfect precision matters more than latency, HNSW for larger sets or when scalability and speed matter. For the document archive of a small or medium company, which rarely exceeds a few tens of thousands of passages, FLAT is the starting choice.

# Schema to adapt, NOT run (no Redis was started or written to).
# Syntax from the Redis documentation on vector search.

# 1) A FLAT index on hashes with the prefix "chunk:".
#    DIM must be identical to the length of the vectors of YOUR embedding model.
FT.CREATE idx:chunk ON HASH PREFIX 1 chunk: SCHEMA
    text TEXT
    source TAG
    vector VECTOR FLAT 6 TYPE FLOAT32 DIM 384 DISTANCE_METRIC COSINE

# 2) Search for the 5 nearest neighbors, filtered by source.
#    $v is the query vector as raw bytes (float32 little-endian).
FT.SEARCH idx:chunk "(@source:{rulebook})=>[KNN 5 @vector $v AS distance]"
    PARAMS 2 v <blob> SORTBY distance DIALECT 2

The schema has three delicate points. DIM 384 must match the length of the vectors of the embedding model: 384 is the hidden layer size in the all-MiniLM-L6-v2 configuration, and it is an example value; your model may have another, and the index does not accept vectors of a different length. The query vector must be computed with the same model used for the passages: changing model means recomputing and reindexing everything. And the DIALECT 2 clause is required by the documentation for vector queries. The schema has not been run: no Redis was started or written to while preparing this article.

Vectors are obtained from Ollama's /api/embed endpoint, which accepts a text or a list of texts and returns the embeddings list. The length of a returned vector is the value to put in DIM: read it from the response, not from memory.

# Schema to adapt, NOT run (no model called).
# Documented endpoint: POST /api/embed. The response contains "embeddings",
# a list of vectors: the length of each one is the DIM of the index.
curl http://127.0.0.1:11434/api/embed -d '{
  "model": "<embedding-model>",
  "input": ["text of the first chunk", "text of the second chunk"]
}'

The call shows a design choice: the input field accepts a list, so texts are sent in groups instead of one at a time, and you pay for a single request per group. The keep_alive parameter of the same API decides how long the model stays in memory after the request; the default is five minutes. If the embedding model and the answering model do not fit in RAM together, the system swaps them at every request and latency explodes: it is a case to measure on your own server, as the article on memory sizing explains.

What goes wrong in a real retrieval

• The embedding model changes and the index does not: the distances become meaningless, but the queries keep answering.
• Documents are updated and the index stays old: the assistant answers with an outdated version of the procedure.
• The right passage is cut in half: the answer lies across two passages, and neither contains it whole.
• Personal data in the passages: the index copies them, and deleting them at the source does not delete them from the index.
• The model ignores the passages: without an explicit instruction it answers from its own memory, not from the context.

The last point is a prompt problem, not a retrieval one: the instruction must say to answer only with the passages provided and to cite their code, and the answer must be checked on a sample. The fourth is a compliance problem that has a dedicated article, privacy by design, with the program that shows how deletion must reach the index too. On measuring the quality of retrieval and of the answer, the series has a piece on RAG evaluation.

Frequently asked questions

What you can do now

You can break a retrieval system down into its five pieces and you know where quality is lost; you have a program that shows the fusion of a word search and a search by meaning, and you know what its surrogate does not prove. You can translate the schema into a Redis index with the right dimension and the two traps of the embedding model. The model that generates the answer is chosen with the criteria of which model to choose. For the next step, the evaluation of retrieval measures whether it works on your data, and the piece on scalability says what happens with more users.

The same series also includes the cost calculation for a server of your own and the recap with the decision sheet.