Ground answers in retrieved context and allow abstention

Condition the model on retrieved passages and instruct it to answer only from that context and to say it does not know when the context is insufficient. This is the single most effective control against hallucination, because it removes the incentive to invent plausible text. Always design the "no good context" path explicitly, or the model will fill the gap with confident fiction.

Do: instruct "Answer only from the context below; if it does not contain the answer, say the context does not cover this."
Don't: send retrieved passages without telling the model it may abstain, then act surprised when it makes things up.

Treat the eval set as immutable and gate releases on it

Build a golden eval set of representative questions with grounded reference answers, and run every change against it the way unit tests gate code. Never edit a reference answer to make the system pass - that defeats the measurement. A faithfulness floor in CI catches regressions before users do.

Do: freeze eval.jsonl, fail the build below a faithfulness threshold calibrated on a baseline run.
Don't: tweak the golden answers until the current model scores well.

Evaluate retrieval and generation separately

Measure whether the retriever surfaced the right passages (context recall) independently from whether the generator used them faithfully. A weak retriever caps answer quality no matter how good the model is, and conflating the two hides the real fault.

Do: log retrieved passages per eval question and inspect them for failing cases.
Don't: blame the LLM for a wrong answer when the needed evidence was never retrieved.

Use hybrid search plus re-ranking, not vectors alone

Combine lexical (BM25) and dense vector retrieval and fuse them, then re-rank the shortlist with a cross-encoder. Vectors miss exact terms and rare tokens; keywords miss paraphrase; the combination plus re-ranking raises precision more than either alone. This is a robust production default.

Do: retrieve a broad top-k with hybrid search, then promote the truly relevant few with a re-ranker.
Don't: ship pure cosine vector search and assume retrieval is solved.

Right-size chunks and tune them empirically

Split documents into self-contained passages of roughly 300 to 600 characters with light overlap, splitting on structure (headings, paragraphs) where possible. Then tune by watching recall and faithfulness on your own data, because chunk size is corpus-dependent.

Do: measure retrieval quality at two or three chunk sizes and pick by metric.
Don't: cargo-cult a chunk size from a blog post without measuring it on your corpus.

Route requests to the cheapest model that meets the bar

Send easy, high-volume requests to a smaller, faster, cheaper model and reserve the strongest model for genuinely hard ones. Most production cost is won here, not in per-token negotiation. Decide routing by task difficulty, measured against your eval set.

Do: classify or score request difficulty and dispatch to a tiered set of models.
Don't: run every request through your most expensive model "to be safe".

Cache aggressively: prompt prefixes and full responses

Use prompt caching for stable prefixes (long system prompts, documents, few-shot blocks) to cut time-to-first-token and cost, and cache full responses for repeated identical queries. The cacheable portion must be an exact, unchanging prefix, so put dynamic content after it.

Do: place the immutable system prompt and reference documents first so they can be cached.
Don't: interleave per-request data into the prefix, defeating the cache.

Treat all model input as potentially adversarial

Assume user text, retrieved documents, and tool results may contain prompt-injection instructions. Separate trusted instructions from untrusted data, give tools least privilege, and never execute model output (SQL, shell, payments) without an independent check. Defence is layered, not a single filter.

Do: validate and constrain every tool call, and run a separate guard model on inputs and outputs.
Don't: concatenate a retrieved document directly after "follow these instructions" and trust the result.

Enforce structured output with a schema and validate it

When output feeds downstream code, constrain it with a tool/function schema or JSON mode and validate the result against that schema in code. Models occasionally emit malformed JSON, so parse defensively and retry or repair on failure.

Do: define a minimal JSON schema and reject or repair non-conforming output at the boundary.
Don't: JSON.parse the model's text and assume it is always valid and complete.

Instrument every request with full tracing

Capture the prompts sent, retrieved context, token usage, tool calls, latency, and final output per request. The same input can yield different outputs, so you cannot debug or evaluate what you did not record. Tracing is the foundation of LLM observability.

Do: store a complete trace per request, including the exact context passed to the model.
Don't: log only aggregate latency and cost, leaving failures impossible to reproduce.

Make agents bounded, scoped, and idempotent

Constrain an agent to the minimum tools it needs, cap its iteration count so it cannot loop forever, and make side-effecting actions idempotent because retries and non-determinism can repeat a call. Reliability comes from tight scoping and validation, not a cleverer prompt.

Do: cap loops, give each tool least privilege, and deduplicate create actions with an idempotency key.
Don't: hand an agent broad tools and an unbounded loop and hope it behaves.

Pin model and prompt versions and re-evaluate on change

Record which model version, prompt, and corpus version produced each result, and re-run the eval set whenever any of them changes - including silent provider model updates. Drift degrades quality quietly, so a scheduled eval run is your early-warning system.

Do: version prompts and corpus, pin model identifiers, and re-evaluate on every change.
Don't: assume "same prompt, same provider" means stable behaviour across time.

Redact PII before logging, indexing, or third-party calls

Detect and remove personally identifiable information before it enters logs, vector indexes, or external API calls, to meet privacy obligations and limit exposure. PII that leaks into an embedding index or a provider log is hard to recall.

Do: run a PII detector and redaction pass at every boundary that persists or forwards data.
Don't: store raw user transcripts containing names, emails, or IDs in your trace store unfiltered.

Set latency budgets and use streaming

Define p95/p99 latency targets, and stream tokens so users see output immediately even when total generation is long. Latency is dominated by output length because tokens are produced sequentially, so shorter outputs and smaller models also help.

Do: stream responses, cap output length, and measure time-to-first-token against an SLO.
Don't: block the UI until a long, non-streamed completion finishes.