Start with the simplest pattern that works
Reach for an agentic loop only when the task genuinely needs the model to decide the number and order of steps at runtime. A single LLM call, or a fixed workflow of LLM and code steps, is cheaper, faster, and far easier to test. Do: prototype as a deterministic workflow and add agentic steps only where you can show they are required. Don't: build a multi-step autonomous agent for a task a structured prompt chain solves, because you inherit all the cost and unpredictability for no benefit.
Ground every factual claim in retrieved context
In RAG and tool-using agents, answers should rest on supplied evidence, not the model's memory, and the model must be allowed to say it does not know. This makes outputs auditable and is the primary defence against hallucination. Do: instruct the model to answer only from the provided context, require citations, and permit an explicit "the context does not cover this" response. Don't: ask the model to "be helpful and answer anyway" when evidence is missing, which manufactures confident, untraceable claims.
Invest in retrieval quality before generation quality
The generator can only use evidence the retriever surfaces, so retrieval sets the ceiling on answer quality. Measure recall and precision of your retriever independently before blaming the prompt. Do: tune chunking, try hybrid (lexical + vector) search, and add reranking when the best passages are not reliably at the top. Don't: keep rewriting the system prompt to fix answers that are actually wrong because the right passage was never retrieved.
Keep the tool set small, distinct, and well documented
Models choose tools from their descriptions and schemas, so a large, overlapping tool set causes wrong or malformed calls. Treat each tool's description and input schema as the contract the model plans against. Do: write precise descriptions, use enums and required fields, and route or group capabilities when you have many. Don't: expose dozens of flat, similar tools and expect the model to disambiguate them reliably.
Always bound the agent loop
Every agentic loop must terminate predictably, or a model that keeps choosing to act will run until it exhausts your budget. Bound the loop on multiple dimensions. Do: set a maximum iteration count, a token or cost ceiling, an error threshold, and a clear "final answer" signal, and log every step. Don't: ship a loop whose only exit is the model deciding it is done, which invites infinite cycles and runaway cost.
Make tool actions idempotent and gate irreversible ones
Because agents loop and retry, a non-idempotent action can fire multiple times and cause real harm. Design for safe replay and put a human in front of anything destructive. Do: pass idempotency keys, make writes safe under repetition, and require human approval before sending money, messages, or modifying production. Don't: let an autonomous agent execute irreversible side effects with no approval gate, trusting that it "usually" gets it right.
Treat evaluations as a regression gate, not an afterthought
A fixed eval set scored on every change is what turns prompt tinkering into engineering. Without it, you cannot tell whether a change helped or quietly broke something. Do: maintain a golden set of questions with grounding contexts, measure faithfulness, relevancy, and context recall (RAGAS / promptfoo / DeepEval), and fail CI below a calibrated floor. Don't: judge changes by spot-checking a few prompts by hand and shipping on vibes.
Engineer the context, not just the prompt
For agents, quality depends on assembling exactly the right information into the window for each step, the relevant passages, tool results, and memory, and excluding distractors. Do: budget the context window across system prompt, history, retrieval, and output; trim or summarise old turns; cap retrieved chunks. Don't: stuff the entire conversation and every retrieved document into the prompt and assume a large window makes context engineering unnecessary.
Isolate and distrust untrusted content
Any text the agent did not author, user input and especially retrieved web pages or documents, can carry prompt-injection payloads. Treat it as data, never as instructions. Do: clearly delimit and label untrusted content, apply least-privilege tools, and keep system instructions authoritative over anything in retrieved text. Don't: concatenate fetched content directly into the instruction region of the prompt where it can override your rules.
Use low temperature and structured output for reliability
Agents that must call tools, extract data, or produce machine-consumable results need reproducible behaviour. Randomness that helps creative tasks hurts here. Do: use a low temperature and constrain outputs to a JSON schema (constrained decoding or the tool-call interface) so downstream code parses reliably. Don't: parse free-form text with brittle regexes, or leave temperature high and then puzzle over flaky, inconsistent runs.
Control cost deliberately across the loop
Cost compounds over iterations and tool calls, so manage it as a first-class metric measured per task. Do: cache the large static prompt prefix, route easy sub-tasks to smaller models, cap retrieved context, parallelise independent tool calls, and track tokens per run alongside quality on your eval set. Don't: send a giant uncached system prompt on every turn and discover the spend only when the invoice arrives.
Make every step observable
You cannot debug or improve what you cannot see, and agents fail in non-obvious places, a bad retrieval, a malformed tool call, an unexpected loop. Do: log each iteration's reasoning, the tool call and its arguments, the observation, and token cost, ideally with tracing that ties a run together end to end. Don't: treat the agent as a black box and try to fix failures by guessing at the prompt without a trace of what actually happened.
Prefer prompting and RAG over fine-tuning for knowledge
Fine-tuning bakes information into weights that go stale and cannot be cited or edited, whereas RAG keeps knowledge fresh, traceable, and updatable. Do: use RAG to inject facts and reserve fine-tuning for stable behaviour like tone, format, or a narrow repeated skill. Don't: fine-tune a model to "teach it your data" when that data changes, which is expensive to maintain and harder to audit than retrieval.