What is prompt engineering?
Prompt engineering is the practice of designing the input to a language model - instructions, examples, context, and formatting - so that the model reliably produces the output you need. It spans choosing the system prompt, structuring the task, providing examples, setting decoding parameters, and validating results. As applications have grown more complex it has broadened into context engineering: curating the entire information state the model sees, not just the wording of a single instruction.
What is the difference between a system prompt and a user prompt?
A system prompt sets durable, conversation-wide behavior - role, constraints, tone, output format, and safety rules - while a user prompt carries the specific request for a single turn. The system prompt is the right place for anything that should hold for the whole session, and with the Anthropic API it is passed in the dedicated system parameter rather than as a message. Keeping stable instructions in the system prompt and variable content in user turns also makes prompt caching effective.
When should I use few-shot instead of zero-shot prompting?
Use zero-shot when the task is common and a clear instruction suffices, since it keeps prompts short and cheap. Switch to few-shot when you need a specific output format, a domain-specific labeling convention, or more consistency than instructions alone deliver - a few correct, diverse examples let the model infer the pattern by analogy. If a couple of examples do not stabilize behavior, the problem is often an ambiguous instruction rather than too few examples.
How do I stop the model from hallucinating?
The most effective defense is grounding: retrieve authoritative context and instruct the model to answer only from that context and to say it does not know when the context is insufficient. Lowering temperature, asking for citations to the provided passages, and validating claims against sources further reduce fabrication. No prompt eliminates hallucination entirely, so production systems pair grounding with programmatic verification of critical facts.
What is the difference between RAG and fine-tuning?
RAG injects relevant documents into the prompt at query time, so knowledge stays current and updatable without retraining, and answers can cite their sources. Fine-tuning changes the model's weights to internalize a behavior, format, or style, which is better for teaching a consistent skill than for supplying facts that change. Many production systems combine them: fine-tune for form and behavior, retrieve for up-to-date facts.
How should I structure a complex prompt?
Separate the prompt into clearly labeled sections - role, instructions, context or documents, examples, and the user request - using explicit delimiters so the model can tell instructions from data. For Claude, Anthropic recommends XML-style tags such as
What temperature should I use?
Use a low temperature (near zero) for extraction, classification, structured output, and factual tasks where you want determinism and consistency. Use a higher temperature for brainstorming, creative writing, or generating diverse alternatives. Tune either temperature or top-p rather than both, and remember that even temperature zero is not perfectly deterministic on most hosted models.
How do I get reliable JSON output from a model?
Prefer the platform's structured-output or tool/function-calling feature, which constrains generation to a schema you define, rather than asking for JSON in free text and parsing it. If you must prompt for JSON directly, provide the exact schema, give an example, prefill the opening brace, and add a stop sequence - then always validate the parsed result and handle failures. Treating model output as untrusted and validating it is non-negotiable in production.
What is prompt injection and how do I defend against it?
Prompt injection is an attack where adversarial instructions hidden in user input or in retrieved/external content (indirect injection) trick the model into ignoring its rules or taking unintended actions; it is the top risk in the OWASP Top 10 for LLM Applications. Defenses are layered: delimit and label untrusted content, instruct the model to treat it as data not instructions, constrain what tools and outputs are possible, and validate outputs in code. No single prompt-level control is sufficient, so least-privilege design around the model is essential.
How do I evaluate whether a prompt is good?
Build an eval: a dataset of representative inputs with expected outputs or graded criteria, and score prompt variants against it rather than judging by feel. For open-ended tasks, use an LLM-as-a-judge with a rubric, calibrated against human labels, and for RAG track faithfulness and answer relevancy. A fixed golden eval set also lets you detect regressions when you change the prompt or upgrade the model.
Does chain-of-thought always improve results?
Chain-of-thought reliably helps on multi-step reasoning, math, and planning tasks, but it adds latency and token cost and can even hurt on simple tasks where it introduces unnecessary deliberation. On modern reasoning models, the extended thinking is often built in and controlled by a reasoning-effort or budget setting rather than by a "think step by step" instruction. Use it where the task genuinely requires intermediate steps, and measure rather than assume.
What is prompt caching and when does it help?
Prompt caching stores the processed representation of a stable prompt prefix so repeated requests that share that prefix skip recomputation, cutting both latency and cost. It helps most when a long system prompt, schema, tool set, or document is reused across many calls - for example a customer-support agent with a fixed policy block. With the Anthropic API you place cache_control breakpoints on the stable content and keep variable content after them.
Should I use an agent or a fixed workflow?
Use a fixed workflow when the steps are known in advance, because hard-coded orchestration is more predictable, testable, and cheaper. Reach for an agent - where the model decides the next action in a loop - only when the path genuinely varies with the input and cannot be enumerated. Anthropic's guidance is to start with the simplest thing that works and add agentic autonomy only when its flexibility earns its cost in reliability and debuggability.
How do I reduce the cost and latency of my prompts?
Cut tokens first: trim the system prompt, retrieve fewer and more relevant passages, and prune conversation history, since every token costs money and time. Use prompt caching for stable prefixes, route easy requests to a smaller model, cap max_tokens, and stream output to lower perceived latency. Measure P95 and P99 latency, not just the average, because the slow tail is what users feel.
How many examples should I include in a few-shot prompt?
Start with three to five diverse, correct examples that cover the variations and edge cases you care about, then add more only if an eval shows it helps. More examples can improve accuracy up to a point but raise cost and can crowd out the actual task in the context window. The right number is empirical, so let your evaluation set decide rather than guessing.
How do I handle conversation history in a long chat?
Treat history as part of the token budget and curate it: keep the system prompt and recent turns, and summarize or drop older content once it stops contributing to the current task. Letting raw history grow unbounded degrades quality (context rot) and inflates cost and latency. Common patterns include rolling summaries, retrieval over past turns, and pinning only the facts that must persist.
Can prompt engineering replace fine-tuning or RAG entirely?
Often yes for behavior, format, and reasoning style, which prompting steers well and cheaply; but no for two things prompts cannot supply - knowledge the model lacks, which needs RAG, and a deeply consistent specialized skill at scale, which fine-tuning delivers more reliably. The practical default is to exhaust prompt engineering first because it is the fastest, cheapest lever, then add retrieval for facts and fine-tuning only when prompting plateaus. Most production systems end up combining all three.