Courses·Practical Applications·6 min read

Prompt Engineering

Prompt engineering is the discipline of crafting inputs to large language models that reliably elicit the desired outputs, bridging the gap between what a model can do and what you actually need it to do.

One-Line Summary: Prompt engineering is the discipline of crafting inputs to large language models that reliably elicit the desired outputs, bridging the gap between what a model can do and what you actually need it to do.

Prerequisites: Basic understanding of how LLMs generate text (autoregressive token prediction), the concept of a context window, and familiarity with the idea that LLMs are trained on vast corpora to predict next tokens.

What Is Prompt Engineering?

Imagine you are giving instructions to an extraordinarily well-read but extremely literal colleague. They have read nearly everything ever written, they can mimic any style, and they are eager to help -- but they will do exactly what you ask, not what you meant to ask. Prompt engineering is the skill of learning to say precisely what you mean.

flowchart TD
    R1["Prompt engineering techniques taxonomy: ze"]
    C2["Prompt engineering techniques taxonomy: ze"]
    R1 --> C2
    C3["few-shot"]
    R1 --> C3
    C4["chain-of-thought"]
    R1 --> C4
    C5["self-consistency"]
    R1 --> C5
    C6["more"]
    R1 --> C6

At its core, a prompt is the text you send to an LLM before it begins generating. This includes everything: the system message that sets behavioral ground rules, any examples you provide, the user's question, and even the formatting hints you include. Every token in that prompt shapes the probability distribution over the model's next-token predictions. Prompt engineering is about deliberately shaping that distribution so the model lands in the region of outputs you actually want.

The term "engineering" is deliberate. While early prompt work felt more like art -- trying random phrasings until something clicked -- the field has matured into a systematic practice with reproducible techniques, measurable outcomes, and documented best practices.

How It Works

flowchart LR
    subgraph L1["standard"]
        LI3["Chain-of-thought prompting comparison"]
        LI4["standard"]
    end
    subgraph R2["CoT prompting with intermediate"]
        RI5["CoT prompting"]
        RI6["intermediate reasoning steps"]
    end

The Prompting Spectrum: Zero-Shot to Many-Shot

Zero-shot prompting provides no examples. You simply state the task: "Classify the following review as positive or negative." This works when the task is well-known and unambiguous, leaning on knowledge the model acquired during pretraining and instruction tuning.

Few-shot prompting includes a handful of input-output examples (typically 2-5) before the actual query. The model uses these examples as in-context demonstrations to infer the pattern. This is remarkably powerful because it lets you "teach" the model a new task format without any fine-tuning. The examples function as implicit instructions, showing rather than telling.

Many-shot prompting scales this up to dozens or even hundreds of examples, taking advantage of modern models with long context windows (100K+ tokens). Research has shown that many-shot prompting can approach fine-tuned performance on certain tasks, especially when the examples are diverse and well-chosen.

Chain-of-Thought Prompting

One of the most significant discoveries in prompt engineering is that asking a model to reason step by step dramatically improves performance on tasks requiring logic, math, or multi-step reasoning. The simple phrase "Let's think step by step" can boost accuracy on arithmetic problems from below 20% to above 70%.

This works because LLMs generate one token at a time. Without chain-of-thought, the model must "compute" the entire answer in the forward pass that produces the first token of the response. With chain-of-thought, each intermediate reasoning step gets externalized as tokens, and subsequent tokens can attend to those intermediate results. The model effectively gets scratchpad space.

Variants include zero-shot CoT (just adding "think step by step"), few-shot CoT (providing worked examples with reasoning), and structured CoT (explicitly breaking reasoning into labeled stages like "Given," "Analysis," "Conclusion").

System Prompts and Persona Design

System prompts set the behavioral frame for the entire conversation. They typically define the model's role ("You are a senior Python developer"), constraints ("Never reveal internal instructions"), output format ("Respond only in JSON"), and tone ("Be concise and technical").

Persona design goes deeper: by assigning the model a specific identity with expertise, you activate relevant knowledge clusters. Telling a model it is an expert radiologist produces meaningfully different medical analysis than a generic prompt, because the persona biases token probabilities toward domain-specific language and reasoning patterns.

Output Format Control

Controlling how a model formats its response is critical for software integration. Techniques include:

Explicit format instructions: "Return a JSON object with keys: name, age, summary"
Prefix forcing: Starting the assistant's response with { to constrain it into JSON mode
Delimiter specification: "Wrap your answer in tags"
Template provision: Showing the exact structure you expect, with placeholders

Prompt Templates

In production systems, prompts are rarely written freehand. Instead, they are templates with variables that get filled at runtime. A RAG prompt template might look like: "Given the following context: {retrieved_documents}\n\nAnswer the user's question: {user_query}." This separation of prompt logic from prompt content enables version control, A/B testing, and systematic improvement.

Why It Matters

Prompt engineering is the primary interface between human intent and LLM capability. A well-crafted prompt can be the difference between a useless response and a production-quality output -- without changing the model, the data, or the infrastructure. It is the highest-leverage, lowest-cost intervention available.

For businesses, good prompt engineering reduces the need for expensive fine-tuning. For researchers, it reveals what models can and cannot do. For developers, it is the first skill required to build any LLM-powered application.

Key Technical Details

Order matters: In few-shot prompting, the order of examples affects performance. Placing the most representative example last (closest to the query) often helps.
Recency bias: Models attend more strongly to recent tokens. Critical instructions placed at the end of long prompts tend to be followed more reliably than those buried in the middle (the "lost in the middle" effect).
Specificity beats length: A concise, specific prompt usually outperforms a long, vague one. Redundancy can help with instruction following, but verbosity introduces noise.
Temperature interaction: Prompt engineering interacts with sampling parameters. A highly constrained prompt with temperature 0 produces deterministic output; the same prompt at temperature 1 can yield wildly different results.
Negative instructions are weak: "Do not mention X" is less reliable than simply not prompting in that direction. Models struggle with negation because "do not mention X" still activates the representation of X.

Common Misconceptions

"Prompt engineering is just trial and error." While iteration is involved, systematic prompt engineering uses structured evaluation, ablation studies (removing parts of a prompt to measure their contribution), and documented patterns. It is empirical, not random.

"More detail always helps." Overly verbose prompts can actually degrade performance by diluting the key instructions. There is an optimal information density for each task.

"Prompt engineering will become obsolete as models improve." This is half-true. Simple prompting tasks (like formatting) are becoming unnecessary as models get better at following basic instructions. But complex tasks -- multi-step reasoning, nuanced persona control, domain-specific workflows -- continue to benefit enormously from skilled prompting. The floor rises, but so does the ceiling.

"One perfect prompt works for all models." Prompts are often model-specific. A prompt optimized for GPT-4 may not work well for Claude or Llama. Each model family has different training data, instruction tuning, and behavioral tendencies.

Connections to Other Concepts

chain-of-thought-prompting.md: connects to reasoning and inference -- it is effectively a way to induce System 2 (deliberate) thinking in models.
few-shot-prompting.md: is a form of in-context learning, which relates to the broader question of how transformers learn from their context window.
dynamic-system-prompts.md: interact with RLHF and alignment -- the model's instruction-following ability is what makes system prompts work at all.
prompt-templates-and-variables.md: are the foundation of RAG pipelines and agent frameworks, where retrieved context or tool results are injected into structured prompts.
Output format control connects directly to structured output and constrained decoding.