Phase 5: Modern LLM Engineering · 70 min · DSPy · PyTorch · OpenAI API

DSPy: Programmatic Prompt Optimization for LLM Applications

Stop hand-writing prompts. Start compiling them.

Hiring signal: DSPy is emerging as the standard for programmatic LLM optimization. DeepLearning.AI already has a course on it. Engineers who understand declarative LLM programming and automatic prompt optimization are at the frontier of LLM engineering.

What you will learn

Understand the shift from manual prompt engineering to programmatic prompt optimization
Write DSPy modules: Signature, Predict, ChainOfThought
Use teleprompters (optimizers) to automatically improve prompts
Define and use metrics to evaluate and optimize LLM programs
Understand when DSPy is better than manual prompting and when it's overkill

The Problem

You've spent 3 weeks hand-tuning a prompt for your RAG system. It works — 87% accuracy on your eval set. Then the model provider releases a new version. Your carefully crafted prompt now gets 72% accuracy. You spend another week re-tuning.

This is the fundamental problem with manual prompt engineering: prompts are brittle. They're tied to a specific model version, specific phrasing, and specific edge cases that you discovered through trial and error. When any of these change, the prompt breaks.

DSPy asks: what if prompts were compiled from declarative specifications, not hand-written?

The Concept

The Analogy: PyTorch for Prompts

In PyTorch, you don't hand-write the gradients for your model. You declare the architecture (layers, connections), define a loss function, and the framework computes gradients automatically via autograd.

DSPy does the same for LLM prompts:

PyTorch	DSPy
`nn.Module`	`dspy.Module`
`forward()`	`forward()`
Weights (learned)	Prompts (compiled)
Loss function	Metric function
Optimizer (Adam, SGD)	Teleprompter (BootstrapFewShot, etc.)
Training loop	Compilation loop
Gradients (automatic)	Prompt examples (automatic)

The Three Building Blocks

1. Signatures: Declarative Input/Output Specs

A Signature declares what a module should do, not how to prompt for it:

import dspy

# Signature: "Given a question, produce an answer"
class QA(dspy.Signature):
    """Answer questions with short factoid answers."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="A factoid answer, 1-3 sentences.")

# Signature for a RAG system
class RAG(dspy.Signature):
    """Answer questions based on retrieved context."""
    context: str = dspy.InputField(desc="Retrieved passages")
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="Answer grounded in context")

# Signature for classification
class SentimentClassifier(dspy.Signature):
    """Classify the sentiment of customer feedback."""
    feedback: str = dspy.InputField()
    sentiment: str = dspy.OutputField(desc="positive, negative, or neutral")
    confidence: float = dspy.OutputField(desc="0.0 to 1.0")

The signature is the contract. DSPy uses it to generate and optimize the actual prompt text.

2. Modules: Composable LLM Operations

Modules are the building blocks of DSPy programs:

# Predict: simplest module — call the LLM with the signature
qa = dspy.Predict(QA)
result = qa(question="What is the capital of France?")
print(result.answer)  # "Paris"

# ChainOfThought: adds reasoning before the answer
cot_qa = dspy.ChainOfThought(QA)
result = cot_qa(question="Why is the sky blue?")
print(result.rationale)  # "Light scattering in the atmosphere..."
print(result.answer)     # "Rayleigh scattering of sunlight..."

# ChainOfThoughtWithHint: adds a hint to guide reasoning
hinted_qa = dspy.ChainOfThoughtWithHint(QA, hint="Think about the physical process")

3. Teleprompters: The Optimizers

Teleprompters automatically optimize prompts by:

Running the module on training examples
Evaluating outputs against a metric
Keeping examples that score well as few-shot demonstrations
Iterating to improve the prompt

# Define a metric
def answer_correctness(example, pred, trace=None):
    """Check if the predicted answer matches the gold answer."""
    return pred.answer.strip().lower() == example.answer.strip().lower()

# Prepare training data
trainset = [
    dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"),
    dspy.Example(question="What is the capital of Japan?", answer="Tokyo").with_inputs("question"),
    dspy.Example(question="Who wrote Hamlet?", answer="William Shakespeare").with_inputs("question"),
    # ... more examples
]

# Choose a teleprompter
from dspy.teleprompt import BootstrapFewShot

teleprompter = BootstrapFewShot(
    metric=answer_correctness,
    max_bootstrapped_demos=4,   # Max examples to auto-generate
    max_labeled_demos=4,        # Max labeled examples to include
    max_rounds=1,               # Optimization rounds
)

# Compile: this is the "training" step
optimized_qa = teleprompter.compile(dspy.Predict(QA), trainset=trainset)

# The optimized module now has auto-selected few-shot examples
# You can inspect what it learned:
print(optimized_qa.demos)  # The examples it chose as demonstrations

You have a manual prompt that achieves 87% accuracy. You switch to DSPy, define the same task as a Signature, and use BootstrapFewShot to compile. The compiled module achieves 91% accuracy. Then the model provider releases a new version and your old manual prompt drops to 72%. What happens with your DSPy module?

Building a Complete DSPy RAG Program

import dspy

# Configure the LLM
lm = dspy.LM('openai/gpt-4o-mini')
dspy.configure(lm=lm)

# Step 1: Define the retrieval module (simulated)
class Retrieve(dspy.Module):
    """Retrieve relevant passages for a question."""
    def __init__(self, k=3):
        super().__init__()
        self.k = k
    
    def forward(self, question: str):
        # In production: call your vector store here
        passages = self._search(question, self.k)
        return dspy.Prediction(context=passages)
    
    def _search(self, query, k):
        # Placeholder — replace with real retrieval
        return ["Passage 1...", "Passage 2...", "Passage 3..."]

# Step 2: Define the RAG signature
class GenerateAnswer(dspy.Signature):
    """Answer questions using retrieved context."""
    context: list[str] = dspy.InputField(desc="Retrieved passages")
    question: str = dspy.InputField()
    answer: str = dspy.OutputField(desc="Answer grounded in context, 1-3 sentences")

# Step 3: Build the RAG program
class RAG(dspy.Module):
    def __init__(self, num_passages=3):
        super().__init__()
        self.retrieve = Retrieve(k=num_passages)
        self.generate = dspy.ChainOfThought(GenerateAnswer)
    
    def forward(self, question: str):
        # Step 1: Retrieve
        retrieval = self.retrieve(question=question)
        
        # Step 2: Generate answer using retrieved context
        prediction = self.generate(
            context=retrieval.context,
            question=question
        )
        return dspy.Prediction(
            context=retrieval.context,
            answer=prediction.answer
        )

# Step 4: Define evaluation metric
def answer_correctness_metric(example, pred, trace=None):
    """Check if answer contains the gold answer string."""
    gold = example.answer.lower()
    return gold in pred.answer.lower()

def answer_groundedness_metric(example, pred, trace=None):
    """Check if answer is grounded in retrieved context."""
    answer_words = set(pred.answer.lower().split())
    context_words = set(" ".join(pred.context).lower().split())
    overlap = len(answer_words & context_words) / max(len(answer_words), 1)
    return overlap > 0.3  # At least 30% of answer words appear in context

# Step 5: Prepare training data
trainset = [
    dspy.Example(question="Who founded Apple?", answer="Steve Jobs").with_inputs("question"),
    dspy.Example(question="What is the speed of light?", answer="299,792,458 meters per second").with_inputs("question"),
    dspy.Example(question="When did WWII end?", answer="1945").with_inputs("question"),
    # ... more examples
]

# Step 6: Compile (optimize) the RAG program
from dspy.teleprompt import BootstrapFewShot

teleprompter = BootstrapFewShot(
    metric=answer_correctness_metric,
    max_bootstrapped_demos=4,
    max_labeled_demos=4,
)

optimized_rag = teleprompter.compile(RAG(), trainset=trainset)

# Step 7: Evaluate
from dspy.evaluate import Evaluate

evaluator = Evaluate(
    devset=trainset,  # In practice: use a separate test set
    metric=answer_correctness_metric,
    num_threads=4,
    display_progress=True,
)

score = evaluator(optimized_rag)
print(f"Accuracy: {score}")

Advanced: Assertions (Constraints)

DSPy Assertions let you define hard constraints that the model must satisfy. If the model violates a constraint, DSPy automatically retries with feedback:

from dspy import Assert, Suggest

class SafeRAG(dspy.Module):
    def __init__(self):
        super().__init__()
        self.retrieve = Retrieve(k=3)
        self.generate = dspy.ChainOfThought(GenerateAnswer)
    
    def forward(self, question: str):
        retrieval = self.retrieve(question=question)
        prediction = self.generate(context=retrieval.context, question=question)
        
        # HARD constraint: answer must be grounded in context
        # If violated, DSPy retries with feedback
        answer_words = set(prediction.answer.lower().split())
        context_words = set(" ".join(retrieval.context).lower().split())
        overlap = len(answer_words & context_words) / max(len(answer_words), 1)
        
        Assert(
            overlap > 0.3,
            "Answer must be grounded in retrieved context. Please re-answer using only information from the context."
        )
        
        # SOFT suggestion: answer should be concise
        Suggest(
            len(prediction.answer.split()) <= 50,
            "Answer should be concise (50 words or fewer)."
        )
        
        return dspy.Prediction(answer=prediction.answer, context=retrieval.context)

When to Use DSPy vs Manual Prompting

Use DSPy when...	Use manual prompting when...
You have an eval set (50+ examples)	You have no eval data
You need to switch models frequently	You're using one fixed model
Your prompt is complex (multi-step)	Your prompt is a single call
You want reproducible optimization	You need a quick prototype
You're building a production system	You're doing exploratory work
You have a metric to optimize against	You can't define a metric

Your team is debating whether to adopt DSPy for a customer support chatbot that handles 50 different intent categories. You have 5,000 labeled examples. The current hand-written prompt achieves 82% accuracy. What's the strongest case for DSPy?

Comparing Teleprompters

from dspy.teleprompt import (
    BootstrapFewShot,
    BootstrapFewShotWithRandomSearch,
    KNNFewShot,
    MIPRO,
)

# 1. BootstrapFewShot: Fast, simple, good baseline
# Automatically generates few-shot examples from training data
teleprompter_1 = BootstrapFewShot(metric=my_metric, max_bootstrapped_demos=4)

# 2. BootstrapFewShotWithRandomSearch: Better quality, slower
# Tries multiple random subsets of demonstrations
teleprompter_2 = BootstrapFewShotWithRandomSearch(
    metric=my_metric, max_bootstrapped_demos=4, num_candidate_programs=10
)

# 3. MIPRO: State-of-the-art, optimizes instructions AND demonstrations
# Uses Bayesian optimization to search over prompt instructions
teleprompter_3 = MIPRO(
    metric=my_metric,
    num_candidates=10,
    init_temperature=1.0,
)

# Rule of thumb:
# Start with BootstrapFewShot → if quality is insufficient, try MIPRO
# MIPRO is slower but produces better prompts for complex tasks

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Ship It, Use It, Common Pitfalls, Sources, Evaluation, Exercises, Key Terms, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.