Phase 7: AI Agents & Tool-Using Systems · 85 min · OpenAI Agents SDK · Hugging Face smolagents · LangChain agents

The Agent Loop: Tools, Actions, and Observations

An agent is not magic. It is a loop that chooses actions, calls tools, observes results, and decides what to do next.

Hiring signal: Can explain and implement the core agent loop without hiding behind a framework

The Problem

Students see demos where an “agent” browses the web, edits files, sends emails, or books meetings. The demos feel like general intelligence. In production, the same pattern often fails for boring reasons: a tool returns malformed JSON, the model calls the wrong function, the loop never stops, the agent acts on stale context, or no one can explain why it took an action.

The professional move is to demystify the system. An agent is usually an LLM wrapped in a control loop:

Once you see the loop, you can engineer it.

The Concept

What Makes Something "Agentic"

When people say "AI agent," they usually mean a system where an LLM is not just generating text — it is making decisions about what to do next. The model looks at the current state (user goal, conversation history, tool outputs so far), chooses an action (call a tool, ask a clarifying question, or produce a final answer), executes that action, observes the result, and then decides again. This cycle — decide, act, observe, decide again — is the agent loop.

The key insight is that the LLM itself is not running the loop. Your code is. The LLM is a decision-making component inside the loop. Your code calls the model, parses its response to extract an action, executes the action, appends the result to the conversation, and calls the model again. The model never "runs" — it is called repeatedly by your control code, which manages the loop, the state, the stop conditions, and the error handling.

This distinction matters because it tells you where bugs live. When an agent fails, the problem is usually not "the model is dumb" — it is "the loop is poorly engineered." The observation format is unclear, the stop condition is ambiguous, the state grows too large, the tool schema is wrong, or there is no error handling when a tool fails. These are all your code's responsibility, not the model's.

The Five Parts of an Agent

An agent needs five components to function:

Goal: what the user wants done. This is the starting input — "find the cheapest flight to Tokyo next Friday" or "summarize this repo's open issues." The goal must be specific enough that the agent can tell when it's finished.
Policy: instructions and constraints that define acceptable behavior. This includes the system prompt ("you are a research assistant, use tools when you need information, never make up facts"), tool descriptions (what each tool does and when to use it), and guardrails ("do not send emails without confirmation").
Tools: callable capabilities such as search, database lookup, file read, code execution, calendar access, or internal APIs. Each tool has a name, a description, an input schema, and a function that executes when called. The model sees the tool descriptions and chooses which to call — but it never executes tools itself. Your code does.
State: the running record of everything that has happened — messages, tool outputs, intermediate decisions, errors, and task status. State grows with every step. Managing state (trimming old context, summarizing long histories, structuring observations) is one of the hardest engineering problems in production agents.
Stop condition: a clear reason to finish, ask a human, or fail safely. Without an explicit stop condition, the agent will loop forever or until it hits a max-steps limit. Stop conditions include: the model says "done," the model produces a final answer (no tool call), a human-approval gate is triggered, a maximum step count is reached, or an unrecoverable error occurs.

Why Agent Loops Fail in Production

The demo looks magical. Production is boring. Here are the most common failure modes:

The model can't tell it's done. The tool already returned the answer, but the observation format is so verbose or unstructured that the model doesn't recognize the answer is there. It calls the same tool again, gets the same result, and loops. Fix: format observations as concise, structured summaries — not raw JSON dumps.

The stop condition is ambiguous. The system prompt says "help the user" but never says "when you have enough information, return a final answer without calling a tool." The model keeps calling tools because it thinks more information might help. Fix: be explicit about when to stop.

Tool schemas are wrong or ambiguous. The model calls a tool with the wrong arguments because the tool description is unclear. Fix: write tool descriptions like API documentation — include parameter types, examples, and when to use each tool.

State grows unbounded. Each step adds messages and tool outputs. By step 8, the context window is full of old observations the model doesn't need. Fix: summarize or trim old state, or use a state management pattern that keeps only relevant history.

No error handling. A tool fails (API timeout, malformed response, permission denied) and the agent crashes or hallucinates a result. Fix: every tool call needs a try/catch, and errors should be fed back to the model as observations so it can adapt.

Your agent has a max_steps limit of 10. It keeps hitting the limit without producing a final answer — it loops between calling the same tool repeatedly. What are two likely causes and fixes?

Agent loops often fail because the model can't tell it's done — either observations are unclear (it doesn't realize the tool already gave the answer) or the prompt doesn't clearly define when to stop. Fix the observation format (structured, concise) and make the stop condition explicit in the system prompt. Also log each step to debug where the loop gets stuck.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Common Mistakes, Common Pitfalls, Exercises, Key Terms, Interview Framing — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Agent