Phase 2: Tool Design & Function Calling · 50 min · Python · asyncio · FastAPI
Streaming Tool Outputs
Streaming is the difference between a demo and a product. Users see progress — they don't watch spinners.
Hiring signal: Every AI product that users actually use (ChatGPT, Claude, Perplexity) streams responses. For agents with tool calls, streaming progress is expected UX. Engineers who have implemented SSE or streaming agent UX have shipped real products — that's what this signals to an interviewer.
What you will learn
- Implement Server-Sent Events (SSE) to stream tool progress to a frontend
- Use async generators to yield tool results as they complete
- Design a progress event schema for multi-tool agent runs
The Problem
A user asks your research agent to summarize 10 documents. The agent calls a summarize_document tool 10 times. Without streaming, the user stares at a spinner for 45 seconds. With streaming, they see results appear one by one — "Document 1: done. Document 2: done..." — and the experience feels responsive even though the total time is the same.
UX research consistently shows users tolerate longer waits when they see progress. A 30-second wait with visible progress feels shorter and more trustworthy than a 30-second wait with nothing but a spinner. For agents that run 5-20 tool calls, streaming individual results is the difference between a usable product and one users abandon.
Async Generators for Tool Streaming
asyncio.as_completed() is the right primitive for streaming tool results as they complete — not in submission order, but in completion order (whichever tool finishes first gets streamed first).
import asyncio
async def stream_tool_results(tool_calls):
tasks = {asyncio.ensure_future(run_tool(tc)): tc for tc in tool_calls}
for completed_future in asyncio.as_completed(tasks.keys()):
result = await completed_future
yield result # Stream this result immediately
The key insight: asyncio.gather() waits for all tools before returning anything. asyncio.as_completed() yields each future as it completes. For a streaming UX, as_completed is almost always the right choice.
You have 5 tools running in parallel with asyncio.gather(). You want to stream each result to the UI as it completes. Which approach is correct?
asyncio.as_completed() is the correct pattern for "stream results in completion order." asyncio.gather() batches all results — nothing is returned until every tool completes. asyncio.as_completed() wraps each coroutine in a future and yields them one by one as they finish. asyncio.wait() with FIRST_COMPLETED in a loop achieves the same result but requires more code.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Server-Sent Events (SSE), Progress Event Schema, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy