Phase 3: AI Document Processing & Data Extraction · 50 min · OpenAI API · Anthropic API · Python
The Problem
An automation workflow calls GPT-4o to extract invoice fields. The prompt says "return JSON with vendor_name, invoice_number, total, and date." 90% of the time it works. 10% of the time, the LLM returns:
- JSON with extra prose: "Here's the extracted data: {...}"
- Wrong field names:
"vendor" instead of "vendor_name" - Wrong types:
"total": "forty-five dollars" instead of "total": 45.00 - Missing fields: no
date field at all - Invalid JSON: unquoted keys, trailing commas
Each of these breaks the downstream Code node that tries to parse the output. The workflow fails, the email is lost, and the team loses trust in automation.
Structured output — forcing the LLM to return valid JSON matching a schema — is the single most important technical skill in AI workflow automation.
Structured output is the difference between demo and production
Free-form LLM output works in demos because you test with clean inputs. In production, you get 10,000 varied inputs and the LLM's output format drifts. JSON mode, Structured Outputs, and schema validation are what make AI workflows reliable enough for production.
The Concept
Three Levels of Structure
Level 1: Free-form text (UNRELIABLE)
────────────────────────────────────
Prompt: "Extract the invoice fields"
Output: "The vendor is Acme Corp, invoice #12345, total $450.00, dated Jan 15"
→ Parsing: regex, fragile, breaks on format changes
Level 2: JSON mode (BETTER)
────────────────────────────────────
Prompt: "Return a JSON object with vendor_name, invoice_number, total, date"
Output: {"vendor_name": "Acme Corp", "invoice_number": "12345", "total": "$450.00", "date": "Jan 15"}
→ Parsing: JSON.parse() works, but field names/types not guaranteed
Level 3: Structured Outputs / Function Calling (RELIABLE)
────────────────────────────────────
Schema: {vendor_name: string, invoice_number: string, total: number, date: string (YYYY-MM-DD)}
Output: {"vendor_name": "Acme Corp", "invoice_number": "12345", "total": 450.00, "date": "2025-01-15"}
→ Parsing: guaranteed valid JSON, correct types, correct field names
OpenAI Structured Outputs vs JSON Mode
| Feature | JSON Mode | Structured Outputs |
|---|
| Valid JSON guaranteed | Yes | Yes |
| Field names enforced | No | Yes |
| Field types enforced | No | Yes |
| Required fields enforced | No | Yes |
| Enum values enforced | No | Yes |
| Nested objects | Not enforced | Enforced |
| Model support | GPT-4o, GPT-4o-mini | GPT-4o-2024-08-06+, GPT-4o-mini |
You need the LLM to return a classification with exactly one of these values: "complaint", "billing", "inquiry", "support", "other". Which approach guarantees the LLM returns only valid values?
Structured Outputs with an enum constraint guarantees the LLM returns only one of the specified values. JSON mode guarantees valid JSON but doesn't enforce field values — the LLM could return "complaintt" (typo) or "feedback" (not in the list). Few-shot examples rely on the LLM following the pattern, which isn't guaranteed. Structured Outputs with enum is the only approach that enforces valid values at the API level.
JSON Schema Design
{
"type": "object",
"properties": {
"category": {
"type": "string",
"enum": ["complaint", "billing", "inquiry", "support", "other"]
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"priority": {
"type": "string",
"enum": ["high", "medium", "low"]
},
"entities": {
"type": "array",
"items": {
"type": "object",
"properties": {
"type": {
"type": "string",
"enum": ["order_number", "email", "phone", "product", "amount"]
},
"value": {
"type": "string"
}
},
"required": ["type", "value"]
}
},
"summary": {
"type": "string"
}
},
"required": ["category", "confidence", "priority", "summary"],
"additionalProperties": false
}
Anthropic Tool Use for Structured Output
Anthropic doesn't have "Structured Outputs" but achieves the same via tool use:
# Define a "tool" that the LLM must call with structured arguments
tool = {
"name": "extract_email_data",
"description": "Extract structured data from an email",
"input_schema": {
"type": "object",
"properties": {
"category": {"type": "string", "enum": ["complaint", "billing", "inquiry", "support", "other"]},
"confidence": {"type": "number"},
"summary": {"type": "string"}
},
"required": ["category", "confidence", "summary"]
}
}