Phase 7: Production, Monitoring & Capstone · 50 min · OpenAI API · Anthropic API · n8n
The Problem
Your AI workflow processes 1,000 emails per day. It uses GPT-4o for everything — classification, extraction, response generation. The monthly LLM cost is $3,000. Your manager says "This is too expensive. Cut it by 70% or we shut it down."
Cost optimization is what makes AI automation sustainable. Without it, workflows get shut down when budgets tighten. With it, you can scale to 10,000 emails per day at the same cost — by using the right model for each task, compressing prompts, caching results, and batching requests.
80% of cost comes from using the wrong model for the task
Classification doesn't need GPT-4o — GPT-4o-mini does it just as well at 1/15th the cost. Complex reasoning needs GPT-4o. The biggest cost optimization is model tiering: cheap models for simple tasks, expensive models only when needed. This alone typically cuts costs by 60–80%.
The Concept
Cost Optimization Strategies
┌──────────────────────────────────────────────────────────────┐
│ COST OPTIMIZATION STRATEGIES (by impact) │
│ │
│ 1. MODEL TIERING (60-80% savings) │
│ Use cheap models for simple tasks, expensive only for complex│
│ │
│ 2. PROMPT COMPRESSION (10-20% savings) │
│ Reduce input tokens without losing meaning │
│ │
│ 3. CACHING (5-30% savings, varies) │
│ Cache LLM responses for identical/repeated queries │
│ │
│ 4. BATCH PROCESSING (50% savings on batched calls) │
│ Group multiple requests into a single LLM call │
└──────────────────────────────────────────────────────────────┘
Model Tiering
| Task | Model | Cost per 1M tokens | Why |
|---|
| Classification | GPT-4o-mini | $0.15 | Simple: pick from categories |
| Entity extraction | GPT-4o-mini | $0.15 | Structured: extract fields |
| Sentiment analysis | GPT-4o-mini | $0.15 | Simple: positive/negative/neutral |
| Response generation | GPT-4o | $2.50 | Complex: write nuanced response |
| Complex reasoning | GPT-4o | $2.50 | Multi-step: analyze, decide, act |
| Embeddings | text-embedding-3-small | $0.02 | Cheapest embedding model |
BEFORE (all GPT-4o):
1,000 emails × (classification + extraction + response)
= 1,000 × 3 calls × ~800 tokens = 2,400,000 tokens
At $2.50/1M (GPT-4o) = $6.00/day = $180/month
AFTER (model tiering):
Classification: GPT-4o-mini → 1,000 × 200 tokens × $0.15/1M = $0.03/day
Extraction: GPT-4o-mini → 1,000 × 400 tokens × $0.15/1M = $0.06/day
Response: GPT-4o → 1,000 × 600 tokens × $2.50/1M = $1.50/day
Total: $1.59/day = $47.70/month
SAVINGS: $180 → $48 = 73% reduction
Your workflow uses GPT-4o for email classification (order, complaint, invoice, other). Each classification call uses ~300 tokens. You process 5,000 emails/day. How much do you save by switching to GPT-4o-mini?
GPT-4o: 5,000 × 300 tokens = 1,500,000 tokens × $2.50/1M = $3.75/day GPT-4o-mini: 5,000 × 300 tokens = 1,500,000 tokens × $0.15/1M = $0.225/day Savings: $3.75 - $0.225 = $3.525/day = ~$106/month Classification is a simple task that GPT-4o-mini handles just as well. The savings are $3.53/day, which is $106/month — significant at scale, and this is just one task. Apply tiering to all simple tasks and savings compound.
Prompt Compression
BEFORE (verbose prompt, 450 tokens):
"You are a helpful assistant that classifies emails. Please read the
following email carefully and determine which category it belongs to.
The categories are: order (a customer placing an order), complaint
(a customer expressing dissatisfaction), invoice (a customer asking
about an invoice), and other (anything that doesn't fit the above).
Please respond with only the category name, nothing else."
AFTER (compressed, 80 tokens):
"Classify email as: order, complaint, invoice, other. Return only category name."
SAVINGS: 370 tokens per call × 5,000 calls = 1,850,000 tokens saved
At $0.15/1M = $0.28/day saved on classification alone
Caching
import hashlib
import json
import redis
r = redis.Redis(host='localhost', port=6379)
def cached_llm_call(prompt: str, model: str = "gpt-4o-mini") -> str:
"""Cache LLM responses for identical prompts."""
cache_key = hashlib.md5(f"{model}:{prompt}".encode()).hexdigest()
# Check cache
cached = r.get(cache_key)
if cached:
return cached.decode()
# Call LLM
response = openai.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)
result = response.choices[0].message.content
# Cache for 24 hours
r.setex(cache_key, 86400, result)
return result
| Cache Hit Scenario | Savings |
|---|
| Same question asked multiple times | 100% (no LLM call) |
| Template emails with same structure | 50–80% (cache classification) |
| Knowledge base queries | 30–60% (common questions repeat) |
| Unique emails | 0% (no cache hits) |
Batch Processing
# Instead of 100 individual LLM calls:
for email in emails:
result = classify_email(email) # 100 API calls
# Batch into fewer calls:
def batch_classify(emails: list, batch_size: int = 20) -> list:
"""Classify multiple emails in a single LLM call."""
results = []
for i in range(0, len(emails), batch_size):
batch = emails[i:i + batch_size]
prompt = f"""Classify each email. Return JSON array.
Emails: {json.dumps(batch)}
Return: [{{"id": 0, "category": "..."}}, ...]"""
response = openai.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": prompt}]
)
batch_results = json.loads(response.choices[0].message.content)
results.extend(batch_results)
return results
# 100 emails → 5 API calls instead of 100