Phase 0: Platforms & Inference Economics · ~45 minutes · Python (stdlib · toy batch-vs-sync cost simulator)

Batch APIs — the 50% Discount as Industry Standard

Every major provider ships an async batch API with a 50% discount and ~24-hour turnaround.

Hiring signal: Can operate batch apis in production

What you will learn

Name the three provider batch APIs (OpenAI, Anthropic, Google) and the common 50% discount + 24h turnaround guarantees.
Compute the cost for stacking batch + cached-input on an overnight classification workload and compare to synchronous-uncached baseline.
Triage a workload into interactive / semi-interactive / batch and justify the lane.
Name the two traps: partial interactivity (user expects faster than 24h) and output-schema drift (batch file format differs per provider).

Introduction

OpenAI, Anthropic, Google, and most of the inference platforms (Fireworks batch tier, Together batch) implement the same pattern. Stack batch with prompt caching and overnight pipelines drop to ~10% of synchronous-uncached cost. The rule is brutally simple: if it is not interactive, it belongs on batch. Content generation pipelines, document classification, data extraction, report generation, bulk labeling, catalog tagging — anything tolerant of 24-hour latency is money left on the table until it moves to batch. The 2026 production pattern is to triage every new LLM workload into three lanes: interactive (synchronous with caching), semi-interactive (async queue with fallback), batch (overnight, cached input stacked). Workloads that pretend to be interactive but tolerate minutes of latency waste most.

Type: Learn Languages: Python (stdlib, toy batch-vs-sync cost simulator) Prerequisites: Phase 17 · 14 (Prompt & Semantic Caching) Time: ~45 minutes

Objective

Learning objectives

Name the three provider batch APIs (OpenAI, Anthropic, Google) and the common 50% discount + 24h turnaround guarantees.
Compute the cost for stacking batch + cached-input on an overnight classification workload and compare to synchronous-uncached baseline.
Triage a workload into interactive / semi-interactive / batch and justify the lane.
Name the two traps: partial interactivity (user expects faster than 24h) and output-schema drift (batch file format differs per provider).

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Inference