Phase 1: AI & ML Literacy for PMs · 45 min · OpenAI Tokenizer · Python

LLM Fundamentals for PMs

The model doesn't see your 200-page PDF. It sees a few hundred thousand tokens, and it's billed and bounded by every one of them.

Hiring signal: PMs who scope LLM features without understanding tokens and context windows ship specs that quietly break in production — the doc doesn't fit, the bill is 40x the estimate, or the model 'forgets' something from earlier in a long session. A PM who can estimate token counts, check them against a model's context window, and produce a rough cost-at-scale number before a single line of code is written is the PM engineering actually wants in the room during scoping.

What you will learn

Explain what a token is and why token count, not word count or character count, drives pricing and context limits
Estimate whether a given document or conversation will fit in a model's context window and know when chunking is required
Explain what embeddings are and why they enable semantic search and retrieval (RAG) even when exact words don't match
Explain attention at a conceptual level: why it lets a model use long-range context, without any matrix math
Produce a rough token-based cost and feasibility estimate for an LLM feature before it's built

The Problem

A PM is scoping a feature: upload a policy document, ask the AI questions about it, get accurate answers with citations. The policy document is 200 pages. In the spec, the PM writes one line: "the AI reads the document and answers questions about it." It sounds obviously feasible — humans read 200-page documents all the time, and the demo with a 10-page sample PDF worked perfectly.

It ships. Within a week, two things go wrong. First, real customer documents are 3-4x longer than the demo file, and some uploads fail outright with an opaque API error — the document plus the conversation history no longer fits in a single call to the model. Second, finance flags the AI feature's bill: it's costing 15x what was estimated, because every single question sent the entire document back to the model as input, every time, and nobody priced that out per token before launch.

Both problems trace back to the same gap: the PM's spec assumed "the AI can just read the whole document" the way a person does — sequentially, once, holding it in memory for free. That's not how it works. An LLM call has a hard ceiling on how much text it can process at once (the context window), and every token of input and output has a real, metered cost. "The AI reads the document" is not a feasible-or-not question you can answer by demoing on a short file. It's a token math question, and it has to be answered before the spec ships, not after the bill arrives.

This lesson is that math — not the deep architecture, but the four concepts (tokens, context windows, embeddings, attention) you need to catch this class of mistake before it becomes a production incident.

Tokens: The Actual Unit of Everything

LLMs don't process text as words or characters. They break text into tokens — sub-word chunks, roughly whole common words, word fragments, punctuation, or parts of longer/rarer words. "Tokenization" is common, but might become "token" + "ization" or something similar depending on the model's vocabulary. This isn't a minor implementation detail: token count, not word count and not character count, is what every LLM API meters. Context window limits are stated in tokens. Pricing is stated in dollars per million tokens. If you estimate feasibility or cost using word counts, you will be wrong, sometimes by a lot, because token-per-word ratios vary with language, formatting, and content type (code and non-English text often tokenize less efficiently than plain English prose).

The rule of thumb for English prose: roughly 4 characters per token, or roughly 0.75 tokens per word. That's a planning heuristic, not a guarantee — dense technical text, tables, and unusual formatting can skew it in either direction. For anything you're actually going to spec or price, don't eyeball it.

Go count real tokens before you scope anything
Open https://platform.openai.com/tokenizer right now and paste in a real document you'd expect a feature to process — an email, a support ticket, a page from a policy doc. Look at two numbers: the token count, and how the tool visually chunks the text into pieces. You'll see tokens don't line up with word boundaries the way you'd assume, and you'll get a real per-document token count instead of a guess. Do this before you write "the AI reads X" in any spec — it takes thirty seconds and it's the single highest-leverage habit in this lesson.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Context Windows: The Ceiling You Can't Spec Around, Embeddings: Finding Meaning, Not Just Matching Words, Transformer Architecture: Attention, at PM Depth, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.