Objective
What you're building
3 real local models running via Ollama, tested both from the terminal and by calling the REST API directly — the same API every later tool in this course (Open WebUI, Continue.dev) is quietly calling underneath its UI.
Install and pull, sized to what lesson 1 found
# Install: curl -fsSL https://ollama.com/install.sh | sh (macOS/Linux)
ollama pull qwen3:8b # general-purpose, fast
ollama pull qwen3-coder:7b # code-focused
ollama pull nomic-embed-text # embeddings, for Lesson 4's RAG
This course uses the Qwen3 family for both general chat and code -- it's the practical default local model family in 2026, balancing quality, size options, and licensing. If you're reading this later and Qwen3 isn't the current best choice anymore, that's expected -- check ollama.com/library for what's actually recommended now rather than trusting this exact model name indefinitely; the STACK PRINCIPLE (pick the tool with real momentum, verify don't assume) applies to model choice as much as any library.
This is exactly the kind of claim the Authenticity Rule exists for
Local model rankings shift every few months as new open-weight releases land. Treat the specific model names in this lesson as "verified accurate as of when this was written," not an eternal fact — the underlying skill (pull, test, benchmark, swap if a better option exists) is what actually transfers.
Testing via CLI vs. the raw REST API
ollama run qwen3:8b "What is 2+2? Answer in one word."
(no surprises here -- this is the friendly interactive mode)
curl http://localhost:11434/api/generate -d '{
"model": "qwen3:8b",
"prompt": "What is 2+2? Answer in one word.",
"stream": false
}'
Same model, same question, same answer -- but now you can see total_duration and eval_count in the raw response, which is exactly what you'll use for benchmarking next. Ollama's server runs on port 11434 by default; every tool that connects to your "local AI" in this course is really just a client of this same REST endpoint.
Benchmark on your actual hardware
You benchmark qwen3:8b on your machine and get roughly 6 tokens/second. A cloud API you've used before felt much faster. Is 6 tokens/second a sign something is misconfigured?
This matches exactly what Lesson 1 predicted for 8GB-class hardware -- local inference genuinely is slower than a cloud API running on server-grade GPUs, and that's the actual tradeoff this course is teaching you to make deliberately (privacy, cost, offline capability) rather than something to debug away. If your number is dramatically below this range (under 1 token/sec, for instance), that's worth investigating; in the expected range, it's just the real cost of local.
What you're building today
ollama_test.sh: pulls all 3 models, tests each via both CLI and raw curl against the REST API, and benchmarks real tokens/second on your machine — recorded honestly, not compared against cloud API speed as if that were a fair comparison.