Objective
What you're building
A hardware check, not a chatbot yet. This decides which model sizes the rest of the course should use for you — a decision every later lesson depends on getting right today.
Quantization: how a 16GB model fits in 8GB of RAM
Quantization trades a little quality for a lot of size
A model's "native" precision (16-bit floating point) takes roughly 2x its parameter count in gigabytes -- an 8B-parameter model is ~16GB at full precision. Quantization to 4-bit (Q4) cuts that to roughly a quarter, ~4-5GB, at a real but usually small quality cost. This is why "Q4" appears everywhere in Ollama's model library: it's the standard tradeoff that makes running real models on consumer hardware possible at all. Going below Q4 starts to show more noticeable quality loss; Q4 is the practical default for local use in 2026.
# A simple hardware check -- run this before picking any model
echo "RAM: $(sysctl -n hw.memsize 2>/dev/null | awk '{print $1/1073741824 " GB"}' || free -h | awk '/Mem:/ {print $2}')"
echo "Disk free: $(df -h . | awk 'NR==2 {print $4}')"
echo "GPU: $(system_profiler SPDisplaysDataType 2>/dev/null | grep Chipset || nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null || echo 'No dedicated GPU detected')"
(exact output depends entirely on your machine -- this is the point)
What your hardware can actually run
Set realistic expectations before you're 3 lessons in and frustrated
On 8GB RAM, a Q4 8B model runs but shares memory with everything else on your machine — expect roughly 5-10 tokens/second, fine for chat, not great for long generation tasks. 16GB+ RAM (or a dedicated GPU with 8GB+ VRAM) runs the same models meaningfully faster and leaves headroom for Open WebUI and Qdrant (Lessons 3-4) running alongside. If you're at the 8GB floor, this course still works — just don't be surprised if it feels slower than a cloud API, because it fundamentally is trading speed for privacy and zero API cost.
Your hardware check shows 8GB total RAM with a modern CPU and no dedicated GPU. You're deciding between an 8B model and a 30B model for your general-purpose chat use. Which fits, and why?
An 8B model at Q4 needs roughly 5GB, leaving room for the OS and Docker services this course adds in later lessons. A 30B model, even at Q4, needs considerably more than 8GB has available once you subtract what the OS and background processes already use -- it would likely fail to load or force heavy swapping. Matching model size to hardware, not aspiration, is the actual skill this lesson teaches.
What you're building today
hardware_check.sh: a script reporting your RAM, disk space, and GPU (if any), plus a written decision (in your own words) on which model sizes fit your hardware — this becomes the actual model choice Lesson 2 installs, not a generic recommendation.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers The Problem, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.
Create a free account — the opening phases of 29 of 35 courses are free, no credit card.