Home › Courses › AI Systems Build Tracks

AI Systems Build Tracks

Eight systems built one tested piece at a time: an agent harness, a research agent, a VLM, advanced RAG, an eval runner, pretraining and distributed training, and a safety gate

Where the capstone projects hand you a whole system at once, a build track assembles one step by step, each step small enough to finish and verify before the next. Build a coding-agent harness from its contract up — tool registry, JSON-RPC transport, dispatcher, plan-and-execute, verification gates, sandbox, traces. Stand up the machinery around a pretraining run, then parallelism from collective ops to ZeRO and pipeline parallel. Assemble an autonomous research agent, a vision-language model, a retrieval system with hybrid search and a reranker, an evaluation runner you can trust, and a deployable safety gate.

8 phases · 56 lessons · 56 worked examples · Early access

Take ML & AI Engineering first — this course builds on it.

Every phase, every lesson, every project

• Agent Harness (10 lessons) — free — Build a coding-agent harness from its contract up: the loop, a schema-validated tool registry, JSON-RPC over stdio, a function-call dispatcher, plan-and-execute control flow, verification gates with an observation budget, a sandbox runner, an eval harness, OpenTelemetry traces, and an end-to-end coding task.
  • Agent Harness Loop Contract
  • Tool Registry with Schema Validation
  • JSON-RPC 2.0 Over Newline-Delimited Stdio
  • Function Call Dispatcher
  • Plan-Execute Control Flow
  • Capstone Lesson 25: Verification Gates and the Observation Budget
  • Capstone Lesson 26: Sandbox Runner with Denylist and Path Jail
  • Capstone Lesson 27: Eval Harness with Fixture Tasks
  • Capstone Lesson 28: Observability with OTel GenAI Spans and Prometheus Metrics
  • Capstone Lesson 29: End-to-End Coding Agent on the Harness
• Pretraining at Scale (8 lessons) — The engineering around a pretraining run: downloading a large corpus, tokenizing it into HDF5, cosine schedules with warmup, gradient clipping and mixed precision, gradient accumulation, checkpoint save and resume, FSDP/DDP, and an LM evaluation harness.
  • Large Corpus Downloader
  • HDF5 Tokenized Corpus
  • Cosine LR with Linear Warmup
  • Gradient Clipping and Mixed Precision
  • Gradient Accumulation
  • Checkpoint Save and Resume
  • Distributed Data Parallel and FSDP from Scratch
  • Language Model Evaluation Harness
• Autonomous Research Agent (8 lessons) — A research agent in stages: hypothesis generation, literature retrieval, an experiment runner, a result evaluator, a paper writer, a critic loop, an iteration scheduler, and the end-to-end demo.
  • Hypothesis Generator
  • Literature Retrieval
  • Experiment Runner
  • Result Evaluator
  • Paper Writer
  • Critic Loop
  • Iteration Scheduler
  • End-to-End Research Demo
• Vision-Language Model (6 lessons) — Assemble a VLM: patch the image, encode it with a ViT, project into the language model's space, fuse with cross-attention, pretrain vision-language alignment, and evaluate it.
  • Vision Encoder Patches
  • Vision Transformer Encoder
  • Projection Layer for Modality Alignment
  • Cross-Attention Fusion
  • Vision-Language Pretraining
  • Multimodal Evaluation
• Advanced RAG (6 lessons) — A retrieval system that holds up: chunking strategies compared, hybrid BM25 plus dense retrieval, a cross-encoder reranker, query rewriting and HyDE, precision/recall evaluation, and the end-to-end system.
  • Chunking Strategies, Compared
  • Hybrid Retrieval with BM25 and Dense Embeddings
  • Cross-Encoder Reranker
  • Query Rewriting: HyDE, Multi-Query, and Decomposition
  • RAG Evaluation: Precision, Recall, MRR, nDCG, Faithfulness, Answer Relevance
  • End-to-End RAG System
• Evaluation Harness (6 lessons) — An eval runner you can trust: a task-spec format, classical metrics, a code-execution metric, perplexity and calibration, leaderboard aggregation, and the end-to-end runner.
  • Task Spec Format
  • Classical Metrics
  • Code Exec Metric
  • Perplexity and Calibration
  • Leaderboard Aggregation
  • End-to-End Eval Runner
• Distributed Training (6 lessons) — Parallelism from first principles: collective ops from scratch, data-parallel DDP, ZeRO parameter sharding, pipeline parallelism, sharded checkpoint resume, and an end-to-end distributed run.
  • Collective Ops From Scratch
  • Data Parallel DDP From Scratch
  • ZeRO Optimizer State Sharding
  • Pipeline Parallel and Bubble Analysis
  • Sharded Checkpoint and Atomic Resume
  • End-to-End Distributed Training
• Safety Gate (6 lessons) — A deployable safety layer: a jailbreak taxonomy, a prompt-injection detector, refusal evaluation, content-classifier integration, a constitutional rules engine, and the end-to-end gate.
  • Capstone 82 — Jailbreak Taxonomy
  • Capstone 83 — Prompt Injection Detector
  • Capstone 84 — Refusal Evaluation
  • Capstone 85 — Content Classifier Integration
  • Capstone 86 — Constitutional Rules Engine
  • Capstone 87 — End-to-End Safety Gate

The technologies you will use

Python · PyTorch · JSON-RPC · HDF5

Common questions

Are the steps graded?

Not by the CLI yet. 34 of the 56 steps ship the upstream test suite alongside the worked example, which is the raw material for a graded lab, but no starter-and-check lab has been authored on top of it. The course is marked early access for that reason.

Do I need a GPU?

Only for the pretraining and distributed-training tracks, and even there the steps are written to run at toy scale first. The agent harness, research agent, RAG, evaluation and safety tracks run on a laptop.

Key terms in this course

Agent · RAG (Retrieval-Augmented Generation) · Chunking · Gradient Accumulation · Hybrid Search · Perplexity · Prompt Injection

Continue your learning path

AI Engineering Capstone Projects · Transformers & LLMs from Scratch · Agentic AI Engineering

Start the AI Systems Build Tracks course

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary