Phase 1: Calculus & Optimization · ~90 minutes · Python

Chain Rule & Automatic Differentiation

The chain rule is the engine behind every neural network that learns.

Hiring signal: Understanding of chain rule & automatic differentiation internals

What you will learn

Build a minimal autograd engine (Value class) that records operations and computes gradients via reverse-mode autodiff
Implement forward and backward passes through a computation graph using topological sort
Construct and train a multi-layer perceptron on XOR using only the from-scratch autograd engine
Verify autodiff correctness using gradient checking against numerical finite differences

Introduction

Type: Build Language: Python Prerequisites: Phase 1, Lesson 04 (Derivatives & Gradients) Time: ~90 minutes

Learning Objectives

Build a minimal autograd engine (Value class) that records operations and computes gradients via reverse-mode autodiff
Implement forward and backward passes through a computation graph using topological sort
Construct and train a multi-layer perceptron on XOR using only the from-scratch autograd engine
Verify autodiff correctness using gradient checking against numerical finite differences

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.