Phase 1: Calculus & Optimization · ~90 minutes · Python

Chain Rule & Automatic Differentiation

The chain rule is the engine behind every neural network that learns.

What you will learn

Build a minimal autograd engine (Value class) that records operations and computes gradients via reverse-mode autodiff
Implement forward and backward passes through a computation graph using topological sort
Construct and train a multi-layer perceptron on XOR using only the from-scratch autograd engine
Verify autodiff correctness using gradient checking against numerical finite differences

Introduction

Type: Build
Language: Python
Prerequisites: Derivatives & Gradients
Time: ~90 minutes

Objective

Learning objectives

Build a minimal autograd engine (Value class) that records operations and computes gradients via reverse-mode autodiff
Implement forward and backward passes through a computation graph using topological sort
Construct and train a multi-layer perceptron on XOR using only the from-scratch autograd engine
Verify autodiff correctness using gradient checking against numerical finite differences

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.