The Problem
You now understand backpropagation from first principles — which is exactly why you'll appreciate PyTorch, because it automates the tedious gradient bookkeeping you just did by hand. But "knows the math" and "can ship a model" are different hires. Job postings list PyTorch by name; take-homes assume you can write a training loop without looking it up. The danger is cargo-culting a loop you don't understand — getting silent bugs like forgetting zero_grad() and never knowing why training stalls.
This lesson makes the framework second nature: tensors, autograd, modules, and the canonical loop.
The Concept
What PyTorch Actually Is
PyTorch is three ideas stacked together, and everything else — CNNs, transformers, LLMs — is built from these three:
1. Tensor: An n-dimensional array, just like NumPy, but with two superpowers: it can live on a GPU (for massive parallelism) and it can track gradients (for automatic differentiation). A tensor is the basic data structure of all deep learning — images are 3D tensors (channels × height × width), sequences are 2D tensors (timesteps × features), and model parameters are tensors too.
2. Autograd: When you perform operations on tensors with requires_grad=True, PyTorch silently records every operation into a computation graph — a DAG (directed acyclic graph) where each node is an operation and each edge is a tensor. When you call .backward(), PyTorch traverses this graph in reverse (reverse-mode autodiff) and computes the gradient of the output with respect to every input. This is exactly what you built by hand in the previous lesson — but PyTorch does it automatically, for any computation, no matter how complex.
3. nn.Module: A container that holds parameters (learnable tensors) and defines a forward() method (the computation). Layers, models, and entire architectures are all nn.Module subclasses. The module system handles parameter registration, device movement, and mode switching (train vs. eval) automatically. Optimizers iterate over a module's parameters and update them using the gradients computed by autograd.
1. Tensor : an n-dim array (like NumPy) that can live on GPU and track gradients
2. Autograd : records operations into a graph; .backward() fills every .grad
3. nn.Module : organizes parameters + forward(); optimizers update params from grads
The canonical training loop ties all three together: zero gradients, forward pass (builds the graph), compute loss, backward pass (fills .grad), optimizer step (updates parameters). Every deep learning training loop in the world follows this pattern.
You're evaluating your model on the validation set. You call model(x) but forget model.eval() or torch.no_grad(). What two problems does this cause?
model.eval() switches BatchNorm/Dropout to inference mode (using running stats, disabling dropout). torch.no_grad() prevents autograd from tracking operations, saving memory. Without both, you get noisy predictions from batch-dependent normalization and waste GPU memory building graphs you'll never backprop through.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Exercises, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.