Deep Learning from Scratch
Build your own mini PyTorch — autograd, layers, and training loops from the ground up
4 phases. 13 lessons. 13 labs. 1 capstone. Neural networks from the ground up — the perceptron, multi-layer networks, backpropagation by hand, activation functions, loss functions, optimizers, regularization, and a mini deep learning framework you build yourself before learning PyTorch. You build an autograd engine and a training loop that trains a real model, then transition to PyTorch and JAX with full understanding of what's happening underneath.
- Lessons: —
- Labs: —
- Projects: —
- Level: Beginner
Curriculum
- Neural Network Foundations — The perceptron — where it all started, multi-layer networks & forward pass, backpropagation from scratch
- The Training Engine — Activation functions (ReLU, Sigmoid, GELU) & why they matter, loss functions (MSE, cross-entropy, contrastive), optimizers (SGD, Momentum, Adam, AdamW), weight initialization & training stability
- Regularization & Debugging — Regularization (dropout, weight decay, batch normalization), learning rate schedules & warmup, debugging neural networks
- Build Your Own Framework + PyTorch & JAX — Build a mini deep learning framework with autograd, introduction to PyTorch (tensors, autograd, nn.Module, training loops), introduction to JAX (functional approach, jit, vmap, grad)
Skills You Will Learn
- Backpropagation from Scratch
- Activation & Loss Functions
- Optimizers (SGD, Adam, AdamW)
- Regularization (Dropout, BatchNorm)
- Autograd Engine Implementation
- PyTorch Fundamentals
- JAX Fundamentals
Related Courses
Browse all courses · View pricing · DeVenture Academy