Home › Courses › Deep Learning from Scratch

Deep Learning from Scratch

Build your own mini PyTorch: autograd, layers and training loops from the ground up

4 phases. 13 lessons. 13 labs. 1 capstone. Neural networks from the ground up — the perceptron, multi-layer networks, backpropagation by hand, activation functions, loss functions, optimizers, regularization, and a mini deep learning framework you build yourself before learning PyTorch. You build an autograd engine and a training loop that trains a real model, then transition to PyTorch and JAX with full understanding of what's happening underneath.

4 phases · 13 lessons · 13 labs · 1 capstone

Take Core ML: Algorithms from Scratch first — this course builds on it.

Outcomes you will have by the end

• Your own mini PyTorch — A working deep learning framework with autograd, layers, optimizers, and a training loop — built from scratch, that trains a real model. The same architecture PyTorch uses, implemented by you.
• A working mental model of how neural networks learn — Backpropagation, autograd, and training dynamics are not black boxes — you built them from scratch. When PyTorch's backward() runs, you know exactly what computation it performs.
• PyTorch and JAX fluency built on deep understanding — You transition to PyTorch and JAX not by learning their APIs, but by recognizing that they implement the same things you already built — just faster and more optimized.
• Verified certificate of completion — Issued by DeVenture Academy, tied to your completion record. Lists the backpropagation, autograd, PyTorch, and training skills you built.

What you will be able to do

Backpropagation from Scratch · Activation & Loss Functions · Optimizers (SGD, Adam, AdamW) · Regularization (Dropout, BatchNorm) · Autograd Engine Implementation · PyTorch Fundamentals · JAX Fundamentals

Every phase, every lesson, every project

• Neural Network Foundations (3 lessons) — free — The perceptron — where it all started, multi-layer networks & forward pass, backpropagation from scratch
  • The Perceptron
  • Multi-Layer Networks and Forward Pass
  • Backpropagation from Scratch
• The Training Engine (4 lessons) — Activation functions (ReLU, Sigmoid, GELU) & why they matter, loss functions (MSE, cross-entropy, contrastive), optimizers (SGD, Momentum, Adam, AdamW), weight initialization & training stability
  • Activation Functions
  • Loss Functions
  • Optimizers
  • Regularization
• Regularization & Debugging (3 lessons) — Regularization (dropout, weight decay, batch normalization), learning rate schedules & warmup, debugging neural networks
  • Weight Initialization and Training Stability
  • Learning Rate Schedules and Warmup
  • Build Your Own Mini Framework
• Build Your Own Framework + PyTorch & JAX (3 lessons) — Build a mini deep learning framework with autograd, introduction to PyTorch (tensors, autograd, nn.Module, training loops), introduction to JAX (functional approach, jit, vmap, grad)
  • Introduction to PyTorch
  • Introduction to JAX
  • Debugging Neural Networks

The technologies you will use

NumPy · PyTorch · JAX · Matplotlib

Roles this course prepares you for

• Deep Learning Engineer ($120k–$180k) — Design and train neural networks for real problems — vision, NLP, speech. Understanding backpropagation and autograd from scratch means you can debug training issues that framework-only engineers can't.
• ML Research Engineer ($140k–$220k) — Implement novel neural network architectures from research papers. The from-scratch understanding of autograd and training loops is essential for modifying architectures beyond what standard libraries support.
• AI Infrastructure Engineer ($125k–$180k) — Build and optimize the training infrastructure. Understanding what happens inside autograd and GPU computation at the lowest level lets you optimize training speed and memory usage.

What deep learning from scratch actually is

It's building neural networks from the ground up — starting with a single perceptron, adding layers, implementing backpropagation by hand, building an autograd engine, and constructing a mini training framework — all before using PyTorch. By the time you reach PyTorch, you've already built a simpler version of it yourself.

What you do every day

You design neural network architectures, write training loops, and debug training dynamics. When something goes wrong — loss explodes, gradients vanish, the model won't learn — you diagnose the issue from first principles because you built the mechanisms yourself. When you need a custom layer or loss function, you implement it without waiting for PyTorch to add it.

Why companies hire for this

Every company doing serious ML uses PyTorch, but most engineers treat it as a black box. The engineer who built an autograd engine from scratch is the one who can debug production training issues, implement custom architectures from papers, and optimize training speed. That's the engineer who gets the senior role.

What this course is not

It is not a PyTorch tutorial. You'll use PyTorch in Phase 4, but only after building a simpler version of it yourself. It is not a computer vision or NLP course — those are the next courses in the chain. This course is purely about how neural networks learn and how to build the machinery that trains them.

Common questions

Do I need Core ML first?

Yes — this course builds on the gradient descent and evaluation foundations from Core ML. You'll use the same optimization concepts, but now applied to neural networks instead of classical algorithms.

Why build a framework from scratch before using PyTorch?

Because PyTorch is an autograd engine with a training loop — and if you've built one yourself, PyTorch is transparent instead of magical. When training breaks, you'll know whether it's the data, the architecture, the optimizer, or the framework. When you need custom gradients or unusual architectures, you'll know how to implement them.

How is this different from the DL phase in ML & AI Engineering?

ML & AI Engineering covers deep learning in 7 lessons as part of a broader curriculum — enough to use PyTorch effectively. This course spends 13 lessons building backpropagation from scratch, implementing an autograd engine, and constructing a mini framework before touching PyTorch. It's the difference between using PyTorch and understanding what PyTorch is.

What do I end up with?

A working mini deep learning framework with autograd, layers, optimizers, and a training loop — plus PyTorch and JAX fluency built on deep understanding of what those frameworks actually do underneath.

Key terms in this course

Backpropagation · Learning Rate

Continue your learning path

Core ML: Algorithms from Scratch · Transformers & LLMs from Scratch · Computer Vision Engineering · NLP & Speech Processing · Generative AI Fundamentals · ML & AI Engineering

Start the Deep Learning from Scratch course

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary