LLM Alignment & RLHF

Master the full LLM alignment pipeline: from RL fundamentals to PPO to RLHF to DPO. Understand exactly how ChatGPT and Claude were trained — and build the same pipeline yourself. For engineers who want to work on LLM training, not just use LLMs.

The route

  1. Pre-training & Fine-tuning — Transformers & LLMs from Scratch (phase tf-03). Understand how base LLMs are pre-trained and fine-tuned — the foundation you need before understanding RLHF.
  2. Reinforcement Learning (full course) — Reinforcement Learning. 3 phases, 12 lessons: MDPs and dynamic programming, Monte Carlo and TD learning (Q-learning, SARSA), DQN, policy gradients, actor-critic and PPO, reward modelling for RLHF, and RL for games (AlphaZero, MuZero).

What you build

RL algorithms you implement yourself, from Q-learning to PPO, and the reward-modelling step that RLHF uses to align an LLM with human preferences.

Every learning path

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary