Phase 3: Scale, Failure & Evaluation · ~90 minutes · Python (stdlib · small NumPy-free implementations)

MARL — MADDPG, QMIX, MAPPO

The reinforcement-learning heritage of multi-agent coordination, which still informs LLM-agent systems in 2026.

Hiring signal: Can operate marl in production

Introduction

MADDPG (Lowe et al., NeurIPS 2017, arXiv:1706.02275) introduced Centralized Training, Decentralized Execution (CTDE): each critic sees all agents' states and actions during training; at test time only local actors run. Works for cooperative, competitive, and mixed settings. QMIX (Rashid et al., ICML 2018, arXiv:1803.11485) is value-decomposition with a monotonic mixing network; per-agent Qs combine into joint Q so argmax distributes cleanly — dominant on StarCraft Multi-Agent Challenge (SMAC). MAPPO (Yu et al., NeurIPS 2022, arXiv:2103.01955) is PPO with a centralized value function; "surprisingly effective" on particle-world, SMAC, Google Research Football, Hanabi with minimal tuning. These underpin training policies for agent teams that must act decentrally. MAPPO is the default 2026 cooperative-MARL baseline. This lesson builds each from a small grid-world toy and lands the three ideas in muscle memory before touching LLM-agent training.

Type: Learn Languages: Python (stdlib, small NumPy-free implementations) Prerequisites: Phase 09 (Reinforcement Learning), Phase 16 · 09 (Parallel Swarm Networks) Time: ~90 minutes

Objective

Learning objectives

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Agent