Phase 5: CV Deployment & Applications · ~75 minutes · Python

World Models & Video Diffusion

A video model that predicts the next seconds of a scene is a world simulator. Condition that prediction on actions and you have a learned game engine.

Hiring signal: Understanding of world models & video diffusion internals

What you will learn

Explain the difference between a pure video generation model (Sora 2) and an action-conditioned world model (Genie 3, DreamerV3)
Describe a video DiT: spatio-temporal patches, 3D position encoding, joint attention across (T, H, W) tokens
Trace how a world model plugs into robotics: VLM plans → video model simulates → inverse dynamics emits actions
Pick between Sora 2, Genie 3, Runway GWM-1 Worlds, Wan-Video, and HunyuanVideo for a given use case (creative video, interactive sim, autonomous-driving synthesis)

Introduction

Type: Learn + Build Languages: Python Prerequisites: Phase 4 Lesson 10 (Diffusion), Phase 4 Lesson 12 (Video Understanding), Phase 4 Lesson 23 (DiT + Rectified Flow) Time: ~75 minutes

Learning Objectives

Explain the difference between a pure video generation model (Sora 2) and an action-conditioned world model (Genie 3, DreamerV3)
Describe a video DiT: spatio-temporal patches, 3D position encoding, joint attention across (T, H, W) tokens
Trace how a world model plugs into robotics: VLM plans → video model simulates → inverse dynamics emits actions
Pick between Sora 2, Genie 3, Runway GWM-1 Worlds, Wan-Video, and HunyuanVideo for a given use case (creative video, interactive sim, autonomous-driving synthesis)

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.