Phase 5: CV Deployment & Applications · ~75 minutes · Python

World Models & Video Diffusion

A video model that predicts the next seconds of a scene is a world simulator. Condition that prediction on actions and you have a learned game engine.

What you will learn

Explain the difference between a pure video generation model (Sora 2) and an action-conditioned world model (Genie 3, DreamerV3)
Describe a video DiT: spatio-temporal patches, 3D position encoding, joint attention across (T, H, W) tokens
Trace how a world model plugs into robotics: VLM plans → video model simulates → inverse dynamics emits actions
Pick between Sora 2, Genie 3, Runway GWM-1 Worlds, Wan-Video, and HunyuanVideo for a given use case (creative video, interactive sim, autonomous-driving synthesis)

Introduction

Type: Learn + Build
Languages: Python
Prerequisites: Stable Diffusion — Architecture & Fine-Tuning, Video Understanding — Temporal Modeling, DiT + Rectified Flow
Time: ~75 minutes

Objective

Learning objectives

Explain the difference between a pure video generation model (Sora 2) and an action-conditioned world model (Genie 3, DreamerV3)
Describe a video DiT: spatio-temporal patches, 3D position encoding, joint attention across (T, H, W) tokens
Trace how a world model plugs into robotics: VLM plans → video model simulates → inverse dynamics emits actions
Pick between Sora 2, Genie 3, Runway GWM-1 Worlds, Wan-Video, and HunyuanVideo for a given use case (creative video, interactive sim, autonomous-driving synthesis)

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.