Phase 2: Audio-Visual Models · ~120 minutes · Python
Show-o and Discrete-Diffusion Unified Models
Transfusion mixes continuous and discrete representations. Show-o (Xie et al., August 2024) goes the other way: text tokens use causal next-token prediction, image tokens use masked discrete diffusion in the spirit of MaskGIT. Both sit inside one transformer with a hybrid attention mask. The result unifies VQA, text-to-image, inpainting, and mixed-modality generation on one backbone, one tokenizer per modality, one loss formulation (next-token extended to masked prediction). This lesson walks the Show-o design — why masked discrete diffusion is a parallel, few-step image generator — and contrasts with Transfusion and Emu3.
Hiring signal: Understanding of show-o and discrete-diffusion unified models internals
What you will learn
- Explain masked discrete diffusion: the schedule that masks tokens uniformly then asks the transformer to recover them.
- Compare parallel image decoding (Show-o, MaskGIT) to autoregressive image decoding (Chameleon, Emu3) on speed and quality.
- Name the three tasks Show-o handles in one checkpoint: T2I, VQA, image inpainting.
- Pick a masking schedule (cosine, linear, truncated) and reason about its effect on sample quality.
Introduction
Type: Learn Languages: Python (stdlib, masked-discrete-diffusion sampler) Prerequisites: Phase 12 · 13 (Transfusion) Time: ~120 minutes
Learning Objectives
- Explain masked discrete diffusion: the schedule that masks tokens uniformly then asks the transformer to recover them.
- Compare parallel image decoding (Show-o, MaskGIT) to autoregressive image decoding (Chameleon, Emu3) on speed and quality.
- Name the three tasks Show-o handles in one checkpoint: T2I, VQA, image inpainting.
- Pick a masking schedule (cosine, linear, truncated) and reason about its effect on sample quality.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy