Phase 2: Audio-Visual Models · ~120 minutes · Python

Show-o and Discrete-Diffusion Unified Models

Transfusion mixes continuous and discrete representations. Show-o (Xie et al., August 2024) goes the other way: text tokens use causal next-token prediction, image tokens use masked discrete diffusion in the spirit of MaskGIT. Both sit inside one transformer with a hybrid attention mask. The result unifies VQA, text-to-image, inpainting, and mixed-modality generation on one backbone, one tokenizer per modality, one loss formulation (next-token extended to masked prediction). This lesson walks the Show-o design — why masked discrete diffusion is a parallel, few-step image generator — and contrasts with Transfusion and Emu3.

What you will learn

Explain masked discrete diffusion: the schedule that masks tokens uniformly then asks the transformer to recover them.
Compare parallel image decoding (Show-o, MaskGIT) to autoregressive image decoding (Chameleon, Emu3) on speed and quality.
Name the three tasks Show-o handles in one checkpoint: T2I, VQA, image inpainting.
Pick a masking schedule (cosine, linear, truncated) and reason about its effect on sample quality.

Introduction

Type: Learn
Languages: Python (stdlib, masked-discrete-diffusion sampler)
Prerequisites: Transfusion: Autoregressive Text + Diffusion Image in One Transformer
Time: ~120 minutes

Objective

Learning objectives

Explain masked discrete diffusion: the schedule that masks tokens uniformly then asks the transformer to recover them.
Compare parallel image decoding (Show-o, MaskGIT) to autoregressive image decoding (Chameleon, Emu3) on speed and quality.
Name the three tasks Show-o handles in one checkpoint: T2I, VQA, image inpainting.
Pick a masking schedule (cosine, linear, truncated) and reason about its effect on sample quality.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.