Phase 3: Vision-Language Model · ~90 minutes · Python

Vision Encoder Patches

A vision model that reads pixels needs a tokenizer for pixels.

Hiring signal: Can build vision encoder patches end to end

What you will learn

Tokenize an image into a fixed-length sequence of patch embeddings.
Implement a `Conv2d`-based patch projection that matches the math of unfold-then-linear.
Build a deterministic 2D sinusoidal position embedding so token order encodes spatial position.
Verify patch count, embedding shape, and `Conv2d`/unfold equivalence on a synthetic fixture.

Introduction

Patch embedding is that tokenizer. Cut the image into a grid of squares, flatten each square, project it through one linear layer, then add a 2D position signal so the transformer knows where each square sat in the original image.

Type: Build Languages: Python Prerequisites: Phase 19 lessons 30-37 (Track B foundations) Time: ~90 minutes

Objective

Learning objectives

Tokenize an image into a fixed-length sequence of patch embeddings.
Implement a Conv2d-based patch projection that matches the math of unfold-then-linear.
Build a deterministic 2D sinusoidal position embedding so token order encodes spatial position.
Verify patch count, embedding shape, and Conv2d/unfold equivalence on a synthetic fixture.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.