Phase 1: Vision-Language Models · ~120 minutes · Python
Any-Resolution Vision: Patch-n'-Pack and NaFlex
Real images are not 224x224 squares. A receipt is 9:16, a chart is 16:9, a medical scan might be 4096x4096, a mobile screenshot is 9:19.5. The pre-2024 VLM answer — resize everything to a fixed square — threw away the signal that makes OCR, document understanding, and high-resolution scene parsing work. NaViT (Google, 2023) showed you could pack variable-resolution patches into a single transformer batch with block-diagonal masking. Qwen2-VL's M-RoPE (2024) dropped absolute positional tables entirely. LLaVA-NeXT's AnyRes tiled high-resolution images into a base + sub-images. SigLIP 2's NaFlex variant (2025) is now the default encoder for open VLMs that want a single checkpoint to serve every aspect ratio. This lesson implements patch-n'-pack end to end.
Hiring signal: Understanding of any-resolution vision: patch-n'-pack and naflex internals
What you will learn
- Pack patches from a batch of variable-resolution images into one sequence and build the block-diagonal attention mask.
- Pick between AnyRes tiling (LLaVA-NeXT), NaFlex (SigLIP 2), and M-RoPE (Qwen2-VL) for a given task.
- Compute token budgets for OCR, charts, and photography without resizing.
- Name the three failure modes of square-resize: squished text, cropped content, wasted tokens on padding.
Introduction
Type: Build Languages: Python (stdlib, patch packer + block-diagonal mask) Prerequisites: Phase 12 · 01 (ViT patches), Phase 12 · 05 (LLaVA) Time: ~120 minutes
Learning Objectives
- Pack patches from a batch of variable-resolution images into one sequence and build the block-diagonal attention mask.
- Pick between AnyRes tiling (LLaVA-NeXT), NaFlex (SigLIP 2), and M-RoPE (Qwen2-VL) for a given task.
- Compute token budgets for OCR, charts, and photography without resizing.
- Name the three failure modes of square-resize: squished text, cropped content, wasted tokens on padding.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy