Phase 5: CV Deployment & Applications · ~60 minutes · Python

Multi-Object Tracking & Video Memory

Tracking is detection plus association. Detect every frame. Match this frame's detections to last frame's tracks by ID.

Hiring signal: Understanding of multi-object tracking & video memory internals

What you will learn

Distinguish tracking-by-detection from query-based tracking and name the algorithm families (SORT, DeepSORT, ByteTrack, BoT-SORT, SAM 2 memory tracker, SAM 3.1 Object Multiplex)
Implement IoU + Hungarian assignment from scratch for classic tracking-by-detection
Explain SAM 2's memory bank and why it handles occlusion better than IoU-based association
Read the three tracking metrics (MOTA, IDF1, HOTA) and pick which one matters for a given use case

Introduction

Type: Build Languages: Python Prerequisites: Phase 4 Lesson 06 (YOLO Detection), Phase 4 Lesson 08 (Mask R-CNN), Phase 4 Lesson 24 (SAM 3) Time: ~60 minutes

Learning Objectives

Distinguish tracking-by-detection from query-based tracking and name the algorithm families (SORT, DeepSORT, ByteTrack, BoT-SORT, SAM 2 memory tracker, SAM 3.1 Object Multiplex)
Implement IoU + Hungarian assignment from scratch for classic tracking-by-detection
Explain SAM 2's memory bank and why it handles occlusion better than IoU-based association
Read the three tracking metrics (MOTA, IDF1, HOTA) and pick which one matters for a given use case

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.