Phase 5: CV Deployment & Applications · ~60 minutes · Python

Multi-Object Tracking & Video Memory

Tracking is detection plus association. Detect every frame. Match this frame's detections to last frame's tracks by ID.

What you will learn

Distinguish tracking-by-detection from query-based tracking and name the algorithm families (SORT, DeepSORT, ByteTrack, BoT-SORT, SAM 2 memory tracker, SAM 3.1 Object Multiplex)
Implement IoU + Hungarian assignment from scratch for classic tracking-by-detection
Explain SAM 2's memory bank and why it handles occlusion better than IoU-based association
Read the three tracking metrics (MOTA, IDF1, HOTA) and pick which one matters for a given use case

Introduction

Type: Build
Languages: Python
Prerequisites: Object Detection — YOLO from Scratch, Instance Segmentation — Mask R-CNN, SAM 3 & Open-Vocabulary Segmentation
Time: ~60 minutes

Objective

Learning objectives

Distinguish tracking-by-detection from query-based tracking and name the algorithm families (SORT, DeepSORT, ByteTrack, BoT-SORT, SAM 2 memory tracker, SAM 3.1 Object Multiplex)
Implement IoU + Hungarian assignment from scratch for classic tracking-by-detection
Explain SAM 2's memory bank and why it handles occlusion better than IoU-based association
Read the three tracking metrics (MOTA, IDF1, HOTA) and pick which one matters for a given use case

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.