Phase 2: Ensembles & Evaluation · ~90 minutes · Python

Model Evaluation

A model is only as good as the way you measure it.

Hiring signal: Understanding of model evaluation internals

What you will learn

Implement K-fold and stratified K-fold cross-validation from scratch and explain why stratification matters for imbalanced data
Compute precision, recall, F1, AUC-ROC, and regression metrics (MSE, RMSE, MAE, R-squared) from scratch
Interpret learning curves to diagnose whether a model suffers from high bias or high variance
Identify common evaluation mistakes including data leakage, wrong metric selection, and test set contamination

Introduction

Type: Build Languages: Python Prerequisites: Phase 1 (Probability & Distributions, Statistics for ML), Phase 2 Lessons 1-8 Time: ~90 minutes

Learning Objectives

Implement K-fold and stratified K-fold cross-validation from scratch and explain why stratification matters for imbalanced data
Compute precision, recall, F1, AUC-ROC, and regression metrics (MSE, RMSE, MAE, R-squared) from scratch
Interpret learning curves to diagnose whether a model suffers from high bias or high variance
Identify common evaluation mistakes including data leakage, wrong metric selection, and test set contamination

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.