Phase 2: Ensembles & Evaluation · ~120 minutes · Python

Ensemble Methods

A group of weak learners, combined correctly, becomes a strong learner. This is not a metaphor. It is a theorem.

What you will learn

Implement AdaBoost and gradient boosting from scratch and explain how boosting sequentially reduces bias
Build a bagging ensemble and demonstrate how averaging decorrelated models reduces variance without increasing bias
Compare bagging, boosting, and stacking in terms of what error component each method targets
Evaluate ensemble diversity and explain why majority voting accuracy improves with more independent weak learners

Introduction

Type: Build
Language: Python
Prerequisites: Bias-Variance Tradeoff
Time: ~120 minutes

Objective

Learning objectives

Implement AdaBoost and gradient boosting from scratch and explain how boosting sequentially reduces bias
Build a bagging ensemble and demonstrate how averaging decorrelated models reduces variance without increasing bias
Compare bagging, boosting, and stacking in terms of what error component each method targets
Evaluate ensemble diversity and explain why majority voting accuracy improves with more independent weak learners

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.