Phase 1: Classification Algorithms · ~90 minutes · Python

Decision Trees and Random Forests

A decision tree is just a flowchart. But a forest of them is one of the most powerful tools in ML.

What you will learn

Implement Gini impurity, entropy, and information gain calculations to find optimal decision tree splits
Build a decision tree classifier from scratch with pre-pruning controls (max depth, min samples)
Construct a random forest using bootstrap sampling and feature randomization, and explain why it reduces variance
Compare MDI feature importance with permutation importance and identify when MDI is biased

Introduction

Type: Build
Language: Python
Prerequisites: Earlier lessons in this course
Time: ~90 minutes

Objective

Learning objectives

Implement Gini impurity, entropy, and information gain calculations to find optimal decision tree splits
Build a decision tree classifier from scratch with pre-pruning controls (max depth, min samples)
Construct a random forest using bootstrap sampling and feature randomization, and explain why it reduces variance
Compare MDI feature importance with permutation importance and identify when MDI is biased

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.