Phase 1: Classification Algorithms · ~90 minutes · Python

Decision Trees and Random Forests

A decision tree is just a flowchart. But a forest of them is one of the most powerful tools in ML.

Hiring signal: Understanding of decision trees and random forests internals

What you will learn

Implement Gini impurity, entropy, and information gain calculations to find optimal decision tree splits
Build a decision tree classifier from scratch with pre-pruning controls (max depth, min samples)
Construct a random forest using bootstrap sampling and feature randomization, and explain why it reduces variance
Compare MDI feature importance with permutation importance and identify when MDI is biased

Introduction

Type: Build Language: Python Prerequisites: Phase 1 (Lessons 09 Information Theory, 06 Probability) Time: ~90 minutes

Learning Objectives

Implement Gini impurity, entropy, and information gain calculations to find optimal decision tree splits
Build a decision tree classifier from scratch with pre-pruning controls (max depth, min samples)
Construct a random forest using bootstrap sampling and feature randomization, and explain why it reduces variance
Compare MDI feature importance with permutation importance and identify when MDI is biased

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.