Phase 1: Training Neural Networks · ~75 minutes · Python

Loss Functions

Your network makes a prediction. The ground truth says otherwise. How wrong is it? That number is the loss. Pick the wrong loss function and your model optimizes for the wrong thing entirely.

Hiring signal: Understanding of loss functions internals

What you will learn

Implement MSE, binary cross-entropy, categorical cross-entropy, and contrastive loss (InfoNCE) from scratch with their gradients
Explain why MSE fails for classification by demonstrating the "predict 0.5 for everything" failure mode
Apply label smoothing to cross-entropy and describe how it prevents overconfident predictions
Choose the correct loss function for regression, binary classification, multi-class classification, and embedding learning tasks

Introduction

Type: Build Languages: Python Prerequisites: Lesson 03.04 (Activation Functions) Time: ~75 minutes

Learning Objectives

Implement MSE, binary cross-entropy, categorical cross-entropy, and contrastive loss (InfoNCE) from scratch with their gradients
Explain why MSE fails for classification by demonstrating the "predict 0.5 for everything" failure mode
Apply label smoothing to cross-entropy and describe how it prevents overconfident predictions
Choose the correct loss function for regression, binary classification, multi-class classification, and embedding learning tasks

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build It, Use It, Ship It, Exercises, Key Terms, Further Reading — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.