Phase 1: Training Neural Networks · ~75 minutes · Python

Loss Functions

Your network makes a prediction. The ground truth says otherwise. How wrong is it? That number is the loss. Pick the wrong loss function and your model optimizes for the wrong thing entirely.

What you will learn

Implement MSE, binary cross-entropy, categorical cross-entropy, and contrastive loss (InfoNCE) from scratch with their gradients
Explain why MSE fails for classification by demonstrating the "predict 0.5 for everything" failure mode
Apply label smoothing to cross-entropy and describe how it prevents overconfident predictions
Choose the correct loss function for regression, binary classification, multi-class classification, and embedding learning tasks

Introduction

Type: Build
Languages: Python
Prerequisites: Activation Functions
Time: ~75 minutes

Objective

Learning objectives

Implement MSE, binary cross-entropy, categorical cross-entropy, and contrastive loss (InfoNCE) from scratch with their gradients
Explain why MSE fails for classification by demonstrating the "predict 0.5 for everything" failure mode
Apply label smoothing to cross-entropy and describe how it prevents overconfident predictions
Choose the correct loss function for regression, binary classification, multi-class classification, and embedding learning tasks

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.