Phase 7: Production Observability & Incident Response · 45 min · Langfuse · Arize Phoenix · Python

The Four Incident Classes

Classify before you act. A PII leak and a subtle quality regression are both 'incidents,' but they call for opposite first moves.

Real-world application: Correctly triaging an incident into the right class and severity tier before acting — instead of treating every alert identically — is exactly the on-call judgment a production AI system needs and most postmortems reveal was missing.

What you will learn

Name and distinguish the four incident classes: hallucination, jailbreak, drift, PII leak
Classify an incident's signals into the correct class before deciding a response
Apply the severity rubric: S1 (safety/regulatory, contain first), S2 (quality regression, RCA first), S3 (edge case, defer)
Explain why containment and root-cause analysis are ordered differently depending on incident class

Introduction

The Four Incident Classes

Objective

Classify a set of incident signals into one of the four classes (or no incident) and assign the correct severity tier and first action.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The Problem, The Concept, Build, Check Yourself, Key Terms & Next — plus a hands-on lab, quiz, and project artifact.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card.

Key terms in this lesson

Hallucination