Phase 3: Adversarial Machine Learning · 60 min · Python · PyTorch

Poisoning Attacks — Label Flipping, Backdoors, and Clean-Label Poisoning

Evasion attacks fool a finished model at inference time; poisoning attacks corrupt the model before it's finished — and a validation accuracy check will not catch it.

Hiring signal: Supply-chain and training-pipeline security is a growing line item in AI security job postings — companies fine-tuning on scraped, crowdsourced, or vendor-supplied data need engineers who understand that a model can pass every accuracy benchmark and still contain a targeted backdoor. Being able to both install and detect a trigger-pattern backdoor is a concrete, demonstrable skill for AI Security Engineer and DevSecOps-for-AI roles.

What you will learn

Implement a label-flipping poisoning attack and measure how it degrades accuracy on the targeted class before it shows up in overall accuracy
Implement a BadNets-style backdoor/trigger-pattern poisoning attack and measure Attack Success Rate separately from clean-input accuracy
Explain clean-label poisoning and gradient-based poisoning (e.g., gradient matching) as attacks that avoid the label-consistency signals that make basic poisoning detectable
Describe statistical anomaly detection approaches (activation clustering, spectral signatures) used to find poisoned samples in a training set

The Problem

Every attack in the previous lesson assumed a fixed, already-trained model: the attacker perturbs an input at inference time and the model's weights never change. Poisoning attacks target something upstream of that — the training data itself. If an attacker can influence even a small fraction of what a model learns from, they can install behavior that no amount of inference-time defense will catch, because the behavior isn't a bug in the model's decision boundary — it is the decision boundary, exactly as trained.

This is not a hypothetical. Modern ML pipelines pull training data from scraped web text, crowdsourced labeling platforms, user feedback loops, and third-party vendors — every one of which is a point where an adversary with far less access than "steal the model weights" can shape what gets learned. Gu et al.'s BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain (2017) demonstrated the sharpest version of this: a model can pass every accuracy benchmark a defender runs, and still contain a targeted backdoor that activates only when the attacker's chosen trigger appears. The OWASP LLM Top 10 lists training data poisoning as its own category for exactly this reason — it's a supply-chain problem as much as an ML problem.

This lesson covers four escalating poisoning techniques — label flipping, backdoor/trigger injection, clean-label poisoning, and gradient-based poisoning — and the statistical methods used to catch them before training, since none of them show up in a normal accuracy report.

Label Flipping: The Simplest Attack That Still Works

Label flipping requires no control over the data pipeline's features at all — only the labels. An attacker (or a compromised/malicious crowdsourced annotator) simply mislabels a fraction of examples from one class as another class. No pixels, no features, nothing else changes.

The reason this is worth taking seriously rather than dismissing as "too crude to matter" is what it does to aggregate accuracy. If you flip labels for a class that's a minority of the overall dataset, the damage concentrates entirely on that class's own accuracy while the overall, class-averaged accuracy barely moves — because the other classes are unaffected and dominate the average. A defender watching a single top-line accuracy number during training will see nothing wrong. Only a per-class accuracy breakdown reveals that one class has effectively collapsed. This is the first lesson poisoning research teaches: never evaluate a model's integrity using only an aggregate metric — the same principle carries through every attack in this lesson.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Backdoor / Trigger Injection: BadNets, Clean-Label Poisoning: Removing the Label-Inconsistency Signal, Gradient-Based Poisoning, Detecting Poisoned Samples, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.