Phase 3: Adversarial Machine Learning · 60 min · Python · PyTorch

Evasion Attacks — FGSM, PGD, and Carlini & Wagner

A model with 100% test accuracy can still be fooled by a perturbation too small to see — accuracy was never the same thing as robustness.

Hiring signal: Adversarial ML and AI red team interviews at frontier labs routinely ask candidates to derive and implement FGSM/PGD from the update rule on a whiteboard or in a take-home — not to call `attack.generate()` from a library. Being able to explain why the sign of the gradient (not the gradient itself) maximizes loss under an L_inf budget is a strong signal of real understanding versus tool familiarity.

What you will learn

Derive and implement the FGSM update rule x' = x + eps * sign(grad_x J(theta, x, y)) from scratch in PyTorch
Implement PGD as iterated FGSM with random start and projection onto the L_inf ball, and explain why it is a strictly stronger attack
Explain the Carlini & Wagner attack as an optimization problem over perturbation norm and misclassification margin, and when it is used instead of FGSM/PGD
Measure and interpret Attack Success Rate, mean L_inf, and mean L2 perturbation norms across a range of epsilon budgets

The Problem

In 2014, Ian Goodfellow and coauthors showed something that unsettled the entire field of deep learning: take an image a convolutional network classifies correctly with high confidence, add a perturbation so small that a human can't perceive it, and the network's prediction flips — often to a wildly wrong class, and often with higher confidence than the original correct prediction. Their paper, Explaining and Harnessing Adversarial Examples (Goodfellow et al., 2014, arXiv:1412.6572), introduced both the phenomenon and the first fast, principled way to construct it: the Fast Gradient Sign Method (FGSM).

This is not a training bug that better data fixes. It's a structural property of how high-dimensional linear (and near-linear) decision boundaries behave: in a space with thousands of pixels, a tiny, coordinated push in every dimension along the gradient direction adds up to a large change in the model's output, even though each individual pixel changes by an amount too small to notice. Three years later, Madry et al. formalized the strongest practical version of this attack — Projected Gradient Descent (PGD) — and proposed using it as the standard yardstick for adversarial robustness (Madry et al., 2017, arXiv:1706.06083). Around the same time, Carlini & Wagner showed that if you frame the problem as an optimization rather than a fixed-budget search, you can find adversarial perturbations with dramatically smaller norms than FGSM/PGD ever find — which made C&W the standard attack for stress-testing claimed defenses (Carlini & Wagner, 2016, arXiv:1608.04644).

If you work in adversarial ML, AI red teaming, or ship any model that makes decisions with real consequences (fraud detection, content moderation, autonomous perception, biometric auth), you need to be able to construct these attacks yourself — not just cite that they exist. This lesson builds FGSM and PGD from the update rule, with no attack library involved.

The Math: FGSM in One Line

FGSM answers a specific optimization question: given a classifier with loss function J(theta, x, y), and a fixed perturbation budget eps measured in the L_inf norm (the max absolute change allowed to any single pixel), what perturbation delta maximizes the loss?

Take the first-order Taylor expansion of the loss around the input:

J(x + delta, y) ≈ J(x, y) + delta^T · grad_x J(theta, x, y)

Maximizing the linear term delta^T · grad_x J subject to ||delta||_inf ≤ eps has a closed-form solution: put the full budget eps on every coordinate, in the direction that matches the sign of that coordinate's gradient. That gives the FGSM update:

x' = x + eps * sign(grad_x J(theta, x, y))

Notice what's not in this formula: no iteration, no line search, no optimizer. FGSM is a single forward pass to get the loss, a single backward pass to get the gradient, and one elementwise sign() — that's the entire attack. It's cheap enough to run against every example in a batch simultaneously, which is why it's still used for adversarial training (generate attacks during training, not just for evaluation) even though it's a weaker attack than PGD.

The reason to use sign(grad) instead of the raw gradient is worth sitting with: under an L_inf constraint, the direction that maximizes a linear function of delta is the vertex of the L_inf ball in the gradient's direction — and the vertices of an L_inf ball are exactly the points where every coordinate is +eps or -eps. sign(grad) picks out which vertex.

Accuracy is not robustness
A model can reach 100% test accuracy and still have an attack success rate near 100% at a perturbation budget invisible to a human eye. These are different axes entirely: accuracy measures performance on the data distribution the model was evaluated on; robustness measures performance in a worst-case neighborhood around each point. Reporting clean accuracy alone, with no adversarial evaluation, tells you nothing about how a model behaves under attack — and in security-sensitive applications (fraud models, content filters, biometric systems), that gap is exactly what an adversary will look for.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers PGD: FGSM Iterated Properly, Carlini & Wagner: Optimization-Based Attacks, Black-Box Transferability and Decision Boundary Attacks, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.