The Problem
In 2014, Ian Goodfellow and coauthors showed something that unsettled the entire field of deep learning: take an image a convolutional network classifies correctly with high confidence, add a perturbation so small that a human can't perceive it, and the network's prediction flips — often to a wildly wrong class, and often with higher confidence than the original correct prediction. Their paper, Explaining and Harnessing Adversarial Examples (Goodfellow et al., 2014, arXiv:1412.6572), introduced both the phenomenon and the first fast, principled way to construct it: the Fast Gradient Sign Method (FGSM).
This is not a training bug that better data fixes. It's a structural property of how high-dimensional linear (and near-linear) decision boundaries behave: in a space with thousands of pixels, a tiny, coordinated push in every dimension along the gradient direction adds up to a large change in the model's output, even though each individual pixel changes by an amount too small to notice. Three years later, Madry et al. formalized the strongest practical version of this attack — Projected Gradient Descent (PGD) — and proposed using it as the standard yardstick for adversarial robustness (Madry et al., 2017, arXiv:1706.06083). Around the same time, Carlini & Wagner showed that if you frame the problem as an optimization rather than a fixed-budget search, you can find adversarial perturbations with dramatically smaller norms than FGSM/PGD ever find — which made C&W the standard attack for stress-testing claimed defenses (Carlini & Wagner, 2016, arXiv:1608.04644).
If you work in adversarial ML, AI red teaming, or ship any model that makes decisions with real consequences (fraud detection, content moderation, autonomous perception, biometric auth), you need to be able to construct these attacks yourself — not just cite that they exist. This lesson builds FGSM and PGD from the update rule, with no attack library involved.
The math: FGSM in one line
FGSM answers a specific optimization question: given a classifier with loss function J(theta, x, y), and a fixed perturbation budget eps measured in the L_inf norm (the max absolute change allowed to any single pixel), what perturbation delta maximizes the loss?
Take the first-order Taylor expansion of the loss around the input:
J(x + delta, y) ≈ J(x, y) + delta^T · grad_x J(theta, x, y)
Maximizing the linear term delta^T · grad_x J subject to ||delta||_inf ≤ eps has a closed-form solution: put the full budget eps on every coordinate, in the direction that matches the sign of that coordinate's gradient. That gives the FGSM update:
x' = x + eps * sign(grad_x J(theta, x, y))
Notice what's not in this formula: no iteration, no line search, no optimizer. FGSM is a single forward pass to get the loss, a single backward pass to get the gradient, and one elementwise sign() — that's the entire attack. It's cheap enough to run against every example in a batch simultaneously, which is why it's still used for adversarial training (generate attacks during training, not just for evaluation) even though it's a weaker attack than PGD.
The reason to use sign(grad) instead of the raw gradient is worth sitting with: under an L_inf constraint, the direction that maximizes a linear function of delta is the vertex of the L_inf ball in the gradient's direction — and the vertices of an L_inf ball are exactly the points where every coordinate is +eps or -eps. sign(grad) picks out which vertex.
Accuracy is not robustness
A model can reach 100% test accuracy and still have an attack success rate near 100% at a perturbation budget invisible to a human eye. These are different axes entirely: accuracy measures performance on the data distribution the model was evaluated on; robustness measures performance in a worst-case neighborhood around each point. Reporting clean accuracy alone, with no adversarial evaluation, tells you nothing about how a model behaves under attack — and in security-sensitive applications (fraud models, content filters, biometric systems), that gap is exactly what an adversary will look for.
PGD: FGSM iterated properly
FGSM takes one big, linear step. But loss surfaces are not linear — the direction that looks best from a first-order approximation at x is often not the direction that maximizes loss once you've actually moved. PGD fixes this by taking many small steps, each locally following the same sign-of-gradient logic as FGSM, and projecting back onto the allowed L_inf ball after every step so the total perturbation never exceeds the budget:
x_0' = x + U(-eps, eps) (random start inside the L_inf ball)
x_t+1' = Proj_{B(x, eps)}( x_t' + alpha * sign(grad_x J(theta, x_t', y)) )
Here alpha is a small per-step size (typically eps / 4 to eps), steps is the number of iterations, and Proj_{B(x, eps)} clips the perturbation back into [-eps, eps] around the original x after every step — not just at the end. That distinction matters: without per-step projection, the perturbation can drift arbitrarily far from x over many iterations before a single final clip drags it back, which produces a much weaker, more distorted attack than repeatedly re-centering on x at every step.
The random start matters too. Starting PGD exactly at x risks the local search getting stuck near a point where the gradient happens to be small or momentarily cancels out. Starting from a random point inside the eps-ball, and — in a full robustness evaluation — running PGD multiple times from different random starts, is what Madry et al. mean when they define a model as empirically PGD-robust: it has to resist many independent attempts, not survive one lucky run.
PGD strictly dominates FGSM in attack strength at the same eps, because FGSM is just PGD with steps=1 and no projection loop needed (there's nowhere to project to after one step). Every additional step gives the optimizer another chance to correct course toward a stronger local perturbation.
You benchmark FGSM and PGD against the same classifier at eps=0.2 (L_inf) and get ASR=18% for FGSM but ASR=50% for PGD. A colleague argues this means your PGD implementation must have a bug, since both attacks use the same eps budget and the same gradient. What's the correct explanation?
This is the expected and well-documented behavior, not a bug. FGSM's one-shot linear approximation is accurate only in a small neighborhood of x; loss surfaces for real classifiers are not globally linear, so the single FGSM step often overshoots or undershoots the true worst-case direction within the eps ball. PGD's iterative local search, refined over multiple steps and started from a random point, explores the eps ball far more thoroughly and consistently finds a stronger adversarial point at the same budget. This is exactly why PGD, not FGSM, is treated as the standard attack for evaluating whether a defense actually works — a defense that only stops FGSM but not PGD is not meaningfully robust.
Carlini & wagner: optimization-based attacks
FGSM and PGD both fix the perturbation budget eps and ask "what's the worst perturbation I can make within this budget?" The Carlini & Wagner (C&W) attack inverts the question: fix the goal of causing misclassification, and search for the smallest perturbation (typically measured in L2) that achieves it. The attack solves:
minimize ||delta||_p + c * f(x + delta)
where f is a margin-based objective built from the model's logits: f becomes negative (satisfied) once the logit for the target class exceeds the logit for the true class by some confidence margin kappa, and stays positive (unsatisfied) otherwise. The constant c trades off perturbation size against attack success — C&W typically searches over c with binary search to find the smallest value that still produces a successful attack. To keep x + delta inside the valid pixel range without explicit box constraints, C&W substitutes a change of variables through tanh, which is solved with gradient-based optimizers like Adam rather than a single sign-of-gradient step.
The trade-off is cost: C&W runs a full optimization loop (often hundreds of Adam steps with an inner binary search over c) per example, versus FGSM's single step or PGD's handful of steps. That expense is exactly why C&W isn't the attack you run across a full test set for a quick robustness number — it's the attack you run against a small, targeted set of examples when you need to know the true minimum-norm adversarial distance, most commonly to check whether a defense that claims robustness against FGSM/PGD actually holds up against a stronger, budget-free search. A defense that passes an FGSM/PGD evaluation but collapses under C&W has likely learned to defeat the specific search pattern of gradient-sign attacks (a phenomenon known as gradient masking) rather than genuinely flattening the loss surface near real data points.
Black-box transferability and decision boundary attacks
Everything above assumes white-box access: you can compute grad_x J because you have the model's weights. Real attacks against production APIs usually don't have that. Two families of attack fill this gap:
Transfer attacks. Train or obtain a surrogate model for a similar task, run white-box FGSM/PGD against the surrogate, and submit the resulting adversarial examples to the real target. This works more often than intuition suggests — adversarial perturbations exploit properties of the data distribution and general architecture family, not just one specific set of weights, so perturbations crafted on one model frequently transfer to a different model solving the same task. Attackers with API-only access (label or confidence score only) routinely use this as step one.
Decision boundary / query-based attacks. When no surrogate is available or transfer fails, attacks like the Boundary Attack or HopSkipJump start from a large, obviously-adversarial perturbation and iteratively shrink it while repeatedly querying the target model's decision boundary, using only the returned label (not gradients) to guide the search. These need far more queries than a white-box attack, which is exactly why query-rate monitoring is a practical, if imperfect, detection signal for this attack family in production.
A defense paper reports that a new training technique reduces FGSM/PGD attack success rate from 80% to 2%, and the authors claim the model is now robust. What's the single most important follow-up test before accepting that claim, and why?
Low ASR against FGSM/PGD is necessary but not sufficient evidence of robustness, because both attacks share a specific search pattern (gradient-sign steps within a fixed L_inf budget) that a defense can learn to specifically defeat — a failure mode called gradient masking — without genuinely removing nearby adversarial points. Carlini & Wagner's optimization-based attack doesn't share that search pattern: it directly searches for the minimum-norm perturbation that flips the prediction, with no fixed step structure to mask against. A defense that collapses under C&W after looking robust against FGSM/PGD has been "solving the benchmark" rather than solving the underlying vulnerability — this is exactly the failure mode the original C&W paper was written to expose in prior published defenses.
Build
Build it
Before the full PyTorch version, here's the FGSM/PGD update rule illustrated on a toy 1D "model": a simple linear scoring function, with the gradient computed by hand instead of autograd, so you can see the sign-of-gradient step in isolation.
import random
random.seed(0)
# A toy linear "classifier": score(x) = sum(w_i * x_i). Positive score = class 1.
weights = [0.8, -0.5, 1.2, -0.3, 0.6]
def score(x):
return sum(w * xi for w, xi in zip(weights, x))
def grad_of_loss(x, true_label):
# For a linear model with hinge-style loss, d(loss)/dx_i = -label * w_i
return [-true_label * w for w in weights]
def fgsm_step(x, true_label, eps):
grad = grad_of_loss(x, true_label)
return [xi + eps * (1 if g > 0 else -1 if g < 0 else 0) for xi, g in zip(x, grad)]
def pgd_attack(x, true_label, eps, alpha, steps):
x_adv = [xi + random.uniform(-eps, eps) for xi in x]
for _ in range(steps):
grad = grad_of_loss(x_adv, true_label)
x_adv = [xi + alpha * (1 if g > 0 else -1 if g < 0 else 0) for xi, g in zip(x_adv, grad)]
# project back into the eps-ball around the original x
x_adv = [max(min(a, o + eps), o - eps) for a, o in zip(x_adv, x)]
return x_adv
x = [1.0, 0.5, -0.2, 0.8, -0.6]
true_label = 1
eps = 0.3
print(f"Original score: {score(x):.3f} (label={true_label})")
x_fgsm = fgsm_step(x, true_label, eps)
print(f"FGSM adversarial score: {score(x_fgsm):.3f}")
x_pgd = pgd_attack(x, true_label, eps, alpha=eps / 4, steps=10)
print(f"PGD adversarial score: {score(x_pgd):.3f}")
print("\nPGD pushes the score down further than FGSM at the same eps budget, "
"because it iterates instead of taking one linear step.")
The code artifact lesson_03_01_evasion_attacks.py implements FGSM and PGD entirely from scratch against a small CNN, with no ART/CleverHans/Foolbox dependency, so you can see every line of the attack math:
- A synthetic four-class shape dataset (generated in-code, no download) and a
TinyCNN classifier trained to near-perfect clean accuracy in a few seconds on CPU fgsm_attack(): the exact one-line update rule from the FGSM sectionpgd_attack(): iterative FGSM with random start and per-step L_inf projection- An evaluation harness that reports Attack Success Rate, mean L_inf, and mean L2 perturbation norm for both attacks across a sweep of epsilon budgets
"""
lesson_03_01_evasion_attacks.py
FGSM and PGD evasion attacks implemented from scratch (no ART/CleverHans/Foolbox)
against a small CNN trained on a synthetic image-classification task.
We generate our own tiny dataset instead of downloading MNIST so this script
trains and attacks in well under a minute on a laptop CPU. The task is
deliberately simple (four separable shape classes) so the classifier reaches
high clean accuracy quickly -- what matters here is the attack math, not the
task difficulty.
Attacks implemented:
- FGSM (Goodfellow et al., 2014): https://arxiv.org/abs/1412.6572
x' = x + eps * sign(grad_x J(theta, x, y))
- PGD (Madry et al., 2017): https://arxiv.org/abs/1706.06083
x_0' = x + U(-eps, eps) (random start inside the L_inf ball)
x_t+1' = Proj_{B(x, eps)}( x_t' + alpha * sign(grad_x J(theta, x_t', y)) )
Metrics reported for each attack / epsilon:
- Attack Success Rate (ASR): fraction of correctly-classified clean examples
that the model misclassifies after perturbation
- Mean L_inf norm of the perturbation (should be <= eps by construction)
- Mean L2 norm of the perturbation
Usage:
python lesson_03_01_evasion_attacks.py
Requirements:
pip install torch
"""
from __future__ import annotations
import time
from dataclasses import dataclass
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
DEVICE = torch.device("cpu")
IMG_SIZE = 20
NUM_CLASSES = 4
# ---------------------------------------------------------------------------
# Synthetic dataset: four separable shape classes on a 20x20 canvas
# ---------------------------------------------------------------------------
def _base_shape(label: int, size: int = IMG_SIZE) -> torch.Tensor:
img = torch.zeros(size, size)
mid = size // 2
if label == 0: # horizontal stripe
img[mid - 2 : mid + 2, :] = 1.0
elif label == 1: # vertical stripe
img[:, mid - 2 : mid + 2] = 1.0
elif label == 2: # diagonal stripe
for i in range(size):
for j in range(max(0, i - 2), min(size, i + 3)):
img[i, j] = 1.0
else: # filled circle
yy, xx = torch.meshgrid(torch.arange(size), torch.arange(size), indexing="ij")
dist = ((yy - mid) ** 2 + (xx - mid) ** 2).float().sqrt()
img[dist <= mid * 0.6] = 1.0
return img
def make_dataset(n_per_class: int, noise_std: float = 0.35) -> tuple[torch.Tensor, torch.Tensor]:
xs, ys = [], []
for label in range(NUM_CLASSES):
base = _base_shape(label)
for _ in range(n_per_class):
noisy = (base + torch.randn(IMG_SIZE, IMG_SIZE) * noise_std).clamp(0.0, 1.0)
xs.append(noisy)
ys.append(label)
x = torch.stack(xs).unsqueeze(1) # (N, 1, H, W)
y = torch.tensor(ys, dtype=torch.long)
perm = torch.randperm(len(y))
return x[perm], y[perm]
# ---------------------------------------------------------------------------
# Small CNN classifier
# ---------------------------------------------------------------------------
class TinyCNN(nn.Module):
def __init__(self, num_classes: int = NUM_CLASSES):
super().__init__()
self.conv1 = nn.Conv2d(1, 8, kernel_size=3, padding=1)
self.conv2 = nn.Conv2d(8, 16, kernel_size=3, padding=1)
self.pool = nn.MaxPool2d(2)
self.fc = nn.Linear(16 * 5 * 5, num_classes)
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.pool(F.relu(self.conv1(x))) # (B, 8, 10, 10)
x = self.pool(F.relu(self.conv2(x))) # (B, 16, 5, 5)
x = x.flatten(1)
return self.fc(x)
def train_model(model: nn.Module, x_train: torch.Tensor, y_train: torch.Tensor, epochs: int = 8) -> None:
optimizer = torch.optim.Adam(model.parameters(), lr=3e-3)
model.train()
for epoch in range(epochs):
optimizer.zero_grad()
logits = model(x_train)
loss = F.cross_entropy(logits, y_train)
loss.backward()
optimizer.step()
if epoch == 0 or (epoch + 1) % 4 == 0:
acc = (logits.argmax(1) == y_train).float().mean().item()
print(f" epoch {epoch + 1:2d}/{epochs} loss={loss.item():.4f} train_acc={acc:.3f}")
@torch.no_grad()
def accuracy(model: nn.Module, x: torch.Tensor, y: torch.Tensor) -> float:
model.eval()
return (model(x).argmax(1) == y).float().mean().item()
# ---------------------------------------------------------------------------
# FGSM -- single-step, first-order attack
# ---------------------------------------------------------------------------
def fgsm_attack(model: nn.Module, x: torch.Tensor, y: torch.Tensor, eps: float) -> torch.Tensor:
"""x' = x + eps * sign(grad_x J(theta, x, y)), then clip to the valid pixel range."""
model.eval()
x_adv = x.clone().detach().requires_grad_(True)
loss = F.cross_entropy(model(x_adv), y)
grad = torch.autograd.grad(loss, x_adv)[0]
x_adv = x_adv.detach() + eps * grad.sign()
return x_adv.clamp(0.0, 1.0)
# ---------------------------------------------------------------------------
# PGD -- iterative FGSM with projection back onto the L_inf ball around x
# ---------------------------------------------------------------------------
def pgd_attack(
model: nn.Module,
x: torch.Tensor,
y: torch.Tensor,
eps: float,
alpha: float,
steps: int = 10,
random_start: bool = True,
) -> torch.Tensor:
model.eval()
if random_start:
delta = (torch.rand_like(x) * 2 - 1) * eps
else:
delta = torch.zeros_like(x)
x_adv = (x + delta).clamp(0.0, 1.0).detach()
for _ in range(steps):
x_adv.requires_grad_(True)
loss = F.cross_entropy(model(x_adv), y)
grad = torch.autograd.grad(loss, x_adv)[0]
with torch.no_grad():
x_adv = x_adv + alpha * grad.sign()
# Project the perturbation back into the eps-ball around the original x
perturbation = (x_adv - x).clamp(-eps, eps)
x_adv = (x + perturbation).clamp(0.0, 1.0)
return x_adv.detach()
# ---------------------------------------------------------------------------
# Evaluation harness
# ---------------------------------------------------------------------------
@dataclass
class AttackReport:
name: str
eps: float
attack_success_rate: float
mean_linf: float
mean_l2: float
def evaluate_attack(
model: nn.Module,
x: torch.Tensor,
y: torch.Tensor,
x_adv: torch.Tensor,
name: str,
eps: float,
) -> AttackReport:
model.eval()
with torch.no_grad():
clean_pred = model(x).argmax(1)
adv_pred = model(x_adv).argmax(1)
correctly_classified = clean_pred == y
n_correct = correctly_classified.sum().item()
if n_correct == 0:
asr = 0.0
else:
flipped = (adv_pred != y) & correctly_classified
asr = flipped.sum().item() / n_correct
delta = (x_adv - x).flatten(1)
mean_linf = delta.abs().max(dim=1).values.mean().item()
mean_l2 = delta.norm(dim=1).mean().item()
return AttackReport(name, eps, asr, mean_linf, mean_l2)
def print_report(report: AttackReport) -> None:
print(
f" {report.name:<10s} eps={report.eps:.3f} "
f"ASR={report.attack_success_rate:6.1%} "
f"mean L_inf={report.mean_linf:.4f} mean L2={report.mean_l2:.4f}"
)
def main() -> None:
print("EVASION ATTACKS: FGSM & PGD FROM SCRATCH")
print("=" * 60)
t0 = time.time()
x_train, y_train = make_dataset(n_per_class=150)
x_test, y_test = make_dataset(n_per_class=40)
print(f"Dataset: {len(x_train)} train / {len(x_test)} test, {NUM_CLASSES} classes")
model = TinyCNN()
print("\nTraining victim model...")
train_model(model, x_train, y_train, epochs=8)
clean_acc = accuracy(model, x_test, y_test)
print(f"\nClean test accuracy: {clean_acc:.1%}")
print("\nRunning attacks across perturbation budgets (eps in L_inf pixel units, [0,1] scale):")
epsilons = [0.05, 0.1, 0.2, 0.3, 0.4]
reports: list[AttackReport] = []
for eps in epsilons:
x_adv_fgsm = fgsm_attack(model, x_test, y_test, eps=eps)
reports.append(evaluate_attack(model, x_test, y_test, x_adv_fgsm, "FGSM", eps))
x_adv_pgd = pgd_attack(model, x_test, y_test, eps=eps, alpha=eps / 4, steps=10)
reports.append(evaluate_attack(model, x_test, y_test, x_adv_pgd, "PGD", eps))
print()
for r in reports:
print_report(r)
print(
"\nTakeaway: PGD's multi-step, randomly-initialized search consistently finds a "
"higher attack success rate than single-step FGSM at the same eps budget -- this is "
"why PGD is the standard 'strong first-order attack' used to benchmark robustness "
"(Madry et al. call a model PGD-robust only if it withstands many random restarts)."
)
print(f"\nDone in {time.time() - t0:.1f}s")
if __name__ == "__main__":
main()