Phase 3: Adversarial Machine Learning · 60 min · PyTorch · Adversarial Robustness Toolbox

Adversarial Defenses & Tooling — ART, CleverHans, Foolbox

A defense that only survives the attack you tested it against isn't a defense — it's a demo.

Hiring signal: Adversarial ML and AI red team roles expect fluency with the standard defense toolkit (IBM's Adversarial Robustness Toolbox, CleverHans, Foolbox) and the ability to correctly distinguish empirical defenses (adversarial training) from certified defenses (randomized smoothing) — a distinction that comes up directly in interviews at labs and security vendors evaluating whether a claimed defense actually holds up against an adaptive attacker.

What you will learn

Implement Madry-style adversarial training with ART's AdversarialTrainerMadryPGD and quantify the robustness/clean-accuracy trade-off it produces
Explain randomized smoothing as a certified defense, including what 'certified radius' guarantees and why that guarantee differs fundamentally from an empirical defense's test-time results
Build a lightweight detection-based defense using smoothed-vote margin, and explain why adversarial inputs produce less decisive votes than clean inputs
Choose between ART, CleverHans, and Foolbox for a given adversarial ML task based on their design goals (production toolkit vs. research reference implementations vs. framework-agnostic attack API)

The Problem

The evasion attacks lesson in this phase covered FGSM, PGD, and Carlini & Wagner — three ways to turn a correctly-classified input into a misclassified one with a perturbation an attacker controls. This lesson is about the other side of that fight: what a defender can actually do about it, and how to evaluate whether a defense claim is real or an artifact of testing against too weak an attack.

The history of adversarial defenses is, bluntly, a history of defenses that looked good against the attack the authors tested and collapsed against a stronger one. Athalye et al.'s 2018 survey Obfuscated Gradients Give a False Sense of Security broke 7 of 9 defenses accepted at a top ML conference that same year, simply by using stronger, adaptive attacks instead of the specific attack each paper's authors had benchmarked against. The pattern behind almost every one of those failures was gradient masking: a defense that makes gradients harder to compute or less informative (through non-differentiable preprocessing, randomization, or numerical instability) looks robust to gradient-based attacks like FGSM/PGD, without actually flattening the loss surface near real data points. A stronger attack — one that works around the obfuscation, or one that doesn't need gradients at all — cuts straight through.

This is why the field has converged on two categories of defense with very different guarantees, and why serious defense evaluation always uses standardized tooling rather than ad hoc scripts: empirical defenses (adversarial training) that are evaluated by testing against known strong attacks and reporting how much they help, and certified defenses (randomized smoothing, among others) that provide a mathematical guarantee — not just an empirical result — that no perturbation within a specified radius can change the prediction. The three libraries this lesson centers on — the Adversarial Robustness Toolbox (ART), CleverHans, and Foolbox — exist specifically so that "we tested our defense" means testing against a shared, vetted, adversarially-maintained set of attacks, not a bespoke implementation an author wrote once and never had scrutinized.

Adversarial Training: Make the Attack Part of Training

The most widely deployed empirical defense is deceptively simple: instead of training only on clean data, train on adversarially perturbed data, generated fresh at every step. Madry et al.'s formulation (2017, arXiv:1706.06083) frames this as a min-max optimization —

min_theta  E_(x,y)~D [ max_{||delta||_inf <= eps} J(theta, x + delta, y) ]

— the inner maximization finds the worst-case perturbation for the current model (this is exactly the PGD attack from the evasion lesson), and the outer minimization updates the model's weights to do well against that worst case. In practice this means: every training batch, run PGD against the current model to generate adversarial examples, then take a gradient step on the loss computed on those adversarial examples instead of (or in addition to) the clean ones. ART implements this directly as AdversarialTrainerMadryPGD, which wraps a classifier and handles the "attack, then train on the attack" loop for you.

The trade-off is real and shows up in every honest adversarial training result: robustness against the attack budget trained against comes at some cost to clean accuracy, because the model's decision boundary is now shaped to be locally flat around every training point within eps, which necessarily uses model capacity that could otherwise fit the clean data more tightly. There is no adversarial training result that gives you "more robust and equally accurate" for free — every reported result is a point on that trade-off curve, and the honest way to report a defense is to show both numbers, at a stated eps, not just the robustness number in isolation.

An empirical defense is only as strong as the attack you tested it against
Adversarial training with AdversarialTrainerMadryPGD at eps=0.75 reduces this lesson's PGD attack success rate meaningfully — but that number is only valid against attacks at or below eps=0.75, and only against the specific attack family (PGD) tested. A model adversarially trained against PGD is not automatically robust against a different attack family, a larger eps, or an adaptive attacker who knows the defense was PGD-trained and designs around it. This is exactly the failure mode Athalye et al. documented at scale: always report the attack, the eps, and the number of attack iterations/restarts used to produce a robustness claim, and treat any defense claim that omits those details with suspicion.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Certified Defenses: Randomized Smoothing, Detection-Based Defenses and Their Limits, Choosing Between ART, CleverHans, and Foolbox, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.