Phase 3: Adversarial Machine Learning · 60 min · PyTorch · Adversarial Robustness Toolbox
The Problem
The evasion attacks lesson in this phase covered FGSM, PGD, and Carlini & Wagner — three ways to turn a correctly-classified input into a misclassified one with a perturbation an attacker controls. This lesson is about the other side of that fight: what a defender can actually do about it, and how to evaluate whether a defense claim is real or an artifact of testing against too weak an attack.
The history of adversarial defenses is, bluntly, a history of defenses that looked good against the attack the authors tested and collapsed against a stronger one. Athalye et al.'s 2018 survey Obfuscated Gradients Give a False Sense of Security broke 7 of 9 defenses accepted at a top ML conference that same year, simply by using stronger, adaptive attacks instead of the specific attack each paper's authors had benchmarked against. The pattern behind almost every one of those failures was gradient masking: a defense that makes gradients harder to compute or less informative (through non-differentiable preprocessing, randomization, or numerical instability) looks robust to gradient-based attacks like FGSM/PGD, without actually flattening the loss surface near real data points. A stronger attack — one that works around the obfuscation, or one that doesn't need gradients at all — cuts straight through.
This is why the field has converged on two categories of defense with very different guarantees, and why serious defense evaluation always uses standardized tooling rather than ad hoc scripts: empirical defenses (adversarial training) that are evaluated by testing against known strong attacks and reporting how much they help, and certified defenses (randomized smoothing, among others) that provide a mathematical guarantee — not just an empirical result — that no perturbation within a specified radius can change the prediction. The three libraries this lesson centers on — the Adversarial Robustness Toolbox (ART), CleverHans, and Foolbox — exist specifically so that "we tested our defense" means testing against a shared, vetted, adversarially-maintained set of attacks, not a bespoke implementation an author wrote once and never had scrutinized.
Adversarial training: make the attack part of training
The most widely deployed empirical defense is deceptively simple: instead of training only on clean data, train on adversarially perturbed data, generated fresh at every step. Madry et al.'s formulation (2017, arXiv:1706.06083) frames this as a min-max optimization —
min_theta E_(x,y)~D [ max_{||delta||_inf <= eps} J(theta, x + delta, y) ]
— the inner maximization finds the worst-case perturbation for the current model (this is exactly the PGD attack from the evasion lesson), and the outer minimization updates the model's weights to do well against that worst case. In practice this means: every training batch, run PGD against the current model to generate adversarial examples, then take a gradient step on the loss computed on those adversarial examples instead of (or in addition to) the clean ones. ART implements this directly as AdversarialTrainerMadryPGD, which wraps a classifier and handles the "attack, then train on the attack" loop for you.
The trade-off is real and shows up in every honest adversarial training result: robustness against the attack budget trained against comes at some cost to clean accuracy, because the model's decision boundary is now shaped to be locally flat around every training point within eps, which necessarily uses model capacity that could otherwise fit the clean data more tightly. There is no adversarial training result that gives you "more robust and equally accurate" for free — every reported result is a point on that trade-off curve, and the honest way to report a defense is to show both numbers, at a stated eps, not just the robustness number in isolation.
An empirical defense is only as strong as the attack you tested it against
Adversarial training with AdversarialTrainerMadryPGD at eps=0.75 reduces this lesson's PGD attack success rate meaningfully — but that number is only valid against attacks at or below eps=0.75, and only against the specific attack family (PGD) tested. A model adversarially trained against PGD is not automatically robust against a different attack family, a larger eps, or an adaptive attacker who knows the defense was PGD-trained and designs around it. This is exactly the failure mode Athalye et al. documented at scale: always report the attack, the eps, and the number of attack iterations/restarts used to produce a robustness claim, and treat any defense claim that omits those details with suspicion.
Certified defenses: randomized smoothing
Randomized smoothing (Cohen, Rosenfeld & Kolter, 2019) takes a fundamentally different approach: instead of trying to train a model whose loss surface is empirically flat near training points, it builds a new, "smoothed" classifier out of any base classifier f, defined as:
g(x) = argmax_c P_{delta ~ N(0, sigma^2 I)} [ f(x + delta) = c ]
In words: to classify x, sample many random Gaussian perturbations of x, run each through the base classifier, and take the majority vote. This alone is already a stronger empirical defense than an unmodified classifier (an attacker fooling one specific point no longer fools the vote over hundreds of noisy neighbors of that point), but the real payoff is that it comes with a provable, closed-form certified radius: for any input where the winning class gets a vote fraction p_A, every point within L2 distance sigma * Phi^-1(p_A) of x (where Phi^-1 is the inverse Gaussian CDF) is guaranteed to receive the same smoothed prediction g(x), for any perturbation whatsoever within that radius — not just perturbations found by a specific attack algorithm.
That "for any perturbation whatsoever" clause is the entire point, and it's why certified defenses matter even though they're more expensive and give guarantees over a smaller radius than adversarial training's empirical results often appear to cover: a certified radius is true even against attacks that don't exist yet. An empirical defense's robustness number is only as good as the attacks used to produce it; a certified radius doesn't need an attack to be evaluated at all — it's a mathematical consequence of the smoothing construction and the observed vote margin.
The cost is real too: classifying one input now requires hundreds of forward passes (one per noise sample) instead of one, and the base classifier must be trained with noise augmentation matching the smoothing sigma, or the vote will be unstable and the certified radius will be small and useless.
A team ships a randomized-smoothing classifier and reports "certified robust accuracy: 92% at radius 1.5." A colleague asks whether this means the model resists all adversarial attacks with perturbation norm larger than 1.5. What is the correct interpretation?
The certified radius is precisely scoped: it guarantees that no perturbation with L2 norm less than or equal to the certified radius (1.5, in this example) can change the smoothed classifier's prediction on that input, for 92% of inputs at that radius. It says absolutely nothing about perturbations larger than 1.5 -- an attacker with a bigger budget may well find a successful attack, and the certificate does not claim otherwise. This distinction -- a certificate is a guarantee within a stated radius, not an unconditional robustness claim -- is exactly what separates a certified defense's honest reporting from an empirical defense's often-overstated one, and is a common point of confusion (or misrepresentation) when certified defenses are marketed.
Detection-based defenses and their limits
A third defense family doesn't try to classify adversarial inputs correctly at all — it tries to notice that an input is adversarial and refuse to answer, escalate to a human, or apply a different (more expensive, more robust) model. Detection is attractive because it doesn't force the robustness/clean-accuracy trade-off that adversarial training does on every single prediction, only on the (hopefully rare) inputs flagged as suspicious.
A practical detection signal that reuses the same machinery as randomized smoothing: PGD and similar gradient attacks push an input toward a decision boundary by construction, since that's what causes misclassification. An input near a decision boundary produces noisy neighbor predictions that split across two or more classes, so the smoothed-vote margin (the gap between the top and second class's vote share) is small. Clean inputs, sitting confidently inside one class's region, produce lopsided votes with a wide margin. Flagging inputs with a small vote margin is a detector that needs no labeled adversarial examples to calibrate — it's a direct consequence of what the attack is geometrically trying to do.
The limits matter as much as the mechanism: like every defense in this lesson, detection is defeatable by an adaptive attacker who knows the detector exists and optimizes against it directly (e.g., adding a term to the attack objective that also keeps the vote margin high). Detection-based defenses are a useful additional layer, not a substitute for evaluating a model's underlying robustness — the same lesson that runs through adversarial training and certified defenses alike: a defense is only as credible as the adaptive-attacker evaluation it has survived.
A team deploys the vote-margin detector from this lesson and reports "flags 95% of PGD adversarial inputs while flagging only 3% of clean inputs — detection solved." What is the key weakness in declaring this defense solved?
This is the same lesson Athalye et al.'s survey established for gradient masking: a defense number is only meaningful against the attack it was measured against. The 95%/3% split describes performance against plain PGD, an attacker with no knowledge of the detector. An adaptive attacker who knows a vote-margin detector is in place can explicitly optimize to keep the smoothed vote decisive (high margin) while still crossing the decision boundary, since nothing in the standard PGD objective prevents that. Declaring the defense "solved" from a non-adaptive number repeats exactly the mistake that got 7 of 9 defenses broken in the Athalye et al. paper.
Choosing between ART, CleverHans, and foolbox
All three libraries implement the same core attack families (FGSM, PGD, C&W, and more), but they exist for different reasons and a working AI security engineer should know which to reach for:
| Library | Design goal | When to use it |
|---|
| ART (IBM) | Production-grade, broad framework support (PyTorch, TensorFlow, scikit-learn, and more), includes attacks, defenses (adversarial training, certified defenses, preprocessing), and evaluation metrics in one package | Building or evaluating a defense pipeline end-to-end, especially outside of pure research (this lesson's code uses ART for exactly this reason) |
| CleverHans | Reference research implementations maintained by the original authors of many core attacks, prioritizing faithfulness to published papers | Reproducing a specific paper's attack exactly, or citing a canonical implementation in a research or red-team report |
| Foolbox | A single, framework-agnostic attack API with strong support for finding minimum-perturbation attacks efficiently (its name for what C&W-style attacks compute) | Benchmarking the true minimum adversarial distance across many models or frameworks with a consistent interface |
In practice, ART is the right default for building and shipping a defense pipeline (it's what this lesson's code artifact uses for both the attack and the adversarial training loop), while CleverHans and Foolbox are the right tools when you specifically need research-grade reproducibility or cross-framework minimum-distance benchmarking.
Build
Build it
The code artifact lesson_03_05_art_defenses.py implements all three defense families from this lesson against a small PyTorch MLP on a synthetic tabular dataset, using the real adversarial-robustness-toolbox package when it's installed, with an automatic, fully-functional pure-PyTorch fallback when it isn't:
- Adversarial training: trains an undefended baseline and a Madry-style adversarially trained model (via ART's
AdversarialTrainerMadryPGD, or a hand-written PGD training loop in the fallback path), then measures clean accuracy and PGD attack success rate for both - Randomized smoothing: trains a noise-augmented classifier, implements the majority-vote smoothed prediction and Cohen et al.'s certified L2 radius formula from scratch, and reports certified accuracy, mean certified radius, and smoothed ASR under the same PGD attack
- Detection: implements the vote-margin detector and compares flagged fractions on clean vs. adversarial inputs
- Everything runs on a synthetic Gaussian-blob dataset (no downloads) and finishes in well under a minute on CPU
"""
lesson_03_05_art_defenses.py
Adversarial defenses, hands-on: adversarial training, randomized smoothing
as a certified defense, and a simple detection-based defense, evaluated
against a PGD evasion attack on a small PyTorch MLP with a tiny synthetic
tabular dataset.
Primary path: uses the real `adversarial-robustness-toolbox` (ART) package
- https://github.com/Trusted-AI/adversarial-robustness-toolbox
for the PGD attack and for Madry-style adversarial training
(art.defences.trainer.AdversarialTrainerMadryPGD). ART is IBM's
production-grade library for building and evaluating adversarial defenses,
and is the tool referenced throughout this course's defense-side lessons,
alongside CleverHans (https://github.com/cleverhans-lab/cleverhans) and
Foolbox (https://github.com/bethgelab/foolbox), which cover the same attack
families with different design philosophies (CleverHans: reference research
implementations; Foolbox: a unified, framework-agnostic attack API).
Fallback path: if ART is not installed, this script automatically falls
back to a pure-PyTorch PGD attack and PGD-based adversarial training loop
that reproduce the same evaluation with no external dependency beyond
torch. The lesson content and the numbers you get are conceptually the
same either way -- only the implementation backing them differs.
What this script demonstrates, end to end:
1. Train an undefended victim model and measure its clean accuracy and
its Attack Success Rate (ASR) under PGD.
2. Adversarially train a second model of the same architecture (Madry
et al., 2017 -- https://arxiv.org/abs/1706.06083) and measure the
same two numbers, to quantify the robustness/clean-accuracy
trade-off directly.
3. Train a third model with Gaussian noise augmentation and evaluate it
under randomized smoothing (Cohen et al., 2019-style majority-vote
prediction with a certified L2 radius), demonstrating a certified
defense that gives a provable robustness guarantee within a radius,
as opposed to the purely empirical guarantee adversarial training
provides.
4. Implement a lightweight detection-based defense: flag inputs whose
smoothed-prediction vote is not decisive (no single class gets a
clear majority across noisy copies), which is a simple, real
proxy for "this input looks adversarial" that requires no labeled
adversarial examples to build.
Runs in well under a minute on a laptop CPU either with or without ART
installed, using a synthetic dataset generated in-code (no downloads).
Usage:
python lesson_03_05_art_defenses.py
Requirements:
pip install torch
pip install adversarial-robustness-toolbox # optional, recommended
pip install scipy # optional, for certified radius
"""
from __future__ import annotations
import time
from dataclasses import dataclass
import torch
import torch.nn as nn
import torch.nn.functional as F
torch.manual_seed(0)
NUM_FEATURES = 8
NUM_CLASSES = 4
CLASS_SEP = 3.0 # distance between per-class Gaussian centroids
EPS = 0.75 # L_inf perturbation budget for PGD
EPS_STEP = EPS / 5
PGD_STEPS = 10
SMOOTHING_SIGMA = 0.5 # Gaussian noise std used for training-with-noise and smoothing
try:
import art # noqa: F401
from art.estimators.classification import PyTorchClassifier
from art.attacks.evasion import ProjectedGradientDescent
from art.defences.trainer import AdversarialTrainerMadryPGD
HAVE_ART = True
except ImportError:
HAVE_ART = False
print(
"adversarial-robustness-toolbox (ART) is not installed -- falling back to a "
"pure-PyTorch PGD attack and PGD adversarial-training loop.\n"
"For the full ART-backed path (recommended): pip install adversarial-robustness-toolbox\n"
)
try:
from scipy.stats import norm as _scipy_norm
HAVE_SCIPY = True
except ImportError:
HAVE_SCIPY = False
print(
"scipy is not installed -- the randomized-smoothing certified radius will be "
"reported as 0.0 (majority-vote predictions still work).\n"
"For certified radii: pip install scipy\n"
)
# ---------------------------------------------------------------------------
# Ground-truth data distribution (synthetic, in-code -- no downloads)
# ---------------------------------------------------------------------------
def _class_centroid(label: int, num_features: int = NUM_FEATURES) -> torch.Tensor:
g = torch.Generator().manual_seed(1000 + label)
direction = torch.randn(num_features, generator=g)
direction = direction / direction.norm()
return direction * CLASS_SEP
def sample_records(n_per_class: int, noise_std: float = 0.7, seed: int | None = None) -> tuple[torch.Tensor, torch.Tensor]:
if seed is not None:
torch.manual_seed(seed)
xs, ys = [], []
for label in range(NUM_CLASSES):
centroid = _class_centroid(label)
noise = torch.randn(n_per_class, NUM_FEATURES) * noise_std
xs.append(centroid.unsqueeze(0) + noise)
ys.append(torch.full((n_per_class,), label, dtype=torch.long))
x = torch.cat(xs)
y = torch.cat(ys)
perm = torch.randperm(len(y))
return x[perm], y[perm]
# ---------------------------------------------------------------------------
# Model
# ---------------------------------------------------------------------------
class MLP(nn.Module):
def __init__(self, num_features: int = NUM_FEATURES, num_classes: int = NUM_CLASSES, hidden: int = 32):
super().__init__()
self.net = nn.Sequential(
nn.Linear(num_features, hidden),
nn.ReLU(),
nn.Linear(hidden, hidden),
nn.ReLU(),
nn.Linear(hidden, num_classes),
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)
@torch.no_grad()
def accuracy(model: nn.Module, x: torch.Tensor, y: torch.Tensor) -> float:
model.eval()
return (model(x).argmax(1) == y).float().mean().item()
# ---------------------------------------------------------------------------
# Pure-PyTorch PGD attack (used directly in the fallback path, and reused
# in the ART path to cross-check the fallback implementation is correct)
# ---------------------------------------------------------------------------
def pgd_attack_torch(
model: nn.Module, x: torch.Tensor, y: torch.Tensor, eps: float, alpha: float, steps: int = PGD_STEPS
) -> torch.Tensor:
"""x_0' = x + U(-eps, eps)
x_t+1' = Proj_{B(x, eps)}( x_t' + alpha * sign(grad_x J(theta, x_t', y)) )
(Madry et al., 2017: https://arxiv.org/abs/1706.06083)
"""
model.eval()
delta = (torch.rand_like(x) * 2 - 1) * eps
x_adv = (x + delta).detach()
for _ in range(steps):
x_adv.requires_grad_(True)
loss = F.cross_entropy(model(x_adv), y)
grad = torch.autograd.grad(loss, x_adv)[0]
with torch.no_grad():
x_adv = x_adv + alpha * grad.sign()
perturbation = (x_adv - x).clamp(-eps, eps)
x_adv = x + perturbation
return x_adv.detach()
@dataclass
class DefenseReport:
name: str
clean_accuracy: float
attack_success_rate: float
@torch.no_grad()
def attack_success_rate(model: nn.Module, x: torch.Tensor, y: torch.Tensor, x_adv: torch.Tensor) -> float:
model.eval()
clean_pred = model(x).argmax(1)
adv_pred = model(x_adv).argmax(1)
correctly_classified = clean_pred == y
n_correct = correctly_classified.sum().item()
if n_correct == 0:
return 0.0
flipped = (adv_pred != y) & correctly_classified
return flipped.sum().item() / n_correct
# ---------------------------------------------------------------------------
# Defense 1: adversarial training (undefended baseline vs. Madry-style AT)
# ---------------------------------------------------------------------------
def train_undefended(x_train: torch.Tensor, y_train: torch.Tensor, epochs: int = 80) -> nn.Module:
model = MLP()
optimizer = torch.optim.Adam(model.parameters(), lr=5e-3)
model.train()
for _ in range(epochs):
optimizer.zero_grad()
loss = F.cross_entropy(model(x_train), y_train)
loss.backward()
optimizer.step()
model.eval()
return model
def train_adversarial_art(x_train: torch.Tensor, y_train: torch.Tensor, epochs: int = 40):
"""Madry-style adversarial training via ART's AdversarialTrainerMadryPGD:
every minibatch is replaced with its PGD-perturbed version before the
gradient step, so the model is trained to be correct on worst-case
local perturbations, not just on clean data."""
model = MLP()
optimizer = torch.optim.Adam(model.parameters(), lr=5e-3)
classifier = PyTorchClassifier(
model=model,
loss=nn.CrossEntropyLoss(),
optimizer=optimizer,
input_shape=(NUM_FEATURES,),
nb_classes=NUM_CLASSES,
clip_values=(-10.0, 10.0),
)
trainer = AdversarialTrainerMadryPGD(
classifier, nb_epochs=epochs, eps=EPS, eps_step=EPS_STEP, max_iter=PGD_STEPS, batch_size=64
)
trainer.fit(x_train.numpy(), y_train.numpy())
return classifier
def train_adversarial_torch(x_train: torch.Tensor, y_train: torch.Tensor, epochs: int = 40) -> nn.Module:
"""Pure-PyTorch fallback for Madry-style adversarial training: generate
a fresh PGD adversarial batch every epoch and train on it directly."""
model = MLP()
optimizer = torch.optim.Adam(model.parameters(), lr=5e-3)
for _ in range(epochs):
model.train()
x_adv_train = pgd_attack_torch(model, x_train, y_train, EPS, EPS_STEP, steps=7)
optimizer.zero_grad()
loss = F.cross_entropy(model(x_adv_train), y_train)
loss.backward()
optimizer.step()
model.eval()
return model
def evaluate_adversarial_training(x_train: torch.Tensor, y_train: torch.Tensor, x_test: torch.Tensor, y_test: torch.Tensor) -> list[DefenseReport]:
reports = []
undefended = train_undefended(x_train, y_train)
x_adv_undef = pgd_attack_torch(undefended, x_test, y_test, EPS, EPS_STEP)
reports.append(
DefenseReport(
"undefended (no defense)",
accuracy(undefended, x_test, y_test),
attack_success_rate(undefended, x_test, y_test, x_adv_undef),
)
)
if HAVE_ART:
defended_classifier = train_adversarial_art(x_train, y_train)
defended_torch_model = defended_classifier.model # underlying nn.Module, for our own PGD + eval helpers
x_adv_def = pgd_attack_torch(defended_torch_model, x_test, y_test, EPS, EPS_STEP)
reports.append(
DefenseReport(
"adversarially trained (ART Madry PGD trainer)",
accuracy(defended_torch_model, x_test, y_test),
attack_success_rate(defended_torch_model, x_test, y_test, x_adv_def),
)
)
else:
defended_model = train_adversarial_torch(x_train, y_train)
x_adv_def = pgd_attack_torch(defended_model, x_test, y_test, EPS, EPS_STEP)
reports.append(
DefenseReport(
"adversarially trained (pure-PyTorch PGD fallback)",
accuracy(defended_model, x_test, y_test),
attack_success_rate(defended_model, x_test, y_test, x_adv_def),
)
)
return reports
# ---------------------------------------------------------------------------
# Defense 2: randomized smoothing (certified defense)
# ---------------------------------------------------------------------------
def train_noise_augmented(x_train: torch.Tensor, y_train: torch.Tensor, sigma: float, epochs: int = 80) -> nn.Module:
"""Randomized smoothing (Cohen et al., 2019) requires the base classifier
to be trained on Gaussian-noised inputs matching the smoothing sigma used
at inference time -- otherwise the majority vote over noisy copies is
just averaging a model that never learned to be robust to that noise."""
model = MLP()
optimizer = torch.optim.Adam(model.parameters(), lr=5e-3)
model.train()
for _ in range(epochs):
noisy_x = x_train + torch.randn_like(x_train) * sigma
optimizer.zero_grad()
loss = F.cross_entropy(model(noisy_x), y_train)
loss.backward()
optimizer.step()
model.eval()
return model
@dataclass
class SmoothingReport:
clean_certified_accuracy: float
mean_certified_radius: float
smoothed_asr_under_pgd: float
@torch.no_grad()
def smoothed_predict_and_certify(
model: nn.Module, x: torch.Tensor, sigma: float, n_samples: int, num_classes: int
) -> tuple[torch.Tensor, torch.Tensor]:
"""Randomized smoothing prediction: classify n_samples independently
Gaussian-noised copies of each input and take the majority vote. The
certified L2 radius around each correctly-and-confidently-classified
point follows Cohen et al.'s formula: radius = sigma * Phi^-1(p_A), where
p_A is a (here, simplified) estimate of the true probability the base
classifier assigns the winning class under the noise distribution --
any input within that radius of x is GUARANTEED to receive the same
smoothed prediction, regardless of what perturbation search an attacker
runs. This is what makes randomized smoothing a *certified* defense:
the guarantee holds for every possible attack, not just the ones tested.
"""
reps = x.unsqueeze(0).expand(n_samples, *x.shape)
noisy = (reps + torch.randn_like(reps) * sigma).reshape(-1, x.shape[-1])
preds = model(noisy).argmax(1).view(n_samples, x.shape[0])
votes = torch.zeros(x.shape[0], num_classes)
for c in range(num_classes):
votes[:, c] = (preds == c).float().sum(0)
top_class = votes.argmax(1)
top_count = votes.max(1).values
p_a_lower_bound = (top_count / n_samples).clamp(0.5 + 1e-6, 1.0 - 1e-6)
if HAVE_SCIPY:
radius = sigma * torch.tensor([_scipy_norm.ppf(p) for p in p_a_lower_bound.tolist()])
else:
radius = torch.zeros_like(p_a_lower_bound)
return top_class, radius
def evaluate_randomized_smoothing(x_train: torch.Tensor, y_train: torch.Tensor, x_test: torch.Tensor, y_test: torch.Tensor) -> SmoothingReport:
smoothed_model = train_noise_augmented(x_train, y_train, SMOOTHING_SIGMA)
preds, radii = smoothed_predict_and_certify(smoothed_model, x_test, SMOOTHING_SIGMA, n_samples=200, num_classes=NUM_CLASSES)
correct = preds == y_test
clean_certified_acc = correct.float().mean().item()
mean_radius = radii[correct].mean().item() if correct.any() else 0.0
# Attack the underlying base classifier directly (the strongest realistic
# threat model against a smoothed classifier is still a gradient attack
# on the base network), then check whether the smoothed vote still holds.
x_adv = pgd_attack_torch(smoothed_model, x_test, y_test, EPS, EPS_STEP)
adv_preds, _ = smoothed_predict_and_certify(smoothed_model, x_adv, SMOOTHING_SIGMA, n_samples=200, num_classes=NUM_CLASSES)
correctly_classified = preds == y_test
n_correct = correctly_classified.sum().item()
if n_correct == 0:
smoothed_asr = 0.0
else:
flipped = (adv_preds != y_test) & correctly_classified
smoothed_asr = flipped.sum().item() / n_correct
return SmoothingReport(clean_certified_acc, mean_radius, smoothed_asr)
# ---------------------------------------------------------------------------
# Defense 3: detection based on vote-margin (indecisive smoothed votes)
# ---------------------------------------------------------------------------
@dataclass
class DetectionReport:
stream_name: str
num_inputs: int
flagged_fraction: float
@torch.no_grad()
def vote_margin_fraction(
model: nn.Module, x: torch.Tensor, sigma: float, n_samples: int, num_classes: int, margin_threshold: float
) -> torch.Tensor:
"""A cheap detection signal: for genuine, confidently-classified inputs,
the smoothed vote should concentrate heavily on one class. Adversarial
inputs sitting near a decision boundary (which is where an attacker
wants them, to flip the base classifier's prediction as cheaply as
possible) produce noisy copies that land on both sides of the boundary,
so the vote margin between the top two classes shrinks. This needs no
labeled adversarial examples to build -- only the smoothing mechanism
already in place for Defense 2."""
reps = x.unsqueeze(0).expand(n_samples, *x.shape)
noisy = (reps + torch.randn_like(reps) * sigma).reshape(-1, x.shape[-1])
preds = model(noisy).argmax(1).view(n_samples, x.shape[0])
votes = torch.zeros(x.shape[0], num_classes)
for c in range(num_classes):
votes[:, c] = (preds == c).float().sum(0)
top2 = votes.topk(2, dim=1).values
margin = (top2[:, 0] - top2[:, 1]) / n_samples
return margin < margin_threshold
def evaluate_detection(model: nn.Module, x_clean: torch.Tensor, x_adv: torch.Tensor, margin_threshold: float = 0.3) -> list[DetectionReport]:
clean_flags = vote_margin_fraction(model, x_clean, SMOOTHING_SIGMA, 200, NUM_CLASSES, margin_threshold)
adv_flags = vote_margin_fraction(model, x_adv, SMOOTHING_SIGMA, 200, NUM_CLASSES, margin_threshold)
return [
DetectionReport("clean_inputs", x_clean.shape[0], clean_flags.float().mean().item()),
DetectionReport("pgd_adversarial_inputs", x_adv.shape[0], adv_flags.float().mean().item()),
]
def main() -> None:
t0 = time.time()
print("ADVERSARIAL DEFENSES: TRAINING, CERTIFICATION, AND DETECTION")
print("=" * 70)
print(f"ART available: {HAVE_ART} scipy available: {HAVE_SCIPY}")
x_train, y_train = sample_records(n_per_class=150, seed=42)
x_test, y_test = sample_records(n_per_class=60, seed=99)
print(f"Dataset: {len(x_train)} train / {len(x_test)} test, {NUM_CLASSES} classes, {NUM_FEATURES} features")
# --- Defense 1: adversarial training ---
print("\nDefense 1: adversarial training (undefended vs. Madry-style PGD training)")
print("-" * 70)
at_reports = evaluate_adversarial_training(x_train, y_train, x_test, y_test)
for r in at_reports:
print(f" {r.name:<48s} clean acc={r.clean_accuracy:6.1%} ASR under PGD(eps={EPS})={r.attack_success_rate:6.1%}")
undefended_asr = at_reports[0].attack_success_rate
defended_asr = at_reports[1].attack_success_rate
print(
f"\nTakeaway: adversarial training cuts ASR from {undefended_asr:.1%} to {defended_asr:.1%} at the same "
f"eps={EPS} budget, at some cost to clean accuracy ({at_reports[0].clean_accuracy:.1%} -> "
f"{at_reports[1].clean_accuracy:.1%}). This is the standard, unavoidable trade-off in empirical "
"adversarial defenses: robustness is bought with some clean-data performance, because the model "
"is now optimizing for worst-case local behavior, not just average-case accuracy."
)
# --- Defense 2: randomized smoothing ---
print("\nDefense 2: randomized smoothing (certified defense)")
print("-" * 70)
smoothing_report = evaluate_randomized_smoothing(x_train, y_train, x_test, y_test)
print(f" Smoothed clean accuracy: {smoothing_report.clean_certified_accuracy:6.1%}")
print(f" Mean certified L2 radius (correct): {smoothing_report.mean_certified_radius:.3f}")
print(f" Smoothed ASR under PGD(eps={EPS}): {smoothing_report.smoothed_asr_under_pgd:6.1%}")
print(
"\nTakeaway: randomized smoothing gives every confidently-classified point a certified radius -- "
"a provable guarantee that NO perturbation within that L2 radius can change the smoothed prediction, "
"which is a fundamentally different (and stronger, in-radius) guarantee than adversarial training's "
"empirical 'we tested against this specific attack and it held.' The cost is inference-time overhead "
"(hundreds of forward passes per prediction instead of one) and a guarantee that only covers the "
"certified radius, not perturbations beyond it."
)
# --- Defense 3: detection based on vote margin ---
print("\nDefense 3: detection via smoothed-vote margin")
print("-" * 70)
smoothed_model = train_noise_augmented(x_train, y_train, SMOOTHING_SIGMA)
x_adv_for_detection = pgd_attack_torch(smoothed_model, x_test, y_test, EPS, EPS_STEP)
detection_reports = evaluate_detection(smoothed_model, x_test, x_adv_for_detection)
for r in detection_reports:
print(f" {r.stream_name:<24s} n={r.num_inputs:4d} flagged fraction={r.flagged_fraction:6.1%}")
print(
"\nTakeaway: PGD pushes inputs toward decision boundaries by construction (that's how it causes "
"misclassification), so adversarial inputs produce far less decisive smoothed votes than clean "
"inputs -- a signal a defender can compute with the same noisy-copy machinery already built for "
"randomized smoothing, with no labeled adversarial examples required to calibrate it."
)
print(f"\nDone in {time.time() - t0:.1f}s")
if __name__ == "__main__":
main()