Phase 6: AI Red Teaming Methodology & Tools · 60 min · Markdown · Python · dataclasses

Writing the Red Team Report

A finding nobody acts on isn't a finding. It's a paragraph.

Hiring signal: Every AI Red Team Engineer and AI Security Engineer job posting eventually asks for 'clear written communication' or 'stakeholder-ready reporting' — because the technical find is worthless if the VP of Engineering can't act on it. Candidates who can produce a structured, severity-ranked report with real attack chains and concrete remediation stand out from candidates who can only run a tool and paste its raw output.

What you will learn

Structure a red team report around an executive summary, methodology, severity-ranked findings, attack chains, remediation, and re-test sections
Write an executive summary a non-technical stakeholder can act on without reading the technical findings
Distinguish an individual finding from an attack chain, and document a chain that combines multiple findings into a single real-world impact narrative
Implement a ReportBuilder that renders structured findings data into a complete, well-formatted markdown report

The Problem

You can run a flawless Garak scan, a devastating PyRIT Crescendo campaign, and lock every fix in with Promptfoo — and still fail the engagement, if the deliverable is a folder of raw JSON reports and a Slack message that says "found some stuff, lmk if questions." Microsoft's public AI red team guidance is explicit that stage 8 of the nine-stage lifecycle (lesson 1) — report findings — is not a formality tacked onto the end of the real work. It's the artifact that determines whether anything the team found actually gets fixed.

The reason this goes wrong constantly: a security engineer writes for other security engineers. A raw finding like "Crescendo escalation via benign-frame drift bypasses the Likert refusal threshold at turn 5" is precise and true, and it is completely useless to the VP of Engineering who has to decide whether this blocks a launch, or the product manager who has to prioritize the fix against three other sprints of work. The report has to serve two audiences at once — a technical audience that needs exact reproduction steps, and a non-technical audience that needs to understand business impact and urgency in under two minutes — without becoming two separate documents that drift out of sync.

Every credible red team report, AI or traditional, converges on the same six-section shape to solve this. This lesson is about building that shape well, because it's the artifact a hiring manager will actually ask to see a sample of.

Executive Summary: Write It Last, Put It First

The executive summary is the only section guaranteed to be read by everyone — the security team, the engineering lead, and the executive who's ultimately accountable for the risk. It needs three things in the first two sentences: what was tested, how many findings at what severity, and what the single most urgent thing is. Everything else in the summary is secondary.

The discipline that makes this work: write the executive summary last, after every finding is written and ranked, but put it first in the document. A summary written before the findings are finalized tends to hedge ("several potential issues were identified") instead of committing to a specific, actionable headline ("a critical finding allows unauthorized refunds up to $500 without identity verification"). The second version is the one a VP can act on without reading further; the first version forces them to read the whole report just to find out if they should be worried.

The executive summary is a different document, not a shorter one
A common failure mode is treating the executive summary as "the findings section, but compressed." That produces a wall of technical detail at lower resolution — still unreadable to a non-technical stakeholder. The executive summary should answer three questions only: what was assessed, what's the worst thing found, and what needs to happen next. Reproduction steps, tool names, and probe hit rates belong in the findings section, not here.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Methodology: Scope, Team, Tools, Timeline, Findings: Severity-Ranked, With Reproduction Steps for Every One, Attack Chains: Where Individual Findings Compound, Remediation and Re-Test: Closing the Loop, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.