Home › Courses › AI Security & Red Teaming

AI Security & Red Teaming

Secure, red-team, and defend production AI systems

10 phases. 55 lessons. 55 labs. 4 projects. The OWASP LLM Top 10, prompt injection and jailbreaking, adversarial ML, RAG and agent security, guardrails, red teaming with Garak and PyRIT, AI governance and compliance, and production security architecture. You build the threat models, detectors, guardrails, and red team reports that AI security teams actually ship.

10 phases · 55 lessons · 55 labs · 4 projects

Take ML & AI Engineering first — this course builds on it.

Outcomes you will have by the end

• 4 GitHub repos with red team & security artifacts — Injection detector, agent security audit, full red team engagement report, and a production security system — each with architecture docs and re-test evidence.
• A working prompt injection detector + jailbreak library — A multi-layer detector (regex + heuristic + ML classifier) and a documented library of 20+ jailbreak techniques with measured bypass rates.
• A full red team engagement report — Garak + PyRIT findings against a production-style LLM app: severity-ranked vulnerabilities, attack chains, remediation guidance, and re-test results.
• Verified AI Security Engineer certificate — Issued by DeVenture Academy, tied to your completion record. Lists the specific security skills, phases, and projects you completed.
• AI Security Readiness Scorecard — Weighted readiness score across threat modeling, red teaming, guardrails, compliance, and production security architecture — exactly what hiring managers test.
• Interview-ready security case studies — Threat models, vulnerability reports, and STAR stories for every project. Walk through them in any AI security interview.

What you will be able to do

OWASP LLM Top 10 · Prompt Injection Defense · Red Teaming · Guardrails · MITRE ATLAS · AI Compliance

Every phase, every lesson, every project

• Security Fundamentals for AI (5 lessons) — free — Threat modeling (STRIDE/PASTA for AI), MITRE ATLAS, the 6-layer AI attack surface, AppSec vs AI security, trust boundaries & blast radius
  • Threat Modeling for AI Systems
  • MITRE ATLAS — The Attack Taxonomy for AI Systems
  • Attack Surface Mapping for LLM Applications & Agents
  • AppSec vs AI Security — Trust Boundaries & Blast Radius
  • The Security Mindset Shift
• OWASP Top 10 for LLM Applications (6 lessons) — free — LLM01 Prompt Injection, LLM02 Insecure Output Handling, LLM03 Training Data Poisoning, LLM04 Model DoS, LLM05 Supply Chain, LLM06 Sensitive Info Disclosure, LLM07 Insecure Plugin/Tool Design, LLM08 Excessive Agency, LLM09 Overreliance, LLM10 Model Theft
  • LLM01: Prompt Injection
  • LLM02 & LLM04: Insecure Output Handling & Model Denial of Service
  • LLM03 & LLM05: Training Data Poisoning & Supply Chain Vulnerabilities
  • LLM06 & LLM07: Sensitive Information Disclosure & Insecure Plugin/Tool Design
  • LLM08 & LLM09: Excessive Agency & Overreliance
  • LLM10: Model Theft — and Mapping a Full System
• Prompt Injection & Jailbreaking (6 lessons) — Direct injection, jailbreak techniques (DAN, roleplay, encoding bypasses), Crescendo, TAP, PAIR, indirect injection & XPIA, EchoLeak, multi-agent injection, assume-breach defense architecture, building a multi-layer detector
  • The Injection & Jailbreak Attack Taxonomy
  • Indirect Injection & Cross-Domain Prompt Injection (XPIA)
  • Multi-Agent Injection, Goal Hijacking & Memory Manipulation
  • Defense Architecture — Assume Breach, Trust Tiers & Structural Separation
  • Building a Multi-Layer Prompt Injection Detector
  • Project: Prompt Injection Detector + Jailbreak Library
• Adversarial Machine Learning (5 lessons) — Evasion attacks (FGSM, PGD, Carlini & Wagner), poisoning attacks (label flipping, backdoors, clean-label), model inversion, membership inference, model extraction, ART/CleverHans/Foolbox
  • Evasion Attacks — FGSM, PGD, and Carlini & Wagner
  • Poisoning Attacks — Label Flipping, Backdoors, and Clean-Label Poisoning
  • Model Inversion & Membership Inference Attacks
  • Model Extraction & Stealing
  • Adversarial Defenses & Tooling — ART, CleverHans, Foolbox
• RAG & Agent Security (6 lessons) — Document poisoning, vector/embedding attacks, retrieval manipulation, multi-tenant isolation, agent privilege escalation, tool poisoning, multi-agent & planner manipulation, MCP security, computer-use safety
  • RAG Attack Surface — Document Poisoning & Vector Attacks
  • Retrieval Manipulation, Tenant Isolation & Access Control
  • Agent Privilege Escalation & Tool Poisoning
  • Multi-Agent, Planner & Memory Attacks
  • MCP Security & Computer Use / Browser Agent Safety
  • Project: Agent Security Audit
• Guardrails & Defensive Controls (5 lessons) — Guardrail architecture patterns, NeMo Guardrails & Colang, Guardrails AI, Lakera Guard, AWS Bedrock Guardrails, Azure Prompt Shields, Llama Guard, custom PII/topic pipelines, Promptfoo regression testing
  • Guardrail Architecture Patterns
  • NeMo Guardrails & Colang
  • Guardrails AI & Hosted Guardrail Platforms
  • Building Custom Guardrail Pipelines
  • Guardrail Regression Testing with Promptfoo
• AI Red Teaming Methodology & Tools (6 lessons) — Microsoft AI Red Team methodology, engagement lifecycle, Garak vulnerability scanning, PyRIT multi-turn orchestration, Promptfoo/Inspect AI/HarmBench, red team report writing, continuous red teaming in CI/CD
  • The Microsoft AI Red Team Methodology & Engagement Lifecycle
  • Garak — Automated LLM Vulnerability Scanning
  • PyRIT — Multi-Turn Attack Orchestration
  • Promptfoo, Inspect AI & HarmBench
  • Writing the Red Team Report
  • Project: Full Red Team Engagement Report
• AI Governance, Compliance & Risk (5 lessons) — NIST AI RMF (Govern/Map/Measure/Manage), EU AI Act risk tiers & GPAI obligations, ISO/IEC 42001, SOC 2 for AI, bias audits & fairness metrics, SHAP/LIME explainability, AI incident response & regulatory reporting
  • NIST AI Risk Management Framework
  • EU AI Act
  • ISO/IEC 42001 & SOC 2 for AI
  • Bias Audits, Fairness Testing & Explainability
  • AI Incident Response & Regulatory Reporting
• Production AI Security Architecture (6 lessons) — Multi-layer security architecture (network to audit layer), observability with LangSmith/Langfuse/Phoenix, CI/CD security gates, supply chain security & SBOM for ML, AI incident response playbooks
  • The Multi-Layer AI Security Architecture
  • Security Monitoring & Observability
  • CI/CD for AI Security
  • Supply Chain Security & SBOM for ML
  • AI Incident Response Playbooks
  • Project: Production AI Security System
• Career, Portfolio & Capstone Demo (5 lessons) — Building an AI security portfolio, resume & LinkedIn for AI security roles, interview prep (system design, live red teaming, code review), certifications (CAISP, AAISM), community (OWASP LLM, MITRE ATLAS, AI Village), staying current & capstone demo
  • Building an AI Security Portfolio
  • Resume & LinkedIn for AI Security Roles
  • AI Security Interview Preparation
  • Certifications & Community
  • Staying Current & Capstone Demo

The technologies you will use

OWASP LLM Top 10 · Garak · PyRIT · Promptfoo · NeMo Guardrails · Guardrails AI · MITRE ATLAS · Adversarial Robustness Toolbox · Langfuse · Docker · Python · FastAPI

Roles this course prepares you for

• AI Security Engineer ($150k–$210k) — Secures AI systems end-to-end: threat models, guardrails, monitoring, compliance, incident response. The broadest and most common AI security role.
• AI Red Team Engineer ($160k–$230k) — Conducts adversarial simulations against LLM applications and agentic systems. Finds vulnerabilities before real adversaries do. Writes red team reports and remediation guidance.
• DevSecOps for AI Pipelines ($150k–$210k) — Integrates security into the MLOps lifecycle. CI/CD security gates, supply chain scanning, automated adversarial testing, model deployment security.
• AI Security Architect ($170k–$250k) — Designs the security architecture for AI systems. Multi-layer defense, trust boundaries, guardrail strategy, compliance framework alignment. Senior role.
• AI Governance & Compliance Engineer ($140k–$200k) — Translates regulatory requirements (EU AI Act, NIST AI RMF, ISO 42001) into technical controls. Conducts bias audits, explainability assessments, risk assessments.
• LLM Application Security Engineer ($150k–$220k) — Specializes in securing LLM-powered applications specifically: prompt injection defense, RAG security, agent safety, output validation. The most hands-on coding role.
• Adversarial ML Researcher/Engineer ($170k–$280k) — Works at the intersection of ML research and security. Develops novel attack techniques and defenses. Found at frontier labs and defense companies.
• AI Security Consultant ($160k–$240k) — Helps organizations assess and improve their AI security posture. Conducts red team engagements, security reviews, compliance audits. Often client-facing.

What AI security engineering actually is

AI security blends offensive security (red teaming) and defensive engineering (guardrails, monitoring, compliance) around a new class of system: one that reasons over natural language and can be manipulated through it. It is not traditional AppSec with an LLM bolted on — the attack surface, the failure modes, and the defenses are structurally different.

What you do every day

You scan models with Garak, run multi-turn attacks with PyRIT, build and tune guardrail pipelines, review agent codebases for excessive agency, write threat models for new AI features, and respond when a guardrail bypass makes it to production. You write red team reports that get read by engineering leadership, not just other security engineers.

Why companies are hiring for this now

Every company shipping an LLM feature now has an attack surface that didn't exist two years ago: prompt injection, jailbreaks, RAG poisoning, agent privilege escalation. AI security engineering job postings grew 124% year-over-year, and 89% of them require prompt injection and jailbreak expertise specifically. There is no large pool of experienced candidates yet — this is a wide-open niche.

What this course is not

It is not a general cybersecurity course with a few AI slides added. It is not a theory course — you will not just read the OWASP LLM Top 10, you will exploit and defend against every category of it. And it does not pretend the field has a finished playbook: you will learn the frameworks that exist (MITRE ATLAS, NIST AI RMF) and the reasoning skills to handle the attack patterns that show up in your job next month, which no course has documented yet.

Common questions

What background do I need for the AI Security & Red Teaming course?

Python proficiency and a basic understanding of LLM APIs (OpenAI, Anthropic) and what RAG and agents are. No prior security experience required — Phase 0 and Phase 1 teach security fundamentals and the OWASP LLM Top 10 from scratch. We recommend the ML & AI Engineering course as a foundation, but it is not required.

Is this standalone or does it require the ML & AI Engineering course?

Fully standalone. If you already know Python and the basics of LLMs, RAG, and agents, you can start here directly. If you're newer to AI engineering, completing Phase 01 of the ML & AI Engineering course first will make Phases 2–4 easier.

How is this different from the security content in the ML & AI Engineering course?

The ML & AI Engineering course covers evaluation and safety in one 6-lesson phase (Phase 08) as part of a broader AI engineering curriculum. This course goes roughly 10x deeper: 55 lessons dedicated entirely to security — the full OWASP LLM Top 10, prompt injection and jailbreaking, adversarial ML, RAG/agent security, guardrail frameworks, red teaming with Garak and PyRIT, AI governance and compliance, and production security architecture.

How long does this course take?

120–160 hours of structured content. Most engineers complete it in 4–6 months at 8–10 hours per week. Phases 0–1 (free) can be completed in about a week and give you a real sense of the field before you commit further.

Do I need a GPU or special hardware?

No. Every lab runs on CPU. The adversarial ML labs (Phase 3) use small models specifically so they run locally without GPU access. Red teaming labs (Phase 6) call hosted or free-tier LLM APIs.

What specific jobs does this course prepare me for?

AI Security Engineer, AI Red Team Engineer, DevSecOps for AI Pipelines, AI Security Architect, AI Governance & Compliance Engineer, LLM Application Security Engineer, Adversarial ML Researcher/Engineer, and AI Security Consultant. Every phase maps to specific hiring signals for these roles.

How is AI red teaming different from traditional penetration testing?

Traditional pentesting focuses on the CIA triad — confidentiality, integrity, availability. AI red teaming focuses on context-specific harms: bias, manipulation, factuality, and safety, alongside traditional security concerns. The attack surface is also different — you're attacking model behavior through natural language and retrieved content, not just network and application boundaries. This course follows the Microsoft AI Red Team methodology, which was built specifically for this difference.

Is this field established enough to have a reliable curriculum?

The field is young and moving fast — there is no 20-year-old playbook to teach from. This course is built from the frameworks that do exist and are stable: OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, and the tools security teams actually use in production (Garak, PyRIT, NeMo Guardrails, Promptfoo). You will also learn to reason from first principles, because you will encounter attack patterns in the job that no course has documented yet.

Key terms in this course

Agent · Guardrails · Prompt Injection · Red Teaming · RAG (Retrieval-Augmented Generation) · Embedding · Inference · MCP (Model Context Protocol)

Continue your learning path

Agentic AI Engineering · Responsible AI Engineering · ML & AI Engineering · Security Review for Vibe Coders

Start the AI Security & Red Teaming course

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary