Phase 5: Guardrails & Defensive Controls · 60 min · Python · re · dataclasses
Building Custom Guardrail Pipelines
Every off-the-shelf guardrail eventually hits a case it wasn't built for. That's the day you write your own.
Hiring signal: Every hosted API and framework from Lesson 3 is trained or configured against generic categories — PII, toxicity, jailbreaks in general. Real production systems almost always need at least one custom layer tuned to their specific domain (internal project codenames, a narrow set of allowed topics, a fixed list of tool actions). Interviewers ask 'how would you build this yourself' precisely because it separates candidates who've only wired up a vendor SDK from candidates who understand what the vendor SDK is doing underneath.
What you will learn
- Build a hybrid PII detector that combines regex pattern matching with an NER-style heuristic for names and organizations regex can't reliably catch
- Implement a topic-restriction classifier using keyword/embedding-adjacent scoring rather than a brittle blocklist
- Design an action-sandboxing gate that evaluates a proposed tool call against an allow-list and parameter constraints before execution
- Combine independent guardrail signals into a single pass/fail/review decision with a clear precedence rule
The Problem
Guardrails AI validators, Lakera Guard, Bedrock Guardrails, Azure Prompt Shields, Llama Guard — every product in Lesson 3 is trained or configured against generic categories: PII in general, toxicity in general, jailbreaks in general. That's genuinely useful, and you should use one or more of them. But production systems accumulate domain-specific requirements none of those products were built to know about: your internal project codenames that count as sensitive even though they're not PII by any standard definition, a fixed set of topics your compliance team has explicitly scoped you into (and everything else is out of scope), and a specific, small set of tool actions your agent is allowed to take with specific parameter limits on each one.
At that point, "call a vendor API" stops being the answer, because no vendor's model was trained on your codenames or your tool schema. You need a custom layer. This lesson builds one from three components that show up in almost every real custom guardrail pipeline: a hybrid PII detector, a topic-restriction classifier, and an action-sandboxing gate — then combines all three into a single decision with an explicit precedence rule, because the hardest part of a multi-layer pipeline is rarely any one layer; it's what happens when two layers disagree.
Layer 1: Hybrid PII Detection — Regex Alone Isn't Enough
Regex is precise and fast for structured PII — anything with a fixed shape: a Social Security number, an email address, a phone number, a credit card number. It will never reliably catch unstructured PII like a person's name or an organization, because names have no fixed shape. "My manager Sarah Chen approved this" contains a name regex cannot find without a list of every name in existence.
That gap is what named entity recognition (NER) is for. A real NER pass — spaCy's en_core_web_sm, a transformer-based NER model, or a hosted service — uses a statistical model trained on labeled text to recognize that "Sarah Chen" functions as a person's name in context, the same way it recognizes "Acme Corp" as an organization. This lesson's code tries a real spaCy model first and, if it isn't installed, falls back to a hand-written heuristic: look for runs of consecutive capitalized words that aren't at the start of a sentence, optionally cued by a title ("Dr.", "Mr.") or a corporate suffix ("Inc.", "LLC"). It's a deliberately crude stand-in — it will miss single-word names and occasionally misfire on a capitalized product name — but it demonstrates the real technique: surface-pattern heuristics can approximate NER when you can't afford the dependency, at a real cost in precision and recall you should be explicit about.
The two approaches are complementary, not competing: regex for the PII categories with a fixed shape, NER (real or heuristic) for the PII categories that only exist in context.
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Layer 2: Topic Restriction Without a Brittle Blocklist, Layer 3: Action Sandboxing — the Model's Output Is a Proposal, Not an Authorization, Combining Three Signals Into One Decision, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.
Browse all courses · View pricing · DeVenture Academy