Phase 4: RAG & Agent Security · 50 min · Python · Vector databases · OWASP LLM Top 10

RAG Attack Surface — Document Poisoning & Vector Attacks

Whoever can write to your knowledge base can write to your model's context window.

Hiring signal: RAG security review is now a standard line item in AI security engineer interviews — companies expect candidates to demonstrate document poisoning end-to-end (ingest, retrieve, inject) rather than describe it abstractly, and to name the ingestion-time control that stops it.

What you will learn

Explain how document poisoning at ingestion time becomes context injection at retrieval time
Build an in-memory vector store and demonstrate a poisoned document out-ranking legitimate content
Implement an ingestion-time scanner that blocks injection-shaped documents before they are embedded
Distinguish ingestion-time defenses from context-time sanitization and explain why both layers matter

The Problem

Every RAG system has an implicit assumption baked into its architecture: the documents in the knowledge base are trustworthy. That assumption breaks the moment any of the following is true — and in production, at least one almost always is:

Customers or the public can submit content that eventually lands in the corpus (support tickets, reviews, community wiki pages, uploaded PDFs)
The corpus is built by crawling external sources (web pages, competitor sites, GitHub issues, email)
Multiple internal teams can write to a shared knowledge base without review

Greshake et al.'s 2023 paper on indirect prompt injection was the first widely cited formalization of this: an attacker doesn't need access to your model, your API keys, or your prompt. They only need one write path into anything your retrieval pipeline will later read. Once a malicious document is embedded and stored, it is retrievable by any query whose embedding is close enough — and once retrieved, its text is concatenated into the model's context window with the same trust level as your system prompt, your policy documents, and your curated FAQ.

This is document poisoning: planting adversarial content in the knowledge base at ingestion time so that it later gets surfaced at retrieval time as "trusted context." It is functionally a supply-chain attack on your RAG pipeline's data plane, and it is why OWASP's GenAI Security guidance treats retrieval-augmented pipelines as first-class attack surface, not just a feature.

How the Attack Actually Works

A poisoning attack against a RAG system has two independent problems the attacker has to solve, and defenders should think about them separately:

1. Ranking — getting the poisoned document retrieved at all. Vector search returns the top-k documents by embedding similarity to the query. An attacker who knows (or guesses) the kinds of queries your users ask can keyword-stuff a document so its embedding sits close to those queries. This doesn't require breaking the embedding model — it's the same SEO logic that has worked against search engines for two decades, just aimed at a vector index instead of a crawler.

2. Payload — what happens once it's in context. Once the poisoned document is retrieved, its raw text is usually concatenated directly into the prompt alongside legitimate retrieved content. If nothing marks retrieved content as untrusted data, the model has no structural signal that "SYSTEM: ignore previous instructions" inside a retrieved chunk is any less authoritative than the instructions in your actual system prompt.

Vector-store-specific variants worth knowing by name:

Attack	Mechanism
Document poisoning	Malicious text injected into the corpus at ingestion, designed to rank for target queries
Embedding manipulation	Crafting input text (not just keywords, but adversarial phrasing) to land a document's vector near a specific query cluster in embedding space
Vector DB injection	Writing directly to the vector index (skipping the embedding step) if the DB's write API is exposed without the same validation as the ingestion pipeline
Embedding exfiltration	Querying a vector DB's nearest-neighbor API repeatedly to reconstruct or approximate the underlying documents' content, when the DB doesn't gate raw-vector access

The common thread: the vector store has no concept of trust. It is a similarity index, not an access-control system. Every security property has to be layered on top of it — nothing about "vector database" implies "input validated" or "content non-executable."

Ranking success does not require breaking the model
The most common mistake in RAG threat models is focusing entirely on "can the model be jailbroken by this text?" The ranking half of the attack is pure information retrieval — an attacker tunes a document against your embedding model the same way an SEO spammer tunes a page against a search ranking algorithm. You can have a perfectly safety-tuned model and still get exploited, because the vulnerability is in what gets handed to the model, not in the model itself.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Ingestion-Time Defense vs. Context-Time Defense, Context-Time Sanitization — the Second Layer, Build It, What to Practice — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.