Phase 1: Test-Driven AI Development · 50 min · Python · pytest

Spec-First Development with AI Assistants

The spec is the contract. The tests are the evidence. The AI is the implementer.

Hiring signal: Engineers who practice spec-first development with AI assistants demonstrate the ability to define behavioral requirements before implementation -- a skill that becomes critical when AI generates the implementation and the spec is the only source of truth for correctness.

What you will learn

Write behavioral specifications from natural language requirements
Translate specifications into pytest test cases with edge cases
Generate code from tests by prompting AI assistants with spec + tests
Handle ambiguous specifications where the AI fills in gaps incorrectly

The Problem

A product manager gives a developer a requirement: "Users should be able to reset their password by entering their email. If the email exists in our system, send a reset link. If not, show a generic message saying 'If an account exists, you will receive an email.'"

The developer asks an AI assistant to implement this. The AI generates code that:

Queries the database for the email
If found, generates a token and sends an email
If not found, returns the generic message

The code works. Tests pass. But the AI made three assumptions the spec didn't state:

It logs "Password reset requested for unknown email: X" -- leaking which emails are registered via logs (CWE-117)
It doesn't rate-limit reset requests -- allowing email bombing
It uses the same token for multiple requests -- allowing token replay

The spec didn't mention logging, rate-limiting, or token uniqueness. The AI filled in those gaps with the most probable implementation -- which happened to introduce three security vulnerabilities. Spec-first development catches this by forcing the developer to think through these concerns before the AI generates code.

Writing Behavioral Specifications

A behavioral specification has five components:

Functional requirements: What the function should do
Input domain: What inputs it should accept (including invalid ones)
Output contract: What it should return and when
Failure modes: How it should handle errors
Invariants: Properties that must always hold

Example specification for the password reset function:

Function: request_password_reset(email: str) -> dict
Functional requirements:
  - If email exists in users table, generate a unique reset token (UUID4),
    store it with a 15-minute expiry, and send a reset email.
  - If email does not exist, do nothing but return the same response.
  - Response must be identical regardless of whether email exists (no user enumeration).

Input domain:
  - Valid email strings (any format)
  - Empty string
  - None
  - Very long strings (>1000 chars)
  - Strings with special characters and SQL injection attempts

Output contract:
  - Always returns {"message": "If an account exists, you will receive an email."}
  - Never returns whether the email was found
  - Never raises an exception (all errors are caught and logged safely)

Failure modes:
  - Database error: log internally, return generic message
  - Email send error: log internally, return generic message
  - Rate limit exceeded: return generic message, do not send email

Invariants:
  - Response is identical for existing and non-existing emails (timing-safe)
  - Reset token is unique per request (UUID4)
  - Reset token expires after 15 minutes
  - No user-identifiable information in logs (CWE-117 compliance)
  - Rate limited: max 3 requests per email per hour

The spec is the only source of truth
When the AI generates code, the spec is the only source of truth for correctness. The AI does not know about your security requirements, your rate-limiting policy, or your logging standards. If the spec doesn't mention them, the AI will make assumptions -- and those assumptions are where vulnerabilities live. The spec must be explicit about security, performance, and operational concerns, not just functional behavior.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Translating Specs into pytest Tests, Handling Ambiguous Specs, Build It — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Token