Glossary › Evaluation

A/B Testing

Comparing two model versions or prompt variants on real user traffic. Statistical significance requires enough samples. Online metric (e.g. thumbs up) vs offline metric (e.g. BLEU).

Where this is taught

• A/B Testing Models & Batch Processing — Phase 7: Multi-Model Pipeline Orchestration
• Prompt Versioning & Management — Phase 1: Advanced Prompt Engineering for Agents
• Bayes' Theorem — Phase 2: Probability & Statistics
• Offline vs Online Evaluation — Phase 5: Evaluation & Quality for AI Products
• Evaluation Framework Design — Phase 7: AI Evaluation & Observability

More in Evaluation

LLM-as-Judge · Golden Set · Regression Testing · BLEU Score

Learn this by building

DeVenture Academy teaches AI engineering through projects — every lesson is paired with a hands-on lab you run on your own machine with real tools.

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary