Glossary › Math

Perplexity

Exponentiated cross-entropy loss: PPL = exp(loss). Measures how "surprised" a model is by the next token. Lower = better. Range: 1 (perfect) to vocab_size (random). Useful for comparing models on same dataset.

Where this is taught

• Information Theory — Phase 3: Advanced Math for AI
• Spotting Hallucinations & Using Web-Connected AI — Phase 4: Research, Analysis & Decision-Making
• Evaluation: Benchmarks, Evals, LM Harness — Phase 3: LLM Architecture
• Text Generation Before Transformers — N-gram Language Models — Phase 2: Sequence Models
• Probability, Statistics & Information Theory for ML — Phase 0: Development Environment & Mathematical Foundations
• Dimensionality Reduction — Phase 3: Advanced Math for AI

Related terms

Cross-Entropy Loss

More in Math

Softmax · Cosine Similarity · Cross-Entropy Loss

Learn this by building

DeVenture Academy teaches AI engineering through projects — every lesson is paired with a hands-on lab you run on your own machine with real tools.

Create a free account — the opening phases of 24 of 30 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary