Glossary › Math

Softmax

Function that converts a vector of real numbers into a probability distribution: softmax(x_i) = e^(x_i) / Σe^(x_j). Used in attention scores and classification outputs. Temperature scales logits before softmax.

Where this is taught

• Where This Shows Up in AI (Preview) — Phase 2: Exponents, Logarithms & Growth
• Activation Functions — Phase 1: Training Neural Networks
• CLIP and Contrastive Vision-Language Pretraining — Phase 0: Multimodal Foundations
• Differential Attention (V2) — Phase 4: Training LLMs
• Logistic Regression — Phase 1: Classification Algorithms
• Numerical Stability — Phase 1: Calculus & Optimization

Related terms

Temperature · Attention Mechanism

More in Math

Cosine Similarity · Cross-Entropy Loss · Perplexity

Learn this by building

DeVenture Academy teaches AI engineering through projects, with hands-on labs you run on your own machine using real tools.

Create a free account — the opening phases of 29 of 35 courses are free, no credit card. Or see Pro pricing.

All courses · Pricing · About · FAQ · Glossary