Objective
Learning objectives
- Recognize exponentials and logarithms inside real AI terminology (softmax, log-loss) without deriving either from scratch
- Observe how exponentials turn raw scores into relative, comparable probabilities
- Explain in plain language why widening the gap between scores makes a softmax output more confident
Hook
Open the source of almost any AI model's output layer and you'll see exp and log everywhere — softmax, log-loss, learning-rate decay schedules. That's not a coincidence, and it's not because AI specifically invented these ideas. It's because "turn a handful of raw scores into a probability distribution" and "measure how wrong a confident guess was" both turn out to be exponent-and-logarithm problems underneath, and this entire phase has been building exactly the vocabulary to recognize that on sight.
This lesson previews — it doesn't teach
Nothing here is derived, and nothing is scored. Softmax, log-loss, and learning-rate schedules all get their full, rigorous treatment later, in this platform's AI/ML-focused courses (specifically the LLM Engineering material). The only goal today is pattern recognition: when you later see exp and log in a real formula, this lesson is why they won't feel like unfamiliar notation.
See it
A model scores a few possible next words — say, 2.0, 1.0, and 0.1 — and needs to turn those raw scores into probabilities that add up to 1. Exponentiating each score first (exp(2.0), exp(1.0), exp(0.1)), then dividing each by their total, does exactly that. This pattern has a name — softmax — and it's built entirely from ideas this phase already covered: exponentials turning differences in score into much larger differences in scale.
Name it
Softmax turns a list of raw scores into a probability distribution by exponentiating each one and dividing by the total — named here only as vocabulary, not derived. Log-loss (also called cross-entropy loss) measures how costly a wrong confident prediction was, using a logarithm specifically because logarithms turn very small probabilities into very large (bad) penalty numbers — a model that was 99.9% confident and wrong gets punished far more than one that was 60% confident and wrong. Learning-rate decay schedules shrink a model's learning rate over training time, following the exact decay pattern from Lesson 03, just applied to a training hyperparameter instead of a battery or a signal.
Code it
import math
def softmax(scores):
exps = [math.exp(s) for s in scores]
total = sum(exps)
return [round(e / total, 4) for e in exps]
print(softmax([2.0, 1.0, 0.1]))
print(softmax([5.0, 1.0, 0.1]))
[0.659, 0.2424, 0.0986]
[0.9749, 0.0179, 0.0073]
Same three relative scores in spirit — one clear leader, two much smaller values — but widening the gap between them (2.0 → 5.0 for the leader) pushes the resulting probability from 66% confident to 97% confident. That's exponentials at work: small differences in the input scores become large differences in the output probabilities, because exponentiating amplifies gaps rather than preserving them.
This part's genuinely ungraded — change the numbers below and run it. There's no right answer being checked here, just a pattern to notice:
my_scores = [3.0, 2.5, 0.0] # try changing these
print(softmax(my_scores))