The Problem
You're asked to build a model that predicts which customers will churn, or which transactions are fraud, or which emails are spam. These are classification problems: the output is a category, not a number. And here's the trap that ends interviews: if 1% of transactions are fraud, a model that predicts "never fraud" is 99% accurate and completely useless.
To build classifiers that work in the real world, you need two things: a model that outputs calibrated probabilities, and the judgment to evaluate it with the right metric for the business cost.
The Concept
From Scores to Probabilities: The Sigmoid Function
Classification problems have a categorical output — spam or not spam, churn or not churn. But the model doesn't output a category directly. It outputs a score (a weighted sum of features, just like linear regression), and then squashes that score into a probability between 0 and 1 using the sigmoid function.
The sigmoid function σ(z) = 1 / (1 + e^(-z)) has a beautiful property: for very negative inputs, it outputs ~0; for very positive inputs, it outputs ~1; and at z=0, it outputs exactly 0.5. This means the model can express uncertainty — a score of 0 gives probability 0.5 (genuinely uncertain), while a score of 5 gives probability 0.993 (very confident it's positive). This calibrated probability is what lets you set a threshold: predict "spam" if probability > 0.8, "not spam" otherwise.
z = w·x + b
p = sigmoid(z) = 1 / (1 + e^(-z)) -> p is in (0, 1)
predict class 1 if p >= threshold (default 0.5)
Why Cross-Entropy, Not MSE?
You might wonder: why not use MSE for classification, like we did for regression? The answer is that MSE with a sigmoid output produces a non-convex loss surface with many local minima — gradient descent gets stuck. Binary cross-entropy (log loss) is the maximum likelihood estimator for binary outcomes, and it has two key properties: it's convex (so gradient descent finds the global minimum), and it heavily penalizes confident wrong predictions.
If the model says "99% sure it's spam" and the email is not spam, cross-entropy imposes a massive penalty (−log(0.01) ≈ 4.6). If the model says "51% sure it's spam" and it's wrong, the penalty is small (−log(0.49) ≈ 0.71). This is exactly the behavior you want: the model is pushed hardest to fix its most confident mistakes.
L = -(1/n) Σ [ yᵢ·log(pᵢ) + (1 - yᵢ)·log(1 - pᵢ) ]
You're building a cancer screening model. Missing a cancer case (false negative) is far worse than a false alarm (false positive). Which metric should you optimize?
In cancer screening, a false negative means a patient with cancer is sent home untreated. Recall = TP/(TP+FN) — maximizing recall minimizes false negatives. Precision matters less here because follow-up tests can rule out false positives.
The confusion matrix is the source of every metric
Predicted 0 Predicted 1
Actual 0 TN (correct) FP (false alarm)
Actual 1 FN (miss) TP (correct hit)
Precision = TP / (TP + FP) -> "of those I flagged, how many were right?"
Recall = TP / (TP + FN) -> "of all real positives, how many did I catch?"
F1 = harmonic mean of precision and recall
Which metric matters depends on the cost of each error:
Fraud / disease screening -> recall matters most (a miss is catastrophic)
Spam filter / ad targeting -> precision matters most (a false alarm annoys users)
Balanced, ranking quality -> ROC-AUC; imbalanced -> PR-AUC
Unlock the full lesson
You've read the first 2 sections. The rest of this lesson covers Build It, Use It, Ship It, Evaluation, Exercises, Key Terms, Common Pitfalls, Interview Framing — plus a hands-on lab, quiz, and project artifact.
Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.