The Concept
The concept
Why ML Coding Is a Separate Skill
Standard DSA interviews test algorithmic thinking. ML coding interviews test something different: can you manipulate numerical data correctly without libraries doing the thinking for you? The interviewer wants to see that you understand what softmax does, not just that you can call torch.nn.Softmax().
This matters because in production ML, you constantly need to: implement metrics from scratch (your eval pipeline doesn't have sklearn), vectorize operations for performance (loops are 100x slower than numpy), manipulate tensors with broadcasting (shape errors are the most common bug), and implement small ML primitives (custom loss functions, custom metrics, data transforms). If you can only do these with library calls, you can't debug when the library doesn't expose what you need.
The three skill areas:
- numpy vectorization: replacing Python loops with array operations — the single most important skill for ML coding
- ML metrics from scratch: precision, recall, F1, confusion matrix, cross-entropy — using only numpy
- ML primitives from scratch: softmax, k-means step, kNN, gradient descent step — proving you understand the math
The patterns
1. numpy vectorisation — never loop over arrays
In an ML coding interview, you're asked to compute cosine similarity between a query vector and 10,000 document vectors. You write a Python for-loop. The interviewer asks "Can you make this faster?" What's the vectorized approach?
The vectorized approach replaces the loop with a single matrix operation. docs @ query computes all 10,000 dot products in one BLAS call. Normalization is also vectorized with broadcasting. No Python overhead per element — numpy hands the entire computation to optimized C/Fortran. This is the #1 pattern in ML coding interviews: if you're looping over arrays, you're doing it wrong.
import numpy as np
# BAD: Python loop
def cosine_sim_slow(a, b):
dot = sum(a[i]*b[i] for i in range(len(a)))
return dot / (sum(x**2 for x in a)**0.5 * sum(x**2 for x in b)**0.5)
# GOOD: vectorised
def cosine_sim(a: np.ndarray, b: np.ndarray) -> float:
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# BATCH: cosine between one query and many docs (broadcasting)
def cosine_sim_batch(query: np.ndarray, docs: np.ndarray) -> np.ndarray:
# query (d,), docs (n, d)
norms = np.linalg.norm(docs, axis=1) * np.linalg.norm(query) # (n,)
return docs @ query / norms # (n,) — one query vs all docs
Broadcasting rule to remember: dimensions align right-to-left; size-1 dims stretch.
2. ML metrics from scratch — numpy only
def precision_recall_f1(y_true: np.ndarray, y_pred: np.ndarray):
tp = np.sum((y_pred == 1) & (y_true == 1))
fp = np.sum((y_pred == 1) & (y_true == 0))
fn = np.sum((y_pred == 0) & (y_true == 1))
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
return precision, recall, f1
def softmax(x: np.ndarray) -> np.ndarray:
x = x - x.max() # numerical stability: subtract max before exp
e = np.exp(x)
return e / e.sum()
Interview tells: numerically stable softmax (subtract max), zero-division guards on precision/recall.
The sandbox here doesn't have numpy, but the numerical-stability trick is the same in plain Python — try it below with an input that would overflow a naive softmax:
import math
def softmax_naive(x: list[float]) -> list[float]:
exps = [math.exp(v) for v in x]
total = sum(exps)
return [e / total for e in exps]
def softmax_stable(x: list[float]) -> list[float]:
m = max(x) # subtract max before exp
exps = [math.exp(v - m) for v in x]
total = sum(exps)
return [e / total for e in exps]
# Try it: a normal input — both should agree
print("normal input:", softmax_stable([1.0, 2.0, 3.0]))
# Try it: a large input that overflows the naive version
big_input = [1000, 1001, 1002]
try:
print("naive on large input:", softmax_naive(big_input))
except OverflowError as e:
print("naive version blew up:", e)
print("stable on large input:", softmax_stable(big_input))
3. pandas feature engineering under time pressure
import pandas as pd
# Rolling feature (no data leakage — shift before rolling window)
df["spend_lag1"] = df.groupby("user_id")["spend"].shift(1)
df["spend_roll3"] = (
df.groupby("user_id")["spend"]
.shift(1) # shift FIRST so current row is excluded
.rolling(3, min_periods=1)
.mean()
.reset_index(level=0, drop=True)
)
# User-level aggregation → merge back
agg = df.groupby("user_id")["spend"].agg(["mean", "max", "std"]).reset_index()
agg.columns = ["user_id", "spend_mean", "spend_max", "spend_std"]
df = df.merge(agg, on="user_id", how="left")
The .shift(1) before .rolling() is the leakage-prevention pattern from SQL window functions — same rule, different syntax.
5. Implement k-NN from Scratch
import numpy as np
def knn_predict(
X_train: np.ndarray, y_train: np.ndarray,
X_test: np.ndarray, k: int = 5,
) -> np.ndarray:
"""k-Nearest Neighbors — no sklearn."""
# Compute all pairwise distances (vectorized!)
# X_train: (n_train, d), X_test: (n_test, d)
# Using ||a - b||^2 = ||a||^2 + ||b||^2 - 2*a·b
train_sq = np.sum(X_train ** 2, axis=1) # (n_train,)
test_sq = np.sum(X_test ** 2, axis=1) # (n_test,)
cross = X_test @ X_train.T # (n_test, n_train)
dists = np.sqrt(test_sq[:, None] + train_sq[None, :] - 2 * cross)
# Get k nearest neighbors for each test point
nearest_idx = np.argsort(dists, axis=1)[:, :k] # (n_test, k)
nearest_labels = y_train[nearest_idx] # (n_test, k)
# Majority vote
from collections import Counter
predictions = np.array([
Counter(labels).most_common(1)[0][0]
for labels in nearest_labels
])
return predictions
# Time: O(n_test × n_train × d) for distances, O(n_test × n_train × log k) for sort
# Interview tip: mention the ||a-b||^2 trick to avoid explicit loop for distances
6. Implement k-Means from Scratch
def kmeans(
X: np.ndarray, k: int, max_iters: int = 100, seed: int = 42,
) -> tuple[np.ndarray, np.ndarray]:
"""k-Means clustering — no sklearn."""
rng = np.random.default_rng(seed)
n, d = X.shape
# Initialize: pick k random points
centroids = X[rng.choice(n, k, replace=False)].copy()
for _ in range(max_iters):
# Assign: each point to nearest centroid
dists = np.linalg.norm(X[:, None, :] - centroids[None, :, :], axis=2) # (n, k)
labels = np.argmin(dists, axis=1) # (n,)
# Update: centroids to mean of assigned points
new_centroids = np.array([
X[labels == j].mean(axis=0) if np.any(labels == j) else centroids[j]
for j in range(k)
])
# Check convergence
if np.allclose(centroids, new_centroids):
break
centroids = new_centroids
return centroids, labels
# Interview tips:
# - Mention random initialization sensitivity → k-means++ is better
# - Empty cluster handling: reinitialize to farthest point
# - Convergence: centroids stop moving, not labels stop changing
7. Confusion Matrix from Scratch
def confusion_matrix(y_true: np.ndarray, y_pred: np.ndarray, n_classes: int = 2) -> np.ndarray:
"""Build a confusion matrix from arrays — no sklearn."""
cm = np.zeros((n_classes, n_classes), dtype=int)
for t, p in zip(y_true, y_pred):
cm[t, p] += 1
return cm
# For binary classification:
# cm = [[TN, FP],
# [FN, TP]]
#
# From the matrix:
# accuracy = (TP + TN) / total
# precision = TP / (TP + FP)
# recall = TP / (TP + FN)
# f1 = 2 * P * R / (P + R)
#
# For multi-class: cm[i, j] = count of true class i predicted as j
# Per-class precision = cm[i, i] / cm[:, i].sum() (column)
# Per-class recall = cm[i, i] / cm[i, :].sum() (row)
4. The timed bug hunt
Common bugs interviewers plant:
# Bug 1: shape mismatch in broadcasting
a = np.array([1, 2, 3]) # (3,)
b = np.array([[1], [2]]) # (2, 1) — this will broadcast to (2, 3), not (3,)!
# Fix: be explicit about shapes; print .shape before any matmul/dot
# Bug 2: NaN propagation silently ruining aggregations
df["feature"].fillna(df["feature"].median(), inplace=True) # or .mean()
# Check: assert df.isnull().sum().sum() == 0
# Bug 3: integer division producing wrong dtype
acc = correct / total # fine in Python 3, but check dtype in numpy:
acc = int(correct) / int(total)
# Bug 4: not resetting index after groupby
result = df.groupby("x")["y"].mean() # result has 'x' as index — .reset_index()
The drill protocol
1. Read the task (2 min)
2. Write the expected input/output shapes and types
3. Code it without looking anything up (15 min)
4. Test with a tiny hand-crafted example
5. State time/space if asked
Do this for every problem in this lesson until you can hit the 15-minute mark without hesitation.