The Problem
Traditional ML requires all training data in one place — a central server. For healthcare, this means every hospital must send patient records to a central location. For finance, every bank must share customer transactions. This creates massive privacy and regulatory challenges:
- GDPR restricts cross-border data transfer
- HIPAA restricts sharing of protected health information
- Competitive concerns: banks don't want to share customer data with each other
- Single point of failure: a central data repository is a high-value target for attackers
Federated learning solves this by flipping the architecture: the model travels to the data, not the data to the model. Each participant trains locally and only shares model updates (gradients/weights) — never raw data.
How federated learning works
Round 1:
Server → sends global model to all clients
Client A → trains locally on their data → sends updated weights to server
Client B → trains locally on their data → sends updated weights to server
Client C → trains locally on their data → sends updated weights to server
Server → averages all weights (FedAvg) → new global model
Round 2:
Server → sends new global model to all clients
... (repeat)
Key properties:
- Raw data never leaves the client device
- Only model weights/gradients are shared
- The server aggregates updates but never sees individual data
- Each client can control how many rounds they participate in
FedAvg (federated averaging)
The most common aggregation algorithm. The server computes a weighted average of client models, weighted by the number of training examples each client has:
$$w_{global} = \sum_{k=1}^{K} \frac{n_k}{n} w_k$$
Where $w_k$ is client $k$'s model weights, $n_k$ is client $k$'s number of examples, and $n$ is the total across all clients.
Limitations
| Limitation | Description | Mitigation |
|---|
| Gradient leakage | Gradients can still reveal information about training data | Add differential privacy to gradients (DP-FedAvg) |
| Communication cost | Sending model weights every round is expensive | Compression, quantization, fewer rounds |
| Non-IID data | Clients have different data distributions | Personalized federated learning, clustering |
| System heterogeneity | Clients have different compute capabilities | Async aggregation, client selection |
| Poisoning attacks | A malicious client can send bad updates | Robust aggregation, Byzantine-resistant algorithms |
Building a federated learning simulation
"""
federated_learning_demo.py
Simulates federated learning with FedAvg on synthetic data.
"""
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
class FederatedClient:
"""Simulates a federated learning client with local data."""
def __init__(self, client_id, X, y):
self.client_id = client_id
self.X = X
self.y = y
self.n_samples = len(X)
def train_local(self, global_weights=None, epochs=5):
"""Train a local model, optionally initialized with global weights."""
model = LogisticRegression(max_iter=1000, random_state=42)
# In a real system, we'd initialize with global weights
# Here we simulate by training on local data
model.fit(self.X, self.y)
return model, self.n_samples
def federated_averaging(client_models_and_weights):
"""
FedAvg: weighted average of client models by sample count.
Args:
client_models_and_weights: List of (model, n_samples) tuples
Returns:
Averaged model
"""
# For logistic regression, we average the coefficients and intercepts
total_samples = sum(n for _, n in client_models_and_weights)
avg_coef = np.zeros_like(client_models_and_weights[0][0].coef_)
avg_intercept = np.zeros_like(client_models_and_weights[0][0].intercept_)
for model, n in client_models_and_weights:
weight = n / total_samples
avg_coef += weight * model.coef_
avg_intercept += weight * model.intercept_
# Create a new model with averaged weights
avg_model = LogisticRegression(max_iter=1000, random_state=42)
avg_model.coef_ = avg_coef
avg_model.intercept_ = avg_intercept
avg_model.classes_ = client_models_and_weights[0][0].classes_
return avg_model
def simulate_federated_learning(n_clients=5, n_rounds=10, data_per_client=400):
"""
Simulate federated learning across multiple clients.
Args:
n_clients: Number of federated clients
n_rounds: Number of communication rounds
data_per_client: Training samples per client
Returns:
Dict with simulation results
"""
np.random.seed(42)
# Generate synthetic data for each client (non-IID: different distributions)
clients = []
for i in range(n_clients):
# Each client has slightly different data distribution
center = np.random.randn(10) * 2
X = np.random.randn(data_per_client, 10) + center
y = (X[:, 0] + X[:, 1] + np.random.randn(data_per_client) * 0.5 > 0).astype(int)
clients.append(FederatedClient(f"client_{i}", X, y))
# Generate a global test set
X_test = np.random.randn(1000, 10)
y_test = (X_test[:, 0] + X_test[:, 1] > 0).astype(int)
# Track accuracy over rounds
round_results = []
for round_num in range(n_rounds):
# Each client trains locally
client_models = []
for client in clients:
model, n = client.train_local()
client_models.append((model, n))
# Server aggregates
global_model = federated_averaging(client_models)
# Evaluate global model
y_pred = global_model.predict(X_test)
global_acc = accuracy_score(y_test, y_pred)
# Evaluate individual client models (for comparison)
client_accs = []
for model, n in client_models:
client_accs.append(accuracy_score(y_test, model.predict(X_test)))
round_results.append({
"round": round_num + 1,
"global_accuracy": round(global_acc, 4),
"avg_client_accuracy": round(np.mean(client_accs), 4),
"min_client_accuracy": round(np.min(client_accs), 4),
"max_client_accuracy": round(np.max(client_accs), 4),
})
# Also train a centralized model for comparison
all_X = np.vstack([c.X for c in clients])
all_y = np.concatenate([c.y for c in clients])
centralized_model = LogisticRegression(max_iter=1000, random_state=42)
centralized_model.fit(all_X, all_y)
centralized_acc = accuracy_score(y_test, centralized_model.predict(X_test))
return {
"rounds": round_results,
"centralized_accuracy": round(centralized_acc, 4),
"n_clients": n_clients,
"data_per_client": data_per_client,
"total_federated_data": n_clients * data_per_client,
}
if __name__ == "__main__":
print("=== Federated Learning Simulation (FedAvg) ===\n")
results = simulate_federated_learning(n_clients=5, n_rounds=10)
print(f"Clients: {results['n_clients']}")
print(f"Data per client: {results['data_per_client']}")
print(f"Total federated data: {results['total_federated_data']}")
print(f"Centralized model accuracy: {results['centralized_accuracy']}\n")
print(f"{'Round':<8} {'Global Acc':<15} {'Avg Client':<15} {'Min Client':<15} {'Max Client':<15}")
print("-" * 68)
for r in results["rounds"]:
print(f"{r['round']:<8} {r['global_accuracy']:<15} {r['avg_client_accuracy']:<15} "
f"{r['min_client_accuracy']:<15} {r['max_client_accuracy']:<15}")
# Generate report
lines = ["# Federated Learning Simulation Report\n"]
lines.append(f"**Clients:** {results['n_clients']}")
lines.append(f"**Data per client:** {results['data_per_client']}")
lines.append(f"**Centralized model accuracy:** {results['centralized_accuracy']}\n")
lines.append("## Round-by-Round Results\n")
lines.append("| Round | Global Accuracy | Avg Client Accuracy | Min Client | Max Client |")
lines.append("|---|---|---|---|---|")
for r in results["rounds"]:
lines.append(f"| {r['round']} | {r['global_accuracy']} | {r['avg_client_accuracy']} | "
f"{r['min_client_accuracy']} | {r['max_client_accuracy']} |")
lines.append(f"\n## Comparison: Federated vs. Centralized\n")
lines.append(f"- Centralized model accuracy: {results['centralized_accuracy']}")
lines.append(f"- Federated model accuracy (final round): {results['rounds'][-1]['global_accuracy']}")
lines.append(f"- Accuracy gap: {round(results['centralized_accuracy'] - results['rounds'][-1]['global_accuracy'], 4)}\n")
lines.append("## Privacy Benefits")
lines.append("- Raw data never left any client — only model weights were shared")
lines.append("- The server never saw individual training examples")
lines.append("- Each client retained full control over their data\n")
lines.append("## Limitations Observed")
lines.append("- Non-IID data: clients have different distributions, causing variance in client accuracy")
lines.append("- The federated model may underperform centralized training due to non-IID data")
lines.append("- Gradient leakage risk: model weights can still reveal information about training data")
lines.append("- Mitigation: combine with differential privacy (DP-FedAvg) for formal privacy guarantees")
with open("federated_learning_report.md", "w") as f:
f.write("\n".join(lines))
print("\nReport saved to federated_learning_report.md")
In the federated learning simulation, the global model's accuracy is lower than the centralized model's accuracy. Why?
1
A hospital consortium wants to train a shared diagnostic model across 8 hospitals without any patient records leaving each hospital's servers. One hospital's IT director objects: "Even if raw data never leaves, couldn't someone still learn something about our patients from the model weights we send to the server?" Is the IT director right to be concerned?
Federated learning is a privacy-improving architecture, not a privacy guarantee. It removes the single point of failure of a central data repository and ensures raw records never leave the client, which is a real and valuable improvement. But the model weights and gradients sent to the server are themselves a function of the local training data, and research has repeatedly shown that gradients can leak information — including, in some cases, reconstructing training examples via gradient inversion, or supporting membership inference against the local dataset. The IT director's instinct is professionally correct: "the data didn't move" is not the same claim as "no information about the data moved." The standard mitigation is DP-FedAvg — adding calibrated differential-privacy noise to the gradients before they're sent to the server — which gives you a formal epsilon-based guarantee on top of federated learning's architectural privacy benefit.