Federated Learning Fundamentals

Federated learning (FL) is a machine learning paradigm where models are trained across multiple decentralized devices or servers holding local data, without exchanging the data itself. Instead of bringing data to the model, federated learning brings the model to the data.

How Federated Learning Works

The basic federated learning process:

1. Server Initialization — a central server initializes a global model
2. Client Selection — server selects a subset of clients for this training round
3. Model Distribution — server sends the current global model to selected clients
4. Local Training — each client trains the model on its local data for several epochs
5. Update Transmission — clients send model updates (gradients or weights) back to the server
6. Aggregation — server combines client updates (typically via FedAvg) to create an improved global model
7. Iteration — repeat steps 2-6 until convergence

FedAvg (Federated Averaging)

The foundational aggregation algorithm:

• Each client trains locally for multiple epochs
• Updates are weighted by the number of local training samples
• Server computes a weighted average of all client models
• Simple yet effective for many IID (independent and identically distributed) scenarios

Types of Federated Learning

• Cross-Device FL — millions of mobile phones or IoT devices (e.g., Google Gboard keyboard prediction)
• Cross-Silo FL — multiple organizations collaborating (e.g., hospitals sharing medical AI models)
• Vertical FL — different organizations have different features about the same entities
• Federated Transfer Learning — transfer knowledge when data distributions and feature spaces differ

Real-World Deployments

Federated learning is already deployed at scale:

• Google Gboard — next-word prediction trained on billions of phones without collecting typing data
• Apple — Siri improvements, QuickType, and facial recognition without uploading photos
• Healthcare — multi-hospital medical imaging AI without sharing patient records
• Finance — cross-bank fraud detection without exposing transaction data