Real-world federated learning faces significant challenges that require advanced techniques to overcome.

Non-IID Data (Statistical Heterogeneity)

The biggest challenge in FL — data across clients is not identically distributed:

• Label Skew — different clients have different label distributions (a hospital may see mostly one disease type)
• Feature Skew — same labels but different feature distributions (medical devices vary)
• Quantity Skew — some clients have much more data than others
• Temporal Skew — data distributions change over time differently on each client

Solutions for non-IID data:

1. FedProx — adds a proximal term to keep local models close to the global model
2. SCAFFOLD — uses control variates to correct client drift
3. Per-FedAvg — meta-learning approach for personalized models
4. Clustered FL — group similar clients and train separate models per cluster

Communication Efficiency

Transmitting model updates is expensive, especially for large models:

• Gradient Compression — quantize or sparsify updates to reduce size
• Federated Distillation — share knowledge through smaller models
• Partial Model Updates — only update and transmit subsets of model parameters
• Asynchronous FL — don't wait for all clients; aggregate updates as they arrive

Systems Heterogeneity

Clients have different computational capabilities:

• Heterogeneous Models — clients train models of different sizes
• Adaptive Computation — adjust local training based on device capability
• Client Selection — intelligently select which clients participate each round
• Timeout Handling — gracefully handle slow or disconnected clients

Privacy Attacks on FL

Federated learning alone doesn't guarantee privacy:

• Gradient Inversion — reconstruct training data from transmitted gradients
• Membership Inference — determine if specific data was used in training
• Model Poisoning — malicious clients corrupt the global model
• Free-Riding — clients benefit from the model without contributing real updates

Defense Mechanisms

Combining FL with additional privacy protections:

• Differential Privacy — add noise to client updates
• Secure Aggregation — cryptographic protocols so server only sees aggregate
• Trusted Execution Environments — process updates in secure enclaves
• Robust Aggregation — detect and filter malicious client updates