The Federated Engine
====================

Scaleout Edge utilizes **Federated Learning (FL)** as its core computational engine. This engine enables the platform to train, fine-tune, and adapt models across sovereign boundaries without ever requiring raw data to leave its source.
    
Why Federated Learning
----------------------

In traditional machine learning, data must be moved to a central repository for processing. This centralization often introduces significant friction due to bandwidth limitations, privacy regulations, and security risks.
The Federated Engine allows for model training directly at the edge. Instead of moving data to the model, the infrastructure **moves the model to the data.**

Core Infrastructure Benefits
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Federated Learning addresses three fundamental infrastructure challenges inherent in distributed environments:  

1. **Data Sovereignty:** Data used by ML algorithms (training and inference) never leaves the edge nodes. Only mathematical model updates (weights/gradients) are transmitted. This ensures compliance with GDPR, HIPAA, and strict data residency laws by design.
2. **Bandwidth Efficiency:** By processing data locally and only sharing model updates, federated learning minimizes the amount of data transmitted over the network.
3. **Scalability:** The architecture can easily scale to accommodate a growing number of edge nodes and data sources without requiring significant changes to the underlying infrastructure.
4. **Continuous Learning:** For unsupervised datasets or using self-supervised techniques, federated learning enables models to be continuously updated as new data becomes available, without the need for retraining from scratch.

The Workflow Cycle
------------------

The engine operates in synchronized rounds managed by the Control Plane. A single round consists of four phases:

1. **Distribution (Control Plane -> Data Plane) -** The current Global Model is broadcasted to selected Edge Nodes.
2. **Local Training (Data Plane): -** Each Edge Node executes the training locally on its private dataset. This computation happens entirely within the node's secure environment.
3. **Model Update Stream (Data Plane -> Aggregation Layer): -** Nodes generate a "Model Update"—a set of mathematical weights representing what was learned. This update is streamed securely (outbound-only) to the Aggregation Layer (Combiners).
4. **Aggregation (Aggregation Layer -> Control Plane) -** The Combiners aggregate thousands of updates into a single "Partial Model." These are then merged by the Reducer to create the next version of the Global Model, which is committed to the Model Registry.

This cycle repeats until the global model converges to the desired performance level.

Supported Algorithms
--------------------

The engine is algorithm-agnostic but comes with built-in support for industry-standard aggregation strategies.

- **Federated Averaging (FedAvg):** The default algorithm where model weights from clients are averaged to form the global model.
- **FedOpt (FedAdam, FedYogi, FedAdaGrad):** Advanced adaptive optimization algorithms designed to handle non-IID data (data that varies significantly between edge nodes) and improve convergence speed.

Custom Aggregation
~~~~~~~~~~~~~~~~~~

For advanced use cases, developers can implement custom aggregation logic and client selection strategies using **server functions** (:ref:`server-functions`). This allows for tailored workflows that meet specific application requirements while still leveraging the core federated learning capabilities of the platform.