Scaleout Edge Clients

This section documents the client implementations available for Scaleout Edge. Clients run on edge nodes and communicate with the Scaleout Edge network to perform local training, evaluation, and model exchange. Multiple client implementations exist to support different environments and programming languages.

Python Client

The Python client provides a high-level API for integrating local training code with the Scaleout Edge network. It is suitable for servers, development machines, notebooks, and lightweight edge nodes.

class scaleout.EdgeClient(train_callback: Callable[[ScaleoutModel, Dict], Tuple[ScaleoutModel | None, Dict]] | None = None, validate_callback: Callable[[ScaleoutModel], Dict] | None = None, runtime: EdgeClientRuntime | None = None)[source]

Bases: object

User-facing interface for an edge client.

Users instantiate this class directly. The runtime defaults to GrpcEdgeClientRuntime; passing runtime= at construction time swaps the implementation — useful for tests and alternative transports.

__init__(train_callback: Callable[[ScaleoutModel, Dict], Tuple[ScaleoutModel | None, Dict]] | None = None, validate_callback: Callable[[ScaleoutModel], Dict] | None = None, runtime: EdgeClientRuntime | None = None) → None[source]: Initialize the EdgeClient.

check_task_abort() → None[source]

Check if the ongoing task has been aborted.

This function should be called periodically from the task callback to ensure that the task can be interrupted if needed. If called from a thread that do not run the task, this function is a no-op.

Raises:: StoppedException – If the task was aborted.

connect_to_api(url: str, json: dict | None = None, token: str | None = None, token_refresh_callback: Callable[[...], None] | None = None) → Tuple[ConnectToApiResult, Any][source]: Connect to the Scaleout API via the runtime.

property current_logging_context: LoggingContext | None: Get the current logging context for the running thread.

get_access_token() → str | None[source]: Return the current access token, if the runtime manages one.

init_grpchandler(config: GrpcConnectionOptions, token: str | None = None, url: str | None = None, token_refresh_callback: Callable[[...], None] | None = None) → bool[source]: Initialize the runtime’s transport handler.

log_attributes(attributes: dict, check_task_abort: bool = True) → bool[source]

Log the attributes to the server.

Parameters:

attributes (dict) – The attributes to log.
check_task_abort (bool, optional) – Whether or not to check for task abort. Defaults to True.

Returns:

True if the attributes were logged successfully, False otherwise.

Return type:

bool

log_metric(metrics: dict, step: int = None, commit: bool = True, check_task_abort: bool = True, context: LoggingContext = None) → bool[source]

Log the metrics to the server.

Parameters:

metrics (dict) – The metrics to log.
step (int, optional) – The step number.
value. (If provided the context step will be set to this)
provided (If not)
used. (the step from the context will be)
commit (bool, optional) – Whether or not to increment the step. Defaults to True.
check_task_abort (bool, optional) – Whether or not to check for task abort. Defaults to True.
context (LoggingContext, optional) – The logging context to use. Defaults to None, which uses the current context.

Returns:

True if the metrics were logged successfully, False otherwise.

Return type:

bool

log_telemetry(telemetry: dict, check_task_abort: bool = True) → bool[source]

Log the telemetry data to the server.

Parameters:

telemetry (dict) – The telemetry data to log.
check_task_abort (bool, optional) – Whether or not to check for task abort. Defaults to True.

Returns:

True if the telemetry data was logged successfully, False otherwise.

Return type:

bool

logging_context(context: LoggingContext)[source]: Set the logging context for the duration of the block.

remove_custom_callback(callback_name: str) → None[source]: Remove a custom task callback.

run(with_heartbeat: bool = False, with_polling: bool = True) → None[source]: Run the client’s event loop via the runtime.

run_inference(model: ScaleoutModel | str = None, params: Dict = None) → None[source]

Run inference using the specified model.

Parameters:

model – The ScaleoutModel or model ID string to use for inference.
params – Additional parameters for inference.

set_client_id(client_id: str) → None[source]: Set the client ID.

set_custom_callback(callback_name: str, callback: Callable[[scaleoututil.grpc.scaleout_pb2.TaskRequest], Dict]) → None[source]: Set a custom task callback.

set_inference_callback(callback: Callable[[ScaleoutModel, Dict], Any]) → None[source]: Set the inference callback.

set_name(name: str) → None[source]: Set the client name.

set_stage_model_callback(callback: Callable[[ScaleoutModel], None]) → None[source]: Set the stage-model callback, invoked after a model is staged for inference.

set_train_callback(callback: callable) → None[source]: Set the train callback.

set_validate_callback(callback: callable) → None[source]: Set the validate callback.

stage_model(model: ScaleoutModel | str) → ScaleoutModel[source]

Stage a model for inference.

Parameters:: model – The ScaleoutModel or model id to stage.

class scaleout.ScaleoutModel[source]

Bases: object

The ScaleoutModel class is the primary model representation in the Scaleout framework.

A ScaleoutModel is a self-describing, immutable container holding: - The serialized model weights (raw bytes, via a helper) - An optional full model representation (e.g. ONNX, .pt, …) - A metadata dict (eagerly loaded): model_id, helper, inference_model_format, and any user-supplied key-value pairs

Models are always immutable after construction. To create a modified copy, use to_builder() to obtain a pre-seeded ScaleoutModelBuilder:

new_model = model.to_builder().set_model_id("m-002").build()
new_model = model.to_builder().set_metadata("tag", "v2").build()

API

Reading metadata:

model.metadata          # shallow copy of the metadata dict
model.model_id          # convenience property

Model parameters:

with model.get_training_model_stream() as s:  # read directly from ZIP, no temp file
    data = s.read()
params = model.get_training_model(helper)     # decodes from ZIP

Full model representation:

model.has_inference_model                         # bool
model.get_inference_model_format()                # format string or None
with model.get_inference_model_stream() as s:    # stream inference_model.bin directly from ZIP
    data = s.read()

On-disk format

Files are ZIP archives (stdlib zipfile, DEFLATED) containing:: metadata.json – eagerly loaded on open training_model.bin – serialized weights inference_model.bin – optional full model representation

Legacy raw-binary streams (no ZIP header, e.g. bare NPZ) are detected automatically and loaded with empty metadata for backward compatibility.

Storage

A single ZIP file (_zip_path) is the canonical representation from construction time onwards. The model always owns its ZIP and deletes it on garbage collection.

get_training_model_stream() and get_inference_model_stream() stream entries directly from the ZIP without extracting to a temp file.

__init__()[source]

property checksum: str

static detect_format(stream: BytesIO) → str[source]

Returns ‘zip’ if data is our ZIP container format, else ‘legacy’.

Legacy NPZ files (produced by NumpyHelper) are also ZIP-based and start with the PK magic bytes. We distinguish them from our container by checking for the presence of ‘metadata.json’ inside the archive.

static detect_format_file(file_path: str) → str[source]

Returns ‘zip’ if the file is our ZIP container format, else ‘legacy’.

Reads only the central directory — does not load the entire file.

static from_file(file_path: str) → ScaleoutModel[source]

Creates a ScaleoutModel from a file (ZIP or legacy raw binary).

Delegates to ScaleoutModelBuilder.

static from_filechunk_stream(filechunk_stream: Iterable[scaleoututil.grpc.scaleout_pb2.FileChunk]) → ScaleoutModel[source]

Creates a ScaleoutModel from a gRPC FileChunk iterator.

Delegates to ScaleoutModelBuilder.

static from_stream(stream: BinaryIO) → ScaleoutModel[source]

Creates a ScaleoutModel from a stream (ZIP or legacy raw binary).

Delegates to ScaleoutModelBuilder.

static from_training_model(model_params, helper=None, metadata: dict | None = None) → ScaleoutModel[source]

Creates a ScaleoutModel from model parameters.

Delegates to ScaleoutModelBuilder. A fresh model_id is always generated; any model_id key present in metadata is ignored.

Parameters:

model_params – Parameters accepted by the helper’s save() method.
helper – Serialization helper. Falls back to NumpyHelper if not provided.
metadata – Optional initial metadata dict.

get_file_stream()[source]: Returns a new read handle to the packed ZIP file.

get_filechunk_stream(chunk_size=1048576)[source]: Yields gRPC FileChunk messages of the raw model bytes.

get_inference_model_format() → str | None[source]: Returns the format string of the full model representation, or None.

get_inference_model_stream()[source]

Context manager yielding a read stream of the full model representation binary.

Reads directly from the ZIP entry — no temp-file extraction.

Usage:

if model.has_inference_model:
    with model.get_inference_model_stream() as stream:
        data = stream.read()

get_model_params(helper=None)[source]: Alias for get_training_model() for backward compatibility.

get_training_model(helper=None)[source]: Decodes and returns the model parameters via the helper.

get_training_model_stream()[source]

Context manager yielding a read stream of the raw model binary.

Reads directly from the ZIP entry — no temp-file extraction.

Usage:

with model.get_training_model_stream() as stream:
    data = stream.read()

property has_inference_model: bool: True if the model contains a full model representation.

property has_training_model: bool: True if the model contains serialized parameters.

property legacy_source: bool: True if this model was loaded from a legacy raw binary (no ZIP).

property metadata: dict: Returns a shallow copy of the metadata dict.

property model_id: str | None

save_to_file(file_path: str)[source]: Saves the model to a file in ZIP format.

sign(private_key, signer_id: str | None = None) → dict[source]

Compute and return a signature dict for this model.

Does not modify the ZIP. POST the returned dict to /api/v1/model-signatures/ to persist it.

Parameters:

private_key – An Ed25519PrivateKey from the cryptography package.
signer_id – Optional free-form string identifying the signer.

Returns:

A dict with keys model_id, algorithm, signature (base64-encoded), and optionally signer_id.

to_builder() → ScaleoutModelBuilder[source]

Return a factory pre-loaded from this model’s current state.

Use this to create a modified copy without mutating the original:

new_model = model.to_builder().set_model_id("m-002").build()
new_model = model.to_builder().set_metadata("tag", "v2").build()

verify_checksum(checksum: str) → bool[source]

verify_signature(public_key, signature: str) → bool[source]

Return True if signature is valid for this model.

Parameters:

public_key – An Ed25519PublicKey from the cryptography package.
signature – Base64-encoded signature string — as returned by GET /api/v1/model-signatures/<id> signature field.

Additional Clients

Support for additional client implementations (e.g., C++ and Kotlin) will be included here in future versions of the documentation.