fl4health.strategies.scaffold module

class OpacusScaffold(*, model, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_eval_losses=True, learning_rate=1.0)[source]

Bases: Scaffold

__init__(*, model, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_eval_losses=True, learning_rate=1.0)[source]

A simple extension of the Scaffold strategy to force the model being federally trained to be an valid Opacus GradSamplingModule and, thereby, ensure that associated the parameters are aligned with those of Opacus based models used by the InstanceLevelDpClient.

Parameters:

  • model (nn.Module) – The model architecture to be federally trained. When using this strategy, the provided model must be of type Opacus GradSampleModule. This model will then be used to set initialize_parameters as the initial parameters to be used by all clients AND the initial_control_variates. NOTE: The initial_control_variates are all initialized to zero, as recommended in the SCAFFOLD paper. If one wants a specific type of control variate initialization, this class will need to be overridden.

  • fraction_fit (float, optional) – Fraction of clients used during training. Defaults to 1.0.

  • fraction_evaluate (float, optional) – Fraction of clients used during validation. Defaults to 1.0.

  • min_available_clients (int, optional) – Minimum number of total clients in the system. Defaults to 2.

  • evaluate_fn (Callable[[int, NDArrays, dict[str, Scalar]], tuple[float, dict[str, Scalar]] | None] | None) – Optional function used for central server-side evaluation. Defaults to None.

  • on_fit_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure training by providing a configuration dictionary. Defaults to None.

  • on_evaluate_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure client-side validation by providing a Config dictionary. Defaults to None.

  • accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

  • fit_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • evaluate_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • weighted_eval_losses (bool, optional) – Determines whether losses during evaluation are linearly weighted averages or a uniform average. FedAvg default is weighted average of the losses by client dataset counts. Defaults to True.

  • learning_rate (float, optional) – Learning rate for server side optimization. Defaults to 1.0.

class Scaffold(*, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_eval_losses=True, learning_rate=1.0, initial_control_variates=None, model=None)[source]

Bases: BasicFedAvg

__init__(*, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_eval_losses=True, learning_rate=1.0, initial_control_variates=None, model=None)[source]

Scaffold Federated Learning strategy. Implementation based on https://arxiv.org/pdf/1910.06378.pdf :type initial_parameters: Parameters :param initial_parameters: Initial model parameters to which all client models are set. :type initial_parameters: Parameters :type fraction_fit: float :param fraction_fit: Fraction of clients used during training. Defaults to 1.0. :type fraction_fit: float, optional :type fraction_evaluate: float :param fraction_evaluate: Fraction of clients used during validation. Defaults to 1.0. :type fraction_evaluate: float, optional :type min_available_clients: int :param min_available_clients: Minimum number of total clients in the system.

Defaults to 2.

Parameters:

  • evaluate_fn (Callable[[int, NDArrays, dict[str, Scalar]], tuple[float, dict[str, Scalar]] | None] | None) – Optional function used for central server-side evaluation. Defaults to None.

  • on_fit_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure training by providing a configuration dictionary. Defaults to None.

  • on_evaluate_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure client-side validation by providing a Config dictionary. Defaults to None.

  • accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

  • fit_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • evaluate_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • weighted_eval_losses (bool, optional) – Determines whether losses during evaluation are linearly weighted averages or a uniform average. FedAvg default is weighted average of the losses by client dataset counts. Defaults to True.

  • learning_rate (float, optional) – Learning rate for server side optimization. Defaults to 1.0.

  • initial_control_variates (Parameters | None, optional) – These are the initial set of control variates to use for the scaffold strategy both on the server and client sides. It is optional, but if it is not provided, the strategy must receive a model that reflects the architecture to be used on the clients. Defaults to None.

  • model (nn.Module | None, optional) – If provided and initial_control_variates is not, this is used to set the server control variates and the initial control variates on the client side to all zeros. If initial_control_variates are provided, they take precedence. Defaults to None.

aggregate(params)[source]

Simple unweighted average to aggregate params, consistent with SCAFFOLD paper. This is “element-wise” averaging.

Parameters:

params (list[NDArrays]) – numpy arrays whose entries are to be averaged together.

Returns:

element-wise average over the list of numpy arrays.

Return type:

NDArrays

aggregate_fit(server_round, results, failures)[source]

Performs server-side aggregation of model weights and control variates associated with the SCAFFOLD method Both model weights and control variates are aggregated through UNWEIGHTED averaging consistent with the paper. The newly aggregated weights and control variates are then repacked and sent back to the clients.

This function also handles aggregation of training run metrics (i.e. accuracy over the local training etc.) through the fit_metrics_aggregation_fn provided in constructing the strategy.

Parameters:

  • server_round (int) – What round of FL we’re on (from servers perspective).

  • results (list[tuple[ClientProxy, FitRes]]) – These are the “successful” training run results. By default these results are the only ones used in aggregation, even if some of the failed clients have partial results (in the failures list).

  • failures (list[tuple[ClientProxy, FitRes] | BaseException]) – This is the list of clients that “failed” during the training phase for one reason or another, including timeouts and exceptions.

Returns:

The aggregated weighted and metrics dictionary. The

parameters are optional and will be none in the even that there are no successful clients or there were failures and they are not accepted.

Return type:

tuple[Parameters | None, dict[str, Scalar]]

compute_parameter_delta(params_1, params_2)[source]

Computes element-wise difference of two lists of NDarray where elements in params_2 are subtracted from elements in params_1.

Parameters:

  • params_1 (NDArrays) – Parameters to be subtracted from.

  • params_2 (NDArrays) – Parameters to subtract from params_1.

Returns:

Element-wise subtraction result across all numpy arrays.

Return type:

NDArrays

compute_updated_control_variates(control_variates_update)[source]

Given the aggregated control variates from the clients, this updates the server control variates in line with the paper. If c is the server control variates and c_update is the client control variates, then this update takes the form

c + |S| / N * c_update,

where |S| is the number of clients that participated and N is the total number of clients |S|/N is the proportion given by fraction fit.

Parameters:

control_variates_update (NDArrays) – Aggregated control variates received from the clients (uniformly averaged).

Returns:

Updated server control variates according to the formula.

Return type:

NDArrays

compute_updated_parameters(scaling_coefficient, original_params, parameter_updates)[source]

Computes updated_params by moving in the direction of parameter_updates with a step proportional the scaling coefficient. Calculates original_params + scaling_coefficient * parameter_updates.

Parameters:

  • scaling_coefficient (float) – Scaling length for the parameter updates (can be thought of as “learning rate”).

  • original_params (NDArrays) – parameters to be updated.

  • parameter_updates (NDArrays) – update direction to update the original_params.

Returns:

Updated numpy arrays according to original_params + scaling_coefficient * parameter_updates.

Return type:

NDArrays

compute_updated_weights(weights)[source]

Computes and update to the current self.server_model_weights. This assumes that the weights represents the raw weights aggregated from the client. Therefore it first needs to be turned into a “delta” with

weights - self.server_model_weights.

Then this is used to update with a learning rate scalar (set by self.learning_rate) as

self.server_model_weights + self.learning_rate * (weights - self.server_model_weights).

Parameters:

weights (NDArrays) – The updated weights (aggregated from the clients).

Returns:

self.server_model_weights + self.learning_rate * (weights - self.server_model_weights)

These are the updated server model weights.

Return type:

NDArrays

configure_fit_all(server_round, parameters, client_manager)[source]

This function configures ALL clients for a training round. That is, it forces the client manager to grab all of the available clients to participate in the training round. By default, the manager will at least wait for the min_available_clients threshold to be met. Thereafter it will simply grab all available clients for participation.

The function follows the standard configuration flow where the on_fit_config_fn function is used to produce configurations to be sent to all clients. These are packaged with the provided parameters and set over to the clients.

Parameters:

  • server_round (int) – Indicates the server round we’re currently on.

  • parameters (Parameters) – The parameters to be used to initialize the clients for the fit round.

  • client_manager (ClientManager) – The manager used to grab all of the clients. Currently we restrict this to be BaseFractionSamplingManager, which has a “sample all” function built in.

Returns:

List of sampled client identifiers and the configuration/parameters to

be sent to each client (packaged as FitIns).

Return type:

list[tuple[ClientProxy, FitIns]]

initialize_control_variates(initial_control_variates, model)[source]

This is a helper function for the SCAFFOLD strategy init function to initialize the server_control_variates. It either initializes the control variates with custom provided variates or using the provided model architecture.

Parameters:

  • initial_control_variates (Parameters | None) – These are the initial set of control variates to use for the scaffold strategy both on the server and client sides. It is optional, but if it is not provided, the strategy must receive a model that reflects the architecture to be used on the clients. Defaults to None.

  • model (nn.Module | None) – If provided and initial_control_variates is not, this is used to set the server control variates and the initial control variates on the client side to all zeros. If initial_control_variates are provided, they take precedence. Defaults to None.

Returns:

This quantity represents the initial values for the control variates for the server and on the client-side.

Return type:

Parameters

Raises:

ValueError – This error will be raised if neither a model nor initial control variates are provided.