fl4health.strategies.fedavg_sparse_coo_tensor module

class FedAvgSparseCooTensor(*, fraction_fit=1.0, fraction_evaluate=1.0, min_fit_clients=2, min_evaluate_clients=2, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters=None, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_aggregation=True, weighted_eval_losses=True)

Bases: BasicFedAvg

__init__(*, fraction_fit=1.0, fraction_evaluate=1.0, min_fit_clients=2, min_evaluate_clients=2, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters=None, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_aggregation=True, weighted_eval_losses=True)

A generalization of the FedAvg strategy where the server can receive any arbitrary subset of parameters from any arbitrary subset of the clients. Weighted average for parameters belonging to each received tensor is performed independently.

Note that this strategy differs from FedAvgDynamicLayer in that it does not require clients to send entire layers (tensors). A client can send an arbitrary set of parameters within a certain tensor, and these parameters are packed according to the sparse COO format.

For more information on the sparse COO format and sparse tensors in PyTorch, please see the following two pages:

1. https://pytorch.org/docs/stable/generated/torch.sparse_coo_tensor.html

2. https://pytorch.org/docs/stable/sparse.html

Parameters:

• fraction_fit (float, optional) – Fraction of clients used during training. Defaults to 1.0. Defaults to 1.0.

• fraction_evaluate (float, optional) – Fraction of clients used during validation. Defaults to 1.0.

• min_fit_clients (int, optional) – _description_. Defaults to 2.

• min_evaluate_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

• min_available_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

• evaluate_fn (Callable[[int, NDArrays, dict[str, Scalar]], tuple[float, dict[str, Scalar]] | None] | None) – Optional function used for central server-side evaluation. Defaults to None.

• on_fit_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure training by providing a configuration dictionary. Defaults to None.

• on_evaluate_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure client-side validation by providing a Config dictionary. Defaults to None.

• accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

• initial_parameters (Parameters | None, optional) – Initial global model parameters. Defaults to None.

• fit_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

• evaluate_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

• weighted_aggregation (bool, optional) – Determines whether parameter aggregation is a linearly weighted average or a uniform average. FedAvg default is weighted average by client dataset counts. Defaults to True.

• weighted_eval_losses (bool, optional) – Determines whether losses during evaluation are linearly weighted averages or a uniform average. FedAvg default is weighted average of the losses by client dataset counts. Defaults to True.

aggregate(results)

Aggregate the different tensors across clients that have contributed to a certain tensor. This aggregation may be weighted or unweighted. The called functions handle tensor alignment.

Parameters:

• results (list[tuple[NDArrays, int]]) – The weight results from each client’s local training

• samples (that need to be aggregated on the server-side and the number of training)

• client. (held on each) – In this scheme, the clients pack the tensor names into the results object along with the weight values to allow for alignment during aggregation.

Returns:

A dictionary mapping the name of the tensor that was aggregated to the aggregated

weights.

Return type:

dict[str, Tensor]

aggregate_fit(server_round, results, failures)

Aggregate the results from the federated fit round. The aggregation requires some special treatment, as the participating clients are allowed to exchange an arbitrary set of parameters. So before aggregation takes place alignment must be done using the tensor names packed in along with the weights in the client results.

More precisely, this method performs the following steps:

1. Align all tensors according to their names.

2. For tensors that have the same name, construct the sparse COO tensors and convert them to dense tensors.

3. Perform averaging on the dense tensors (can be weighted or unweighted).

4. For every aggregated dense tensor, discard the zero values and retain all information needed to represent it in the sparse COO format.

Parameters:

• server_round (int) – Indicates the server round we’re currently on.

• results (list[tuple[ClientProxy, FitRes]]) – The client identifiers and the results of their local training that need to be aggregated on the server-side. In this scheme, the clients pack the tensor names into the results object along with the weight values to allow for alignment during aggregation.

• failures (list[tuple[ClientProxy, FitRes] | BaseException]) – These are the results and exceptions from clients that experienced an issue during training, such as timeouts or exceptions.

Returns:

The aggregated model weights and the metrics dictionary.

For sparse tensor exchange we also pack in the names of all of the tensors that were aggregated in this phase to allow clients to insert the values into the proper areas of their models.

Return type:

tuple[Parameters | None, dict[str, Scalar]]

unweighted_aggregate(results)

“results” consist of four parts: the exchanged (nonzero) parameter values, their coordinates within the tensor to which they belong, the shape of that tensor, and finally the name of that tensor.

The first three items constitute the information that is needed to construct the tensor in the sparse COO format and convert it to a regular dense tensor. The tensor name is used to align tensors to ensure that averaging is performed only among tensors with the same name.

This method performs the following steps:

1. Align all tensors according to their names.

2. For tensors that have the same name, construct the sparse COO tensors and convert them to dense tensors.

3. Perform uniform averaging on the dense tensors across all clients.

Note: this method performs uniform averaging.

Parameters:

results (list[tuple[NDArrays, int]]) – The weight results from each client’s local training that need to be aggregated on the server-side and the number of training samples held on each client. The weight results consist of four parts, as detailed above. In this scheme, the clients pack the layer names into the results object along with the weight values to allow for alignment during aggregation.

Returns:

A dictionary mapping the name of the tensor that was aggregated to the aggregated

weights.

Return type:

dict[str, Tensor]

weighted_aggregate(results)

“results” consist of four parts: the exchanged (nonzero) parameter values, their coordinates within the tensor to which they belong, the shape of that tensor, and finally the name of that tensor.

The first three items constitute the information that is needed to construct the tensor in the sparse COO format and convert it to a regular dense tensor. The tensor name is used to align tensors to ensure that averaging is performed only among tensors with the same name.

This method performs the following steps:

1. Align all tensors according to their names.

2. For tensors that have the same name, construct the sparse COO tensors and convert them to dense tensors.

3. Perform weighted averaging on the dense tensors according to the number of training examples each client has.

Note: this method performs weighted averaging.

Parameters:

results (list[tuple[NDArrays, int]]) – The weight results from each client’s local training that need to be aggregated on the server-side and the number of training samples held on each client. The weight results consist of four parts, as detailed above. In this scheme, the clients pack the layer names into the results object along with the weight values to allow for alignment during aggregation.

Returns:

A dictionary mapping the name of the tensor that was aggregated to the aggregated

weights.

Return type:

dict[str, Tensor]