fl4health.strategies.fedavg_with_adaptive_constraint module

class FedAvgWithAdaptiveConstraint(*, fraction_fit=1.0, fraction_evaluate=1.0, min_fit_clients=2, min_evaluate_clients=2, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, initial_loss_weight=1.0, adapt_loss_weight=False, loss_weight_delta=0.1, loss_weight_patience=5, weighted_aggregation=True, weighted_eval_losses=True, weighted_train_losses=False)[source]

Bases: BasicFedAvg

__init__(*, fraction_fit=1.0, fraction_evaluate=1.0, min_fit_clients=2, min_evaluate_clients=2, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, initial_loss_weight=1.0, adapt_loss_weight=False, loss_weight_delta=0.1, loss_weight_patience=5, weighted_aggregation=True, weighted_eval_losses=True, weighted_train_losses=False)[source]

A generalization of the fedavg strategy for approaches that use a penalty constraint that we might want to adapt based on the loss trajectory. A quintessential example is FedProx, which uses an l2 penalty on model weight drift and potentially adapts the coefficient based on the aggregated loss. In addition to the model weights, the server also receives the training loss from the clients. If adaptation is enabled, these losses are used to update the loss weight parameter according to the FedProx paper recommendations.

NOTE: Initial parameters are NOT optional. They must be passed for this strategy.

The aggregation strategy for weights is the same as in FedAvg.

Implementation based on https://arxiv.org/abs/1602.05629.

Parameters:

  • fraction_fit (float, optional) – Fraction of clients used during training. Defaults to 1.0.

  • fraction_evaluate (float, optional) – Fraction of clients used during validation. Defaults to 1.0.

  • min_fit_clients (int, optional) – _description_. Defaults to 2.

  • min_evaluate_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

  • min_available_clients (int, optional) – Minimum number of total clients in the system. Defaults to 2.

  • evaluate_fn (Callable[[int, NDArrays, dict[str, Scalar]], tuple[float, dict[str, Scalar]] | None] | None) – Optional function used for central server-side evaluation. Defaults to None.

  • on_fit_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure training by providing a configuration dictionary. Defaults to None.

  • on_evaluate_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure client-side validation by providing a Config dictionary. Defaults to None.

  • accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

  • initial_parameters (Parameters) – Initial global model parameters.

  • fit_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • evaluate_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

  • initial_loss_weight (float) – Initial loss weight (mu in FedProx). If adaptivity is false, then this is the constant weight used for all clients.

  • adapt_loss_weight (bool, optional) – Determines whether the value of mu is adaptively modified by the server based on aggregated train loss. Defaults to False.

  • loss_weight_delta (float, optional) – This is the amount by which the server changes the value of mu based on the modification criteria. Only applicable if adaptivity is on. Defaults to 0.1.

  • loss_weight_patience (int, optional) – This is the number of rounds a server must see decreasing aggregated train loss before reducing the value of mu. Only applicable if adaptivity is on. Defaults to 5.

  • weighted_aggregation (bool, optional) – Determines whether parameter aggregation is a linearly weighted average or a uniform average. FedAvg default is weighted average by client dataset counts. Defaults to True.

  • weighted_eval_losses (bool, optional) – Determines whether losses during evaluation are linearly weighted averages or a uniform average. FedAvg default is weighted average of the losses by client dataset counts. Defaults to True.

  • weighted_train_losses (bool, optional) – Determines whether the training losses from the clients should be aggregated using a weighted or unweighted average. These aggregated losses are used to adjust the proximal weight in the adaptive setting. Defaults to False.

aggregate_fit(server_round, results, failures)[source]

Aggregate the results from the federated fit round and, if applicable, determine whether the constraint weight should be updated based on the aggregated loss seen on the clients.

Parameters:

  • server_round (int) – Indicates the server round we’re currently on.

  • results (list[tuple[ClientProxy, FitRes]]) – The client identifiers and the results of their local training that need to be aggregated on the server-side. For adaptive constraints, the clients pack the weights to be aggregated along with the training loss seen during their local training cycle.

  • failures (list[tuple[ClientProxy, FitRes] | BaseException]) – These are the results and exceptions from clients that experienced an issue during training, such as timeouts or exceptions.

Returns:

The aggregated model weights and the metrics dictionary.

For adaptive constraints, the server also packs a constraint weight to be sent to the clients. This is sent even if adaptive constraint weights are turned off and the value simply remains constant.

Return type:

tuple[Parameters | None, dict[str, Scalar]]