fl4health.strategies.client_dp_fedavgm module

class ClientLevelDPFedAvgM(*, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters=None, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_aggregation=False, weighted_eval_losses=True, per_client_example_cap=None, adaptive_clipping=False, server_learning_rate=1.0, clipping_learning_rate=1.0, clipping_quantile=0.5, initial_clipping_bound=0.1, weight_noise_multiplier=1.0, clipping_noise_multiplier=1.0, beta=0.9)

Bases: BasicFedAvg

__init__(*, fraction_fit=1.0, fraction_evaluate=1.0, min_available_clients=2, evaluate_fn=None, on_fit_config_fn=None, on_evaluate_config_fn=None, accept_failures=True, initial_parameters=None, fit_metrics_aggregation_fn=None, evaluate_metrics_aggregation_fn=None, weighted_aggregation=False, weighted_eval_losses=True, per_client_example_cap=None, adaptive_clipping=False, server_learning_rate=1.0, clipping_learning_rate=1.0, clipping_quantile=0.5, initial_clipping_bound=0.1, weight_noise_multiplier=1.0, clipping_noise_multiplier=1.0, beta=0.9)

This strategy implements the Federated Learning with client-level DP approach discussed in Differentially Private Learning with Adaptive Clipping. This function provides a noised version of unweighted FedAvgM. NOTE: It assumes that the models are packaging clipping bits along with the model parameters. If adaptive clipping is false, these bits will simply be 0.

Paper: https://arxiv.org/abs/1905.03871 If enabled, it performs adaptive clipping rather than fixed threshold clipping.

Parameters:

• fraction_fit (float, optional) – Fraction of clients used during training. Defaults to 1.0.

• fraction_evaluate (float, optional) – Fraction of clients used during validation. Defaults to 1.0.

• min_available_clients (int, optional) – Minimum number of clients used during validation. Defaults to 2.

• evaluate_fn (Callable[[int, NDArrays, dict[str, Scalar]], tuple[float, dict[str, Scalar]] | None] | None) – Optional function used for central server-side evaluation. Defaults to None.

• on_fit_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure training by providing a configuration dictionary. Defaults to None.

• on_evaluate_config_fn (Callable[[int], dict[str, Scalar]] | None, optional) – Function used to configure client-side validation by providing a Config dictionary. Defaults to None.

• accept_failures (bool, optional) – Whether or not accept rounds containing failures. Defaults to True.

• initial_parameters (Parameters | None, optional) – Initial global model parameters. This strategy assumes that the initial parameters is not None. So they need to be set in spite of the optional tag.

• fit_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

• evaluate_metrics_aggregation_fn (MetricsAggregationFn | None, optional) – Metrics aggregation function. Defaults to None.

• weighted_aggregation (bool, optional) – Determines whether the FedAvg update is weighted by client dataset size or unweighted. Defaults to False.

• weighted_eval_losses (bool, optional) – Determines whether losses during evaluation are linearly weighted averages or a uniform average. FedAvg default is weighted average of the losses by client dataset counts. Defaults to True.

• per_client_example_cap (float | None, optional) – The maximum number samples per client. hat{w} in https://arxiv.org/pdf/1710.06963.pdf. Defaults to None.

• adaptive_clipping (bool, optional) – If enabled, the model expects the last entry of the parameter list to be a binary value indicating whether or not the batch gradient was clipped. Defaults to False.

• server_learning_rate (float, optional) – Learning rate for the server side updates. Defaults to 1.0.

• clipping_learning_rate (float, optional) – Learning rate for the clipping bound. Only used if adaptive clipping is turned on. Defaults to 1.0.

• clipping_quantile (float, optional) – Quantile we are trying to estimate in adaptive clipping. i.e. P(||g|| < C_t) pprox clipping_quantile. Only used if adaptive clipping is turned on. Defaults to 0.5.

• initial_clipping_bound (float, optional) – Initial guess for the clipping bound corresponding to the clipping quantile described above. NOTE: If Adaptive clipping is turned off, this is the clipping bound through out FL training.. Defaults to 0.1.

• weight_noise_multiplier (float, optional) – Noise multiplier for the noising of gradients. Defaults to 1.0.

• clipping_noise_multiplier (float, optional) – Noise multiplier for the noising of clipping bits. Defaults to 1.0.

• beta (float, optional) – Momentum weight for previous weight updates. If it is 0, there is no momentum. Defaults to 0.9.

aggregate_fit(server_round, results, failures)

Aggregate fit using averaging of weights (can be unweighted or weighted) and inject noise and optionally perform adaptive clipping updates. NOTE: This assumes that the model weights sent back by the clients are UPDATES rather than raw weights. That is they are theta_client - theta_server rather than just theta_client. NOTE: this function packs the clipping bound for clients as the last member of the parameters list.

Parameters:

• server_round (int) – Indicates the server round we’re currently on.

• results (list[tuple[ClientProxy, FitRes]]) – The client identifiers and the results of their local training that need to be aggregated on the server-side. In this strategy, the clients pack the weights to be aggregated along with a clipping bit calculated during their local training cycle.

• failures (list[tuple[ClientProxy, FitRes] | BaseException]) – These are the results and exceptions from clients that experienced an issue during training, such as timeouts or exceptions.

Returns:

The aggregated model weights and the metrics dictionary.

For this strategy, the server also packs a clipping bound to be sent to the clients. This is sent even if adaptive clipping is turned off and the value simply remains constant.

Return type:

tuple[Parameters | None, dict[str, Scalar]]

calculate_update_with_momentum(weights_update)

Performs a weight update with momentum. That is, combining some weighted value of the previous update with the current update.

Parameters:

• weights_update (NDArrays) – The current update after the weights have been aggregated from the training

• round.

Return type:

None

configure_evaluate(server_round, parameters, client_manager)

This function configures a sample of clients for an eval round. Due to the privacy accounting, this strategy requires that the sampling manager be of type BaseFractionSamplingManager.

The function follows the standard configuration flow where the on_evaluate_config_fn function is used to produce configurations to be sent to all clients. These are packaged with the provided parameters and set over to the clients.

Parameters:

• server_round (int) – Indicates the server round we’re currently on.

• parameters (Parameters) – The parameters to be used to initialize the clients for the eval round.

• client_manager (ClientManager) – The manager used to grab all of the clients. Currently we restrict this to be BaseFractionSamplingManager, which has a sample_fraction function built in.

Returns:

List of sampled client identifiers and the configuration/parameters

to be sent to each client (packaged as EvaluateIns)

Return type:

list[tuple[ClientProxy, EvaluateIns]]

configure_fit(server_round, parameters, client_manager)

This function configures a sample of clients for a training round. Due to the privacy accounting, this strategy requires that the sampling manager be of type BaseFractionSamplingManager.

The function follows the standard configuration flow where the on_fit_config_fn function is used to produce configurations to be sent to all clients. These are packaged with the provided parameters and set over to the clients.

Parameters:

• server_round (int) – Indicates the server round we’re currently on.

• parameters (Parameters) – The parameters to be used to initialize the clients for the fit round.

• client_manager (ClientManager) – The manager used to sample the clients. Currently we restrict this to be BaseFractionSamplingManager, which has a sample_fraction function built in.

Returns:

List of sampled client identifiers and the configuration/parameters to

be sent to each client (packaged as FitIns).

Return type:

list[tuple[ClientProxy, FitIns]]

modify_noise_multiplier()

Modifying the noise multiplier as in Algorithm 1 of Differentially Private Learning with Adaptive Clipping. This is done to ensure the privacy accountant computes the correct privacy values.

Raises:

ValueError – If the noise multiplier and the clipping noise multiplier are not well related then we’ll end up with a sqrt of a negative number. If this happens a value error is raised.

Returns:

The modified noise multiplier when performing adaptive clipping.

Return type:

float

split_model_weights_and_clipping_bits(results)

Given results from an FL round of training, this function splits the result into sets of (weights, training counts) and clipping bits. The split is required because the clipping bits are packed with the weights in order to communicate them back to the server. The parameter packer facilitates this splitting.

Parameters:

results (list[tuple[ClientProxy, FitRes]]) – The client identifiers and the results of their local training that need to be aggregated on the server-side. In this strategy, the clients pack the weights to be aggregated along with a clipping bit calculated during training.

Returns:

The first tuple is the set of (weights, training counts) per

client. The second is a set of clipping bits, one for each client.

Return type:

tuple[list[tuple[NDArrays, int]], NDArrays]

update_clipping_bound(clipping_bits)

This adds noise to the clipping bits returned by the clients and then updates the server-side clipping bound using this information.

Parameters:

• clipping_bits (NDArrays) – Bits associated with each of the clients. These are to be noised and aggregated

• side. (in order to update the clipping bound on the server)

Return type:

None

update_current_weights()

This function updates each of the layer weights using the server learning rate and the m_t values (computed with or without momentum). NOTE: It assumes that the values in m_t are UPDATES rather than raw weights.

Return type:

None