fl4health.parameter_exchange.parameter_selection_criteria module

class LayerSelectionFunctionConstructor(norm_threshold, exchange_percentage, normalize=True, select_drift_more=True)

Bases: object

__init__(norm_threshold, exchange_percentage, normalize=True, select_drift_more=True)

This class leverages functools.partial to construct layer selection functions, which are meant to be used by the DynamicLayerExchanger class.

Parameters:

• norm_threshold (float) – A nonnegative real number used to select those layers whose drift in \(\ell^2\) norm exceeds (or falls short of) it.

• exchange_percentage (float) – Indicates the percentage of layers that are selected.

• normalize (bool, optional) – Indicates whether when calculating the norm of a layer, we also divide by the number of parameters in that layer. Defaults to True.

• select_drift_more (bool, optional) – Indicates whether layers with larger drift norm are selected. Defaults to True.

select_by_percentage()

Return type:

Callable[[Module, Module | None], tuple[list[ndarray[Any, dtype[Any]]], list[str]]]

select_by_threshold()

Return type:

Callable[[Module, Module | None], tuple[list[ndarray[Any, dtype[Any]]], list[str]]]

largest_final_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

largest_increase_in_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

largest_magnitude_change_scores(model, initial_model)

Return type:

dict[str, Tensor]

select_layers_by_percentage(exchange_percentage, normalize, select_drift_more, model, initial_model)

Return type:

tuple[list[ndarray[Any, dtype[Any]]], list[str]]

select_layers_by_threshold(threshold, normalize, select_drift_more, model, initial_model)

Return those layers of model that deviate (in \(\ell^2\) norm) away from corresponding layers of self.initial_model by at least (or at most) self.threshold.

Parameters:

• threshold (float) – Drift threshold to be used for selection. It is an fixed value.

• normalize (bool) – Whether to divide the difference between the tensors by their number of elements.

• select_drift_more (bool) – Whether we are selecting parameters that have drifted further (True) or less far from their comparison values

• model (nn.Module) – Model after training/modification.

• initial_model (nn.Module) – Model that we started with to which we are comparing parameters.

Returns:

Layers selected by the process and their corresponding names in the model’s state_dict.

Return type:

tuple[NDArrays, list[str]]

select_scores_and_sample_masks(model, initial_model)

Selection function that first selects the weight_scores and bias_scores parameters for the masked layers, and then samples binary masks based on those scores to send to the server. This function is meant to be used for the FedPM algorithm.

NOTE: in the current implementation, we always exchange the score tensors for all layers. In the future, we might support exchanging a subset of the layers (for example, filtering out the masks that are all zeros).

Return type:

tuple[list[ndarray[Any, dtype[Any]]], list[str]]

smallest_final_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

smallest_increase_in_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

smallest_magnitude_change_scores(model, initial_model)

Return type:

dict[str, Tensor]