fl4health.parameter_exchange.parameter_selection_criteria module

class LayerSelectionFunctionConstructor(norm_threshold, exchange_percentage, normalize=True, select_drift_more=True)

Bases: object

__init__(norm_threshold, exchange_percentage, normalize=True, select_drift_more=True)

This class leverages functools.partial to construct layer selection functions, which are meant to be used by the DynamicLayerExchanger class.

Parameters:

• norm_threshold (float) – A nonnegative real number used to select those layers whose drift in l2 norm exceeds (or falls short of) it.

• exchange_percentage (float) – Indicates the percentage of layers that are selected.

• normalize (bool, optional) – Indicates whether when calculating the norm of a layer, we also divide by the number of parameters in that layer. Defaults to True.

• select_drift_more (bool, optional) – Indicates whether layers with larger drift norm are selected. Defaults to True.

select_by_percentage()

Return type:

Callable[[Module, Module | None], tuple[List[ndarray[Any, dtype[Any]]], list[str]]]

select_by_threshold()

Return type:

Callable[[Module, Module | None], tuple[List[ndarray[Any, dtype[Any]]], list[str]]]

largest_final_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

largest_increase_in_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

largest_magnitude_change_scores(model, initial_model)

Return type:

dict[str, Tensor]

select_layers_by_percentage(exchange_percentage, normalize, select_drift_more, model, initial_model)

Return type:

tuple[List[ndarray[Any, dtype[Any]]], list[str]]

select_layers_by_threshold(threshold, normalize, select_drift_more, model, initial_model)

Return those layers of model that deviate (in l2 norm) away from corresponding layers of self.initial_model by at least (or at most) self.threshold.

Return type:

tuple[List[ndarray[Any, dtype[Any]]], list[str]]

select_scores_and_sample_masks(model, initial_model)

Selection function that first selects the “weight_scores” and “bias_scores” parameters for the masked layers, and then samples binary masks based on those scores to send to the server. This function is meant to be used for the FedPM algorithm.

Return type:

tuple[List[ndarray[Any, dtype[Any]]], list[str]]

Note: in the current implementation, we always exchange the score tensors for all layers. In the future, we might

support exchanging a subset of the layers (for example, filtering out the masks that are all zeros).

smallest_final_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

smallest_increase_in_magnitude_scores(model, initial_model)

Return type:

dict[str, Tensor]

smallest_magnitude_change_scores(model, initial_model)

Return type:

dict[str, Tensor]