fl4health.parameter_exchange.parameter_packer module

class ParameterPacker[source]

Bases: ABC, Generic[T]

abstract pack_parameters(model_weights, additional_parameters)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

abstract unpack_parameters(packed_parameters)[source]
Return type:

tuple[List[ndarray[Any, dtype[Any]]], TypeVar(T)]

class ParameterPackerAdaptiveConstraint[source]

Bases: ParameterPacker[float]

pack_parameters(model_weights, extra_adaptive_variable)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

unpack_parameters(packed_parameters)[source]
Return type:

tuple[List[ndarray[Any, dtype[Any]]], float]

class ParameterPackerWithClippingBit[source]

Bases: ParameterPacker[float]

pack_parameters(model_weights, additional_parameters)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

unpack_parameters(packed_parameters)[source]
Return type:

tuple[List[ndarray[Any, dtype[Any]]], float]

class ParameterPackerWithControlVariates(size_of_model_params)[source]

Bases: ParameterPacker[List[ndarray[Any, dtype[Any]]]]

__init__(size_of_model_params)[source]

Class to handle the exchange of control variates for the SCAFFOLD FL method

NOTE model params exchanged and control variates can be different sizes, for example, when layers are frozen or the state dictionary contains things like Batch Normalization layers.

Parameters:

size_of_model_params (int) – This is the number of layers that are associated with the parameters of the model itself. This is used to split the covariates from the model parameters during unpacking.

pack_parameters(model_weights, additional_parameters)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

unpack_parameters(packed_parameters)[source]
Return type:

tuple[List[ndarray[Any, dtype[Any]]], List[ndarray[Any, dtype[Any]]]]

class ParameterPackerWithLayerNames[source]

Bases: ParameterPacker[list[str]]

pack_parameters(model_weights, weights_names)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

unpack_parameters(packed_parameters)[source]

Function to separate the model parameters from the layer names that have been packed with them.

Parameters:

packed_parameters (NDArrays) – packed_parameters is a list containing model parameters followed by an NDArray that contains the corresponding names of those parameters.

Returns:

tuple of model parameters and the names of the layers to which they correspond

Return type:

tuple[NDArrays, list[str]]

class SparseCooParameterPacker[source]

Bases: ParameterPacker[tuple[List[ndarray[Any, dtype[Any]]], List[ndarray[Any, dtype[Any]]], list[str]]]

This parameter packer is responsible for selecting an arbitrary set of parameters and then representing them in the sparse COO tensor format, which requires knowing the indices of the parameters within the tensor to which they belong, the shape of that tensor, and also the name of it.

For more information on the sparse COO format and sparse tensors in PyTorch, please see the following two pages:

  1. https://pytorch.org/docs/stable/generated/torch.sparse_coo_tensor.html

  2. https://pytorch.org/docs/stable/sparse.html

static extract_coo_info_from_dense(x)[source]

Take a dense tensor x and extract the information required (namely, its nonzero values, their indices within the tensor, and the shape of x) in order to represent it in the sparse coo format.

The results are converted to numpy arrays.

Parameters:

x (Tensor) – Input dense tensor.

Returns:

The nonzero values of x, the indices of those values within x, and the shape of x.

Return type:

tuple[NDArray, NDArray, NDArray]

pack_parameters(model_parameters, additional_parameters)[source]
Return type:

List[ndarray[Any, dtype[Any]]]

unpack_parameters(packed_parameters)[source]
Return type:

tuple[List[ndarray[Any, dtype[Any]]], tuple[List[ndarray[Any, dtype[Any]]], List[ndarray[Any, dtype[Any]]], list[str]]]