fl4health.parameter_exchange.sparse_coo_parameter_exchanger module¶
- class SparseCooParameterExchanger(sparsity_level, score_gen_function)[source]¶
Bases:
PartialParameterExchanger[tuple[List[ndarray[Any,dtype[Any]]],List[ndarray[Any,dtype[Any]]],list[str]]]- __init__(sparsity_level, score_gen_function)[source]¶
Parameter exchanger for sparse tensors.
This exchanger is responsible for selecting an arbitrary subset of a model’s parameters via some selection criterion and then packaging them into the COO sparse tensor format for exchanging.
For more information on the sparse COO format and sparse tensors in PyTorch, please see the following two pages:
- Parameters:
sparsity_level (float) – The level of sparsity. Must be between 0 and 1.
score_gen_function (ScoreGenFunction) – Function that is responsible for
selection. (generating a score for every parameter inside a model in order to facilitate parameter)
cases (In most)
model (this function takes as inputs a current model and an initial)
:param : :param and it returns a dictionary that maps the name of each of the current model’s tensors to: :param another tensor which contains the parameter scores.:
- generate_parameter_scores(model, initial_model)[source]¶
Calling the score generating function to produce parameter scores.
- select_parameters(model, initial_model=None)[source]¶
Select model parameters according to the sparsity level and pack them into the sparse COO format to be exchanged.
First, this method leverages a score generating function to generate scores for all parameters of model.
Next, these scores are used to select the parameters to be exchanged by performing a thresholding operation on each of the model’s tensors. A threshold is determined according to the desired sparsity level, then for each model tensor, parameters whose scores are less than this threshold are set to zero, while parameters whose scores are greater than or equal to this threshold retain their values.
Finally, the method extracts all the information required to represent the selected parameters in the sparse COO tensor format. More specifically, the information consists of the indices of the parameters within the tensor to which they belong, the shape of that tensor, and also the name of it.