fl4health.privacy.moments_accountant module

class FixedSamplingWithoutReplacement(population_size, sample_size)

Bases: SamplingStrategy

get_dp_event(noise_event)

Return type:

DpEvent

class MomentsAccountant(moment_orders=None)

Bases: object

__init__(moment_orders=None)

Moment orders are equivalent to lambda from Deep Learning with Differential Privacy (Abadi et. al. 2016). They form the set of moments to estimate the infimum of Theorem 2 part 2. The default values were taken from the tensorflow federated DP tutorial notebook:

https://github.com/tensorflow/federated/blob/main/docs/tutorials/federated_learning_with_differential_privacy.ipynb

In the paper above, they state that trying lambda <= 32 is usually sufficient.

Sampling type is the data point sampling strategy: i.e. examples from dataset for a batch with probability q Noise type specifies whether Gaussian or Laplacian noise is added to the updates

Parameters:

moment_orders (list[float] | None, optional) – See description above. Defaults to None.

get_delta(sampling_strategies, noise_multiplier, updates, epsilon)

If the parameters are lists, then it is assumed that the training applied the parameters in a sequence of updates.

Ex.

sampling_strategies = [PoissonSampling(q_1), PoissonSampling(q_2)]

noise_multiplier = [z_1, z_2]

updates = [t_1, t_2]

implies that q_1, z_1 were applied for t_1 updates, followed by q_2, z_2 for t_2 updates.

Parameters:

• sampling_strategies (SamplingStrategy | Sequence[SamplingStrategy]) –

  Are the type of sampling done for each datapoint or client in the DP procedure. This is either Poisson sampling with a sampling rate specified or Fixed ratio sampling with a fixed number of selections performed over a specified population size. For non-FL DP-SGD: This is the ratio of batch size to dataset size (L/N, from Deep Learning with Differential Privacy). For FL with client-side DP-SGD (no noise on server side, instance level privacy): This is the ratio of client sampling probability to client data point probability q*(b_k/n_k) For FL with client privacy: This is the sampling of clients from the client population.

  NOTE: If a sequence of strategies is given, they must be all of the same kind (that is poisson or subset, but may have different parameters)

• noise_multiplier (float | list[float]) – Ratio of the noise standard deviation to clipping bound (sigma in Deep Learning with Differential Privacy, z in some other implementations).

• updates (int | list[int]) – This is the number of noise applications to the update weights. For non-FL DP-SGD: This is the number of updates run (epochs*batches per epoch) For FL w/ client-side DP-SGD (instance DP): This is the number of batches run per client (if selected every time), server_updates*epochs per server update*batches per epoch For FL with client privacy: Number of server updates

• epsilon (float) – This is the epsilon in (epsilon, delta)-Privacy, that we require.

Returns:

delta for the provided epsilon

Return type:

float

get_epsilon(sampling_strategies, noise_multiplier, updates, delta)

If the parameters are lists, then it is assumed that the training applied the parameters in a sequence of updates.

Ex.

sampling_strategies = [PoissonSampling(q_1), PoissonSampling(q_2)]

noise_multiplier = [z_1, z_2]

updates = [t_1, t_2]

implies that q_1, z_1 were applied for t_1 updates, followed by q_2, z_2 for t_2 updates.

Parameters:

• sampling_strategies (SamplingStrategy | Sequence[SamplingStrategy]) –

  Are the type of sampling done for each datapoint or client in the DP procedure. This is either Poisson sampling with a sampling rate specified or Fixed ratio sampling with a fixed number of selections performed over a specified population size. For non-FL DP-SGD: This is the ratio of batch size to dataset size (L/N, from Deep Learning with Differential Privacy). For FL with client-side DP-SGD (no noise on server side, instance level privacy): This is the ratio of client sampling probability to client data point probability q*(b_k/n_k) For FL with client privacy: This is the sampling of clients from the client population

  NOTE: If a sequence of strategies is given, they must be all of the same kind (that is poisson or subset, but may have different parameters)

• noise_multiplier (float | list[float]) – Ratio of the noise standard deviation to clipping bound (sigma in Deep Learning with Differential Privacy, z in some other implementations).

• updates (int | list[int]) – This is the number of noise applications to the update weights. For non-FL DP-SGD: This is the number of updates run (epochs*batches per epoch) For FL w/ client-side DP-SGD (instance DP): This is the number of batches run per client (if selected every time), server_updates*epochs per server update*batches per epoch. For FL with client privacy: Number of server updates.

• delta (float) – This is the delta in (epsilon, delta)-Privacy, that we require.

Returns:

The corresponding epsilon

Return type:

float

class PoissonSampling(sampling_ratio)

Bases: SamplingStrategy

get_dp_event(noise_event)

Return type:

DpEvent

class SamplingStrategy(neighbor_relation)

Bases: ABC

abstract get_dp_event(noise_event)

Return type:

DpEvent