fl4health.model_bases.masked_layers.masked_conv module¶

class MaskedConv1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: Conv1d

__init__(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Implementation of masked Conv1d layers.

Like regular Conv1d layers (i.e., nn.Conv1d module), a masked convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained.

In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int, tuple or str, optional) – Padding added to both sides of the input. Default: 0
padding_mode (str, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

forward(input)[source]¶

Forward for the mask 1D Convolution

Parameters:: input (Tensor) – input tensor for the layer
Returns:: output tensor for the convolution
Return type:: Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConv1d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.Conv1d) – Module to be converted
Returns:: Module with masked layers added to enable FedPM training
Return type:: MaskedConv1d

class MaskedConv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: Conv2d

__init__(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Implementation of masked Conv2d layers.

Like regular Conv2d layers (i.e., nn.Conv2d module), a masked convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained. In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int, tuple or str, optional) – Padding added to all four sides of the input. Default: 0
padding_mode (str, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

forward(input)[source]¶

Forward for the Masked 2D Convolution

Parameters:: input (Tensor) – input tensor for the layer
Returns:: output tensor for the convolution
Return type:: Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConv2d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.Conv2d) – Module to be converted
Returns:: Module with masked layers to enable FedPM
Return type:: MaskedConv2d

class MaskedConv3d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: Conv3d

__init__(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', device=None, dtype=None)[source]¶

Implementation of masked Conv2d layers.

Like regular Conv3d layers (i.e., nn.Conv3d module), a masked convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained. In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int, tuple or str, optional) – Padding added to all six sides of the input. Default: 0
padding_mode (str, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

forward(input)[source]¶

Forward for the Masked 3D Convolution

Parameters:: input (Tensor) – input tensor for the layer
Returns:: output tensor for the convolution
Return type:: Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConv3d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.Conv3d) – Module to convert
Returns:: Module with mask layers added to enable FedPM
Return type:: MaskedConv3d

class MaskedConvTranspose1d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: ConvTranspose1d

__init__(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)[source]¶

Implementation of masked ConvTranspose1d layers. For more information on transposed convolution, please see the PyTorch implementation of nn.Conv1d.

(https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#ConvTranspose1d)

Like regular ConvTranspose1d layers (i.e., nn.ConvTranspose1d module), a masked transpose convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained. In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the transposed convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int or tuple, optional) – dilation * (kernel_size - 1) - padding zero-padding will be added to both sides of the input. Default: 0
output_padding (int or tuple, optional) – Additional size added to one side of the output shape. Default: 0
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

forward(input, output_size=None)[source]¶

Forward for the MaskedConvTranspose1D

Parameters:

input (Tensor) – input to be mapped with the module
output_size (list[int] | None, optional) – Desired output from the transpose. Defaults to None.

Raises:

ValueError – If something other than “zeros” padding has been requested.

Returns:

Output tensors.

Return type:

Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConvTranspose1d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.ConvTranspose1d) – Target module to be converted
Returns:: Module with masked layers to enable FedPM
Return type:: MaskedConvTranspose1d

class MaskedConvTranspose2d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: ConvTranspose2d

Implementation of masked ConvTranspose2d layers. For more information on transposed convolution, please see the PyTorch implementation of nn.Conv2d.

(https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#ConvTranspose2d)

Like regular ConvTranspose2d layers (i.e., nn.ConvTranspose2d module), a masked transpose convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained. In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the transposed convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int or tuple, optional) – dilation * (kernel_size - 1) - padding zero-padding will be added to both sides of each dimension in the input. Default: 0
output_padding (int or tuple, optional) – Additional size added to one side of each dimension in the output shape. Default: 0
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

forward(input, output_size=None)[source]¶

Maps input tensor through the MaskedConvTranspose2D module

Parameters:

input (Tensor) – tensor to be mapped
output_size (list[int] | None, optional) – Desired output size from the module. Defaults to None.

Raises:

ValueError – Thrown if anything except “zeros” padding is requested

Returns:

Mapped tensor

Return type:

Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConvTranspose2d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.ConvTranspose2d) – Target module to be converted
Returns:: Module with mask layers added to enable FedPM
Return type:: MaskedConvTranspose2d

class MaskedConvTranspose3d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros', device=None, dtype=None)[source]¶

Bases: ConvTranspose3d

Implementation of masked ConvTranspose3d layers. For more information on transposed convolution, please see the PyTorch implementation of nn.Conv3d.

(https://pytorch.org/docs/stable/_modules/torch/nn/modules/conv.html#ConvTranspose3d)

Like regular ConvTranspose3d layers (i.e., nn.ConvTranspose3d module), a masked transpose convolutional layer has a weight (i.e., convolutional filter) and a (optional) bias. However, the weight and the bias do not receive gradient in back propagation. Instead, two score tensors - one for the weight and another for the bias - are maintained. In the forward pass, the score tensors are transformed by the Sigmoid function into probability scores, which are then used to produce binary masks via Bernoulli sampling. Finally, the binary masks are applied to the weight and the bias. During training, gradients with respect to the score tensors are computed and used to update the score tensors.

NOTE: The scores are not assumed to be bounded between 0 and 1.

Parameters:

in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the transposed convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int or tuple, optional) – dilation * (kernel_size - 1) - padding zero-padding will be added to both sides of each dimension in the input. Default: 0
output_padding (int or tuple, optional) – Additional size added to one side of each dimension in the output shape. Default: 0
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

forward(input, output_size=None)[source]¶

Maps the input tensor with MaskedConvTranspose3D

Parameters:

input (Tensor) – Tensor to be mapped
output_size (list[int] | None, optional) – Desired output size from the transpose. Defaults to None.

Raises:

ValueError – Throws if anything except “zeros” padding is requested

Returns:

Mapped tensor

Return type:

Tensor

classmethod from_pretrained(conv_module)[source]¶

Return an instance of MaskedConvTranspose3d whose weight and bias have the same values as those of conv_module.

Parameters:: conv_module (nn.ConvTranspose3d) – Target module to be converted
Returns:: Module with masked layers added to enable FedPM
Return type:: MaskedConvTranspose3d