mmlearn.modules.layers.mlp.MLP

class MLP(in_dim, out_dim=None, hidden_dims=None, hidden_dims_multiplier=None, apply_multiplier_to_in_dim=False, norm_layer=None, activation_layer=<class 'torch.nn.modules.activation.ReLU'>, bias=True, dropout=0.0)[source]

Bases: Sequential

Multi-layer perceptron (MLP).

This module will create a block of Linear -> Normalization -> Activation -> Dropout layers.

Parameters:

  • in_dim (int) – The input dimension.

  • out_dim (Optional[int], optional, default=None) – The output dimension. If not specified, it is set to in_dim.

  • hidden_dims (Optional[list], optional, default=None) – The dimensions of the hidden layers. The length of the list determines the number of hidden layers. This parameter is mutually exclusive with hidden_dims_multiplier.

  • hidden_dims_multiplier (Optional[list], optional, default=None) – The multipliers to apply to the input dimension to get the dimensions of the hidden layers. The length of the list determines the number of hidden layers. The multipliers will be used to get the dimensions of the hidden layers. This parameter is mutually exclusive with hidden_dims.

  • apply_multiplier_to_in_dim (bool, optional, default=False) – Whether to apply the hidden_dims_multiplier to in_dim to get the dimensions of the hidden layers. If False, the multipliers will be applied to the dimensions of the previous hidden layer, starting from in_dim. This parameter is only relevant when hidden_dims_multiplier is specified.

  • norm_layer (Optional[Callable[..., torch.nn.Module]], optional, default=None) – The normalization layer to use. If not specified, no normalization is used. Partial functions can be used to specify the normalization layer with specific parameters.

  • activation_layer (Optional[Callable[..., torch.nn.Module]], optional, default=torch.nn.ReLU) – The activation layer to use. If not specified, ReLU is used. Partial functions can be used to specify the activation layer with specific parameters.

  • bias (bool, optional, default=True) – Whether to use bias in the linear layers.

  • dropout (float, optional, default=0.0) – The dropout probability to use.

Raises:

ValueError – If both hidden_dims and hidden_dims_multiplier are specified or if the lengths of bias and hidden_dims do not match or if the lengths of dropout and hidden_dims do not match.

Methods

Attributes