mmlearn.datasets.core.modalities

Module for managing supported modalities in the library.

Classes

Modality	A representation of a modality in the library.
ModalityRegistry	Modality registry.

class Modality(name, modality_specific_properties=None)

A representation of a modality in the library.

This class is used to represent a modality in the library. It contains the name of the modality and the properties that can be associated with it. The properties are dynamically generated based on the name of the modality and can be accessed as attributes of the class.

Parameters:

• name (str) – The name of the modality.

• modality_specific_properties (Optional[dict[str, str]], optional, default=None) – Additional properties specific to the modality, by default None

Raises:

ValueError – If the property already exists for the modality or if the format string is invalid.

add_property(name, format_string)

Add a new property to the modality.

Parameters:

• name (str) – The name of the property.

• format_string (str) – The format string for the property. The format string should contain a placeholder that will be replaced with the name of the modality when the property is accessed.

Warns:

UserWarning – If the property already exists for the modality. It will overwrite the existing property.

Raises:

ValueError – If format_string is invalid. A valid format string contains at least one placeholder enclosed in curly braces.

Return type:

None

attention_mask: str

ema_embedding: str

Embedding from an Exponential Moving Average (EMA) encoder associated with the modality.

embedding: str

Embedding associated with the modality. This will return name_embedding.

mask: str

Input mask associated with the modality. This will return name_mask.

masked_embedding: str

modality_specific_properties: Optional[dict[str, str]] = None

Other properties specific to the modality.

name: str

The name of the modality.

property properties: dict[str, str]

Return the properties associated with the modality.

target: str

Target/label associated with the modality. This will return name_target.

class ModalityRegistry

Modality registry.

A singleton class that manages the supported modalities (and their properties) in the library. The class provides methods to add new modalities and properties, and to access the existing modalities. The class is implemented as a singleton to ensure that there is only one instance of the registry in the library.

add_default_property(name, format_string)

Add a new property that is applicable to all modalities.

Parameters:

• name (str) – The name of the property.

• format_string (str) – The format string for the property. The format string should contain a placeholder that will be replaced with the name of the modality when the property is accessed.

Warns:

UserWarning – If the property already exists for the default properties. It will overwrite the existing property.

Raises:

ValueError – If the format string is invalid. A valid format string contains at least one placeholder enclosed in curly braces.

Return type:

None

get_modality(name)

Get the modality name from the registry.

Parameters:

name (str) – The name of the modality.

Returns:

The modality object from the registry.

Return type:

Modality

get_modality_properties(name)

Get the properties of a modality from the registry.

Parameters:

name (str) – The name of the modality.

Returns:

The properties associated with the modality.

Return type:

dict[str, str]

has_modality(name)

Check if the modality exists in the registry.

Parameters:

name (str) – The name of the modality.

Returns:

True if the modality exists in the registry, False otherwise.

Return type:

bool

list_modalities()

Get the list of supported modalities in the registry.

Returns:

The list of supported modalities in the registry.

Return type:

list[Modality]

register_modality(name, modality_specific_properties=None)

Add a new modality to the registry.

Parameters:

• name (str) – The name of the modality.

• modality_specific_properties (Optional[dict[str, str]], optional, default=None) – Additional properties specific to the modality.

Warns:

UserWarning – If the modality already exists in the registry. It will overwrite the existing modality.

Return type:

None