mmlearn.datasets.core.data_collator

Data collators for batching examples.

Functions

collate_example_list(examples)

Collate a list of Example objects into a batch.

Parameters:

examples (list[Example]) – list of examples to collate.

Returns:

Dictionary of batched examples.

Return type:

dict[str, Any]

Classes

DefaultDataCollator

Default data collator for batching examples.

class DefaultDataCollator(batch_processors=None)

Default data collator for batching examples.

This data collator will collate a list of Example objects into a batch. It can also apply processing functions to specified keys in the batch before returning it.

Parameters:

batch_processors (Optional[dict[str, Callable[[Any], Any]]], optional, default=None) – Dictionary of callables to apply to the batch before returning it.

Raises:

ValueError – If the batch processor for a key does not return a dictionary with the key in it.

__call__(examples)

Collate a list of Example objects and apply processing functions.

Return type:

dict[str, Any]

batch_processors: Optional[dict[str, Callable[[Any], Any]]] = None

Dictionary of callables to apply to the batch before returning it. The key is the name of the key in the batch, and the value is the processing function to apply to the key. The processing function must take a single argument and return a single value. If the processing function returns a dictionary, it must contain the key that was processed in it (all the other keys will also be included in the batch).

collate_example_list(examples)

Collate a list of Example objects into a batch.

Parameters:

examples (list[Example]) – list of examples to collate.

Returns:

Dictionary of batched examples.

Return type:

dict[str, Any]