mmlearn.datasets.core.combined_dataset.CombinedDataset

class CombinedDataset(datasets)

Bases: Dataset[Example]

Combine multiple datasets into one.

This class is similar to ConcatDataset but allows for combining iterable-style datasets with map-style datasets. The iterable-style datasets must implement the __len__() method, which is used to determine the total length of the combined dataset. When an index is passed to the combined dataset, the dataset that contains the example at that index is determined and the example is retrieved from that dataset. Since iterable-style datasets do not support random access, the examples are retrieved sequentially from the iterable-style datasets. When the end of an iterable-style dataset is reached, the iterator is reset and the next example is retrieved from the beginning of the dataset.

Parameters:

datasets (Iterable[Union[torch.utils.data.Dataset, torch.utils.data.IterableDataset]]) – Iterable of datasets to combine.

Raises:

• TypeError – If any of the datasets in the input iterable are not instances of Dataset or IterableDataset.

• ValueError – If the input iterable of datasets is empty.

Methods

__getitem__(idx)

Return an example from the combined dataset.

Return type:

Example