dataset module

class dataset.Audio_Dataset(train_transformation=None, test_transformation=None)[source]

Bases: object

Avalanche audio datasets wrapper.

MLCommons(root='../dataset/', sub_folder='subset2', subset='training', transforms=None)[source]

MLCommons dataset wrapper function for avalanche lib.

Parameters:

  • root (str, optional) – dataset root location. Defaults to ‘../dataset/’.

  • sub_folder (str, optional) – dataset subset. Defaults to “subset2”.

  • subset (str, optional) – one of ‘training’, ‘validation’, ‘testing’. Defaults to “training”.

  • transforms (_type_, optional) – transformations applied to the data. Defaults to None.

Returns:

Avalanche’s classification dataset

Return type:

ClassificationDataset

SpeechCommands(root=PosixPath('/home/joe/.avalanche/data/speechcommands'), url='speech_commands_v0.02', download=True, subset=None, transforms=None, pre_process=True, output_shape=[])[source]

SpeechCommands dataset wrapper function for avalanche lib.

Parameters:

  • root (str, optional) – dataset root location. Defaults to default_dataset_location(“speechcommands”).

  • url (str, optional) – version name of the dataset. Defaults to “speech_commands_v0.02”.

  • download (bool, optional) – automatically download the dataset, if not present. Defaults to True.

  • subset (str, optional) – one of ‘training’, ‘validation’, ‘testing’. Defaults to None.

  • transforms (torch.nn.Module, optional) – transformations applied to the data. Defaults to None.

  • pre_process (bool, optional) – Enable prior preprocessing and saving of the dataset. Defaults to True.

  • output_shape (list) – Output shape of a transformed element.

Raises:

ValueError – If an unkown subset is chosen

Returns:

Avalanche’s classification dataset

Return type:

ClassificationDataset

class dataset.CachedAudio(subset, train_cache_path='../dataset_cache/', test_cache_path='../dataset_cache/')[source]

Bases: Dataset

Wrapper for cached hdf5 audio datasets.

class dataset.MLcommonsData(root, sub_folder, subset, folder_in_archive='MLCommons')[source]

Bases: object

Wrapper for a subset of the MlCommons Multilingual Spoken Words dataset

get_metadata(n: int) → Tuple[str, int, str, str, int][source]
class dataset.SpeechCommandsData(root, url, download, subset)[source]

Bases: SPEECHCOMMANDS

Wrapper for torchaudio’s speechcommand dataset.

dataset.preprocess_and_save_dataset(dataset, save_path: str, transformation, output_shape=[])[source]

Function for preprocessing and saving datasets.

Important

This function only works for the SpeechCommands dataset, or for dataset that have those specific entries : wave, label, rate, speaker_id, utterance_number

Parameters:

  • dataset (torch.utils.data.Dataset) – The dataset to be processed

  • save_path (str) – Save path for the preprocessed dataset

  • transformation (torch.nn.Module) – The transformations that will be applied to the data

  • output_shape (list) – Output shape of an element of the transformation. Defaults to [].

Raises:

AttributeError – If given output shape is not a list or is an empty list

dataset.speech_commands_collate(batch)[source]

Collate function for setting up the dataloader

Parameters:

batch (int) – batch size

Returns:

return batched data in the form ; audio_tensor,target,task_label

Return type:

batch