mlvbench.datasets¶
Datasets.
DataModule
¶
Base class for data modules.
A data module encapsulates all data processing steps for a dataset. It is inspired by the DataModule API in PyTorch Lightning.
configure(model)
¶
Configure the data module for the given model.
Called by the trainer before [prepare_data][]. Override to store
model-specific attributes (e.g., input size) that are needed during data
preparation or dataset construction.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
Model
|
The model to configure the data module for. |
required |
prepare_data()
¶
Download, preprocess or generate the stimuli.
setup()
¶
Perform additional setup after prepare_data (defaults to no-op).
subsets()
¶
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the requested test dataset.
This method is called by the default implementation of test_dataloader().
Override this method to provide the test dataset, or override
test_dataloader() to build a custom data loader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training dataset.
This method is called by the default implementation of train_dataloader().
Override this method to provide the training dataset, or override
train_dataloader() to build a custom data loader.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation dataset.
This method is called by the default implementation of val_dataloader().
Override this method to provide the validation dataset, or override
val_dataloader() to build a custom data loader.
FigureGroundConvexityDataModule
¶
Bases: ProceduralDataModule
Data module for testing convexity as a figure-ground cue.
Each image is split into two regions by a parabolic curve. The convex region is labelled as the foreground, and the concave region as the background. Both regions are filled with different textures from the DTD dataset.
__init__(seed=0)
¶
Initialize the data module.
after_generate()
¶
Clean up the textures after generating the samples.
before_generate(seed)
¶
Load the DTD textures.
configure(model)
¶
Configure the data module for the given model.
generate_sample(subset, seed)
¶
Generate a single figure-ground convexity sample.
generate_sample_seeds(num_samples, rng)
staticmethod
¶
Generate the sample seeds.
generate_subset(subset, sample_seeds, path)
¶
Generate the samples for the given subset.
prepare_data()
¶
Generate and cache the dataset subsets.
setup()
¶
Perform additional setup after prepare_data (defaults to no-op).
subsets()
¶
Return the names of all available subsets.
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the requested test split (standard, reversed, or figure-split).
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training split.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation split.
FigureGroundSurroundednessDataModule
¶
Bases: ProceduralDataModule
Data module for testing surroundedness as a figure-ground cue.
Each image contains a random foreground shape from the idsprites dataset. The foreground and background regions are filled with different textures from the DTD dataset.
__init__(seed=0)
¶
Initialize the data module.
after_generate()
¶
Clean up the textures after generating the samples.
before_generate(seed)
¶
Load the DTD textures.
configure(model)
¶
Configure the data module for the given model.
generate_sample(subset, seed)
¶
Generate a single figure-ground surroundedness sample.
generate_sample_seeds(num_samples, rng)
staticmethod
¶
Generate the sample seeds.
generate_subset(subset, sample_seeds, path)
¶
Generate the samples for the given subset.
prepare_data()
¶
Generate and cache the dataset subsets.
setup()
¶
Perform additional setup after prepare_data (defaults to no-op).
subsets()
¶
Return the names of all available subsets.
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the requested test split (standard, reversed, or figure-split).
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training split.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation split.
FigureGroundSymmetryDataModule
¶
Bases: ProceduralDataModule
Data module for testing symmetry as a figure-ground cue.
__init__(seed=0)
¶
Initialize the data module.
after_generate()
¶
Clean up the textures after generating the samples.
before_generate(seed)
¶
Load the DTD textures.
configure(model)
¶
Configure the data module for the given model.
generate_sample(subset, seed)
¶
Generate a single figure-ground symmetry sample.
generate_sample_seeds(num_samples, rng)
staticmethod
¶
Generate the sample seeds.
generate_subset(subset, sample_seeds, path)
¶
Generate the samples for the given subset.
prepare_data()
¶
Generate and cache the dataset subsets.
setup()
¶
Perform additional setup after prepare_data (defaults to no-op).
subsets()
¶
Return the names of all available subsets.
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the requested test split (standard or reversed variant).
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training split.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation split.
MSRA10KDataModule
¶
Bases: DataModule
Data module for the MSRA-10K dataset.
__init__(seed=0, to_numpy=False)
¶
configure(model)
¶
Configure the data module for the given model.
prepare_data()
¶
Download the dataset.
setup()
¶
Create the train/val/test index splits.
subsets()
¶
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the test split.
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training split.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation split.
PrecomputedMSRA10KDataModule
¶
Bases: DataModule
Precomputed data module for the MSRA-10K dataset.
__init__(seed=0)
¶
Initialize the data module.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
seed
|
int
|
Seed used for splitting the dataset into train, val, and test subsets. |
0
|
configure(model)
¶
Configure the data module for the given model.
prepare_data()
¶
Precompute and cache the dataset.
setup()
¶
Perform additional setup after prepare_data (defaults to no-op).
subsets()
¶
test_dataloader(batch_size, num_workers=0, device='cpu', subset='test')
¶
Return the data loader for the requested test subset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
batch_size
|
int
|
The batch size. |
required |
num_workers
|
int
|
The number of worker processes. |
0
|
device
|
str | device
|
The device the batches are placed on. |
'cpu'
|
subset
|
str
|
The name of the test subset to construct, as returned by
|
'test'
|
test_dataset(subset='test')
¶
Return the test split.
train_dataloader(batch_size, num_workers=0, repeat=True, device='cpu')
¶
Return the data loader for the training set.
train_dataset()
¶
Return the training split.
val_dataloader(batch_size, num_workers=0, device='cpu')
¶
Return the data loader for the validation set.
val_dataset()
¶
Return the validation split.