kitcar_ml.utils.data.data_loader package

Submodules

kitcar_ml.utils.data.data_loader.base_data_loader module

Classes:

BaseDataLoader(*, dataset[, max_dataset_size])

Wrapper class of Dataset class that performs multi-threaded data loading.
class BaseDataLoader(*, dataset: Union[UnlabeledDataset, LabeledDataset], max_dataset_size: Optional[int] = None, **kwargs)[source]

Bases: DataLoader

Wrapper class of Dataset class that performs multi-threaded data loading.

Methods:

prepare_batch(batch)

Attributes:

prepare_batch(batch)[source]
dataset: Dataset[T_co]
batch_size: Optional[int]
num_workers: int
pin_memory: bool
drop_last: bool
timeout: float
sampler: Union[Sampler, Iterable]
pin_memory_device: str
prefetch_factor: int
_iterator: Optional[_BaseDataLoaderIter]

kitcar_ml.utils.data.data_loader.bbox_data_loader module

Classes:

BBoxDataLoader(*, transforms[, torch_tf])

Dataloader for bbox datasets that applies transformations.
class BBoxDataLoader(*, transforms: List[BBoxImageTransform], torch_tf=ToTensor(), **kwargs)[source]

Bases: BaseDataLoader

Dataloader for bbox datasets that applies transformations.

Attributes:

super_sampling_factor

How many images are consumed to produce one training image.

complete_transform

Concatenate all transforms.

Methods:

prepare_batch(batch)

Prepare a batch before it is converted to torch.
property super_sampling_factor

How many images are consumed to produce one training image.

property complete_transform

Concatenate all transforms.

prepare_batch(batch)[source]

Prepare a batch before it is converted to torch.

dataset: Dataset[T_co]
batch_size: Optional[int]
num_workers: int
pin_memory: bool
drop_last: bool
timeout: float
sampler: Union[Sampler, Iterable]
pin_memory_device: str
prefetch_factor: int
_iterator: Optional[_BaseDataLoaderIter]

kitcar_ml.utils.data.data_loader.example module

kitcar_ml.utils.data.data_loader.utils module

Functions:

load_labeled_dataset(label_file, ...)

Create dataloader for a labeled dataset.

load_unpaired_unlabeled_datasets(dir_a, ...)

Create dataloader for two unpaired and unlabeled datasets.

sample_generator(dataloader[, n_samples])

Generator that samples from a dataloader.

unpaired_sample_generator(dataloader_a, ...)

Generator that samples pairwise from both dataloaders.
load_labeled_dataset(label_file: str, max_dataset_size: Optional[int], batch_size: int, sequential: bool, num_workers: int) → BaseDataLoader[source]

Create dataloader for a labeled dataset.

Parameters

  • label_file – Path to a file containing all labels

  • max_dataset_size – Maximum amount of images to load; None means infinity

  • batch_size – Batch size

  • sequential – If true, takes images in order, otherwise takes them randomly

  • num_workers – Threads for loading data

load_unpaired_unlabeled_datasets(dir_a: Union[str, List[str]], dir_b: Union[str, List[str]], max_dataset_size: Optional[int], batch_size: int, sequential: bool, num_workers: int) → Tuple[BaseDataLoader, BaseDataLoader][source]

Create dataloader for two unpaired and unlabeled datasets.

E.g. used by cycle gan with data from two domains.

Parameters

  • dir_a – path to images of domain a

  • dir_b – path to images of domain b

  • max_dataset_size (int) – maximum amount of images to load; -1 means infinity

  • batch_size (int) – input batch size

  • sequential (bool) – if true, takes images in order, otherwise takes them randomly

  • num_workers (int) – workers for loading data

sample_generator(dataloader: BaseDataLoader, n_samples: Optional[int] = None)[source]

Generator that samples from a dataloader.

Parameters

  • dataloader – Dataloader.

  • n_samples – Number of batches of samples. None means infinity

unpaired_sample_generator(dataloader_a: BaseDataLoader, dataloader_b: BaseDataLoader, n_samples: Optional[int] = None)[source]

Generator that samples pairwise from both dataloaders.

Parameters

  • dataloader_a – Domain a dataloader.

  • dataloader_b – Domain b dataloader.

  • n_samples – Number of batches of samples.

Module contents