Noesis
3.3.4 · D5Deep Learning Frameworks

Question bank — Datasets and DataLoaders

1,273 words6 min read↑ Back to topic

Before we start, three words we lean on:

  • Lazy loading — loading a sample only when it is asked for by index, never all at once.
  • Epoch — one full pass through every sample in the dataset.
  • Collation — the act of gluing several individual samples into one rectangular batch tensor.

True or false — justify

A Dataset.__getitem__(idx) must return the same sample every time it is called with the same idx.
Mostly true — the identity of the sample is deterministic, but if a random transform (crop, flip) runs inside __getitem__, the returned tensor values differ each call. The parent's "deterministic" property refers to which sample, not its post-augmentation pixels.
Setting shuffle=True changes the contents of your dataset.
False — shuffling only changes the order in which indices are visited each epoch; the underlying Dataset and every (x_i, y_i) pair are untouched.
If num_workers=4 your training runs roughly 4× faster.
False — workers only overlap data loading (disk I/O, decode) with GPU compute. If the GPU is the bottleneck, extra workers help little; they parallelise I/O, not the forward/backward pass.
With drop_last=False and N=100, B=32, you get 4 batches.
True — , the last holding samples. With drop_last=True you'd get full batches and throw those 4 away.
A DataLoader must be re-created before each epoch to reshuffle.
False — iterating the same DataLoader again (a new for batch in loader loop) triggers a fresh shuffle automatically. Re-creating it wastes setup work.
Shuffling once at the start is just as good as shuffling every epoch.
False — a single fixed order means the model always sees batch boundaries in the same place, letting it learn spurious sequence correlations; per-epoch reshuffling decorrelates consecutive samples across passes.
pin_memory=True makes your model more accurate.
False — it only speeds the CPU→GPU copy by using page-locked RAM. It touches transfer speed, never the numbers being transferred, so accuracy is unchanged.
The DataLoader loads all samples into RAM when constructed.
False — construction just wraps the dataset and stores config. Actual loading happens during iteration, one batch at a time, via __getitem__ — this is the whole point of lazy loading.

Spot the error

class D(Dataset): def __getitem__(self, idx): return self.images[idx] where self.images was filled in __init__ with every decoded image.
The images are eagerly loaded in __init__, defeating lazy loading — RAM blows up before training. Store paths in __init__, decode inside __getitem__.
A student sets batch_size=32 but forgets __len__ returns len(self.img_paths) and instead returns a hard-coded 1000 while only 800 files exist.
__getitem__ will be asked for indices up to 999, hitting an out-of-range file access and crashing. __len__ must equal the true sample count so batching and epoch iteration stay in bounds.
DataLoader(dataset, batch_size=32, shuffle=True) on a validation set used to compute a confusion matrix.
Shuffling on validation is not wrong mechanically, but it makes per-sample results unaligned with any external ordered reference. For deterministic evaluation/logging you usually set shuffle=False.
Variable-length text samples are passed to the default collate function.
The default torch.stack requires identical shapes, so unequal lengths raise a shape error. You must supply a collate_fn that pads sequences to a common length first.
padded = pad(seq, (0, 0, pad_amount)) where the intent was to pad the feature dimension.
The (0, 0, pad_amount) layout pads the length dimension (dim 0). Padding usually should target length, not features — but if you truly wanted features you'd have used a different pad tuple; the mistake is a mismatch between comment and code.
Someone calls .backward() inside __getitem__.
__getitem__ is data-preparation only and runs inside worker processes with no model/graph. Gradient computation belongs in the training loop after the forward pass (see 3.3.05-Training-Loop-Basics).

Why questions

Why store file paths instead of decoded images in __init__?
A path is a few bytes; a decoded image is megabytes. Storing paths keeps RAM roughly constant regardless of dataset size, decoding only the ~one image currently requested.
Why do we batch at all instead of one sample at a time?
A batch averages the gradient over samples, , reducing its variance, and lets the GPU process many samples in parallel — both speed and stability. See 2.2.03-Mini-Batch-Gradient-Descent.
Why put data augmentation inside __getitem__ rather than precomputing it?
Per-access augmentation means every epoch sees a fresh random crop/flip of each image, effectively enlarging the dataset. Precomputing would freeze one fixed variant, losing that regularisation.
Why can num_workers>0 sometimes slow training on a tiny dataset?
Spawning worker processes and pickling data between them has overhead. If loading is already trivial, that inter-process cost outweighs any overlap benefit.
Why does drop_last=True exist if it throws away data?
Some layers or fixed-shape optimisations require every batch to be the same size; a ragged final batch of size would break them. Dropping it guarantees uniform shape at the cost of a few samples per epoch.
Why is the data pipeline often the training bottleneck, not the model?
The GPU can finish a batch faster than the CPU can decode and deliver the next one from disk. If loading can't keep up, the GPU sits idle — hence workers, pin_memory, and overlapping prefetch (relevant too in 3.3.08-Distributed-Training).
Why does Dataset deliberately separate __len__ and __getitem__ from the training loop?
This clean interface lets you swap images for text, or a local folder for a pretrained-model's dataset, without touching training code — the same abstraction powers 4.1.02-Transfer-Learning workflows.

Edge cases

What happens with N=0 (empty dataset)?
__len__ returns 0, so the DataLoader yields no batches and the inner loop body never runs — no error, but silently zero training, a bug worth asserting against.
N=32, B=32: how many batches with drop_last=True vs False?
Both give exactly 1 — since is an exact multiple of there is no incomplete tail, so drop_last has no effect here.
B > N (batch bigger than dataset), drop_last=False: what do you get?
One single batch containing all samples (since ). With drop_last=True you'd get zero batches, because the only batch is "incomplete" and gets dropped.
A batch straddles two epochs — is that possible?
No — batches never cross the epoch boundary. Each epoch re-partitions the freshly shuffled indices from scratch, so the last (possibly short) batch always closes the epoch cleanly.
Two consecutive calls dataset[5] with a random transform: same tensor?
No — the sample chosen is identical (index 5's file), but the random augmentation reseeds each call, so pixel values generally differ. This is intended, not a bug.
Does shuffling guarantee every sample appears exactly once per epoch?
Yes — shuffling permutes the index list , so each index is visited once per epoch; it changes order, never multiplicity.