List - dict comprehensions and generators: Exercises

L1 — Recognition

You are not writing code yet. You are reading it and stating what it produces. This is the "which angle has this tan?" skill: recognise the shape before you compute.

Exercise 1.1 — Read a list comprehension

What is the value of out?

out = [x + 1 for x in [10, 20, 30]]

Recall Solution 1.1

Read left-to-right: "the expression x + 1, for each x drawn from the list [10, 20, 30]."

x = 10 → 11
x = 20 → 21
x = 30 → 31

Answer: [11, 21, 31]. The brackets [...] mean the result is a list — a fully-built collection sitting in memory.

Exercise 1.2 — List vs. generator by punctuation alone

Two lines differ by one character. What type of object does each produce, and what does print show?

a = [x for x in range(3)]
b = (x for x in range(3))
print(a)
print(b)

Recall Solution 1.2

The only difference is [...] vs (...).

a is a list comprehension → a real list. print(a) shows [0, 1, 2].
b is a generator expression → a lazy iterator that has computed nothing yet. print(b) shows something like <generator object <genexpr> at 0x...> — not the numbers.

Why the difference matters: the list has already spent memory storing 0, 1, 2. The generator is a promise: "ask me and I'll make one number at a time." Printing it prints the promise, not the goods.

Exercise 1.3 — Read a dict comprehension

What dictionary does this build?

d = {c: ord(c) for c in "abc"}

(ord('a') is 97, ord('b') is 98, ord('c') is 99.)

Recall Solution 1.3

{key: value for item in iterable} — the colon separates key from value. Iterating a string yields its characters one at a time.

c = 'a' → key 'a', value ord('a') = 97
c = 'b' → key 'b', value 98
c = 'c' → key 'c', value 99

Answer: {'a': 97, 'b': 98, 'c': 99}.

L2 — Application

Now you write the comprehension from a specification.

Exercise 2.1 — Squares of odd numbers

Using a single list comprehension, build the squares of the odd numbers from 0 to 9 inclusive.

Recall Solution 2.1

Two decisions: filter to odds (if x % 2 == 1), then transform with x**2. Order inside the machine: iterate → filter → transform → collect.

result = [x**2 for x in range(10) if x % 2 == 1]

Odd values 1, 3, 5, 7, 9 survive the filter, then get squared: 1, 9, 25, 49, 81.

Answer: [1, 9, 25, 49, 81].

Exercise 2.2 — Invert a dictionary

Given prices = {'pen': 5, 'book': 40, 'bag': 200}, build a new dict mapping price → name using a dict comprehension.

Recall Solution 2.2

.items() yields (key, value) pairs. We want the value as the new key and the key as the new value, so we swap them:

prices = {'pen': 5, 'book': 40, 'bag': 200}
inv = {v: k for k, v in prices.items()}

('pen', 5) → 5: 'pen'
('book', 40) → 40: 'book'
('bag', 200) → 200: 'bag'

Answer: {5: 'pen', 40: 'book', 200: 'bag'}. (This only works cleanly because the values are unique — see the L3 trap.)

Exercise 2.3 — Flatten and filter together

Flatten grid = [[1, -2], [3, -4], [5, -6]] into a single list keeping only the positive numbers.

Recall Solution 2.3

Nested comprehension reads like nested loops, outer-first: for row in grid then for n in row. The if filter goes at the end.

grid = [[1, -2], [3, -4], [5, -6]]
flat = [n for row in grid for n in row if n > 0]

Walk it: 1 (keep), -2 (drop), 3 (keep), -4 (drop), 5 (keep), -6 (drop).

Answer: [1, 3, 5].

L3 — Analysis

Predict behaviour, especially edge cases: empty inputs, exhaustion, duplicate keys.

Exercise 3.1 — The double-drain

What does this print, and why?

g = (x for x in range(4))
print(sum(g))
print(sum(g))

Recall Solution 3.1

sum(g) walks the generator to the end: 0 + 1 + 2 + 3 = 6. During this walk the generator is consumed — its internal cursor is now parked at "end."

First print: 6.
Second print: the generator has nothing left, so sum of an empty stream is 0.

Answer: prints 6 then 0. Lesson: a generator is a one-shot. If you need the values twice, either store them in a list or rebuild the generator.

Exercise 3.2 — Duplicate keys in a dict comprehension

What is the length of d?

d = {x % 3: x for x in range(6)}

Recall Solution 3.2

Keys are x % 3 for x = 0..5: 0, 1, 2, 0, 1, 2. Only three distinct keys exist: 0, 1, 2. When a key repeats, the later value overwrites the earlier one.

key 0: written by x=0, then overwritten by x=3
key 1: x=1, then x=4
key 2: x=2, then x=5

Final dict: {0: 3, 1: 4, 2: 5}.

Answer: len(d) == 3.

Exercise 3.3 — Empty-iterable edge case

What are a and b?

a = [y*2 for y in [] if y > 0]
b = list(z + 1 for z in range(0))

Recall Solution 3.3

Both iterables are empty ([] and range(0) produce zero items). A comprehension over an empty iterable does zero work and yields an empty collection — no error.

a = []
b = []

Answer: a == [] and b == []. Degenerate inputs are safe; comprehensions never raise on emptiness, they just produce nothing.

L4 — Synthesis

Combine generators, comprehensions, and laziness into pipelines.

Exercise 4.1 — Lazy pipeline with early stop

Trace this. How many times is x**2 actually computed?

def squares(n):
    for x in range(n):
        yield x**2
 
first_two = list(squares(1000))[:2]

Then rewrite it so that only two squares are ever computed using itertools.islice.

Recall Solution 4.1

list(squares(1000)) fully drains the generator first — it computes all 1000 squares into a list, then slices [:2]. So x**2 runs 1000 times. first_two = [0, 1], but the work was wasteful.

The lazy fix stops the generator after two values:

from itertools import islice
first_two = list(islice(squares(1000), 2))

islice pulls exactly 2 items, so x**2 runs only twice. Same answer [0, 1], far less work.

Answer: original computes x**2 1000 times; the islice version computes it 2 times. Both give [0, 1].

Exercise 4.2 — Build a word-length index

From words = ['hi', 'cat', 'to', 'dog', 'a'], build a dict mapping each length → list of words of that length, using a comprehension over the set of lengths.

Recall Solution 4.2

Step 1: the distinct lengths are {1, 2, 3}. Step 2: for each length L, collect every word whose length is L — an inner comprehension.

words = ['hi', 'cat', 'to', 'dog', 'a']
index = {L: [w for w in words if len(w) == L]
         for L in {len(w) for w in words}}

L = 1 → ['a']
L = 2 → ['hi', 'to']
L = 3 → ['cat', 'dog']

Answer: {1: ['a'], 2: ['hi', 'to'], 3: ['cat', 'dog']}. (This scans words once per length — fine for small data; a single pass with a plain loop is cheaper at scale, which is the L5 theme.)

L5 — Mastery

Reason about memory, laziness, exhaustion, and correctness all at once.

The two figures below make these laws visual. Study each before its exercise.

Figure 1 — where the items live. The blue row is a list: all five squares 0, 1, 4, 9, 16 are built at once and sit on a shelf in memory (yellow arrow: "all 5 in memory"). The green row is a generator: a small "magic box" that, when asked, hands you one value (here 0) and keeps only $O (1)$ bytes of state (red label). The red note reminds you it is one-shot — a value taken is gone from the box.

Figure — List - dict comprehensions and generators

Figure 2 — who wins depends on passes. The x-axis is $k$ , the number of times you iterate over the whole collection. The blue curve is the list's total cost $n T_{c} + n k$ (build once, then cheap re-reads); the green curve is the generator's $n T_{c} k$ (recompute everything every pass). At $k = 1$ (green arrow) the generator is cheaper — no upfront build. As $k$ grows, the green line climbs faster and the list wins; the yellow marker at $k = 4$ shows list 14000 vs generator 40000.

Exercise 5.1 — Memory reasoning

A record is ~200 bytes. You must process 50,000,000 records but only need one pass (feed each to train_model, then discard). Estimate the memory of the list approach vs the generator approach, and state which to use.

Recall Solution 5.1

Use the two laws above: list memory $\approx n \times item_size$ ; generator memory $\approx O (1)$ (a constant, independent of $n$ ).

List: $50, 000, 000 \times 200 bytes = 1.0 \times 1 0^{10}$ bytes $= 10$ GB. That won't fit in typical RAM.
Generator: a few hundred bytes of iterator state, independent of $n$ (this is what $O (1)$ means).

Because $k = 1$ (single pass) and $n$ is huge, the generator wins decisively:

stream = (process(line) for line in open('huge.csv'))
for record in stream:
    train_model(record)

Answer: list $\approx 10$ GB; generator $\approx O (1)$ (hundreds of bytes). Use the generator. This is the mini-batch streaming pattern.

Exercise 5.2 — List vs generator cost model

Using the parent's cost model — list cost $\approx n T_{c} + n k$ , generator cost $\approx n T_{c} k$ (the three symbols $n$ , $T_{c}$ , $k$ are defined in the callout above) — decide which is cheaper when you must iterate the collection $k = 4$ times, with $n = 1000$ items, each costing $T_{c} = 10$ "time units" to compute.

Recall Solution 5.2

Plug in $n = 1000$ , $T_{c} = 10$ , $k = 4$ .

List: $n T_{c} + n k = 1000 \cdot 10 + 1000 \cdot 4 = 10000 + 4000 = 14000$ .
Generator: $n T_{c} k = 1000 \cdot 10 \cdot 4 = 40000$ .

The list computes each item once ( $n T_{c}$ ) then pays only cheap $O (1)$ accesses across the 4 passes. The generator recomputes everything on every pass, so it pays $T_{c}$ four times per item.

Answer: list cost 14000, generator cost 40000. With multiple passes ( $k > 1$ ) and small $n$ , the list wins. (This is the crossover shown in Figure 2.)

Exercise 5.3 — Custom generator: sliding windows

Write a generator windows(seq, size) that yields consecutive overlapping slices of length size. For windows([1,2,3,4], 2) it should yield [1,2], [2,3], [3,4]. How many windows come out in general — and what happens for the degenerate inputs size > len(seq), size == 0, and size < 0?

Recall Solution 5.3

A window starting at index i covers seq[i:i+size]. The last valid start is len(seq) - size. So valid starts are i = 0, 1, ..., len(seq) - size, giving len(seq) - size + 1 windows.

def windows(seq, size):
    for i in range(len(seq) - size + 1):
        yield seq[i:i+size]
 
list(windows([1, 2, 3, 4], 2))
# -> [[1, 2], [2, 3], [3, 4]]

Count check: len([1,2,3,4]) - 2 + 1 = 4 - 2 + 1 = 3. ✓

Degenerate cases — walk every one:

size > len(seq) (e.g. windows([1,2], 5)): len(seq) - size + 1 = 2 - 5 + 1 = -2, and range(-2) is empty, so the generator yields nothing. Sensible: you can't fit a window bigger than the data.
size == 0 (e.g. windows([1,2], 0)): the formula gives 2 - 0 + 1 = 3 windows, each seq[i:i+0] which is the empty list []. So you get [[], [], []] — a surprising, probably-unwanted result: three empty windows. This is the trap of the naive formula.
size < 0 (e.g. windows([1,2], -1)): 2 - (-1) + 1 = 4 "windows," and seq[i:i-1] produces odd partial/empty slices — clearly nonsense.

Hardened version — reject non-positive size up front so the degenerate cases can't sneak through:

def windows(seq, size):
    if size <= 0:
        raise ValueError("size must be positive")
    for i in range(len(seq) - size + 1):
        yield seq[i:i+size]

Answer: [[1, 2], [2, 3], [3, 4]]; in general len(seq) - size + 1 windows for size >= 1. For size > len(seq) you get zero windows (fine); for size == 0 the naive formula wrongly yields empty windows and for size < 0 it yields nonsense, so guard with size <= 0. This relies on the iterator protocol under the hood.

Exercise 5.4 — The exhaustion bug in a data loader

A student writes a loader and reuses it for two epochs. What goes wrong, and how do you fix it?

loader = (process(line) for line in open('data.csv'))
for epoch in range(2):
    for record in loader:
        train(record)

Recall Solution 5.4

loader is a generator — a one-shot (recall Figure 1's red "one-shot" note, and Exercise 3.1's double-drain). Epoch 0 drains it completely. When epoch 1 starts, loader is already exhausted, so the inner for iterates zero times. Only one epoch of training actually happens.

Fix A — rebuild the generator each epoch (re-open the file), so every epoch gets a fresh stream:

for epoch in range(2):
    loader = (process(line) for line in open('data.csv'))
    for record in loader:
        train(record)

Fix B — materialise once into a list if the data fits in RAM (a list is reusable, a generator is single-use):

data = [process(line) for line in open('data.csv')]
for epoch in range(2):
    for record in data:
        train(record)

Fix A keeps memory at $O (1)$ but pays the read/parse cost every epoch; Fix B pays memory $n \times item_size$ once but reuses it cheaply — exactly the $k > 1$ tradeoff of Exercise 5.2. Details on streaming files live in 1.4.04-File-IO-and-data-loading.

Answer: the second epoch trains on 0 records because the generator was consumed in epoch 0; re-create the generator per epoch (Fix A) or use a reusable list (Fix B).

Recall One-line summary to test yourself

[...] builds now and stores; (...) promises later and forgets — choose by passes and memory. ::: Correct if you also said: generators are one-shot and recompute per pass; lists are reusable but cost $n \times$ item_size in RAM.

Connections

Parent: List - dict comprehensions and generators
Prereqs: 1.4.01-Basic-Python-syntaxand-data-structures · 1.4.02-NumPy-arrays-and-vectorization
Next: 1.4.04-File-IO-and-data-loading · 1.5.01-Batch-processing-and-mini-batches · 2.3.02-Iterator-protocol
Hinglish: 1.4.03 List - dict comprehensions and generators (Hinglish)