Generator expressions — memory efficiency: Exercises

1.2.36 · D4 · Coding › Introduction to Programming (Python) › Generator expressions — memory efficiency

Shuru karne se pehle, ek picture jo poore time dimaag mein rakhni hai.

Figure — Generator expressions — memory efficiency

Left waali list ke saare boxes ek saath bhare hue hain. Right waala generator exactly ek hi box rakhta hai; baaki sab sirf ek promise hain (dotted). Neeche ki har exercise asal mein yahi pooch rahi hai: "ab tum is picture ke kis side ho?"

Level 1 — Recognition

Goal: kya tum ek generator aur ek list mein instantly fark kar sakte ho, aur padh sakte ho ki usmein kya hai?

Recall Solution 1.1

Sirf b ek generator expression hai.

a mein square brackets [] hain → ek list.
b mein round brackets () hain → ek generator (yahan ka woh ek lazy object).
c mein curly brackets {} hain → ek set (eager, sab abhi ban gaya).
d mein ek generator expression hai, lekin list(...) use turant drain karke ek list mein daal deta hai. Final object d ek list hai.

Rule: comprehension ke around ka punctuation type decide karta hai. Eager [] cousin ke liye List comprehensions dekho.

Recall Solution 1.2

<class 'generator'>. Ek common galat guess hai tuple, kyunki round brackets usually tuples banate hain. Lekin ek tuple mein koi for andar nahi hota — (1, 2, 3) ek tuple hai, jabki (n for n in range(5)) ek generator hai. Parentheses ke andar for ... in ... ka hona hi use flip karta hai.

Level 2 — Application

Goal: generator ko next(), sum(), for, aur filters ke saath sahi se use karo.

Recall Solution 2.1

0, phir 10, phir 20. Har next(g) ek value khींchta hai, internal position aage badhata hai, aur picchla bhool jaata hai — bilkul upar waali single-box picture ki tarah. Ek chautha next(g) StopIteration raise karega, kyunki dene ke liye sirf teen values hain.

Recall Solution 2.2

9. Filter if x % 2 == 1 1..5 se odd numbers rakhta hai: yaani 1, 3, 5. Unka sum $1 + 3 + 5 = 9$ hai. sum kabhi list nahi dekhta — use ek-ek karke har odd number milta hai aur woh use running total mein add karta jaata hai.

Recall Solution 2.3

biggest = max(len(w) for w in ["hi", "world", "ok"])

Jab ek generator expression kisi call ka only argument ho, toh Python inner parentheses hataane deta hai. Answer value: word lengths 2, 5, 2 hain, toh max 5 return karta hai.

Level 3 — Analysis

Goal: memory cost aur exhaustion ke baare mein reasoning karo — sirf code chalana nahi, balki explain karna.

Worked example Exercise 3.1

Do programs ek million squares ka same sum compute karte hain:

A = sum([x * x for x in range(10**6)])   # square brackets
B = sum( x * x for x in range(10**6))    # generator

Dono same number print karte hain. Toh fark kya hai, aur Big-O mein dono ki space complexity batao.

Recall Solution 3.1

Same result, bahut alag peak memory.

A pehle ek million integers ki list banata hai, phir sum karta hai. Peak extra memory $O (n)$ hai — woh saare $n$ items ek saath hold karta hai (~8 MB+ pointers plus ints).
B ek square ko ek baar mein sum mein stream karta hai; kisi bhi instant par sirf ek value exist karti hai, toh peak extra memory $O (1)$ hai.

Dono same total compute karte hain, toh time comparable hai; space hi saara point hai. Big-O space complexity dekho.

Recall Solution 3.2

Line 1: [0, 2]. Line 2: []. Pehla list(g) generator ko end tak walk karta hai, even numbers 0, 2 collect karta hai. Woh single pass generator ko exhaust kar deta hai. Doosra list(g) kuch yield karne ko nahi paata, toh empty list return karta hai. Generator rewind nahi karta. Yeh parent note ka exhaustion behaviour hai — Iterators and the iterator protocol bhi dekho.

Worked example Exercise 3.3

Neeche, read_lines() ek generator hai jo ek huge file ki lines yield karta hai. Kaunsa version file kam padhta hai, aur kyun?

# Version 1
found = next(line for line in read_lines() if "ERROR" in line)
 
# Version 2
found = [line for line in read_lines() if "ERROR" in line][0]

Recall Solution 3.3

Version 1 kam padhta hai.

Version 1 next ke saath generator use karta hai. Yeh pehli "ERROR" line milte hi ruk jaata hai aur aage nahi padhta. Yeh Lazy evaluation action mein hai.
Version 2 har error line ki poori list banata hai (poori file scan karke), phir [0] chhod ke sab kuch phenk deta hai. Ek item rakhne ke liye maximum kaam karta hai.

Same answer, lekin Version 2 shayad laakhon lines padh le jo Version 1 kabhi nahi chhoota.

Level 4 — Synthesis

Goal: generators combine karo, chain karo, aur sahi tool choose karo.

Worked example Exercise 4.1

Do generators chain karo: numbers ko square karo, phir sirf 20 se chhote squares rakho. list(small) kya print karta hai?

squares = (x * x for x in range(6))
small   = (s for s in squares if s < 20)
print(list(small))

Recall Solution 4.1

[0, 1, 4, 9, 16]. squares 0, 1, 4, 9, 16, 25 yield karta. Doosra generator small squares se ek-ek value khींchta hai aur sirf woh pass karta hai jo < 20 hain. 25 drop ho jaata hai. Importantly, squares ki koi poori list kabhi nahi banti — values ek-ek karke poori pipeline mein flow karti hain. Yeh lazy iterators ka composition hai.

Worked example Exercise 4.2

Tumhe same filtered data do baar chahiye (ek baar print karne, ek baar sum karne ke liye). Yeh broken hai — ise fix karo:

evens = (n for n in range(10) if n % 2 == 0)
print(list(evens))
print(sum(evens))     # prints 0 — bug!

Recall Solution 4.2

Generator pehle list(evens) se exhaust ho jaata hai. Agar tumhe genuinely data do baar chahiye, toh ek baar list mein materialise karo:

evens = [n for n in range(10) if n % 2 == 0]   # ek real list, reusable
print(evens)          # [0, 2, 4, 6, 8]
print(sum(evens))     # 20

Ab evens ek list hai, toh dono passes poora data dekhte hain. Sum: $0 + 2 + 4 + 6 + 8 = 20$ . Rule of thumb: ek baar lazily chahiye → generator. Kai baar chahiye → list.

Recall Solution 4.3

lengths = (len(w) for w in words if len(w) > 2)
print(list(lengths))   # [3, 3, 8]

2 se zyada letters wale words: "cat" (3), "dog" (3), "elephant" (8). "ox" (2) filter ho jaata hai. Result: [3, 3, 8].

Level 5 — Mastery

Goal: subtle edge cases — empty inputs, late binding, side effects, no subscripting.

Recall Solution 5.1

list(x for x in range(0)) → []. range(0) kuch yield nahi karta, toh generator janm se hi empty hai.
sum(x for x in []) → 0. Zero numbers ka sum additive identity 0 deta hai.
next((x for x in []), "none") → "none". Jab generator empty ho, toh next default ke saath StopIteration raise karne ki jagah woh default return karta hai.

Edge cases matter karte hain: ek empty generator legal aur common hai — hamesha safe, kabhi error nahi — jab tak tum usse next bina default ke na call karo.

Recall Solution 5.2

TypeError: 'generator' object is not subscriptable. Generator mein index karne ke liye koi stored elements nahi hain — memory mein koi "slot 0" nahi hai. Pehla item lene ke liye next(g) use karo; kisi arbitrary position ke liye tumhe ussi tak iterate karna hoga (ya pehle list mein convert karo, memory cost chukaakar).

Recall Solution 5.3

[0, 100, 200]. Yeh deep wala hai. Generator creation time par kuch compute nahi karta — yeh lazy hai. Yeh apna body tabhi chalata hai jab list(g) use iterate karta hai, aur tab tak m 100 ho chuka hota hai. Toh yeh free variable m ki current value use karta hai, naki woh value jo generator likhte waqt thi. Kyunki iteration deferred hai, woh environment bhi deferred hai jise woh padhta hai. Contrast: ek list comprehension [x * m for x in range(3)] turant m == 2 ke saath evaluate hota, [0, 2, 4] deta.

Worked example Exercise 5.4 (side effects reveal laziness)

print("made") kitni baar chalta hai, aur doosre prints ke relative to kya order hai?

def make(x):
    print("made", x)
    return x
 
g = (make(x) for x in range(3))
print("built generator")
print("first:", next(g))

Recall Solution 5.4

Output order:

built generator
made 0
first: 0

make zero baar chalta hai jab generator build hota hai — creation koi kaam nahi karta. Yeh ek baar pehle next(g) par chalta hai, made 0 print karta hai, phir 0 yield karta hai. Values made 1 / made 2 kabhi appear nahi hote kyunki humne unhe kabhi maanga hi nahi. Yeh printer-prints-on-demand model visible ho gaya. Lazy evaluation aur Generator functions and yield dekho.

Recall Feynman check — ek saanss mein kaho

Ek generator ek recipe hai jo demand par chalta hai: yeh koi data nahi rakhta, koi kaam nahi karta, aur apne variables sirf tabhi padhta hai jab tum ek value pull karo. Isi liye yeh $O (1)$ memory cost karta hai, ek pass ke baad exhaust ho jaata hai, index nahi ho sakta, empty inputs se koi problem nahi, aur build karne ke baad badla hua variable phir bhi output affect karta hai. Ye paancho behaviours ek hi fact hain — "baad mein compute karo, ek ek karke" — alag angles se dekha gaya.

Connections

Generator expressions — memory efficiency — woh parent concept jise ye exercises drill karti hain
List comprehensions — eager [] counterpart (Exercises 1.1, 4.2, 5.3)
Iterators and the iterator protocol — exhaustion aur next yahaan se aate hain (Ex 3.2, 5.2)
Generator functions and yield — def/yield cousin (Ex 5.4)
Lazy evaluation — early-exit aur late binding ke peeche ka principle (Ex 3.3, 5.3, 5.4)
Big-O space complexity — $O (1)$ vs $O (n)$ (Ex 3.1)
Memory management in Python — Ex 3.1 mein list allocation kyun costly hai