Generator expressions — memory efficiency — Hinglish notes

1.2.36 · Coding › Introduction to Programming (Python)

WHAT hai ek generator expression?

list_comp = [x*x for x in range(1_000_000)]   # builds 1,000,000 ints NOW
gen_expr  = (x*x for x in range(1_000_000))   # builds NOTHING yet

list_comp ek real list hai jisme ek million numbers stored hain.
gen_expr ek chhota sa generator object hai jo yaad rakhta hai kaise unhe produce karna hai.

WHY save hoti hai memory isse?

Toh memory cost yeh hai:

Object	Memory used	$n$ ke saath badhti hai?
$n$ items ki List	saare $n$ items store karta hai	Haan — $O (n)$
Generator	1 item + recipe store karta hai	Nahi — $O (1)$

HOW karte hain actually use?

Aap ise iterate karke consume karte hain — for, sum(), next(), any(), etc. Har ek values ko ek baar consume karta hai.

# next() pulls ONE value
g = (x*x for x in range(5))
next(g)   # 0
next(g)   # 1   <- state advanced, 0 is gone
 
# A function that takes an iterable consumes the whole generator
total = sum(x*x for x in range(1_000_000))   # parentheses optional as sole arg

Common mistake Steel-man: "Main generator ko do baar reuse kar lunga"

Kyun sahi lagta hai: lists ko baar baar loop kiya ja sakta hai, toh aap assume karte hain ki generators bhi ho sakte hain. Actually kya hota hai: ek generator ek full pass ke baad exhausted ho jaata hai — doosri baar kuch yield nahi karta.

g = (x for x in range(3))
list(g)   # [0, 1, 2]
list(g)   # []   <-- empty! already consumed

Fix: ise dobara banao (g = (...) phir se), ya agar genuinely do baar chahiye, toh list use karo.

Figure — Generator expressions — memory efficiency

Worked Examples

Worked example Example 1 — Million-element list ke bina squares ka sum karna

total = sum(x*x for x in range(1_000_000))

Yeh step kyun? sum ko har square sirf momentarily chahiye hota hai running total mein add karne ke liye. Generator unhe ek ek karke deta hai, toh peak memory chhoti rehti hai jabki ek list ~8 MB+ allocate kar deti.

Worked example Example 2 — Ek badi file ko lazily filter karna

with open("huge.log") as f:
    error_lines = (line for line in f if "ERROR" in line)
    first = next(error_lines)   # reads only until the FIRST error

Yeh step kyun? Laziness ki wajah se, Python file padhna tab rok deta hai jab use pehla match milta hai — poori file kabhi load nahi hoti. List comprehension pehle poori file scan kar leta.

Worked example Example 3 — Forecast-then-Verify

Output pehle predict karo aage padhne se:

g = (n for n in range(4) if n % 2 == 0)
print(list(g))
print(sum(g))

Forecast: pehli line [0, 2], doosri line 0 (NOT 2). Verify / Kyun? list(g) generator ko poora consume karta hai → [0, 2]. Ab g exhausted hai, toh sum(g) kuch nahi add karta → 0. Trap yeh hai ki aap assume karte hain ki g reset ho jaata hai.

Recall Feynman: ek 12-saal ke bacche ko explain karo

Socho tum 100 sandwiches khaana chahte ho. List tarika: pehle saari 100 banao, table par pile kar do (bada mess, bahut jagah). Generator tarika: kitchen next sandwich tabhi banata hai jab tum pichli kha lo. Aapke paath kabhi ek se zyada nahi hoti — toh almost koi table space nahi chahiye, chahe 100 hon ya million sandwiches. Ek baat: jab sab kha lo, toh gone — aap same wali dobara nahi kha sakte.

Active Recall

What punctuation distinguishes a generator expression from a list comprehension?

Parentheses () instead of square brackets [].

What is the memory complexity of a generator expression vs a list comprehension over n items?

Generator is O(1) (one item at a time); list is O(n) (all items stored).

Why does (x*x for x in range(n)) use almost no memory?

It stores only the recipe/state and yields each square on demand, never building the full sequence.

What happens when you iterate a generator a second time?

Nothing is produced — it is exhausted after one full pass.

When summing 1,000,000 squares, why prefer sum(x*x for x in range(10**6)) over a list comprehension?

The generator avoids allocating a million-element list; peak memory stays tiny.

Can you index a generator like g[0]?

No — generators are not subscriptable; you must use next() or iterate.

If a generator expression is the sole argument to a function, what can you drop?

The extra parentheses, e.g. sum(x*x for x in data).

Connections

List comprehensions — same syntax, eager evaluation
Iterators and the iterator protocol — __iter__ / __next__ generators ko underpin karte hain
Generator functions and yield — generator expressions ka def/yield cousin
Lazy evaluation — "compute when needed" ka general principle
Big-O space complexity — kyun O(1) vs O(n) matter karta hai
Memory management in Python — lists ki heap allocation