Generators — yield, generator functions, send(), next()

1.3.9 · Coding › Python Intermediate

WHAT is a generator?

def count_up(n):
    i = 0
    while i < n:
        yield i        # pause here, hand out i
        i += 1         # resume here next time
 
g = count_up(3)        # NOTHING runs yet; g is a generator object
print(next(g))   # 0   <- runs until first yield, pauses
print(next(g))   # 1   <- resumes after yield, runs to next yield
print(next(g))   # 2
print(next(g))   # raises StopIteration  (loop ended)

HOW does pausing actually work? (derive it from scratch)

Tum ek plain class se generator ka concept build kar sakte ho, jo prove karta hai ki isme koi magic nahi hai:

class CountUp:
    def __init__(self, n):
        self.n = n
        self.i = 0
    def __iter__(self):
        return self
    def __next__(self):
        if self.i >= self.n:
            raise StopIteration        # the "end of body"
        val = self.i                   # the "yield value"
        self.i += 1                    # the "resume" bookkeeping
        return val

Toh ek generator exactly yeh hai: ek object jisme __iter__ aur __next__ hain, lekin position ka saving interpreter karta hai, tum nahi.

The protocol: `next()`, `StopIteration`, aur `for`

for x in count_up(3):   # calls next() until StopIteration, then stops
    print(x)            # 0, 1, 2

`send()` — generator ke ANDAR baat karna

Figure — Generators — yield, generator functions, send(), next()

Worked example A running-average coroutine

def averager():
    total = 0.0
    count = 0
    average = None
    while True:
        x = yield average      # send 'average' OUT, get next 'x' IN
        total += x
        count += 1
        average = total / count
 
g = averager()
next(g)            # PRIME it: run up to first yield (yields None)
print(g.send(10)) # 10.0   -> x became 10
print(g.send(20)) # 15.0
print(g.send(30)) # 20.0

next(g) se pehle prime kyun karo? Kyunki pehli run mein generator abhi yield tak pahuncha nahi hai, toh koi paused yield expression nahi hai jo value receive kare. Prime karne se pehle send(non-None) karna TypeError raise karta hai. Yeh step kyun? Tum ek sawaal ka "jawab" tabhi de sakte ho jab generator ne woh "poochha" ho (yaani yield tak pahuncha ho).

Worked examples

Worked example 1 — Lazy infinite sequence (memory win)

def naturals():
    n = 1
    while True:        # infinite! safe because it's lazy
        yield n
        n += 1
 
def take(gen, k):
    out = []
    for _ in range(k):
        out.append(next(gen))   # pull exactly k values
    return out
 
print(take(naturals(), 5))   # [1, 2, 3, 4, 5]

Yeh step kyun? while True ek normal function ko hang kar deta. Ek generator mein yeh theek hai — values tabhi materialize hoti hain jab pull karo, toh hum control karte hain kitni chahiye.

Worked example 2 — Pipelining (chaining generators)

def read_numbers(lines):
    for line in lines:
        yield int(line.strip())
 
def squares(nums):
    for n in nums:
        yield n * n
 
data = ["1\n", "2\n", "3\n"]
pipe = squares(read_numbers(data))   # nothing computed yet
print(list(pipe))   # [1, 4, 9]

Yeh step kyun? Har generator pichle wale se ek item tab hi pull karta hai jab maanga jaata hai. Poori pipeline ek element at a time stream karti hai — kabhi saara data load nahi hota.

Worked example 3 —

yield from (delegation)

def chain(*iterables):
    for it in iterables:
        yield from it        # re-yield every value of it
 
print(list(chain([1,2], (3,4), "ab")))   # [1, 2, 3, 4, 'a', 'b']

Yeh step kyun? yield from it ek manual for x in it: yield x loop ki jagah le leta hai aur send()/exceptions/return values bhi sahi se forward karta hai.

Common mistakes (steel-manned)

80/20 — woh 20% jo 80% result deta hai

Recall Feynman: explain to a 12-year-old

Ek kahani sunane wale ki kalpana karo jo tumhe EK sentence sunata hai, phir bich mein freeze ho jaata hai aur wait karta hai. Jab tum "next!" bolte ho, woh exactly wahi word se continue karta hai jahan ruka tha — woh kabhi nahi bhoolta kahan tha. Yahi ek generator hai: yield hai freeze, next() hai tumhara "next!", aur send() hai tumhara woh word whisper karna jo woh decide karne ke liye use karta hai ki aage kya bolna hai. Kyunki woh ek time par ek hi sentence bolta hai, use poori kitaab dimag mein rakhne ki zaroorat nahi — isliye memory bachti hai.

Flashcards

What makes a function a generator function?

Uski body mein yield keyword hota hai.

What does calling a generator function actually do?

Turant ek generator object return karta hai; body sirf pehle next() par chalti hai.

What does next(g) do?

Execution ko next yield tak resume karta hai (woh value return karta hai) ya agar body khatam ho jaaye toh StopIteration raise karta hai.

Difference between yield and return?

yield pause karta hai aur state save karta hai taaki baad mein resume ho sake; return function khatam kar deta hai (generator mein yeh StopIteration raise karta hai).

Why are generators memory-efficient?

Yeh demand par lazily values produce karte hain, poori sequence build/store karne ki jagah.

Can you iterate a generator twice?

Nahi — yeh single-use hai; ek baar exhaust hone ke baad kuch nahi yield karta.

What does g.send(v) do?

Generator ko resume karta hai, paused yield expression ko v evaluate karata hai, aur next yielded value return karta hai.

Why must you prime a generator before send(non-None)?

Woh kisi bhi yield tak pahuncha nahi hai, toh koi paused yield expression nahi hai jo value receive kare; warna TypeError.

next(g) is equivalent to which send call?

g.send(None).

What does yield from iterable do?

Sub-iterable ki har value yield karta hai aur send/exceptions forward karta hai aur uski return value capture karta hai.

How does a for loop know when to stop on a generator?

Woh __next__ ke zariye raise hone wale StopIteration ko catch karta hai aur silently band ho jaata hai.

Is a generator its own iterator?

Haan — iter(g) is g, aur iske paas __iter__ aur __next__ dono hain.

Connections

Iterators and the Iterator Protocol — generators ek banane ka sabse aasaan tarika hai.
List Comprehensions vs Generator Expressions — (x for x in ...) ek generator hai.
Lazy Evaluation — generators kaam ko zaroorat padne tak delay karte hain.
Coroutines and async — send() coroutine-style code ki neenv hai.
StopIteration and Exceptions — woh signal jo iteration khatam karta hai.
Memory and Big Data Streaming — kyun pipelines generators use karti hain.