Generators — yield, generator functions, send(), next(): Worked examples

The scenario matrix

Before any code, let's list every distinct behaviour a generator can show. Think of each row as a trap the language can spring on you. Every worked example below is tagged with the cell it hits, so by the end you'll have touched all of them.

#	Case class	The specific edge	Example that covers it
A	Empty output	generator that `yield`s zero times	Ex 1
B	Exactly one value	one `yield`, then `StopIteration`	Ex 1
C	Finite many	normal loop, ends cleanly	Ex 2
D	Infinite / lazy	`while True`, never ends by itself	Ex 3
E	Single-use exhaustion	looping the same generator twice	Ex 4
F	`send()` before priming	the `TypeError` trap	Ex 5
G	`send()` after priming	value flows in and out	Ex 5
H	`return` inside a generator	value hidden in `StopIteration.value`	Ex 6
I	Delegation `yield from`	forwarding + capturing the sub-return	Ex 7
J	Real-world word problem	streaming a huge file, memory win	Ex 8
K	Exam-style twist	ordering of side-effects (when does code run?)	Ex 9

Related vault ideas we'll lean on: Iterators and the Iterator Protocol, Lazy Evaluation, StopIteration and Exceptions, Coroutines and async, Memory and Big Data Streaming, and the contrast in List Comprehensions vs Generator Expressions.

Example 1 — Empty and one-value cases (cells A & B)

Worked example Statement

def empty_gen():
    if False:
        yield 1          # makes it a generator, but never runs
 
def one_gen():
    yield 42
 
print(list(empty_gen()))
print(list(one_gen()))

Forecast: guess both printed lines before reading on. How many values does each produce?

Steps:

empty_gen contains the word yield, so it is a generator function even though the yield is unreachable.
- Why this step? The rule is purely syntactic: the presence of yield anywhere in the body turns the function into a generator. Reachability doesn't matter.
Calling list(empty_gen()) repeatedly calls next(). The very first next() runs the body, hits no yield, falls off the end → raises StopIteration immediately.
- Why this step? list() swallows StopIteration and returns whatever it collected — here, nothing. So the result is [].
one_gen yields once. First next() → 42, pauses. Second next() → body ends → StopIteration.
- Why this step? This is the minimal non-empty generator; it proves "one yield = one value, then stop".

Verify: list(empty_gen()) == [] and list(one_gen()) == [42]. Sanity: number of items = number of times yield actually executed (0 and 1). ✓

Example 2 — Finite many, counted (cell C)

Worked example Statement

def count_up(n):
    i = 0
    while i < n:
        yield i
        i += 1
 
print(list(count_up(4)))

Forecast: how many items, and what are they?

Steps:

Loop condition i < n with n = 4 runs for i = 0,1,2,3.
- Why this step? The loop guards the number of yields; count them by counting loop iterations.
Each iteration yields the current i, then increments.
- Why this step? The value comes before the increment, so we get 0,1,2,3, not 1,2,3,4.
When i reaches 4, i < n is false → loop ends → StopIteration.
- Why this step? This clean end is exactly what list()/for relies on to stop.

Verify: list(count_up(4)) == [0, 1, 2, 3]; length 4, sum 0+1+2+3 = 6. ✓

Example 3 — Infinite / lazy, take only what you need (cell D)

Here's the picture of why an infinite generator doesn't hang: values are produced one at a time, on demand, never all at once.

Figure — Generators — yield, generator functions, send(), next()

Worked example Statement

def naturals():
    n = 1
    while True:          # never ends on its own
        yield n
        n += 1
 
g = naturals()
first_five = [next(g) for _ in range(5)]
print(first_five)

Forecast: does while True freeze your program? What does first_five hold?

Steps:

naturals() builds the generator object — no code runs yet.
- Why this step? Lazy start (Lazy Evaluation) is what makes an infinite loop safe here.
We call next(g) exactly 5 times via the list comprehension.
- Why this step? We control how many values materialize; the generator only advances when pulled. Look at the figure: each pull crosses the "tap" once.
It never raises StopIteration — we simply stop asking.
- Why this step? An infinite generator ends only when you stop calling next; there is no natural end.

Verify: first_five == [1, 2, 3, 4, 5]. ✓

Example 4 — Single-use exhaustion (cell E)

Worked example Statement

def count_up(n):
    i = 0
    while i < n:
        yield i
        i += 1
 
g = count_up(3)
a = list(g)     # first pass
b = list(g)     # second pass on the SAME object
print(a, b)

Forecast: will b equal a? Or something else?

Steps:

list(g) drains g to exhaustion; after this g has already raised StopIteration.
- Why this step? A generator's position only moves forward — there is no rewind. See Iterators and the Iterator Protocol.
The second list(g) immediately gets StopIteration (the generator is spent).
- Why this step? This is the defining difference from a list, which you can iterate again.
To iterate again you must call count_up(3) afresh to build a new generator.
- Why this step? A fresh call = fresh frozen frame with i = 0.

Verify: a == [0, 1, 2] and b == []. ✓

Example 5 — `send()`: before vs after priming (cells F & G)

This is the two-way street: the picture shows a value flowing out at yield, and a value flowing in as the result of that same yield expression.

Worked example Statement

def averager():
    total, count, average = 0.0, 0, None
    while True:
        x = yield average      # send average OUT, receive x IN
        total += x
        count += 1
        average = total / count
 
g = averager()
# BAD (cell F): g.send(10)  -> TypeError, generator not primed
next(g)                      # prime: run to first yield, yields None
r1 = g.send(10)
r2 = g.send(20)
r3 = g.send(30)
print(r1, r2, r3)

Forecast: why must we next(g) first, and what three numbers come out?

Steps:

g.send(10) before any next(g) raises TypeError — the generator has not yet reached a yield, so there is no paused expression to receive 10.
- Why this step? (cell F) You can only answer a question after it's been asked; priming makes the generator ask.
next(g) runs to x = yield average. Since average is None, this first pull hands out None and freezes inside the yield.
- Why this step? Priming = send(None); it gets the generator to the point where it can receive.
g.send(10) sets x = 10, updates total=10, count=1, average=10.0, loops, yields 10.0 out.
- Why this step? Now the injected value flows in (see the blue arrow in the figure) and the new average flows out.
g.send(20): total=30, count=2, average=15.0. g.send(30): total=60, count=3, average=20.0.
- Why this step? Each send both delivers input and receives the running result — that's the coroutine pattern (Coroutines and async).

Verify: r1, r2, r3 == 10.0, 15.0, 20.0. Cross-check: means of {10}, {10,20}, {10,20,30} = 10, 15, 20. ✓

Example 6 — `return` inside a generator (cell H)

Worked example Statement

def gen_with_return():
    yield 1
    yield 2
    return "done"      # NOT a yielded value
 
g = gen_with_return()
values = []
try:
    while True:
        values.append(next(g))
except StopIteration as e:
    final = e.value
print(values, final)

Forecast: does "done" appear in values? Where does it end up?

Steps:

next(g) twice gives 1 then 2.
- Why this step? Only yield produces iteration values.
The third next(g) hits return "done", which ends the generator by raising StopIteration.
- Why this step? In a generator, return x is not a normal return — it's the exhaustion signal.
The returned value is stashed on the exception as e.value.
- Why this step? This is how a generator can hand back a summary without it being iterated. Loops using for/list discard this value; only manual next or yield from captures it. See StopIteration and Exceptions.

Verify: values == [1, 2] and final == "done". ✓

Example 7 — `yield from` delegation (cell I)

Worked example Statement

def sub():
    yield "a"
    yield "b"
    return "sub-result"
 
def main():
    result = yield from sub()   # re-yields a,b; captures return
    yield f"got:{result}"
 
print(list(main()))

Forecast: what's in the final list, and where does "sub-result" surface?

Steps:

yield from sub() transparently re-yields every value of sub: "a", then "b".
- Why this step? yield from it replaces a manual for x in it: yield x — but it also forwards send/exceptions and captures the sub-generator's return value.
When sub hits return "sub-result", that value becomes the value of the whole yield from expression, bound to result.
- Why this step? This is the one place a generator's return value is not thrown away — delegation catches it.
main then yields "got:sub-result".
- Why this step? Proves the captured value can be reused downstream.

Verify: list(main()) == ["a", "b", "got:sub-result"]. ✓

Example 8 — Real-world: streaming a huge log, memory win (cell J)

The whole point: a generator holds one line at a time, not the whole file. The figure contrasts a list (all rows in memory) with a generator (a single moving window).

Worked example Statement

A 10-million-line server log. Count how many lines contain the word "ERROR", without loading the file into memory.

def lines_from(source):           # source: any iterable of strings
    for line in source:
        yield line.strip()
 
def only_errors(lines):
    for line in lines:
        if "ERROR" in line:
            yield line
 
log = ["ok\n", "ERROR disk\n", "ok\n", "ERROR net\n", "warn\n"]
pipe = only_errors(lines_from(log))     # nothing computed yet
count = sum(1 for _ in pipe)
print(count)

Forecast: how many errors, and how many lines sit in memory at peak?

Steps:

only_errors(lines_from(log)) builds a pipeline but computes nothing yet (lazy).
- Why this step? Each stage pulls one item from the previous only when asked — Memory and Big Data Streaming.
sum(1 for _ in pipe) pulls items one at a time; only the current line is ever "live".
- Why this step? Peak memory ≈ one line, whether the log has 5 lines or 10 million. That's the win over list (List Comprehensions vs Generator Expressions).
Two lines contain "ERROR" → count = 2.
- Why this step? We count matches, not total lines.

Verify: count == 2 for the sample; peak-in-memory = 1 line regardless of file size. Units check: a count is dimensionless (a plain integer). ✓

Example 9 — Exam twist: WHEN does the code run? (cell K)

Worked example Statement

def noisy():
    print("START")
    yield 1
    print("MIDDLE")
    yield 2
    print("END")
 
g = noisy()
print("created")
print(next(g))
print(next(g))

Forecast: write down the exact order of everything printed before peeking.

Steps:

g = noisy() prints nothing — only the generator object is built.
- Why this step? Body execution is deferred to the first next(). So "created" prints first.
First next(g) runs the body up to the first yield: prints "START", then hands out 1.
- Why this step? Everything before the first yield runs on the first pull, not at call time.
Second next(g) resumes after yield 1: prints "MIDDLE", hands out 2.
- Why this step? Execution continues from the frozen line, so "MIDDLE" appears between the two values. "END" would only print on a third next, which we never call.

Verify: printed order is exactly: created, START, 1, MIDDLE, 2. "END" does not print. ✓

80/20 — the scenarios that catch people

Recall Self-test

A generator yields 3 values then return "z". What does list(g) give? ::: [v1, v2, v3] — the "z" is discarded by list. g.send(5) on a brand-new (unprimed) generator does what? ::: raises TypeError. How many lines of a 10M-line file are in memory when streamed by a generator pipeline? ::: essentially one at a time. After list(g) exhausts g, what is list(g) the second time? ::: [] — generators are single-use.