Generator Patterns

Introduction

Generators provide powerful patterns for lazy evaluation, streaming data, and processing infinite sequences. They produce values on-demand rather than storing them in memory.

Generator Basics

def number_generator(n):
    for i in range(n):
        yield i

gen = number_generator(5)
print(list(gen))  # [0, 1, 2, 3, 4]

# Or use directly in loop
for num in number_generator(3):
    print(num)  # 0, 1, 2

Pipeline Pattern

def numbers():
    for i in range(1, 10):
        yield i

def squares(iterable):
    for x in iterable:
        yield x ** 2

def filter_even(iterable):
    for x in iterable:
        if x % 2 == 0:
            yield x

# Chain generators
result = filter_even(squares(numbers()))
print(list(result))  # [4, 16, 36, 64]

Infinite Sequence

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
for i in range(10):
    print(next(fib))  # 0, 1, 1, 2, 3, 5, 8, 13, 21, 34

Streaming Data

def process_large_file(filename):
    with open(filename, "r") as f:
        for line in f:
            yield line.strip()

# Memory efficient - doesn't load entire file
for line in process_large_file("large.txt"):
    if "ERROR" in line:
        print(line)

Generator Send

def bounded_counter(limit):
    count = 0
    while count < limit:
        value = yield count
        if value is not None:
            count = value
        else:
            count += 1

gen = bounded_counter(5)
print(next(gen))  # 0
print(next(gen))  # 1
gen.send(10)      # Reset to 10
print(next(gen))  # 11

Practice Problems

1. Create generator pipeline
2. Generate infinite prime sequence
3. Implement streaming data processor
4. Use generator send for bidirectional communication
5. Build generator for file processing