Python Generators, Iterators, and Decorators
Python is famous for writing clean, readable, and expressive code. Three concepts that often confuse beginners—but are used extensively in real-world Python applications—are iterators, generators, and decorators.
Understanding these concepts will help you write more efficient code and better understand popular frameworks like Flask, Django, FastAPI, and many AI libraries.
1. Iterators
An iterator is an object that returns one element at a time while keeping track of its current position.
Instead of manually accessing elements by index, an iterator provides a standard way to traverse a collection.
Example
numbers = [1, 2, 3]
iterator = iter(numbers)
print(next(iterator)) # 1
print(next(iterator)) # 2
print(next(iterator)) # 3
After all elements have been returned, calling next() again raises a StopIteration exception.
print(next(iterator))
# StopIteration
How does a for loop work?
Most Python developers don't realize that every for loop uses an iterator internally.
When you write:
for number in numbers:
print(number)
Python roughly does this behind the scenes:
iterator = iter(numbers)
while True:
try:
number = next(iterator)
print(number)
except StopIteration:
break
This is why almost every iterable object in Python can be used directly inside a for loop.
Why use iterators?
- Traverse collections one item at a time
- Standard interface used by Python loops
- Work with custom iterable objects
- Foundation of generators
2. Generators
A generator is a special type of iterator created by a function that uses the yield keyword.
Unlike a normal function, a generator doesn't return all values at once. Instead, it produces values one at a time and remembers where it left off.
Example
def countdown(n):
while n > 0:
yield n
n -= 1
for number in countdown(5):
print(number)
Output
5
4
3
2
1
Each time next() is called, execution resumes from the previous yield statement.
Unlike return, which immediately ends a function, yield pauses execution and preserves the function's state until the next value is requested.
Why are generators useful?
Imagine reading a file with 10 million lines.
One approach loads the entire file into memory:
lines = open("large_file.txt").readlines()
A generator reads one line at a time:
with open("large_file.txt") as file:
for line in file:
print(line)
This uses significantly less memory because only one line is processed at a time.
Common use cases
- Reading large files
- Processing streaming data
- Working with APIs
- Creating pipelines
- Generating infinite sequences
3. Decorators
A decorator is a function that adds functionality to another function without modifying its original source code.
Decorators are commonly used for logging, authentication, caching, validation, timing, and much more.
Example
from functools import wraps
def logger(func):
@wraps(func)
def wrapper(*args, **kwargs):
print(f"Calling {func.__name__}...")
result = func(*args, **kwargs)
print(f"{func.__name__} finished.")
return result
return wrapper
@logger
def greet(name):
print(f"Hello, {name}!")
greet("Alice")
Output
Calling greet...
Hello, Alice!
greet finished.
The @logger syntax tells Python to wrap the original function with additional behavior before and after it executes.
Using functools.wraps preserves the original function's metadata, such as its name and documentation, which is considered a best practice.
Common use cases
- Logging
- Authentication
- Authorization
- Input validation
- Performance monitoring
- Caching
- Rate limiting
Iterator vs Generator vs Decorator
| Feature | Iterator | Generator | Decorator |
|---|---|---|---|
| Purpose | Traverse data one item at a time | Produce values lazily | Extend function behavior |
| Created using | iter() or implementing __iter__() and __next__() |
Function with yield |
Function with @decorator syntax |
| Stores state | ✅ | ✅ | N/A |
| Memory efficient | Depends on the iterable | Yes, for lazy evaluation | N/A |
| Common use cases | Loops, collections | Large datasets, streaming, pipelines | Logging, authentication, caching |
When should you use each?
Use an iterator when you need a standard way to traverse a collection.
Use a generator when producing values one at a time is more efficient than creating an entire collection in memory.
Use a decorator when you want to add functionality to existing functions without changing their implementation.
Key Takeaways
- An iterator produces one element at a time and powers Python's
forloop. - A generator is an easy way to create iterators using the
yieldkeyword. - A decorator extends a function's behavior without modifying its original code.
Although these concepts seem unrelated, they are all examples of Python's emphasis on writing reusable, efficient, and expressive code.
Mastering them will make you a stronger Python developer and help you better understand many modern Python frameworks and AI libraries.
