Skip to main content

Command Palette

Search for a command to run...

Python Generators, Iterators, and Decorators

Updated
•4 min read•View as Markdown
A
I'm a Software Engineer

Python is famous for writing clean, readable, and expressive code. Three concepts that often confuse beginners—but are used extensively in real-world Python applications—are iterators, generators, and decorators.

Understanding these concepts will help you write more efficient code and better understand popular frameworks like Flask, Django, FastAPI, and many AI libraries.


1. Iterators

An iterator is an object that returns one element at a time while keeping track of its current position.

Instead of manually accessing elements by index, an iterator provides a standard way to traverse a collection.

Example

numbers = [1, 2, 3]

iterator = iter(numbers)

print(next(iterator))  # 1
print(next(iterator))  # 2
print(next(iterator))  # 3

After all elements have been returned, calling next() again raises a StopIteration exception.

print(next(iterator))
# StopIteration

How does a for loop work?

Most Python developers don't realize that every for loop uses an iterator internally.

When you write:

for number in numbers:
    print(number)

Python roughly does this behind the scenes:

iterator = iter(numbers)

while True:
    try:
        number = next(iterator)
        print(number)
    except StopIteration:
        break

This is why almost every iterable object in Python can be used directly inside a for loop.

Why use iterators?

  • Traverse collections one item at a time
  • Standard interface used by Python loops
  • Work with custom iterable objects
  • Foundation of generators

2. Generators

A generator is a special type of iterator created by a function that uses the yield keyword.

Unlike a normal function, a generator doesn't return all values at once. Instead, it produces values one at a time and remembers where it left off.

Example

def countdown(n):
    while n > 0:
        yield n
        n -= 1

for number in countdown(5):
    print(number)

Output

5
4
3
2
1

Each time next() is called, execution resumes from the previous yield statement.

Unlike return, which immediately ends a function, yield pauses execution and preserves the function's state until the next value is requested.

Why are generators useful?

Imagine reading a file with 10 million lines.

One approach loads the entire file into memory:

lines = open("large_file.txt").readlines()

A generator reads one line at a time:

with open("large_file.txt") as file:
    for line in file:
        print(line)

This uses significantly less memory because only one line is processed at a time.

Common use cases

  • Reading large files
  • Processing streaming data
  • Working with APIs
  • Creating pipelines
  • Generating infinite sequences

3. Decorators

A decorator is a function that adds functionality to another function without modifying its original source code.

Decorators are commonly used for logging, authentication, caching, validation, timing, and much more.

Example

from functools import wraps

def logger(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}...")

        result = func(*args, **kwargs)

        print(f"{func.__name__} finished.")
        return result

    return wrapper


@logger
def greet(name):
    print(f"Hello, {name}!")


greet("Alice")

Output

Calling greet...
Hello, Alice!
greet finished.

The @logger syntax tells Python to wrap the original function with additional behavior before and after it executes.

Using functools.wraps preserves the original function's metadata, such as its name and documentation, which is considered a best practice.

Common use cases

  • Logging
  • Authentication
  • Authorization
  • Input validation
  • Performance monitoring
  • Caching
  • Rate limiting

Iterator vs Generator vs Decorator

Feature Iterator Generator Decorator
Purpose Traverse data one item at a time Produce values lazily Extend function behavior
Created using iter() or implementing __iter__() and __next__() Function with yield Function with @decorator syntax
Stores state ✅ ✅ N/A
Memory efficient Depends on the iterable Yes, for lazy evaluation N/A
Common use cases Loops, collections Large datasets, streaming, pipelines Logging, authentication, caching

When should you use each?

Use an iterator when you need a standard way to traverse a collection.

Use a generator when producing values one at a time is more efficient than creating an entire collection in memory.

Use a decorator when you want to add functionality to existing functions without changing their implementation.


Key Takeaways

  • An iterator produces one element at a time and powers Python's for loop.
  • A generator is an easy way to create iterators using the yield keyword.
  • A decorator extends a function's behavior without modifying its original code.

Although these concepts seem unrelated, they are all examples of Python's emphasis on writing reusable, efficient, and expressive code.

Mastering them will make you a stronger Python developer and help you better understand many modern Python frameworks and AI libraries.