Python - Python yield from and Advanced Generator Delegation

The yield from statement in Python is used inside a generator to delegate part or all of its iteration work to another iterable or generator. It provides a convenient way for one generator to yield values produced by another generator without writing an explicit loop. It was introduced in Python 3.3 and is especially useful when working with nested generators, generator pipelines, and complex iteration logic.

1. Understanding yield from

Normally, if one generator wants to yield values from another iterable, you might write:

def numbers():
    for number in [1, 2, 3]:
        yield number

Using yield from, the same operation can be written more directly:

def numbers():
    yield from [1, 2, 3]

When the generator is executed, it produces:

1
2
3

The statement:

yield from iterable

means that values from the specified iterable should be yielded one by one by the current generator.

2. Delegating to Another Generator

The main purpose of yield from becomes clearer when one generator delegates its work to another generator.

def child_generator():
    yield 10
    yield 20
    yield 30

def parent_generator():
    yield from child_generator()

for value in parent_generator():
    print(value)

Output:

10
20
30

Here, parent_generator() delegates its iteration to child_generator(). The parent generator does not need to manually loop through the child generator.

Without yield from, the code would normally be:

def parent_generator():
    for value in child_generator():
        yield value

Both approaches produce the same values, but yield from expresses the delegation more clearly and handles additional generator behavior automatically.

3. Delegating to Multiple Generators

A generator can delegate to several generators sequentially.

def first():
    yield "A"
    yield "B"

def second():
    yield "C"
    yield "D"

def combined():
    yield from first()
    yield from second()

print(list(combined()))

Output:

['A', 'B', 'C', 'D']

The first generator is completely processed before the second generator begins.

This makes yield from useful for combining multiple sources of data into a single generator.

4. Using yield from with Regular Iterables

yield from is not limited to generators. It can delegate to lists, tuples, strings, sets, dictionaries, and other iterable objects.

For example:

def characters():
    yield from "Python"

print(list(characters()))

Output:

['P', 'y', 't', 'h', 'o', 'n']

Similarly:

def values():
    yield from [10, 20, 30]
    yield from (40, 50)

The resulting sequence is:

10
20
30
40
50

5. yield from and return Values

One of the most important features of yield from is that it can receive a value returned by the delegated generator.

Consider:

def child():
    yield 1
    yield 2
    return "Finished"

def parent():
    result = yield from child()
    print("Child returned:", result)

for value in parent():
    print(value)

Output:

1
2
Child returned: Finished

The return "Finished" statement in the child generator does not become another yielded value. Instead, it becomes the result of the yield from expression.

Therefore:

result = yield from child()

stores the value returned by child() in result.

This is an important distinction between yield and yield from.

6. Sending Values Through yield from

Generators can receive values using the send() method. yield from allows these values to be passed through the delegating generator to the delegated generator.

Example:

def child():
    value = yield "Ready"
    return value * 2

def parent():
    result = yield from child()
    print("Result:", result)

generator = parent()

print(next(generator))
print(generator.send(5))

The child generator receives 5 through send(). It calculates:

5 * 2 = 10

and returns 10 to the parent generator.

The parent receives that returned value through:

result = yield from child()

This allows generators to communicate values in both directions.

7. Why yield from Is Better Than a Manual Loop

Consider this code:

def parent():
    for item in child_generator():
        yield item

It works, but it manually forwards yielded values.

With yield from:

def parent():
    yield from child_generator()

Python handles the delegation more completely. In addition to forwarding yielded values, yield from can also manage values sent into the generator, exceptions, and the return value of the delegated generator.

Therefore, yield from is more than a shorter version of a for loop. It creates a formal delegation relationship between two generators.

8. Building Generator Pipelines

yield from can be useful when building a sequence of generators that process data in stages.

For example:

def numbers():
    yield from range(1, 6)

def squares():
    for number in numbers():
        yield number * number

for value in squares():
    print(value)

Output:

1
4
9
16
25

The numbers() generator produces the input values, while squares() processes them.

Generator delegation can become particularly useful when a large application separates different stages of data processing into independent generators.

9. Nested Generator Delegation

Generators can delegate through several levels.

def level_three():
    yield "C1"
    yield "C2"

def level_two():
    yield from level_three()

def level_one():
    yield from level_two()

print(list(level_one()))

Output:

['C1', 'C2']

Here, level_one() delegates to level_two(), which delegates to level_three().

This type of structure can be useful when breaking a complicated data-generation process into smaller, reusable components.

10. Exception Handling with Delegated Generators

yield from also participates in exception handling between generators.

For example:

def child():
    try:
        yield 1
        yield 2
    except ValueError:
        yield "Error handled by child"

def parent():
    yield from child()

The delegating generator can allow the delegated generator to handle exceptions appropriately.

This is one reason yield from is more powerful than simply writing a for loop around another generator.

11. Important Difference Between yield and yield from

The difference can be summarized as follows:

yield [1, 2, 3]

produces the entire list as one value.

The output is conceptually:

[1, 2, 3]

Whereas:

yield from [1, 2, 3]

produces each element separately:

1
2
3

Therefore, yield from is particularly useful when the intention is to delegate iteration over another iterable.

12. Practical Example

Suppose an application receives information from several data sources:

def database_records():
    yield "Database record 1"
    yield "Database record 2"

def file_records():
    yield "File record 1"
    yield "File record 2"

def all_records():
    yield from database_records()
    yield from file_records()

for record in all_records():
    print(record)

Output:

Database record 1
Database record 2
File record 1
File record 2

The all_records() generator provides one unified interface while keeping each data source separate.

Conclusion

yield from is an advanced Python generator feature used for generator delegation. It allows one generator to delegate iteration to another iterable or generator while also supporting more advanced communication, including passing values with send(), handling exceptions, and receiving a delegated generator's return value.

Its simplest form is:

yield from iterable

For basic iteration, it provides cleaner code than manually writing a for loop with yield. For advanced generator programming, however, its main importance is that it creates a complete delegation mechanism between generators. This makes yield from valuable for building reusable generators, combining multiple data sources, constructing generator pipelines, and organizing complex iteration logic.