Skip to content

Conversation

@eendebakpt
Copy link
Contributor

@eendebakpt eendebakpt commented Jan 13, 2023

This PR optimizes the closing of generators by switching the match statements in the expression PyErr_ExceptionMatches(PyExc_StopIteration) || PyErr_ExceptionMatches(PyExc_GeneratorExit) from the gen_close method. The case of a PyExc_GeneratorExit is more common.

Benchmark (with optimizations including PGO) updated May 9th:

all: Mean +- std dev: [main2] 283 ns +- 11 ns -> [pr] 280 ns +- 6 ns: 1.01x faster
all with comparison: Mean +- std dev: [main2] 570 ns +- 14 ns -> [pr] 563 ns +- 9 ns: 1.01x faster

Benchmark hidden because not significant (3): generator, list comprehension, generator exhaust

Geometric mean: 1.00x faster

Benchmark script:

import pyperf
runner = pyperf.Runner()

setup="""
l=[1,2]
"""

runner.timeit(name=f"generator", stmt=f"(x for x in l)", setup=setup)
runner.timeit(name=f"list comprehension", stmt=f"[x for x in l]", setup=setup)
stmt="""
for x in (_ for _ in ()):
    pass
"""
runner.timeit(name=f"generator exhaust", stmt=stmt, setup=setup)
runner.timeit(name=f"all", stmt=f"all( (x for x in l) ) ", setup=setup)
runner.timeit(name=f"all with comparison", stmt=f"all( (x==1 for x in l) ) ", setup=setup)

Notes:

def f():
    try: 
        yield None
        print('here')
    except GeneratorExit:
        print("exiting")
    print('done')

g = f()
next(g)
g.close()

Copy link
Member

@brandtbucher brandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good. Let's wait for 3.13 in two weeks, since performance improvements are effectively frozen right now.

@eendebakpt
Copy link
Contributor Author

Seems good. Let's wait for 3.13 in two weeks, since performance improvements are effectively frozen right now.

@brandtbucher Thanks for looking into this. The main branch has changed in the meantime, and there is currently only a single match instead of the two before. This makes this PR irrelevant, I will close it.

@eendebakpt eendebakpt closed this Jun 9, 2023
@eendebakpt eendebakpt deleted the gen_close branch June 26, 2025 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants