Skip to content

[mypyc] Faster set creation with generator expression #10261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

97littleleaf11
Copy link
Collaborator

Description

Closes mypyc/mypyc#771 and #9707

Changes:

Test Plan

Both generated IR test and functional test:

  • Set built from iterators (list tuple dict range)
  • Set built from nested generator

@JukkaL
Copy link
Collaborator

JukkaL commented Mar 29, 2021

Thanks for the PR! Have you benchmarked this against master? (If not, no worries, I can do some quick benchmarking tomorrow.)

@97littleleaf11
Copy link
Collaborator Author

Thanks for the PR! Have you benchmarked this against master? (If not, no worries, I can do some quick benchmarking tomorrow.)

Not yet.

@@ -143,6 +143,9 @@ def translate_safe_generator_call(
val = tuple_from_generator_helper(builder, expr.args[0])
if val is not None:
return val
if callee.fullname == "builtins.set":
return translate_set_comprehension(builder, expr.args[0])

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should move builtins.set to a new specialize function ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds reasonable, since the implementation isn't sharing much with the other cases.

@JukkaL
Copy link
Collaborator

JukkaL commented Mar 30, 2021

I measured the performance and now the speed of calling set with a generator expression is the same as a set comprehension. Nice! Calling set was 62% faster in the ad-hoc benchmark that I used, compared to master.

@97littleleaf11 97littleleaf11 requested a review from JukkaL March 31, 2021 16:11
Copy link
Collaborator

@JukkaL JukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good! This is a nice performance win.

@JukkaL JukkaL merged commit 2787d49 into python:master Mar 31, 2021
@97littleleaf11 97littleleaf11 deleted the faster-creation-of-set-with-generator branch March 31, 2021 17:33
sthagen added a commit to sthagen/python-mypy that referenced this pull request Apr 2, 2021
[mypyc] Faster set creation with generator expression (python#10261)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Faster set creation with generator expression
2 participants