Add support for using zstd-compressed nightly bundles#2442
Merged
Conversation
henrymercer
force-pushed
the
henrymercer/zstd-bundles
branch
3 times, most recently
from
August 28, 2024 15:25
4b5b9e6 to
7ce060f
Compare
Using the downloaded path is unreliable since we may have removed the file extension.
henrymercer
force-pushed
the
henrymercer/zstd-bundles
branch
from
August 29, 2024 17:12
7ce060f to
27dbb1a
Compare
henrymercer
marked this pull request as ready for review
August 29, 2024 17:16
angelapwen
approved these changes
Aug 29, 2024
angelapwen
left a comment
Contributor
There was a problem hiding this comment.
Looks good, couple of questions:
- is there a reason we're only supporting for nightly bundles so far? (maybe I've missed the context behind that)
- should we expose this in an
experimentalchangenote or will we do that once we've deemed the experiment a success?
Contributor
Author
|
Thanks for the review!
Only that we haven't had a stable release with a zstd bundle yet. My thinking was that it would be easier to write and test the functionality for stable versions once we have a stable zstd bundle.
I was thinking no since this only applies to nightly bundles which we haven't advertised before, but I don't feel strongly about it. |
Contributor
When we're confident about this being a positive improvement, we will go through a proper release channel just to make sure we put out a changelog note and update documentation. |
8 tasks
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We are currently experimenting with using bundles compressed using Zstandard (zstd) rather than gzip to potentially save a significant chunk of the time we spend extracting the bundle. This could lead to a noticeable analysis speedup, particularly for small codebases where fixed costs represent a higher proportion of the total runtime.
This PR is one of the first steps. Specifically:
toolsinput toinittar xz,xfor extract,zfor gzip, we just runtar x.Merge / deployment checklist