Repository navigation
Conversation
Author
|
Closing in favor of the earlier #164, which proposes the same core directory index. I’ve shared focused parsing and close-cleanup follow-ups there, with the subclass compatibility question kept separate. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recursive
Path.iterdir()traversal currently scans every archive entry andconstructs a
Pathfor each candidate at every directory. Cache immediate childnames by parent on the first read-only listing, then construct paths only for
that directory's children. A complete traversal avoids repeated archive-wide
scans.
The index is owned by the archive and released on
ZipFile.close(). Nonlistingoperations do not build it. Writable and closed archives scan without caching;
Pathsubclasses retain the original implementation to preserve their hooks.Entry order, duplicates, and unusual-name behavior are preserved. Closing clears
the cached index without invalidating existing iterators.
Performance and trade-off
CPython main at
4a9b98115fec4d93c32c554aa541d6c3fc4c3b27(3.16.0a1+dev), GCC 13.3,
-O3, without PGO/LTO. Both implementations usethis interpreter; the baseline is its unmodified
zipfile.Path.Medians of five rotating runs, with ZIP opening excluded. The recursive case
has 500 directories and 2,500 files. The first traversal includes name
preparation and index construction; the repeated traversal follows a complete
walk.
The first listing builds the entire index even if only one child is requested.
The extra retained allocations were approximately 41 KiB for the flat archive
and 99 KiB for the recursive case. Directory-heavy layouts can have
proportionally larger overhead. The improvement favors repeated or recursive
listings at the expense of first-listing work and additional metadata storage.
Validation
On the same CPython main build:
process; two skipped.
names, pickling, concurrent construction, and cleanup during and after close.
Ruff, formatting, mypy, and
git diff --checkpassed.