Repository navigation
Closed
Labels
OS-windowsstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-pathlibtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
| BPO | 33428 |
|---|---|
| Nosy | @pfmoore, @pitrou, @tjguk, @jreese, @zware, @zooba, @emilyemorehouse, @BrianMSheldon |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2018-05-05.06:23:00.558>
labels = ['type-bug', 'library', 'OS-windows']
title = 'pathlib.Path.glob does not follow symlinks'
updated_at = <Date 2020-04-29.11:12:49.344>
user = 'https://github.com/BrianMSheldon'bugs.python.org fields:
activity = <Date 2020-04-29.11:12:49.344>
actor = 'Danya.Alexeyevsky'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2018-05-05.06:23:00.558>
creator = 'brianmsheldon'
dependencies = []
files = []
hgrepos = []
issue_num = 33428
keywords = []
message_count = 4.0
messages = ['316197', '316724', '316880', '367640']
nosy_count = 9.0
nosy_names = ['paul.moore', 'pitrou', 'tim.golden', 'jreese', 'Danya.Alexeyevsky', 'zach.ware', 'steve.dower', 'emilyemorehouse', 'brianmsheldon']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33428'
versions = ['Python 3.6']Linked PRs
Metadata
Metadata
Assignees
Labels
OS-windowsstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytopic-pathlibtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Projects
- StatusShow more project fieldsDone
Activity
BrianMSheldon commentedon May 5, 2018
Given a
pathlib.Paththat contains symlinked subfolders,Path.glob(and.rglob) do not follow symlinks. This is not consistent withglob.globwhich does.For example given the following:
C:\Folder
C:\Folder\Subfolder -> D:\Subfolder
D:\Subfolder\File.txt
pathlib.Path('C:/Folder').glob('**/*')yields the following paths:WindowsPath('C:/Folder/Subfolder')
glob.glob('C:/Folder/**/*')yields the following paths:'C:/Folder\Subfolder'
'C:/Folder\Subfolder\File.txt'
Notice how the contents of Subfolder are present in the
glob.globresults but not forPath.glob.I would expect
Path.globto be consistent withglob.glob. This is not the only inconsistency (e.g. bpo-22276, bpo-31202) and perhapsPath.globshould be re-implemented usingglob.glob.jreese commentedon May 15, 2018
This looks like an issue specific to Windows? I can't replicate on Mac, and given Windows' method of implementing "symlinks" as junctions.
BrianMSheldon commentedon May 17, 2018
Windows does not implement symlinks as junctions. Windows has hardlinks, symlinks and junctions which are all distinctly different in behaviour.
I don't doubt that this is a Windows-specific issue, although I have not tested other platforms. Path.glob and .rglob does work for junctions and hardlinks but glob.glob works consistently for all three.
DanyaAlexeyevsky commentedon Apr 29, 2020
I can reproduce the bug with Linux and python 3.7.5:
Expected result:
dlukes commentedon Nov 24, 2022
Following symlinks was disabled on purpose as a fix for #70200:
cpython/Lib/pathlib.py
Line 319 in 69f6cc7
To re-enable it, we'd have to come up with a different mitigation for the symlink loops problem.
Or possibly, if hidden behind a
follow_symlinksarg which would beFalseby default, it might be enough to just document it as a limitation whenfollow_symlinks=True? I don't know.barneygale commentedon Jan 28, 2023
glob()still follows symlinks when matching segments likefoo/,*/and so forth; it only refuses to follow symlinks when it encounters a**wildcard. So the current behaviour is roughlyfollow_symlinks=Sometimes.I realised this because I've been trying to implement an iterative version of
rglob(), building uponwalk()and filtering paths with a compiledre.Patternobject.walk()has its ownfollow_symlinksargument that acceptsTrueandFalse, but neither option aligns with the existing behaviour, so it seems like a dead end unless this "bug" is "fixed".barneygale commentedon Jan 29, 2023
zsh has a
***wildcard that recurses into symlinks to directories, see Recursive Globbing here: https://linux.die.net/man/1/zshexpnbarneygale commentedon Feb 6, 2023
Having thought about this some more, I'd like to propose that we add follow_symlinks arguments to
glob()andrglob(), where:follow_symlinks=Falsetreats symlinks as files (default)follow_symlinks=Truefollows symlinks to directoriesNote that the default behaviour would therefore change: a pattern like
foo/baror*/barwould no longer matchfoo/bariffoois a symlink. This is consistent with how**/barworks today.With that in place, we could implement
glob()iteratively atopwalk(), which would address #89727 and substantially improve performance!Thoughts?
24 remaining items
Merge branch 'main' into pythongh-77609-glob-follow-symlinks
Merge branch 'main' into pythongh-77609-glob-follow-symlinks
Merge branch 'main' into pythongh-77609-glob-follow-symlinks
Merge branch 'main' into pythongh-77609-glob-follow-symlinks
GH-77609: Add follow_symlinks argument to `pathlib.Path.glob()` (GH-1…
barneygale commentedon May 29, 2023
I've added a keyword-only
follow_symlinksargument, which will be available in Python 3.13. See 9700dd1 / #102616Thank you everyone, and especially @zooba, for the help in getting this through. Resolving!
gpshead commentedon Mar 23, 2024
Lets change the new
follow_symlinks=Nonetofollow_symlinks=NotRecursive(or some other explicitly named sentinel value defined in the pathlib module).People reading code that explicitly specifies
follow_symlinks=Noneas would be needed if we ever decide to change the default would not be able to reason about what that does and why=Noneisn't the same behavior as=Falseor=0. Seeing a named constant makes it much more clear to those not intimately familiar with the API.This is good even if we never decide to work towards a default change.
pythonGH-77609: Add recurse_symlinks argument to `pathlib.Path.glob()`
GH-77609: Add recurse_symlinks argument to `pathlib.Path.glob()` (#11…
barneygale commentedon Apr 7, 2024
Re-resolving - the argument is now
recurse_symlinksand accepts a boolean.pythonGH-77609: Add recurse_symlinks argument to `pathlib.Path.glob()` (