Skip to content

Decouple the C compilers from distutils build/filesystem helpers#417

Open
jaraco wants to merge 6 commits into
mainfrom
feature/5267-vendor-build-helpers
Open

Decouple the C compilers from distutils build/filesystem helpers#417
jaraco wants to merge 6 commits into
mainfrom
feature/5267-vendor-build-helpers

Conversation

@jaraco

@jaraco jaraco commented Jul 20, 2026

Copy link
Copy Markdown
Member

Removes the C compilers' dependence on distutils' build/filesystem helpers, per the analysis in pypa/setuptools#5267 (Phase 1 of pypa/setuptools#5264).

What changed

Vendored into the compilers namespace package (stdlib-only, siblings of the logging.py from #5266):

  • compilers/_modified.pynewer/newer_group mtime helpers.
  • compilers/_util.pysplit_quoted, vendored verbatim (a switch to shlex.split is not behavior-equivalent and is left for later).

Subprocess execution reworked into a modern call + deprecated spawn:

  • New Compiler.call(cmd, *, env=None, **kwargs) — the modern entry point: logs the command, injects the macOS deployment target, and runs subprocess.check_call, letting native subprocess exceptions propagate.
  • Compiler.spawn is now a deprecation shim — it warns, delegates to call, and translates subprocess/OSError failures to DistutilsExecError (imported late, to keep the package off distutils.errors at module scope).
  • Internal call sites across unix/cygwin/zos/msvc moved to call() and now catch subprocess.CalledProcessError/OSError. MSVC's PATH injection moved from a spawn override to a call override.
  • The macOS env injection moved out of distutils.spawn into compilers/platform/macos.py (_inject_ver) — the compiler path is the only place it applies (verified: no non-compiler spawn call site invokes a compiler/linker).
  • The previously-vendored compilers/spawn.py is removed.

distutils.spawn.spawn reduced to a thin wrapper — it now only logs, forwards cmd/**kwargs to subprocess.check_call, and maps OSError/CalledProcessError to DistutilsExecError; no shutil.which (subprocess searches PATH itself) and no macOS injection. Command.spawn no longer forwards the now-defunct search_path.

Modernized in place — the public Compiler methods keep their signatures for backward compatibility, but their bodies now use the stdlib:

  • mkpathos.makedirs(name, mode, exist_ok=True) (empty-name guarded).
  • move_fileshutil.move.
  • execute → trivial log-and-call, no longer used internally.

Eliminated — the cygwin .def-file write is inlined (pathlib.Path.write_text), removing distutils.file_util.write_file and the execute indirection. Also removed the MSVC _fallback_spawn monkeypatch shim (numpy.distutils <1.19, per #15) and its now-dead pytest filter.

Net: compilers/C/* no longer import distutils.spawn, dir_util, file_util, _modified, or util.execute/split_quoted.

Deliberately out of scope

Two transitive couplings remain, by design, as they belong to sibling sub-issues:

  • the vendored _modified / deprecated spawn still reference Distutils*Error (native compilers.errors is #5270 / S6);
  • _inject_ver still calls distutils.util for the target-version helpers (platform/util decoupling is #5268 / S4).

Verification

  • Full distutils suite: 246 passed, 22 skipped. Compiler + build_ext + spawn subset green.
  • Verified call() raises native exceptions without warning; spawn() warns and translates to DistutilsExecError; distutils.spawn.spawn still translates.
  • ruff check --select F,I clean on all changed files.

🤖 Generated with Claude Code

jaraco and others added 5 commits July 20, 2026 08:36
Add stdlib-only copies of the generic build helpers the C compilers
rely on (spawn, newer/newer_group, split_quoted) as siblings of
compilers.logging, so the package can provide them without importing
distutils.spawn/_modified/util. Errors still route through
distutils.errors pending pypa/setuptools#5270; the macOS deployment
env injection still defers to distutils.util pending
pypa/setuptools#5268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Point base/unix/cygwin at compilers.spawn/_modified/_util, and
reimplement Compiler.mkpath, move_file, and execute directly on the
stdlib (os.makedirs, shutil.move). Inline the cygwin .def-file write
(dropping distutils.file_util.write_file and the execute indirection).
The public methods are retained for backward compatibility.

Ref pypa/setuptools#5267, pypa/setuptools#5264.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The _fallback_spawn guard tolerated third-party spawn overrides that
predate the 'env' keyword (numpy.distutils before 1.19, per
#15). numpy.distutils has accepted env since then and does
not run on Python 3.12+, so the shim now protects essentially nothing.
Remove it, narrow Compiler.spawn to spawn(cmd, *, env=None), and trim
the vendored primitive to match.

Ref pypa/setuptools#5267.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the macOS deployment-target injection (_inject_macos_ver) and the
shutil.which PATH resolution. subprocess.check_call searches PATH itself,
and the macOS injection only ever mattered for compiler/linker
invocations, which now carry it in the compilers package. spawn now just
logs the command, forwards cmd and **kwargs to check_call, and maps
OSError/CalledProcessError to DistutilsExecError. Command.spawn no longer
forwards the now-defunct search_path.

Ref pypa/setuptools#5267.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce Compiler.call as the modern subprocess entry point: it logs,
injects the macOS deployment target (now compilers.platform.macos._inject_ver,
moved out of distutils.spawn), and calls check_call, letting native
subprocess exceptions propagate. Compiler.spawn becomes a deprecation
shim that translates them to DistutilsExecError (imported late).

Internal call sites across unix/cygwin/zos/msvc move to call() and catch
subprocess.CalledProcessError/OSError; MSVC's PATH injection moves from a
spawn override to a call override. Eliminates the vendored compilers.spawn
module.

Ref pypa/setuptools#5267.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread distutils/compilers/C/base.py Outdated
from .._modified import newer_group
from .._util import split_quoted
from ..logging import get_logger
from ..platform.macos import _inject_ver

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use from ..platform import macos, so that the macos name is reflected in the call below.

Comment thread distutils/compilers/C/base.py Outdated
Comment on lines +1188 to +1199
from ...errors import DistutilsExecError

try:
self.call(cmd, env=env, **kwargs)
except OSError as exc:
raise DistutilsExecError(
f"command {cmd[0]!r} failed: {exc.args[-1]}"
) from exc
except subprocess.CalledProcessError as err:
raise DistutilsExecError(
f"command {cmd[0]!r} failed with exit code {err.returncode}"
) from err

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's unify this behavior with distutils.spawn.spawn, either by extracting a context manager to do the exception translation or by having "inner" spawn that's called after the logging is done.

Import the macos module (from ..platform import macos) so the injection
reads as macos._inject_ver at the call site. Extract the subprocess ->
DistutilsExecError translation into a _translate_errors context manager
in distutils.spawn, shared by distutils.spawn.spawn and the deprecated
Compiler.spawn (imported late).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant