Skip to content

fix(python-bridge): harden sync and async execution boundaries - #39332

Merged
yujonglee-berri merged 1 commit into
litellm_python_bridge_modulesfrom
litellm_python_bridge_runtime_boundaries
Sep 2, 2026
Merged

yujonglee-berri merged 1 commit into
litellm_python_bridge_modulesfrom
litellm_python_bridge_runtime_boundaries

Conversation

@yujonglee-berri

@yujonglee-berri yujonglee-berri commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Layer 6 of 17 in the reorganized Rust bridge stack

Scope

Adds Pythonized conversion, panic mapping, and shared sync/async execution helpers for GIL release, nested-runtime rejection, signal polling, cancellation, and safe result conversion

The helpers live in crates/python-bridge/src/execution.rs, outside routes/, with inline tests. Routes retain request assembly and tracing. Packaged bridge tests cover sync and async success, traced responses, upstream errors, concurrency, and SIGINT cancellation

Review boundary

Rebased onto the current litellm_python_bridge_modules base, preserving its shared request assembly and tracing support. Later behavior remains outside this PR

Supersedes #39263, whose record GitHub automatically marked merged when its former stacked base temporarily became a descendant during the guarded branch rewrite. The original discussion remains there; this PR is the live review head

Validation

Formatting, focused strict Clippy for core/interop/bridge, and all 264 Rust workspace tests pass. The release wheel builds and passes the packaged route tests, including traced sync/async responses and SIGINT cancellation. Ruff passes for the packaged test script

Full gateway Clippy with the server feature reports three existing collapsible_if warnings in unchanged realtime code

@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from 3a23a95 to 2b0f7e6 Compare September 2, 2026 15:01
@yujonglee-berri
yujonglee-berri marked this pull request as ready for review September 2, 2026 15:26
@yujonglee-berri
yujonglee-berri requested a review from a team September 2, 2026 15:26
@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from 2b0f7e6 to a5b9fba Compare September 2, 2026 15:26
@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR centralizes Python bridge route execution behind shared sync and async runtime helpers, adding GIL release, signal handling, nested-runtime rejection, panic conversion, and Python result conversion

  • Migrates OCR, transcription, messages, and chat-completion routes to the shared runtime
  • Adds packaged-wheel coverage for successful calls, mapped errors, concurrency, cancellation, and signal handling
  • Updates PyO3 and adds the Rust dependencies required by runtime and WebSocket tests

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified

The shared runtime preserves route conversion and error contracts while adding bounded signal handling, panic containment, nested-runtime protection, and packaged-wheel regression coverage

Important Files Changed

Filename Overview
litellm-rust/crates/python-bridge/src/routes/runtime.rs Adds a well-covered shared boundary for synchronous and asynchronous route execution, panic mapping, signal polling, and result conversion
litellm-rust/crates/python-interop/src/marshal.rs Adds reusable panic-safe Pythonization while preserving the existing pythonize-based response shape and conversion errors
litellm-rust/crates/python-bridge/src/lib.rs Updates module initialization for free-threaded Python support and preserves WebSocket header conversion and public registration
tests/test_litellm/rust_bridge/native_route_wheel_test.py Adds bounded loopback-only packaged-wheel tests covering route behavior, concurrency, SIGINT propagation, and upstream cancellation
.github/workflows/test-rust.yml Runs the new native route test against the single release wheel produced by the existing workflow

Reviews (1): Last reviewed commit: 2b0f7e6 | Re-trigger Greptile

Comment thread litellm-rust/crates/python-bridge/src/execution.rs
@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from a5b9fba to d69da84 Compare September 2, 2026 19:16
@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from d69da84 to 80b7371 Compare September 2, 2026 22:03
@yujonglee-berri yujonglee-berri changed the title fix(python-bridge): harden sync and async route boundaries fix(python-bridge): harden sync and async execution boundaries Sep 2, 2026
@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from 80b7371 to 469f6c1 Compare September 2, 2026 22:05
@yujonglee-berri
yujonglee-berri force-pushed the litellm_python_bridge_runtime_boundaries branch from 469f6c1 to b10282f Compare September 2, 2026 22:21
@yujonglee-berri
yujonglee-berri merged commit 62e318d into litellm_internal_staging Sep 2, 2026
95 of 132 checks passed
@yujonglee-berri
yujonglee-berri deleted the litellm_python_bridge_runtime_boundaries branch September 2, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants