Fix undefined kwargs in InFlightRequestsMiddleware

[jquinter]

Summary

• kwargs was referenced on lines 67 and 71 of in_flight_requests_middleware.py but never initialized, causing F821 Undefined name lint errors
• Initialize kwargs as an empty dict before the conditional block

Note: The router.py:1721 PLR0915 Too many statements error is a pre-existing style warning and is not addressed here to keep this PR scoped.

Test plan

• Verify make lint-ruff passes for this file

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Building	Preview, Comment	Mar 1, 2026 8:23pm

[greptile-apps]

Greptile Summary

Fixes a NameError bug in InFlightRequestsMiddleware._get_gauge() where kwargs was referenced but never initialized. This would cause the Prometheus gauge to fail to initialize on every first request, falling into the except Exception handler and leaving _gauge as None permanently — meaning no Prometheus metrics would ever be emitted for in-flight requests.

• Initializes kwargs as an empty dict[str, Any] before the conditional block that optionally sets multiprocess_mode
• Removes the # type: ignore[arg-type] comment that is no longer needed since kwargs is now properly typed

Confidence Score: 5/5

• This PR is safe to merge — it's a minimal, clearly correct bug fix with no behavioral side effects.
• The change is a one-line initialization of a variable that was previously undefined, fixing a clear runtime NameError. The fix is minimal, scoped, and correct. The code path before this fix would always throw an exception and silently disable Prometheus metrics.
• No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/middleware/in_flight_requests_middleware.py	Initializes kwargs as an empty dict before the conditional block, fixing a NameError that would occur at runtime when _get_gauge() is called. Also removes the now-unnecessary # type: ignore[arg-type] comment.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["_get_gauge() called on first HTTP request"] --> B{"Already attempted init?"}
    B -- Yes --> C["Return cached _gauge"]
    B -- No --> D["Mark init attempted"]
    D --> E["Import prometheus_client.Gauge"]
    E --> F["Initialize kwargs = {}"]
    F --> G{"PROMETHEUS_MULTIPROC_DIR set?"}
    G -- Yes --> H["kwargs['multiprocess_mode'] = 'livesum'"]
    H --> I["Create Gauge(**kwargs)"]
    G -- No --> I
    I --> J["Return _gauge"]
    E -- ImportError --> K["_gauge = None"]
    K --> J

Loading

_{Last reviewed commit: 22140b0}