fix(mcp): cap tools preview and test-connection at the listing timeout and name the unreachable upstream

[mateo-berri]

TLDR

Problem this solves:

• MCP tools preview and test-connection hang about 62s when the upstream is unreachable
• Load balancers give up first, so the UI shows "Received non-JSON response (504: )"
• The generic error never says which upstream URL the proxy could not reach

How it solves it:

• Caps OAuth discovery, connect, handshake, and the operation at MCP_TOOL_LISTING_TIMEOUT (30s)
• Same cap the by-server_id tools listing already uses, so a proxy JSON error beats the LB timeout
• The timeout error names the upstream URL and the budget and points at proxy egress

User Flow

Before: an admin registering an OAuth MCP server behind a load balancer sees "Received non-JSON response (504: )" in the tools preview and never learns that the proxy cannot reach the upstream

1. They open https://litellm-domain/ui/?page=mcp-servers, click their github_mcp server (URL https://api.githubcopilot.com/mcp/), open Edit, pick OAuth2 auth, and click Authorize; the upstream consent page completes and the form comes back holding the new access token
2. The form sends POST https://litellm-domain/mcp-rest/test/tools/list with their UI session key and Authorization: Bearer <upstream oauth token> and the server config in the body
3. The proxy pod cannot get an MCP reply from https://api.githubcopilot.com/mcp/ (blocked egress), so it sits on the request for about 62s; the load balancer in front times out at 60s and answers 504 with an empty body
4. The tools panel shows "Failed to fetch tools: Received non-JSON response (504: )", the proxy log has nothing tied to their click, and they end up chasing unrelated 401 lines

After: the same click comes back inside 30s with a JSON error naming the unreachable upstream URL

1. They open https://litellm-domain/ui/?page=mcp-servers, click their github_mcp server (URL https://api.githubcopilot.com/mcp/), open Edit, pick OAuth2 auth, and click Authorize; the upstream consent page completes and the form comes back holding the new access token
2. The form sends POST https://litellm-domain/mcp-rest/test/tools/list with their UI session key and Authorization: Bearer <upstream oauth token> and the server config in the body
3. After 30s the proxy answers 200 {"status":"error","error":true,"message":"Failed to connect to MCP server: no response from https://api.githubcopilot.com/mcp/ within 30s. Check that the LiteLLM proxy can reach this URL from its network (DNS, egress rules, firewalls) and that the server answers MCP requests."}, well before the load balancer's 60s
4. The tools panel shows that message, they open egress from the proxy pods to api.githubcopilot.com, click Authorize again, and the same POST returns 200 with the GitHub tool catalog in about 3s

The Test Connection button on the same form also fires POST https://litellm-domain/mcp-rest/test/tools/list, so it gets the same 30s cap and message

Relevant issues

Linear ticket

Resolves LIT-6477

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs boot the proxy with --num_workers 2 from the exact commit named in the heading, each with its own Postgres database and master key, on a random free port (before 35306, after 29349). The QA box's shared Postgres sat at its client cap, so both legs export DATABASE_URL with connection_limit=2 and the after leg ran against its own postgres:16 container; the four requests only touch the database for key auth. The requests below are byte-for-byte what the Admin UI's MCP server edit form sends after Authorize: x-litellm-api-key carries the UI session key and Authorization: Bearer carries the upstream OAuth token. The browser was not available on the QA box for this run, so the click-through is listed at the end of this section for a hand run

Config:

model_list:
  - model_name: gpt-5.2
    litellm_params:
      model: openai/gpt-5.2
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  store_model_in_db: true
  enable_mcp_registry: true
  use_x_forwarded_for: true

Boot (per leg, from that leg's checkout): python litellm/proxy/proxy_cli.py --config config.yaml --port <port> --num_workers 2 --detailed_debug

blackhole.json is the form's server config pointed at an address that drops SYNs, standing in for a proxy pod with no egress to the upstream:

{
  "server_id": "",
  "server_name": "github_mcp",
  "url": "http://10.255.255.1/mcp/",
  "transport": "http",
  "auth_type": "oauth2",
  "oauth2_flow": "authorization_code",
  "authorization_url": "https://github.com/login/oauth/authorize",
  "token_url": "https://github.com/login/oauth/access_token"
}

github.json is the same config with "url": "https://api.githubcopilot.com/mcp/", the customer's real upstream, and its bearer is a real GitHub token ($(gh auth token))

Before (7114302)

Tools preview, upstream unreachable from the proxy

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:35306/mcp-rest/test/tools/list -H 'x-litellm-api-key: sk-lit6486-before' -H 'Authorization: Bearer dummy-upstream-oauth-token' -H 'Content-Type: application/json' --data-binary @blackhole.json
2. Output, 62 seconds later, past the customer's 60s load balancer; staging's fix(mcp): follow tools/list pagination from upstream servers #39172 now names its listing deadline but still not the URL that never answered:

{"status":"error","error":true,"message":"Timed out listing tools after 60.0 seconds. The MCP server may be responding slowly or paginating excessively."}
HTTPSTATUS=200 TIME=62.445961s

Tools preview, real GitHub MCP upstream

1. curl -m 150 -s -o github_before.json -w 'HTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:35306/mcp-rest/test/tools/list -H 'x-litellm-api-key: sk-lit6486-before' -H "Authorization: Bearer $(gh auth token)" -H 'Content-Type: application/json' --data-binary @github.json
2. Output: HTTPSTATUS=200 TIME=4.005674s, and github_before.json holds 44 tools starting with add_comment_to_pending_review, add_issue_comment, add_reply_to_pull_request_comment

/mcp-rest/test/connection, upstream unreachable from the proxy

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:35306/mcp-rest/test/connection -H 'x-litellm-api-key: sk-lit6486-before' -H 'Authorization: Bearer dummy-upstream-oauth-token' -H 'Content-Type: application/json' --data-binary @blackhole.json
2. Output:

{"status":"error","error":true,"message":"Failed to connect to MCP server. Check proxy logs for details."}
HTTPSTATUS=200 TIME=4.454631s

/mcp-rest/test/connection, real GitHub MCP upstream

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:35306/mcp-rest/test/connection -H 'x-litellm-api-key: sk-lit6486-before' -H "Authorization: Bearer $(gh auth token)" -H 'Content-Type: application/json' --data-binary @github.json
2. Output:

{"status":"error","error":true,"message":"Failed to connect to MCP server. Check proxy logs for details."}
HTTPSTATUS=200 TIME=1.169445s

After (cdf8383)

Tools preview, upstream unreachable from the proxy

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:29349/mcp-rest/test/tools/list -H 'x-litellm-api-key: sk-lit6486-after' -H 'Authorization: Bearer dummy-upstream-oauth-token' -H 'Content-Type: application/json' --data-binary @blackhole.json
2. Output, 30 seconds later, inside the load balancer's budget, naming the URL the proxy could not reach:

{"status":"error","error":true,"message":"Failed to connect to MCP server: no response from http://10.255.255.1/mcp/ within 30s. Check that the LiteLLM proxy can reach this URL from its network (DNS, egress rules, firewalls) and that the server answers MCP requests."}
HTTPSTATUS=200 TIME=30.724341s

Tools preview, real GitHub MCP upstream

1. curl -m 150 -s -o github_after.json -w 'HTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:29349/mcp-rest/test/tools/list -H 'x-litellm-api-key: sk-lit6486-after' -H "Authorization: Bearer $(gh auth token)" -H 'Content-Type: application/json' --data-binary @github.json
2. Output: HTTPSTATUS=200 TIME=2.143061s, and github_after.json holds the same 44 tools starting with add_comment_to_pending_review, add_issue_comment, add_reply_to_pull_request_comment

/mcp-rest/test/connection, upstream unreachable from the proxy

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:29349/mcp-rest/test/connection -H 'x-litellm-api-key: sk-lit6486-after' -H 'Authorization: Bearer dummy-upstream-oauth-token' -H 'Content-Type: application/json' --data-binary @blackhole.json
2. Output:

{"status":"error","error":true,"message":"Failed to connect to MCP server. Check proxy logs for details."}
HTTPSTATUS=200 TIME=0.088182s

/mcp-rest/test/connection, real GitHub MCP upstream

1. curl -m 150 -s -w '\nHTTPSTATUS=%{http_code} TIME=%{time_total}s\n' -X POST http://localhost:29349/mcp-rest/test/connection -H 'x-litellm-api-key: sk-lit6486-after' -H "Authorization: Bearer $(gh auth token)" -H 'Content-Type: application/json' --data-binary @github.json
2. Output:

{"status":"error","error":true,"message":"Failed to connect to MCP server. Check proxy logs for details."}
HTTPSTATUS=200 TIME=0.945831s

Observed alongside the fix:

• /mcp-rest/test/connection rejects OAuth2 staged bearers instantly on both legs; PR leaves it alone
• The dashboard's Test Connection button calls /mcp-rest/test/tools/list, not /test/connection
• Before leg held the request 62s single-hop, now surfacing fix(mcp): follow tools/list pagination from upstream servers #39172's listing-deadline text; a 60s load balancer answers first

Admin UI click-through, for a hand run with a proxy at the tip and a pod that cannot reach the upstream (for example a firewall rule dropping traffic to api.githubcopilot.com from the proxy host):

1. Open http://localhost:4000/ui/?page=mcp-servers and click Add MCP Server: name github_mcp, URL https://api.githubcopilot.com/mcp/, transport HTTP, auth OAuth2, authorization URL https://github.com/login/oauth/authorize, token URL https://github.com/login/oauth/access_token, plus a GitHub OAuth app client id and secret whose callback is <PROXY_BASE_URL>/callback
2. Click Authorize, approve on GitHub, and land back on the form
3. Watch the tools panel: at the tip it reads Failed to connect to MCP server: no response from https://api.githubcopilot.com/mcp/ within 30s. Check that the LiteLLM proxy can reach this URL from its network (DNS, egress rules, firewalls) and that the server answers MCP requests. after about 30s, where the merge base reads Received non-JSON response (504: ) after 60s behind a load balancer, or Timed out listing tools after 60.0 seconds. The MCP server may be responding slowly or paginating excessively. after about 62s without one
4. Remove the firewall rule, click Authorize again, and the panel lists the GitHub tools

Type

🐛 Bug Fix

Caveats (if any)

Medium

• Upstreams that list tools in 30 to 60s now fail the preview by design
  • Only the two preview endpoints change
  • The runtime tools listing already enforced the same 30s cap, so those tools never loaded anyway
  • LITELLM_MCP_TOOL_LISTING_TIMEOUT raises both caps together
• QA replayed the Admin UI's exact request via curl, not browser clicks
  • Headers and body match the edit form byte for byte
  • The click-through above covers the hand run

Low

• With the timeout set above 60s, the message names the configured budget
  • The MCP SDK's own 60s read timeout fires first in that config
• Tests patch the private _create_mcp_client instead of injecting it
  • Matches the ten existing patches in that test class
• Cancellation driven live on HTTP transport only
  • stdio and SSE share the same anyio cancel scope but were not exercised
• fix(mcp): follow tools/list pagination from upstream servers #39172's per-server listing deadline now sits inside this PR's 30s outer cap
  • _list_tools_operation's move_on_after waits at least 60s, so its "Timed out listing tools" message is unreachable through the endpoint and a per-server timeout above 30s does not extend the preview
  • fix(mcp): follow tools/list pagination from upstream servers #39172 only needed the preview bounded; its tests bypass _execute_with_mcp_client and still pass
• A timed-out preview logs the CancelledError and TimeoutError chain at ERROR through the pre-existing Error in MCP operation handler; the response body stays the one-line message

live-pr-risk CHECKED at cdf8383: PASS. Checked on the merged tree (staging 7114302 folded in). Dependents close over two in-file routes, /mcp-rest/test/connection and /mcp-rest/test/tools/list, both driven live base 7114302 vs head cdf8383 with 2 workers each against a real GitHub MCP upstream and a blocked-egress address; the one new edge from the merge, #39172's _list_tools_operation deadline now running inside this PR's 30s fail_after scope, is what request 1 of the after leg exercises; the dashboard reads the unchanged {status, message} shape; no registry, subclass, kwargs, schema, or DB dependents; tests/llm_translation and tests/local_testing have zero references to the changed surface. Not verified live: stdio-transport cancellation and the above-60s budget mislabel

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Low Risk
Scoped to admin MCP test/preview helpers and error messaging; behavior change is faster failure for slow/unreachable upstreams, aligned with existing runtime tool listing limits.

Overview
Admin MCP preview/test paths (_execute_with_mcp_client, used by /mcp-rest/test/tools/list and related flows) now enforce a combined deadline on client creation and the preview operation via anyio.fail_after, defaulting to MCP_TOOL_LISTING_TIMEOUT (30s). That replaces ~60s hangs that often lose to load balancers and return opaque 504 responses.

On timeout, _connection_error_message returns a dedicated message that includes the configured upstream URL, the timeout budget, and hints about proxy egress/DNS/firewalls; other error branches keep their existing wording but use the updated helper signature.

Tests cover hanging operations and client creation, the default timeout parameter, and timeout-specific error text.

^{Reviewed by Cursor Bugbot for commit d3268e4. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

The PR bounds temporary MCP client creation and preview operations with the existing tool-listing timeout, returning a more actionable error when an upstream does not respond.

• Applies one combined timeout to client creation, connection setup, and the preview operation.
• Includes the configured upstream URL and timeout budget in timeout responses.
• Adds coverage for hanging client creation, hanging operations, default timeout selection, and error formatting.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/rest_endpoints.py	Adds a bounded timeout around temporary MCP client setup and execution and improves timeout diagnostics without leaving a blocking issue.
tests/test_litellm/proxy/_experimental/mcp_server/test_rest_endpoints.py	Adds focused timeout regression tests and now conforms to the repository’s 120-character line-length requirement.

_{Reviews (3): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d3268e4. Configure here.}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_mcp_oauth_tool_fetch_auth (cdf8383) with litellm_internal_staging (86ca146)}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cdf8383. Configure here.}

[tin-berri]

LGTM