Repository navigation
fix(bedrock): honor BEDROCK_MANTLE_API_BASE on bedrock/mantle messages and chat URLs - #39364
Merged
mateo-berri merged 2 commits intoSep 2, 2026
Conversation
Contributor
Contributor
Greptile SummaryThe PR makes Bedrock Mantle messages and chat requests honor
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/common_utils.py | Adds normalized environment endpoint fallback while retaining existing explicit override precedence. |
| tests/test_litellm/llms/bedrock/test_mantle.py | Covers supported environment endpoint forms and verifies explicit endpoint overrides continue to win. |
Reviews (2): Last reviewed commit: "fix(bedrock): drop the OpenAI base suffi..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…E before the mantle messages path
Contributor
Author
|
bugbot run |
Contributor
Author
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 49c69c4. Configure here.
mateo-berri
enabled auto-merge
September 2, 2026 19:03
mateo-berri
deleted the
litellm_fix_bedrock_mantle_messages_env_api_base
branch
September 2, 2026 23:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
bedrock/mantle/*deployments ignoredBEDROCK_MANTLE_API_BASEand always called the publicbedrock-mantle.<region>.api.awshost/v1/messagesroute and/v1/chat/completionson those models were affected, since they share one URL builderapi_baseon every deployment instead of setting the documented env var onceHow it solves it:
build_mantle_messages_urlnow falls back toBEDROCK_MANTLE_API_BASEafterapi_baseandaws_bedrock_runtime_endpointbedrock_mantle/*routes, which need it to end in/v1or/openai/v1for plain chat models, so that suffix is dropped before/anthropic/v1/messagesis appended and one value serves every Mantle routebedrock_mantle/*provider already honored the env var, so the two Mantle spellings now agreeUser Flow
Before: an operator whose Mantle endpoint is a private host sets
BEDROCK_MANTLE_API_BASE=https://bedrock-mantle.us-east-1.api.aws.corp.example, adds abedrock/mantle/anthropic.claude-haiku-4-5deployment with noapi_base, and every call still goes to the public AWS host their network cannot reachBEDROCK_MANTLE_API_BASE=https://bedrock-mantle.us-east-1.api.aws.corp.exampleand start the proxy with abedrock/mantle/anthropic.claude-haiku-4-5deployment namedmantle-claude{"model": "mantle-claude", "max_tokens": 32, "messages": [{"role": "user", "content": "Say pong"}]}bedrock-mantle.us-east-1.api.aws, and their private endpoint's access log shows no requestapi_baseon the deployment do POST https://litellm-domain/v1/messages and POST https://litellm-domain/v1/chat/completions reach their private hostAfter: the env var alone is enough, and both routes come back 200 through the private host
BEDROCK_MANTLE_API_BASE=https://bedrock-mantle.us-east-1.api.aws.corp.exampleand start the proxy with abedrock/mantle/anthropic.claude-haiku-4-5deployment namedmantle-claude{"model": "mantle-claude", "max_tokens": 32, "messages": [{"role": "user", "content": "Say pong"}]}"type": "message"and"text": "pong", and their private endpoint's access log showsPOST /anthropic/v1/messages"content": "Pong", again through their private hostapi_basekeeps using that value, so POST https://litellm-domain/v1/messages against it behaves exactly as beforeRelevant issues
Linear ticket
Resolves LIT-6641
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup, identical on both sides: a local TLS reverse proxy plays the customer-shaped private host
https://bedrock-mantle.us-east-1.api.aws.localtest.me:9443(resolving to 127.0.0.1) and forwards to the real public Mantle host, appending one line per hit to its access log. Real calls, real spend,AWS_BEARER_TOKEN_BEDROCKfrom the CI account. Both proxies run with no DB, 2 uvicorn workers each (--num_workers 2), on a random port with this env and config. The Before proxy runs the merge base on port 44776 and the After proxy runs the PR tip on port 42383, which is why the curls below name those ports instead of 4000Each case shows the curl, the response, and the new lines in the private host's access log, which is the operator's own evidence of whether the request reached their endpoint. Cases 1, 2 and 4 are the env-var-only deployment on the messages, chat completions and responses routes; case 3 is the control with
api_baseon the deployment, which must not change. Cases 5 and 6 rerun both proxies with the same env exceptBEDROCK_MANTLE_API_BASE=https://bedrock-mantle.us-east-1.api.aws.localtest.me:9443/v1, the shape plain chat models onbedrock_mantle/*need: case 5 is the messages route again and case 6 is the existing gpt-oss chat route that must keep working with that valueBefore (ffc0a8e)
Case 1: Anthropic messages with
BEDROCK_MANTLE_API_BASEonly (mantle-claude)Case 2: Chat completions with
BEDROCK_MANTLE_API_BASEonly (mantle-claude)Case 3: Anthropic messages with
api_baseon the deployment, control (mantle-claude-apibase)Case 4: Responses with
BEDROCK_MANTLE_API_BASEonly, chat bridge (mantle-claude)Case 5: Anthropic messages with
BEDROCK_MANTLE_API_BASEending in/v1(mantle-claude)Case 6: Plain chat completions on gpt-oss with
BEDROCK_MANTLE_API_BASEending in/v1, control (mantle-oss)After (49c69c4)
Case 1: Anthropic messages with
BEDROCK_MANTLE_API_BASEonly (mantle-claude)Case 2: Chat completions with
BEDROCK_MANTLE_API_BASEonly (mantle-claude)Case 3: Anthropic messages with
api_baseon the deployment, control (mantle-claude-apibase)Case 4: Responses with
BEDROCK_MANTLE_API_BASEonly, chat bridge (mantle-claude)Case 5: Anthropic messages with
BEDROCK_MANTLE_API_BASEending in/v1(mantle-claude)Case 6: Plain chat completions on gpt-oss with
BEDROCK_MANTLE_API_BASEending in/v1, control (mantle-oss)Observations from the run, none caused by this PR:
Before, cases 1, 2, 4 and 5 still return 200 because this machine can reach the public host; the operator's private endpoint sees nothing
After, cases 1, 2, 4 and 5 hit the private host with
POST /anthropic/v1/messagesand return the same bodiesCase 3 hits the private host on both sides, so
api_baseon the deployment is unchangedCase 6 hits the private host with
POST /v1/chat/completionson both sides, so the/v1env value keeps serving plain chatNot verified live: SigV4 signing against the private host (the QA account authenticates with a bearer token) and the
aws_bedrock_runtime_endpointoverride, both covered by the unit tests onlyType
🐛 Bug Fix
Caveats
Low: only a trailing
/v1or/openai/v1is dropped fromBEDROCK_MANTLE_API_BASEbefore the messages path is appended. A value that ends in some other path keeps it, so a private gateway that mounts Mantle under a different prefix needsapi_baseon thebedrock/mantle/*deployment, which keeps winning over the env varFinal Attestation