Repository navigation
fix(responses): keep namespace tools intact when a guardrail returns them unchanged - #39366
Conversation
…them unchanged Any pre_call guardrail on /v1/responses flattened Codex namespace tools into ns__member functions and wrote the flattened list back to the request, so the model called mcp__server__tool with no namespace and Codex rejected the call as unsupported. The handler now keeps the client's original tools, hands the guardrail a deep copy of the flattened ones, and rebuilds data["tools"] by matching the guardrail's output to the originals by type and name. Unchanged tools go back as the original objects, a dropped or edited namespace member changes only that member, and tools the guardrail injects are still appended. Fixes #39183
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThe PR preserves original Responses API tool shapes while applying guardrail changes selectively.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains, and the previously reported redundant source comment has been removed.
|
| Filename | Overview |
|---|---|
| litellm/llms/openai/responses/guardrail_translation/handler.py | Deep-copies flattened tools for guardrail inspection and merges returned changes back into the original Responses tool shapes. |
| litellm/llms/openai/responses/guardrail_translation/tool_merge.py | Implements identity-aware reconciliation of original, flattened, modified, removed, and injected tools. |
| litellm/responses/litellm_completion_transformation/transformation.py | Refactors tool conversion into per-tool chat forms and removes the previously flagged redundant source comment. |
| tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_handler.py | Extends handler coverage for passthrough, mutation, injection, and namespace-tool behavior. |
| tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_tool_merge.py | Exercises reconciliation behavior across unchanged, edited, removed, duplicate, Pydantic, and unsupported tool shapes. |
| tests/test_litellm/responses/litellm_completion_transformation/test_litellm_completion_responses.py | Verifies the refactored conversion retains expected Responses-to-chat behavior. |
Reviews (6): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
Sort the chat-tool keys once and number duplicates with groupby instead of rescanning every preceding key per position, so the guardrail merge stays O(n log n) on client-supplied tool lists. Drop the comment that restated the unsupported-tool warning in the Responses-to-chat transformation.
|
bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Namespace drop removes custom members
- Reordered
_merged_originalso the namespace rebuild path runs before the empty-guardrailed shortcut, letting_rebuilt_namespacepreserve non-function members when every function is pruned, and added a regression test.
- Reordered
Or push these changes by commenting:
@cursor push 23f3b9ab10
Preview (23f3b9ab10)
diff --git a/litellm/llms/openai/responses/guardrail_translation/tool_merge.py b/litellm/llms/openai/responses/guardrail_translation/tool_merge.py
--- a/litellm/llms/openai/responses/guardrail_translation/tool_merge.py
+++ b/litellm/llms/openai/responses/guardrail_translation/tool_merge.py
@@ -147,11 +147,11 @@
guardrailed_group: Final = tuple(guardrailed_by_key[key] for key in group_keys if key in guardrailed_by_key)
if guardrailed_group == tuple(flattened_group):
return (original,)
- if not guardrailed_group:
- return ()
members: Final = _namespace_members(original) if original.get("type") == "namespace" else ()
if members and sum(map(_is_function, members)) == len(flattened_group):
return _rebuilt_namespace(original, members, flattened_group, group_keys, guardrailed_by_key)
+ if not guardrailed_group:
+ return ()
return tuple(
LiteLLMCompletionResponsesConfig.transform_chat_completion_tool_params_to_responses_api_tools(guardrailed_group)
)
diff --git a/tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_tool_merge.py b/tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_tool_merge.py
--- a/tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_tool_merge.py
+++ b/tests/test_litellm/llms/openai/responses/test_openai_responses_guardrail_tool_merge.py
@@ -132,6 +132,20 @@
assert merged[0]["tools"][1] == custom_member
+def test_namespace_keeps_non_function_members_when_every_function_is_dropped():
+ custom_member = {"type": "custom", "name": "grep", "description": "Grep", "format": {"type": "text"}}
+ original = [
+ {"type": "namespace", "name": "ns", "description": "NS", "tools": [_function("read", "Read"), custom_member]}
+ ]
+ groups = _groups(original)
+
+ merged = merge_guardrailed_tools(original, groups, [])
+
+ assert len(merged) == 1
+ assert [member["name"] for member in merged[0]["tools"]] == ["grep"]
+ assert merged[0]["tools"][0] == custom_member
+
+
def test_member_extras_edited_by_the_guardrail_land_on_that_member():
original = [{"type": "namespace", "name": "ns", "tools": [_function("read")]}]
groups = _groups(original)You can send follow-ups to the cloud agent here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 7cbc743. Configure here.
…itellm_responses_guardrail_namespace_tools # Conflicts: # basedpyright-code-budget.json # type-discipline-budget.json
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 1636fdd. Configure here.
…itellm_responses_guardrail_namespace_tools # Conflicts: # type-discipline-budget.json
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3f9e44f. Configure here.
…itellm_responses_guardrail_namespace_tools # Conflicts: # type-discipline-budget.json
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2842e90. Configure here.
TLDR
Problem this solves:
/v1/responsesflattens Codexnamespace(MCP) tools intons__memberfunctionsmcp__server__toolwith no namespace, which Codex rejects as unsupportedcustomtools got rewritten and computer_use/image_generation/shell tools got dropped the same wayHow it solves it:
User Flow
Before: a developer running the Codex CLI through the gateway with a guardrail on gets every MCP tool call refused as unsupported
mode: pre_call,default_on: true) to the proxy config and registers an MCP server (deepwiki) on the proxybase_url = "http://litellm-domain/v1",wire_api = "responses") with[mcp_servers.deepwiki]athttp://litellm-domain/mcp/deepwikinamespacetool namedmcp__deepwikiwhose members aredeepwiki_read_wiki_structure,deepwiki_read_wiki_contents, anddeepwiki_ask_questionfunction_callnamedmcp__deepwiki__deepwiki_read_wiki_structurewith nonamespace, which matches nothing Codex registeredunsupported call: mcp__deepwiki__deepwiki_read_wiki_structurex-headroom-bypass: trueon the provider gets the same refusal while Headroom's request counter stays flatAfter: the same prompt runs the MCP tool and answers
mode: pre_call,default_on: true) to the proxy config and registers an MCP server (deepwiki) on the proxybase_url = "http://litellm-domain/v1",wire_api = "responses") with[mcp_servers.deepwiki]athttp://litellm-domain/mcp/deepwikinamespacetool namedmcp__deepwikiwhose members aredeepwiki_read_wiki_structure,deepwiki_read_wiki_contents, anddeepwiki_ask_questionfunction_callnameddeepwiki_read_wiki_structureinside namespacemcp__deepwiki, exactly what Codex registered/statsstill counts the request, so the guardrail ran; when it rewrites or drops one member tool, only that member changes on the way to the modelRelevant issues
Fixes #39183
Supersedes #39193, which reached the same diagnosis; this PR also carries a guardrail's edits to individual tools back onto the client's original shapes and keeps
customand other non-function tools intactLinear ticket
Resolves LIT-6691
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Real Codex CLI 0.145.0 TUI (driven under tmux) -> LiteLLM proxy
/v1/responseswith a Headroompre_callguardrail (default_on: true, self-hostedheadroom proxy0.37.0) and the public DeepWiki MCP server registered on the proxy -> OpenAIgpt-5.3-codex. Each leg runs from its own fresh worktree and venv with two uvicorn workers, no DB. The MCP server, the trigger, and the model are stand-ins for the reporter's Confluence server and per-request header; the symptom is about thenamespacetool shape Codex sends, not the server.proxy_config.yaml(the Before and After legs differ only in the Headroom port and master key):Codex
config.toml(CODEX_HOME=$L/codex_on;codex_offaddshttp_headers = { "x-headroom-bypass" = "true" }under[model_providers.litellm]):Before (merge base 55e9e4c)
Codex screen (run A, Headroom on):
Codex session log
codex_on/sessions/2026/09/02/rollout-2026-09-02T11-45-10-*.jsonl: the model's call carries the flattened name and nonamespace, and Codex rejects itHeadroom
/statsafter run A:api_requests=3, requests_compressed=0, uncompressed_requests={"no_compressible_content": 3}. Headroom compressed nothing and injected nothing; the rename happened anyway.Run B (same flow with
x-headroom-bypass: true): same refusal on screen (unsupported call: mcp__deepwiki__deepwiki_read_wiki_structure), the same flatfunction_callwith nonamespaceincodex_off/sessions/2026/09/02/rollout-2026-09-02T11-48-52-*.jsonl, and Headroomapi_requestsstays at 3, so the bypass was honored by Headroom while the proxy still rewrote the tools.Run C, the Responses-to-chat bridge with a plain function tool, works on the base too (
get_weathercalled with{"city": "Paris"}), so the After leg's run C is a regression check, not a fix.After (this PR at 7cbc743)
Same proxy config, Codex config, and prompt as the Before leg; the proxy runs from 7cbc743 with a single uvicorn worker (
--num_workers 1), Headroom on 48046 and the proxy on 25206.The proxy log for this leg shows 0 uvicorn worker deaths (
grep -c 'Child process .* died'), against nine in the earlier two-worker run.Codex screen (run A, Headroom on), no nudge needed; the one-time "Allow the deepwiki MCP server to run tool" consent prompt was answered with Enter while polling and had scrolled off by the final capture:
Codex session log
codex_on/sessions/2026/09/02/rollout-2026-09-02T15-41-05-01a06448-7411-7303-aa3b-9abe5fb878f9.jsonl: the model's call keeps the namespace, and it is the only tool name and the onlynamespacevalue in the fileNo
function_call_outputcontainsunsupported call. Headroom/statsafter run A:api_requests=2, requests_compressed=0, uncompressed_requests={"no_compressible_content": 2}, so the guardrail ran on both turns and the tools still reached the model intact.Run B (same flow with
x-headroom-bypass: truein the Codex provider headers): same screen and the same"name":"deepwiki_read_wiki_structure","namespace":"mcp__deepwiki"call incodex_off/sessions/2026/09/02/rollout-2026-09-02T15-42-19-01a06449-979e-7212-b847-a662e88d0a2c.jsonl, and Headroomapi_requestsstays at 2.Run C, the Responses-to-chat bridge with a plain function tool (the
transformation.pychange also serves this path), still returns the tool call:Type
🐛 Bug Fix
Caveats (if any)
Low
customtool gets afunctiontool back, same as before this PRGuardrailToolParamincluded); anything else is dropped with a warning instead of being forwarded to the provider. Base answered 400 for a guardrail returning pydantic models, so no working caller changes behaviortoolsvalue that is not a list is treated as no tools: the guardrail sees none, and anything it injects becomes the whole listFinal Attestation
The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR
2842e90 passes /live-pr-risk