feat(datadog_llm_obs): cost tag dimensions, router decision fields, reasoning token metric, redaction gating by yucheng-berri · Pull Request #39402 · BerriAI/litellm

yucheng-berri · 2026-09-02T20:56:43Z

TLDR

Problem this solves:

Datadog LLM Obs spans cannot be grouped by user, key, or model group
Auto-router decisions are invisible in Datadog
Reasoning token counts never reach Datadog
Turning message logging off still ships tool definitions, the routing decision record that quotes the prompt, and raw prompts on failure spans
Excluding messages via standard_logging_payload_excluded_fields crashes the callback and drops the whole span

How it solves it:

Adds user, key_alias, and model_group span tags next to team
Declares the emitted dimensions under meta.metadata._dd.cost_tags
Flattens the routing decision into router_* metadata plus low cardinality tags, including router_escalated
Emits reasoning_output_tokens from either provider spelling of the usage details
Under any redaction route, redacts success and failure span messages while keeping their count and their recognized roles, drops tool definitions, and drops the metadata records that quote the prompt: the routing decision, requester metadata, prompt management variables, MCP tool arguments and results, vector store queries, and the guardrail's copy of the request
A redacted message only keeps its role when the role is one the proxy actually speaks, since the role is caller controlled text that would otherwise ride onto the span verbatim
The redaction gate also honors the deprecated message_logging=False spelling and truthy non-bool settings, matching the shared redaction hook
Ships the span with empty input messages when the operator excludes messages from the logging payload
Forwards a ddtrace integer parent_id as its string form: the old callback shipped the raw int and the Datadog intake rejected the whole batch with a 400

User Flow

Before: an operator sending traffic through an auto-router cannot slice LLM Observability by who spent the money, and cannot see which tier the router chose

They POST http://localhost:4000/v1/chat/completions with "model": "smart-router" using a key that has a key alias and a user id
They open the trace in Datadog LLM Observability, and the span carries only env, service, version and the pod tags, so grouping by user, key or model group is impossible
They POST http://localhost:4000/v1/responses with "reasoning": {"effort": "high"}, and the span reports output tokens with no reasoning breakdown, so the reasoning share of the bill is unknown
They turn message logging off for the Datadog LLM Obs callback only, resend a request carrying tools, and the span still lists every tool the model was offered; a failed request ships the raw prompt on the error span

After: the same operator can group spend by user, key and model group, see the router's choice including whether it escalated, and keep prompts and tool schemas out of Datadog while still getting token and cost metrics

They POST http://localhost:4000/v1/chat/completions with "model": "smart-router" using the same key
The span in Datadog now carries user:..., key_alias:..., model_group:smart-router, router_tier:simple, router_cause:heuristic_scorer, router_escalated:false and routed_model:gpt-4o-mini, and _dd.cost_tags declares exactly those dimensions, so they show up as cost dimensions instead of plain tags
They POST http://localhost:4000/v1/responses with "reasoning": {"effort": "high"}, and the span reports reasoning_output_tokens: 1152 alongside the usual token metrics
They turn message logging off for the Datadog LLM Obs callback only, resend the request carrying tools, and the span shows redacted prompts, no tool definitions, no routing decision record, and the token and cost metrics still intact; a failed request now redacts the error span's messages too

Relevant issues

Fixes #39320

Linear ticket

Resolves LIT-6728

Pre-Submission checklist

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy on real Postgres, real OpenAI calls, and a real Datadog intake, run at 4 workers on both sides. Same config, same requests, same order

Config used for the first two cases:

model_list:
  - model_name: smart-router
    litellm_params:
      model: auto_router/complexity_router
      complexity_router_config:
        tiers:
          SIMPLE: ["gpt-4o-mini"]
          MEDIUM: ["gpt-4o-mini"]
          COMPLEX: ["gpt-4o-mini"]
          REASONING: ["reasoner"]
        default_model: gpt-4o-mini
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
  - model_name: reasoner
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY
litellm_settings:
  callbacks: ["datadog_llm_observability", "datadog"]
general_settings:
  master_key: sk-1234

The third case adds this to litellm_settings, so message logging is off for the Datadog LLM Obs callback alone and left on globally:

  datadog_llm_observability_params:
    turn_off_message_logging: true

The fourth case instead adds this to litellm_settings:

  standard_logging_payload_excluded_fields: ["messages"]

Every case mints its own key so the identity dimensions have something to carry:

curl -sS -X POST http://localhost:4000/key/generate \
  -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
  -d '{"key_alias":"obs-key","user_id":"obs-user","models":["smart-router","gpt-4o-mini","reasoner"]}'

The span shown per case is the one the proxy sent to Datadog, read back out of the proxy's own --detailed_debug log

Before (`ffc0a8e`)

Auto-router request: cost dimensions and routing fields

Send the request

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"smart-router","messages":[{"role":"user","content":"Design a distributed consensus protocol and prove its safety property rigorously."}],"max_tokens":40}'

status=200

Read the span's tags: no identity or routing dimensions at all

["env:unknown", "service:litellm-server", "version:unknown", "HOSTNAME:", "POD_NAME:unknown",
 "request_tag:user-agent:_curl", "request_tag:user-agent:_curl/8.7.1"]

Read the span's metadata for routing and cost declarations: nothing there

{}

Reasoning request on /v1/responses

Send the request

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"reasoner","reasoning":{"effort":"high"},"input":"Count the distinct prime factors of 987654321987654321. Final answer only.","max_output_tokens":1200}'

status=200

Read the span's metrics: output tokens are there, the reasoning share is not

{"input_tokens": 24.0, "output_tokens": 1152.0, "total_tokens": 1176.0,
 "total_cost": 0.00231, "time_to_first_token": 17.140908002853394}

Redacted request carrying tools

Send the request with message logging off for this callback

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"smart-router","messages":[{"role":"user","content":"Design a distributed consensus protocol and prove its safety property rigorously."}],"max_tokens":40,"tools":[{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]}'

status=200

Read the span: prompts are redacted, but the tool definition still went to Datadog, and a failed request (send the same body with "max_tokens":-5, provider returns 400) ships the raw prompt on the error span

{"in": "redacted-by-litellm", "out": "redacted-by-litellm", "tool_defs": 1,
 "router": {}, "cost_tags": null,
 "metrics": {"input_tokens": 53.0, "output_tokens": 40.0, "total_tokens": 93.0,
             "total_cost": 3.195e-05, "time_to_first_token": 0.9272160530090332},
 "usage_object_present": true}

Excluding messages from the logging payload

Send any request with standard_logging_payload_excluded_fields: ["messages"] configured

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"smart-router","messages":[{"role":"user","content":"Design a distributed consensus protocol and prove its safety property rigorously."}],"max_tokens":30}'

status=200

The request succeeds but no span reaches Datadog at all; the proxy log shows the callback crashing

LiteLLM:ERROR: datadog_llm_obs.py:337 - DataDogLLMObs: Error logging success event - 'messages'
KeyError: 'messages'

After (`4df14cb`)

Auto-router request: cost dimensions and routing fields

Send the same request

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"smart-router","messages":[{"role":"user","content":"Design a distributed consensus protocol and prove its safety property rigorously."}],"max_tokens":40}'

status=200

Read the span's tags: identity and routing dimensions are on the span, including whether the router escalated

["env:unknown", "service:litellm-server", "version:unknown", "HOSTNAME:", "POD_NAME:unknown",
 "request_tag:user-agent:_curl", "request_tag:user-agent:_curl/8.7.1",
 "user:obs-user", "key_alias:obs-key", "model_group:smart-router",
 "router_tier:simple", "router_cause:heuristic_scorer", "router_escalated:false",
 "routed_model:gpt-4o-mini"]

Read the span's metadata: the routing decision is flat, and only the dimensions actually on the span are declared

{"router_tier": "SIMPLE", "router_cause": "heuristic_scorer", "router_score": 0.125,
 "router_signals": ["technical (distributed, protocol)"], "router_escalated": false,
 "routed_model": "gpt-4o-mini",
 "_dd": {"cost_tags": ["user", "key_alias", "model_group", "router_tier", "router_cause",
                       "router_escalated", "routed_model"]}}

Send a request whose message contains the escalation keyword: the span flips to router_escalated:true in the tags, the metadata, and the declaration
Repeat the request 8 times against a 4 worker proxy: all 8 spans carry the same dimensions, so nothing depends on which worker served the request

Reasoning request on /v1/responses

Send the same request

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/responses \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"reasoner","reasoning":{"effort":"high"},"input":"Count the distinct prime factors of 987654321987654321. Final answer only.","max_output_tokens":1200}'

status=200

Read the span's metrics: the reasoning share is reported

{"input_tokens": 24.0, "output_tokens": 1152.0, "total_tokens": 1176.0, "total_cost": 0.00231,
 "time_to_first_token": 14.018558979034424, "reasoning_output_tokens": 1152.0}

Send a plain gpt-4o-mini request, whose provider usage reports no reasoning tokens: the span has no reasoning_output_tokens key rather than a zero

Redacted request carrying tools

Send the same request with message logging off for this callback

curl -sS -o /dev/null -w 'status=%{http_code}\n' http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"smart-router","messages":[{"role":"user","content":"Design a distributed consensus protocol and prove its safety property rigorously."}],"max_tokens":40,"tools":[{"type":"function","function":{"name":"get_weather","description":"Get weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]}'

status=200

Read the span: prompts redacted, no tool definitions, no routing decision record, no requester metadata, no prompt quoting router signals, and the token, cost and usage data all still there

{"in": "redacted-by-litellm", "out": "redacted-by-litellm", "tool_defs": 0,
 "router": {"router_tier": "MEDIUM", "router_cause": "heuristic_scorer", "router_score": 0.125,
            "router_escalated": true, "routed_model": "gpt-4o-mini"},
 "routing_decision_present": false, "requester_metadata_present": false,
 "guardrail_information": null,
 "cost_tags": ["user", "key_alias", "model_group", "router_tier", "router_cause",
               "router_escalated", "routed_model"],
 "metrics": {"input_tokens": 60.0, "output_tokens": 40.0, "total_tokens": 100.0,
             "total_cost": 3.3e-05, "time_to_first_token": 1.5417299270629883},
 "usage_object_present": true}

Send the same body with "max_tokens":-5 so the provider returns 400: the error span's input messages read redacted-by-litellm, its output messages are empty rather than a made up redacted reply, and its cost tags and router fields stay on the span

{"status": "error", "in": [["user", "redacted-by-litellm"]], "out": [],
 "tool_defs": 0, "routing_decision_present": false,
 "error": {"type": "BadRequestError",
           "message": "litellm.BadRequestError: OpenAIException - Invalid 'max_tokens' ..."}}

Repeat with global turn_off_message_logging: true instead of the per callback setting: same result, and the usage object survives either way

Excluding messages from the logging payload

Send the same request with standard_logging_payload_excluded_fields: ["messages"] configured

status=200

The span now reaches Datadog with empty input messages and every dimension intact

{"in": null, "router_tags": ["user:obs-user", "key_alias:obs-key", "model_group:smart-router",
 "router_tier:simple", "router_cause:heuristic_scorer", "router_escalated:false",
 "routed_model:gpt-4o-mini"],
 "cost_tags": ["user", "key_alias", "model_group", "router_tier", "router_cause",
               "router_escalated", "routed_model"]}

Type

🆕 New Feature
🐛 Bug Fix

Caveats (if any)

Severe

Tool definitions and the metadata records that quote the prompt no longer reach Datadog under any redaction route: the nested routing_decision, requester_metadata, prompt_management_metadata, mcp_tool_call_metadata, vector_store_request_metadata, and guardrail_information
- deliberate: the issue asks for prompts, tool arguments and tool results to stay redacted, and each of these records carries at least one of them verbatim
- the flat router_* fields minus router_signals still ship, so the router's choice stays visible, and guardrail latency stays in latency_metrics
- an operator who relied on seeing tool schemas, router signals, or guardrail request records while redacting prompts loses them

Medium

Failure spans now redact their messages under per callback redaction, matching what global redaction already did
- before this PR the per callback setting shipped raw prompts on error spans
- this fix covers the Datadog LLM Obs callback only; other callbacks with per callback redaction keep their existing behavior, tracked separately
- the error message and stack still ship on redacted failure spans, matching every other callback; a provider error that quotes the prompt would leak through them, tracked in the same follow-up

Low

Tag values go through Datadog tag normalization, so router_tier:simple is lowercase while the flat metadata keeps SIMPLE
router_escalated ships as false when the router did not escalate, since false is a real dimension value
The user tag reads user_api_key_user_id, the id the key was minted for; the sibling datadog cost management callback tags by user alias, and unifying them is left to a follow-up
The router's classifier cost is not emitted as a spend metric here; follow-up
Datadog's Tools view reads the tool call fields on the LLM span itself, so no kind: "tool" child spans are needed, and producing them would need a trace context the proxy does not have today
Global turn_off_message_logging does not strip the usage object, so cache and token metrics survive it; the issue's premise about losing usage data did not reproduce
The shared tag builder emits a bare HOSTNAME: tag when the env var is unset, on both sides, and is left alone here
A redacted message whose role is outside the recognized set (agent, assistant, developer, function, model, system, tool, user) ships with an empty role, so an operator who redacts through this callback and sends a custom role such as human loses that value in Datadog queries
- deliberate: callers pick the role text, so a redacted span would otherwise carry whatever they wrote, and the global redaction path already blanked roles

Live proxy coverage

The final live regression run used four workers on both the staging base and PR head. It exercised successful tool calls, provider failures, malformed string/list/object roles, streaming requests, and the x-litellm-disable-message-redaction override. Successful and failed redacted spans kept metrics while removing prompt text and tool definitions; streaming responses returned 200 on both sides, failure requests returned 400 on both sides, and no callback errors occurred on the head.

Datadog UI evidence

The public dashboard shows fresh PR-head traffic in Datadog LLM Observability. It breaks token volume and estimated cost down by model group, adds router-tier cost grouping, reports reasoning output tokens, and documents that input text is redacted while those dimensions remain queryable.

Public dashboard: https://p.us5.datadoghq.com/sb/9b1eff10-830b-11f0-b7cb-de02bd9ba50c-972cc0d9b6013a3d8dd05eb0da05f60e

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

ran /live-pr-risk and found no regressions/backward incompatible risks

Note

Medium Risk
Changes observability payload shape and redaction semantics (operators may lose tool schemas and prompt-quoting metadata in Datadog when redacting); batch intake behavior changes for parent_id and excluded messages.

Overview
Datadog LLM Observability spans now carry richer grouping and safer payloads when message logging is off.

Spans get cost-dimension tags (user, key_alias, model_group, plus flattened router fields) with matching meta.metadata._dd.cost_tags so Datadog can break down token/cost metrics. Auto-router decisions are flattened into router_* metadata and tags (including router_escalated from escalated or context_escalated). reasoning_output_tokens is emitted from either completion_tokens_details or output_tokens_details when present and non-zero.

Redaction is unified via _payload_logging_is_off (turn_off_message_logging, deprecated message_logging, and should_redact_message_logging): messages are redacted on success and failure spans while preserving recognized roles; tool definitions and prompt-carrying metadata blocks are stripped; prompt-quoting router fields are omitted. Robustness fixes: missing messages no longer crashes span creation; integer parent_id from ddtrace is stringified; init params use model_dump(exclude_unset=True).

Type/linter budget files are nudged down slightly. Extensive unit tests cover the new behavior.

^{Reviewed by Cursor Bugbot for commit b303caf. Bugbot is set up for automated code reviews on this repo. Configure here.}

…easoning token metric, redaction gating

codspeed · 2026-09-02T20:58:48Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_lit6728 (b303caf) with litellm_internal_staging (7a81ae9)}

greptile-apps · 2026-09-02T20:59:33Z

Greptile Summary

The PR expands Datadog LLM Observability spans with cost dimensions, router decision metadata, and reasoning-token metrics while tightening message-redaction behavior.

Adds normalized identity and routing tags with Datadog cost-tag declarations.
Redacts prompt-bearing messages, tools, and metadata across success and failure spans.
Handles excluded messages and integer parent IDs without dropping spans.
Adds focused coverage for metrics, routing dimensions, redaction paths, and payload robustness.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/integrations/datadog/datadog_llm_obs.py	Adds cost and router dimensions, reasoning-token metrics, comprehensive redaction gating, and safer handling of missing messages and parent IDs.
litellm/types/integrations/datadog_llm_obs.py	Extends the Datadog metric payload type with optional reasoning output tokens.
tests/test_litellm/integrations/datadog/test_datadog_llm_obs.py	Adds broad regression coverage for the new Datadog span behavior.

_{Reviews (8): Last reviewed commit: "Merge remote-tracking branch 'origin/lit..." | Re-trigger Greptile}

greptile-apps · 2026-09-02T20:59:36Z

+def _redacting_logger(**kwargs: Any) -> DataDogLLMObsLogger:
+    with patch.dict(os.environ, {"DD_API_KEY": "k", "DD_SITE": "us5.datadoghq.com"}, clear=True):
+        with patch("asyncio.create_task"):
+            return DataDogLLMObsLogger(**kwargs)


Test helper typing and mutation

The new helper accepts unrestricted Any values, while the failure fixture mutates nested payload state, weakening static guarantees and test clarity

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

devin-ai-integration

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 1 potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

devin-ai-integration · 2026-09-02T21:00:35Z

+        input_messages: Final = _to_dd_messages(standard_logging_payload.get("messages"))
+        output_messages: Final = self._get_response_messages(
+            standard_logging_payload=standard_logging_payload,
+            call_type=standard_logging_payload.get("call_type"),
+        )
+        input_meta: Final = InputMeta(messages=_redact_messages(input_messages) if redact_payload else input_messages)


🟡 Redaction destroys conversation shape

On successful calls with per-callback turn_off_message_logging, the base hook collapses input before _redact_messages runs. Datadog records one assistant message regardless of the conversation

Was this helpful? React with 👍 or 👎 to provide feedback.

Live A/B on staging and a536037 produce one redacted assistant message on successful per-callback calls; identical behavior proves this finding predates this PR.

yucheng-berri · 2026-09-02T21:02:39Z

@greptileai please review the current head 79db71b

codecov · 2026-09-02T21:05:04Z

Codecov Report

❌ Patch coverage is 98.14815% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/integrations/datadog/datadog_llm_obs.py	98.11%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

yucheng-berri · 2026-09-02T21:18:11Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 79db71b. Configure here.}

yucheng-berri · 2026-09-02T23:18:47Z

@greptileai please review the current head 729d73c, which merges staging and adds a parent_id string coercion fix

yucheng-berri · 2026-09-02T23:27:46Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 729d73c. Configure here.}

…itellm_lit6728 # Conflicts: # basedpyright-code-budget.json # type-discipline-budget.json

yucheng-berri · 2026-09-03T00:27:55Z

@greptileai review latest head

yucheng-berri · 2026-09-03T00:27:59Z

bugbot run

veria-ai · 2026-09-03T00:28:40Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 1845ac9. Configure here.}

…itellm_lit6728

yucheng-berri · 2026-09-03T01:06:17Z

@greptileai please review the current head 808973b

devin-ai-integration

Devin Review found 1 new potential issue.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

devin-ai-integration · 2026-09-03T01:06:50Z

+    def _payload_logging_is_off(self, kwargs: Mapping[str, Any]) -> bool:
+        return (
+            bool(self.turn_off_message_logging)
+            or self.message_logging is not True
+            or should_redact_message_logging(dict(kwargs))
+        )


🟨 Redaction override is ignored

With callback or global redaction enabled, litellm-disable-message-redaction cannot disable Datadog redaction. Authorized requests still lose their logged content

Was this helpful? React with 👍 or 👎 to provide feedback.

Verified live on base and head, per callback turn_off_message_logging ignores this header on both sides, so the override behavior is pre-existing, not introduced here

yucheng-berri · 2026-09-03T01:09:52Z

@greptileai please review the current head a536037

yucheng-berri · 2026-09-03T01:14:42Z

@greptileai review latest head

yucheng-berri · 2026-09-03T01:14:46Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a536037. Configure here.}

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

…itellm_lit6728

yucheng-berri · 2026-09-03T02:21:27Z

@greptileai review latest head

yucheng-berri · 2026-09-03T02:21:33Z

bugbot run

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit b303caf. Configure here.}

feat(datadog_llm_obs): cost tag dimensions, router decision fields, r…

4df14cb

…easoning token metric, redaction gating

greptile-apps Bot reviewed Sep 2, 2026

View reviewed changes

devin-ai-integration Bot reviewed Sep 2, 2026

View reviewed changes

test(datadog_llm_obs): satisfy test quality gate

79db71b

cursor Bot reviewed Sep 2, 2026

View reviewed changes

yucheng-berri added 2 commits September 2, 2026 15:37

merge staging into lit6728

4f7cdf2

fix: forward integer parent_id as its string form

729d73c

cursor Bot reviewed Sep 2, 2026

View reviewed changes

tin-berri approved these changes Sep 2, 2026

View reviewed changes

yucheng-berri added 2 commits September 2, 2026 17:15

merge staging into lit6728

e52ce00

Merge remote-tracking branch 'origin/litellm_internal_staging' into l…

1845ac9

…itellm_lit6728 # Conflicts: # basedpyright-code-budget.json # type-discipline-budget.json

This comment was marked as resolved.

Sign in to view

cursor Bot reviewed Sep 3, 2026

View reviewed changes

fix(datadog): sanitize redacted message roles

f266708

This comment was marked as resolved.

Sign in to view

yucheng-berri added 5 commits September 2, 2026 17:48

fix(datadog): keep the A2A agent role on redacted spans

8bab419

Merge remote-tracking branch 'origin/litellm_internal_staging' into l…

5396f8a

…itellm_lit6728

Merge remote-tracking branch 'origin/litellm_internal_staging' into l…

343a17a

…itellm_lit6728

fix(datadog): merge current staging budget

92f5dab

style(datadog): format redaction tests

808973b

devin-ai-integration Bot reviewed Sep 3, 2026

View reviewed changes

fix(datadog): handle malformed redacted roles

a536037

cursor Bot reviewed Sep 3, 2026

View reviewed changes

test(datadog): put the test quality suppression on the reported line

a1a082a

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

devin-ai-integration Bot mentioned this pull request Sep 3, 2026

chore(crowdstrike): expect the deduped end-of-stream scan in crowdstrike cadence test #39467

Merged

4 of 6 tasks

Merge remote-tracking branch 'origin/litellm_internal_staging' into l…

b303caf

…itellm_lit6728

cursor Bot reviewed Sep 3, 2026

View reviewed changes

yucheng-berri merged commit 291e84e into litellm_internal_staging Sep 3, 2026
83 checks passed

yucheng-berri deleted the litellm_lit6728 branch September 3, 2026 02:46

yucheng-berri mentioned this pull request Sep 4, 2026

fix(datadog_llm_obs): keep the guardrail audit record under message redaction #39702

Merged

4 of 5 tasks

Conversation

yucheng-berri commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Before (ffc0a8e)

Auto-router request: cost dimensions and routing fields

Reasoning request on /v1/responses

Redacted request carrying tools

Excluding messages from the logging payload

After (4df14cb)

Auto-router request: cost dimensions and routing fields

Reasoning request on /v1/responses

Redacted request carrying tools

Excluding messages from the logging payload

Type

Caveats (if any)

Severe

Medium

Low

Live proxy coverage

Datadog UI evidence

Final Attestation

Uh oh!

codspeed Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

greptile-apps Bot Sep 2, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yucheng-berri Sep 3, 2026

Choose a reason for hiding this comment

Uh oh!

yucheng-berri commented Sep 2, 2026

Uh oh!

codecov Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

yucheng-berri commented Sep 2, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

yucheng-berri commented Sep 2, 2026

Uh oh!

yucheng-berri commented Sep 2, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

yucheng-berri commented Sep 3, 2026

Uh oh!

yucheng-berri commented Sep 3, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

veria-ai Bot commented Sep 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

yucheng-berri commented Sep 2, 2026 •

edited

Loading

Before (`ffc0a8e`)

After (`4df14cb`)

codspeed Bot commented Sep 2, 2026 •

edited

Loading

greptile-apps Bot commented Sep 2, 2026 •

edited

Loading

devin-ai-integration Bot left a comment •

edited

Loading

devin-ai-integration Bot Sep 2, 2026 •

edited

Loading

codecov Bot commented Sep 2, 2026 •

edited

Loading

veria-ai Bot commented Sep 3, 2026 •

edited

Loading