Skip to content

.NET: Surface x-ms-served-model header as ChatResponse.ModelId for Foundry agents#5979

Merged
rogerbarreto merged 5 commits into
microsoft:mainfrom
rogerbarreto:issues/x-ms-served-model-support
May 21, 2026
Merged

.NET: Surface x-ms-served-model header as ChatResponse.ModelId for Foundry agents#5979
rogerbarreto merged 5 commits into
microsoft:mainfrom
rogerbarreto:issues/x-ms-served-model-support

Conversation

@rogerbarreto
Copy link
Copy Markdown
Member

.NET twin of python PR #5910.

Foundry Responses API send back x-ms-served-model header. Header carry real snapshot (e.g. gpt-5-nano-2025-08-07). JSON body only carry deployment alias. OTel span and ChatResponse.ModelId show wrong name without fix.

Fix:

  • New internal ServedModelPolicy (SCM PipelinePolicy) read header after ProcessNext.
  • AsyncLocal<StrongBox<string?>> carry value across async boundary (box mutation propagate, plain AsyncLocal write do not).
  • ServedModelChatClient (internal DelegatingChatClient) push box before inner call, overwrite ChatResponse.ModelId after. Sit between OTel and MEAI OpenAIResponsesChatClient.
  • Wire policy + client into every AsAIAgent path in Microsoft.Agents.AI.Foundry (Responses direct, hosted agent endpoint, versioned, AgentRecord, AgentReference).
  • All new types internal. MEAI.Core untouched (provider-agnostic).

Tests:

  • 14 unit tests covering scope, policy, client (sync + streaming), end-to-end with real OpenAI SDK + mock HTTP.
  • 2 integration tests against live Foundry endpoint validate ModelId differ from deployment alias.

All green: 283 unit pass, 2 integration pass, build --warnaserror clean, CI parity dotnet format clean.

Copilot AI review requested due to automatic review settings May 20, 2026 20:50
@moonbox3 moonbox3 added the .NET label May 20, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Surfaces the Azure OpenAI Responses API x-ms-served-model response header (the real served snapshot model, e.g. gpt-5-nano-2025-08-07) as ChatResponse.ModelId / ChatResponseUpdate.ModelId for Foundry-based agents, so telemetry and callers see the actual model instead of the deployment alias from the JSON body.

Changes:

  • Add an internal SCM PipelinePolicy (ServedModelPolicy) + AsyncLocal<StrongBox<string?>> carrier (ServedModelScope) to capture x-ms-served-model after the HTTP roundtrip.
  • Add an internal DelegatingChatClient (ServedModelChatClient) that overwrites ModelId on responses/updates using the captured header.
  • Wire this behavior into Foundry agent creation paths and add unit + integration coverage.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
dotnet/src/Microsoft.Agents.AI.Foundry/ServedModelScope.cs AsyncLocal StrongBox carrier for propagating the served-model header across async boundaries.
dotnet/src/Microsoft.Agents.AI.Foundry/ServedModelPolicy.cs SCM pipeline policy capturing x-ms-served-model into the scope after ProcessNext.
dotnet/src/Microsoft.Agents.AI.Foundry/ServedModelChatClient.cs Delegating chat client that applies the captured served model onto ModelId for non-streaming and streaming.
dotnet/src/Microsoft.Agents.AI.Foundry/FoundryAgent.cs Wires served-model policy/client into agent creation (responses + agent endpoint paths).
dotnet/src/Microsoft.Agents.AI.Foundry/AzureAIProjectChatClientExtensions.cs Wires served-model policy/client into AIProjectClient.AsAIAgent extension creation paths.
dotnet/tests/Microsoft.Agents.AI.Foundry.UnitTests/ServedModelTests.cs New unit tests covering scope/policy/client behavior and end-to-end behavior via real OpenAI SCM pipeline + mock HTTP.
dotnet/tests/Microsoft.Agents.AI.Foundry.UnitTests/Microsoft.Agents.AI.Foundry.UnitTests.csproj Excludes ServedModelTests for legacy TFMs (net472).
dotnet/tests/Foundry.IntegrationTests/ResponsesAgentServedModelTests.cs New live integration tests validating ChatResponse.ModelId reflects served-model header.
Comments suppressed due to low confidence (1)

dotnet/src/Microsoft.Agents.AI.Foundry/ServedModelChatClient.cs:69

  • GetStreamingResponseAsync sets ServedModelScope.Current but never restores it after enumeration completes or errors. Even if AsyncLocal sometimes restores across awaited boundaries, async iterators can complete synchronously or be disposed early, leaving the scope set unexpectedly. Wrap the await-foreach in try/finally (saving/restoring the previous ServedModelScope.Current) so the scope is always cleared/restored when the enumerator is disposed.
    {
        var box = new StrongBox<string?>(null);
        ServedModelScope.Current = box;

        await foreach (var update in base.GetStreamingResponseAsync(messages, options, cancellationToken).ConfigureAwait(false))
        {
            if (box.Value is { } servedModel)
            {
                update.ModelId = servedModel;
            }

            yield return update;
        }
    }

Comment thread dotnet/src/Microsoft.Agents.AI.Foundry/ServedModelChatClient.cs Outdated
Comment thread dotnet/tests/Foundry.IntegrationTests/ResponsesAgentServedModelTests.cs Outdated
Comment thread dotnet/src/Microsoft.Agents.AI.Foundry/AzureAIProjectChatClientExtensions.cs Outdated
rogerbarreto and others added 5 commits May 21, 2026 11:53
…undry agents

Mirrors Python PR microsoft#5910. Adds an internal SCM PipelinePolicy that reads the x-ms-served-model HTTP response header on Azure OpenAI Responses calls and writes it into an AsyncLocal box. A DelegatingChatClient sits between OpenTelemetry and the MEAI OpenAIResponsesChatClient and overwrites ChatResponse.ModelId with the served snapshot so OTel spans report the actual model rather than the deployment alias. Wired through all AsAIAgent paths in Microsoft.Agents.AI.Foundry.
- Restore previous ServedModelScope in finally to avoid AsyncLocal leak into caller execution context.
- Make served-model integration test assertion robust to deployment names that already match the snapshot pattern.
- Broaden UnitTests csproj comment to cover all conditional removals (net8.0+ requirement).
Split the combined ServedModelTests.cs into one test class per SUT:

- ServedModelScopeTests.cs (AsyncLocal carrier)
- ServedModelPolicyTests.cs (SCM pipeline policy)
- ServedModelChatClientTests.cs (delegating client, with regions for Non-streaming / Streaming / End-to-end)

Shared helpers and fake clients moved into ServedModelTestHelpers.cs.

Csproj net8.0+ exclusion list updated accordingly.
Move x-ms-served-model header capture from the standalone ServedModelChatClient
decorator directly into FoundryChatClient, eliminating a separate wrapper that
had to be applied at every Foundry entry point via WireServedModel().

- Register ServedModelPolicy in FoundryChatClient constructors (alongside the
  existing AgentFrameworkUserAgentPolicy registration)
- Add StrongBox push/read logic to FoundryChatClient.GetResponseAsync and
  GetStreamingResponseAsync
- Delete ServedModelChatClient.cs and its unit tests
- Remove WireServedModel() from FoundryAgent and AIProjectClientExtensions
- Update ServedModelPolicy/Scope XML docs to reference FoundryChatClient
- Simplify ServedModelTestHelpers to use FoundryChatClient directly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@chetantoshniwal chetantoshniwal enabled auto-merge May 21, 2026 17:23
@rogerbarreto rogerbarreto force-pushed the issues/x-ms-served-model-support branch from 72e1d0c to 6c2f8e4 Compare May 21, 2026 17:41
@chetantoshniwal chetantoshniwal added this pull request to the merge queue May 21, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 21, 2026
@chetantoshniwal chetantoshniwal added this pull request to the merge queue May 21, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 21, 2026
@rogerbarreto rogerbarreto added this pull request to the merge queue May 21, 2026
Merged via the queue into microsoft:main with commit afd2739 May 21, 2026
26 checks passed
@github-project-automation github-project-automation Bot moved this from In Review to Done in Agent Framework May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants