test(e2e): repair the wildcard readiness probe and the semantic auto-router spend assertion

[yuneng-berri]

TLDR

Two e2e suites have been red in litellm-e2e builds 119 through 123. Neither is a product regression — both are test assumptions that an intentional behaviour change invalidated. This repairs them.

Not covered here: batches/test_batches_e2e.py::test_batch_lifecycle is failing on the same builds and is a real regression (batches created through the model-encoded and model-param paths no longer get a LiteLLM_ManagedObjectTable row, so they never appear in GET /v1/batches). That is being handled separately.

1. Wildcard readiness probe — 6 setup errors in access_control/test_model_access_group_e2e.py

#31731 moved models_to_remove.add(model) out of the fallback branches of _get_wildcard_models and made it unconditional. Before it, a wildcard route with a matching router deployment stayed in the /v1/models list, and only the no-router / no-deployment paths removed it.

ProxyClient._await_model_servable polls /v1/models for an exact id match. The fixture registers openai/gpt-5.4* via /model/new, so the poll now can never resolve, times out at model_servable_timeout, and every test in TestKeyScopedToAccessGroup errors in setup:

model 'openai/gpt-5.4*' was created but never became servable on the data plane
within 40.0s of first listing ... after /model/new

return_wildcard_routes=True still re-adds the route, so the probe asks for it. Set unconditionally rather than sniffing for * in the name: the flag only ever adds wildcard entries, so it is a no-op for a concrete model name, and one code path is easier to keep honest than two.

2. Semantic auto-router spend assertion — test_responses_input_reaches_the_semantic_auto_router

#39532 bills the routing embedding to the caller's key on purpose. The key's spend logs now legitimately carry an openai/text-embedding-3-small row, and _assert_served_only_by rejects it:

expected every request to be served by one of ['anthropic/claude-haiku-4-5', ...],
spend logs show ('anthropic/claude-haiku-4-5', 'openai/text-embedding-3-small')

Adding EMBEDDING_MODEL to the allowlist would have fixed the red at the cost of the thing this test exists to prove — that the request actually reached the target deployment. So the embedding row is split off and asserted on its own, which turns the break into coverage for #39532 instead of a hole in it. The poll gains a predicate so it waits for the embedding row rather than racing whichever row is written first.

Verification

• uv run ruff check --config ruff-tests.toml tests — passes (this is the lint gate that covers tests/; ruff.toml excludes the tree)
• ModelsListParams().model_dump(by_alias=True, exclude_none=True) → {'return_wildcard_routes': True}
• The suites themselves are AWS-gated and only run in litellm-e2e; they have not been run green locally. The next scheduled build is the real check.

Loose end worth a follow-up

TestTeamScopedToAccessGroup registers openai/gpt-5.6* through the same helper (with listed_for=key) and its probe passes today. I have not traced why the team-scoped path still lists its wildcard when the proxy-wide one does not. This change is safe either way, but that asymmetry may itself be a bug.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR repairs two E2E assumptions after intentional behavior changes:

• Wildcard model readiness polls now request wildcard routes from /v1/models
• The semantic auto-router test waits for both spend records, verifies embedding attribution, and excludes embedding rows from the deployment allowlist assertion

Confidence Score: 5/5

The PR appears safe to merge and preserves the intended regression coverage while repairing both stale E2E assumptions

The wildcard query restores the readiness poll’s model-listing contract, and the spend-log changes wait for and independently validate both expected records without allowing an unrelated routed deployment

Important Files Changed

Filename	Overview
tests/e2e/models.py	Adds the typed /v1/models query parameter needed to include configured wildcard routes
tests/e2e/proxy_client.py	Uses the wildcard-aware query for model propagation polling without changing concrete-model behavior
tests/e2e/router/test_auto_router_regressions_e2e.py	Separately validates embedding spend attribution while retaining a nonempty target-deployment allowlist assertion

_{Reviews (1): Last reviewed commit: "test(e2e): repair two suites broken by i..." | Re-trigger Greptile}