🐛 Fix proxy caching between requests in aiohttp transport

[lcfyi]

Title

Fix proxy caching between requests in aiohttp transport

Relevant issues

Couldn't find one at a glance, this may have just been an edge case we encountered and nobody else has.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Will let CI pass for (2) and (3). My change may have hit a linter whammy so there are a fair number of whitespace / formatting changes from ruff.

Type

🐛 Bug Fix
✅ Test

Changes

Background

This PR fixes a bug in the LiteLLMAiohttpTransport proxy cache when multiple endpoints are sent through the same session. A concrete example of where this can happen is in the /v1/messages Anthropic adapter, which fetches an anthropic client from the cache here:

litellm/litellm/llms/custom_httpx/llm_http_handler.py

Lines 1795 to 1797 in be97073

	async_httpx_client = get_async_httpx_client(
	llm_provider=litellm.LlmProviders.ANTHROPIC
	)

If we have multiple backends that have mixed proxy settings (e.g. amazonaws.com and azure.com, NO_PROXY=amazonaws.com HTTP_PROXY=foo.com), we'll get unwanted behaviour depending on when the requests land on the process:

1. If an amazonaws.com request lands first, we'll cache the NO_PROXY, and subsequent requests will not pass in a proxy configuration (there's some extra weird behaviour here because we do pass in trust_env to aiohttp, meaning it'll will actually use the HTTP_PROXY variable if it's properly formed)
2. If an azure.com request lands first, we'll cache the proxy configuration, and subsequent requests will use foo.com

The fix

This PR basically makes the proxy cache depend on the request URL rather than caching it once ~forever. I chose the full URL as the cache key since I don't think there are any request URLs that have constantly rotating query params that'd bust the cache, but correct me if I'm wrong.

I ended up caching with the host as key, see the last 2 commits on this PR. I wrote a test to reflect this behaviour.

[vercel]

@lcfyi is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

[ishaan-jaff]

@AlexsanderHamir please take a look when possible

[lcfyi]

I rebased on main, looks like CI is running and not dying on lockfile issues anymore.

[AlexsanderHamir]

I benchmarked the behavior before and after this PR, and found no performance regressions.