Skip to content

fix(images): forward gpt-image supported params like background to OpenAI and Azure - #39525

Merged
mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_fix_gpt_image_background_dropped
Sep 3, 2026
Merged

mateo-berri merged 3 commits into
litellm_internal_stagingfrom
litellm_fix_gpt_image_background_dropped

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • background, output_format, moderation, output_compression silently dropped for gpt-image models, on both openai/gpt-image-* and azure/gpt-image-*
  • Response came back with "background": null even when the provider echoed it
  • On the Azure path the response reported output_format and quality from request defaults, so a jpeg request came back labelled png and the quality tier Azure actually used was overwritten with high

How it solves it:

  • Seed the image-gen default param table with the provider's supported params so they survive filtering, and reuse that same list when forwarding instead of computing it twice
  • ImageResponse now keeps the background field the provider returned
  • The gpt-image response transform keeps the provider's echoed size, quality, and output_format and falls back to the request values (folds in the response-side fix from fix(image-generation): preserve requested output format #34262)

User Flow

Before: a developer asking gpt-image-2 for a transparent PNG gets an opaque image back

  1. They send POST https://litellm-domain/v1/images/generations with {"model": "gpt-image-2", "prompt": "a red apple", "background": "transparent", "output_format": "png"}
  2. They get a 200 with an image, but the JSON says "background": null and the PNG has a solid background
  3. Same thing with "model": "azure-gpt-image-2": the request that reaches Azure has no background or output_format key
  4. Asking either model for "output_format": "jpeg" returns PNG bytes labelled "output_format": "png"

After: the same request produces a transparent PNG

  1. They send the same POST https://litellm-domain/v1/images/generations with "background": "transparent" and "output_format": "png"
  2. The request that reaches OpenAI (or Azure) carries background and output_format, and the image comes back with an alpha channel
  3. The 200 response now says "background": "transparent"
  4. Asking for "output_format": "jpeg" returns JPEG bytes labelled "output_format": "jpeg", and quality is the tier the provider reports

Relevant issues

Fixes #38649

Linear ticket

Resolves LIT-6836

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Real OpenAI and real Azure OpenAI (a Sweden Central gpt-image-2 deployment), real cost. Two worktrees with their own venvs, each proxy booted from its own tree with PYTHONPATH pointed at it (litellm.__file__ checked inside that tree), --num_workers 2, no DATABASE_URL so both run DB-less. Before is the merge base 658f506 on port 53213, after is the tip ec2e35b on port 42828, same config, same script, same order

model_list:
  - model_name: gpt-image-2
    litellm_params:
      model: openai/gpt-image-2
      api_key: os.environ/OPENAI_API_KEY
  - model_name: azure-gpt-image-2
    litellm_params:
      model: azure/img-dep-6413
      api_base: os.environ/AZURE_SWEDEN_API_BASE
      api_key: os.environ/AZURE_SWEDEN_API_KEY
      api_version: "2025-04-01-preview"
general_settings:
  master_key: sk-1234
PYTHONPATH=<worktree> .venv/bin/python litellm/proxy/proxy_cli.py --config config.yaml --port <port> --num_workers 2
PORT=$1; OUT=$2; mkdir -p "$OUT"
B=http://localhost:$PORT/v1/images/generations
H1='Authorization: Bearer sk-1234'; H2='Content-Type: application/json'
show() { jq -c '{background, output_format, quality, size, n_images: (.data|length), b64_len: (.data[0].b64_json|length)}'; }
for m in gpt-image-2 azure-gpt-image-2; do
  echo "== $m transparent png"
  curl -s $B -H "$H1" -H "$H2" -d "{\"model\":\"$m\",\"prompt\":\"a red apple\",\"size\":\"1024x1024\",\"n\":1,\"background\":\"transparent\",\"output_format\":\"png\"}" > "$OUT/$m.json"
  show < "$OUT/$m.json"
  jq -r '.data[0].b64_json' "$OUT/$m.json" | base64 -d > "$OUT/$m.png"
  sips -g format -g hasAlpha "$OUT/$m.png" | tail -2 | tr -s ' ' | paste -sd' ' -
  echo "== $m jpeg"
  curl -s $B -H "$H1" -H "$H2" -d "{\"model\":\"$m\",\"prompt\":\"a red apple\",\"size\":\"1024x1024\",\"n\":1,\"output_format\":\"jpeg\"}" > "$OUT/$m-jpeg.json"
  show < "$OUT/$m-jpeg.json"
  echo "magic bytes: $(jq -r '.data[0].b64_json' "$OUT/$m-jpeg.json" | base64 -d | head -c 4 | xxd -p)"
done
echo "== gpt-image-2 with response_format (unsupported, unchanged)"
curl -s $B -H "$H1" -H "$H2" -d '{"model":"gpt-image-2","prompt":"a red apple","response_format":"b64_json"}' | jq -c '.error.message' | cut -c1-140

89504e47 is the PNG magic, ffd8ffe0 is JPEG, and hasAlpha comes from sips

Before (658f506, port 53213)

== gpt-image-2 transparent png
{"background":null,"output_format":"png","quality":"low","size":"1024x1024","n_images":1,"b64_len":2017812}
 format: png  hasAlpha: no
== gpt-image-2 jpeg
{"background":null,"output_format":"png","quality":"low","size":"1024x1024","n_images":1,"b64_len":2068368}
magic bytes: 89504e47
== azure-gpt-image-2 transparent png
{"background":null,"output_format":"png","quality":"high","size":"1024x1024","n_images":1,"b64_len":1811692}
 format: png  hasAlpha: no
== azure-gpt-image-2 jpeg
{"background":null,"output_format":"png","quality":"high","size":"1024x1024","n_images":1,"b64_len":1780060}
magic bytes: 89504e47
== gpt-image-2 with response_format (unsupported, unchanged)
"litellm.UnsupportedParamsError: Setting `response_format` is not supported by openai, gpt-image-2. To drop it from the call, set `litellm.d

After (ec2e35b, port 42828)

== gpt-image-2 transparent png
{"background":"transparent","output_format":"png","quality":"medium","size":"1024x1024","n_images":1,"b64_len":2125332}
 format: png  hasAlpha: yes
== gpt-image-2 jpeg
{"background":"opaque","output_format":"jpeg","quality":"low","size":"1024x1024","n_images":1,"b64_len":142640}
magic bytes: ffd8ffe0
== azure-gpt-image-2 transparent png
{"background":"transparent","output_format":"png","quality":"medium","size":"1024x1024","n_images":1,"b64_len":1574368}
 format: png  hasAlpha: yes
== azure-gpt-image-2 jpeg
{"background":"opaque","output_format":"jpeg","quality":"low","size":"1024x1024","n_images":1,"b64_len":484944}
magic bytes: ffd8ffe0
== gpt-image-2 with response_format (unsupported, unchanged)
"litellm.UnsupportedParamsError: Setting `response_format` is not supported by openai, gpt-image-2. To drop it from the call, set `litellm.d

The same azure jpeg request sent straight to Azure with no quality echoes "quality":"low", which is what the before leg overwrote with high

Observations from the run:

  • Before, Azure quality echo became high; this PR fixes it
  • OpenAI SDK path skips the transform; unchanged here
  • dall-e-3 gone from our OpenAI key; path not exercised
  • response_format rejection for gpt-image-2 identical on both legs
  • Azure cost header empty both legs, deployment unmapped, pre-existing

Type

🐛 Bug Fix

Caveats (if any)

Low

  • dall-e-3 path not exercised live; model gone from our key
  • Providers listing params beyond the old default set now forward them
  • OpenAI SDK image path skips this transform; echo already kept
  • Azure gpt-image-1 cost without quality now uses the echoed tier

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/e2daaef76ead48c687fd33ad2c258672
Open in Devin Desktop: https://app.devin.ai/desktop/session/e2daaef76ead48c687fd33ad2c258672?variant=devin

…enAI and Azure

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

CLAassistant commented Sep 3, 2026 •

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ devin-ai-integration[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR forwards the full parameter set supported by GPT image-generation adapters and preserves image metadata echoed by OpenAI and Azure.

  • Seeds image parameter normalization with the selected provider configuration’s supported parameters.
  • Preserves provider-returned background, size, quality, and output format in ImageResponse.
  • Adds OpenAI and Azure regression coverage for request forwarding and response metadata.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
litellm/utils.py Extends image-generation parameter normalization so adapter-supported parameters survive filtering and are forwarded through the existing provider mapping path.
litellm/llms/openai/image_generation/gpt_transformation.py Preserves provider-echoed image metadata and falls back to requested or default values when metadata is absent.
litellm/types/utils.py Retains the provider-returned background field when constructing and serializing an ImageResponse.
tests/test_litellm/test_utils.py Covers forwarding GPT image parameters for both OpenAI and Azure while retaining unknown parameters in extra_body.
tests/test_litellm/llms/openai/image_generation/test_gpt_transformation.py Covers preservation of metadata echoed by OpenAI and Azure image-generation responses.
tests/test_litellm/types/test_types_utils.py Verifies that ImageResponse retains and serializes the background field.

Reviews (3): Last reviewed commit: "fix(image_gen): keep the provider's echo..." | Re-trigger Greptile

@codspeed

codspeed Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_fix_gpt_image_background_dropped (ec2e35b) with litellm_internal_staging (fa533f7)

Open in CodSpeed

@codecov

codecov Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@mateo-berri mateo-berri added run-ci and removed run-ci labels Sep 3, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit ec2e35b. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mateo-berri
mateo-berri merged commit 4b1e24e into litellm_internal_staging Sep 3, 2026
125 of 127 checks passed
@mateo-berri
mateo-berri deleted the litellm_fix_gpt_image_background_dropped branch September 3, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Pass-through background parameter for gpt-image-2 (OpenAI & Azure)

2 participants