Repository navigation
Add atexit handlers to flush callbacks for async completions - #16487
Merged
1 commit merged intoNov 12, 2025
Merged
1 commit merged into
1 commit merged into
Conversation
Fixes BerriAI#16486 ## Problem Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith) were not being invoked for litellm.acompletion() in short-lived scripts. The callbacks worked correctly for synchronous completions but async completions would queue callbacks that were lost when the script exited before GLOBAL_LOGGING_WORKER could process them. Root cause: asyncio.run() closes the event loop immediately after the async function completes, preventing the background worker from processing queued callbacks. ## Solution Implemented a two-level atexit handler approach: 1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py): - Processes remaining callbacks from queue before exit - Creates new event loop to run pending coroutines synchronously - Applies time and iteration limits to prevent blocking shutdown 2. Integration-specific atexit handlers (posthog.py as example): - Flushes internal queue to external service - Uses synchronous HTTP client for reliable delivery - Each integration needs its own handler due to varying sync APIs ## Changes - litellm/litellm_core_utils/logging_worker.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds - litellm/integrations/posthog.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Groups events by credentials for batch sending - Uses sync_client for reliable HTTP delivery - tests/logging_callback_tests/test_posthog.py: - Added test_async_callback_atexit_handler_exists() - Added test_posthog_atexit_flushes_internal_queue() - Added test_sync_callback_not_affected_by_atexit() ## Testing - All existing tests pass - Manual end-to-end testing confirms fix: - Async events now arrive in PostHog - Sync events continue working (no regression) - Unit tests verify atexit handlers registered and functional ## Impact - Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER - No breaking changes - only adds missing functionality - Sync path unchanged - no performance impact
|
@andrewm4894 is attempting to deploy a commit to the CLERKIEAI Team on Vercel. A member of the Team first needs to authorize it. |
|
|
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…rriAI#16487) Fixes BerriAI#16486 ## Problem Callbacks configured via litellm.success_callback (e.g., PostHog, LangSmith) were not being invoked for litellm.acompletion() in short-lived scripts. The callbacks worked correctly for synchronous completions but async completions would queue callbacks that were lost when the script exited before GLOBAL_LOGGING_WORKER could process them. Root cause: asyncio.run() closes the event loop immediately after the async function completes, preventing the background worker from processing queued callbacks. ## Solution Implemented a two-level atexit handler approach: 1. GLOBAL_LOGGING_WORKER atexit handler (logging_worker.py): - Processes remaining callbacks from queue before exit - Creates new event loop to run pending coroutines synchronously - Applies time and iteration limits to prevent blocking shutdown 2. Integration-specific atexit handlers (posthog.py as example): - Flushes internal queue to external service - Uses synchronous HTTP client for reliable delivery - Each integration needs its own handler due to varying sync APIs ## Changes - litellm/litellm_core_utils/logging_worker.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Processes up to MAX_ITERATIONS_TO_CLEAR_QUEUE events - Time-limited to MAX_TIME_TO_CLEAR_QUEUE seconds - litellm/integrations/posthog.py: - Added _flush_on_exit() method - Registered atexit handler in __init__ - Groups events by credentials for batch sending - Uses sync_client for reliable HTTP delivery - tests/logging_callback_tests/test_posthog.py: - Added test_async_callback_atexit_handler_exists() - Added test_posthog_atexit_flushes_internal_queue() - Added test_sync_callback_not_affected_by_atexit() ## Testing - All existing tests pass - Manual end-to-end testing confirms fix: - Async events now arrive in PostHog - Sync events continue working (no regression) - Unit tests verify atexit handlers registered and functional ## Impact - Fixes async callback delivery for ALL integrations using GLOBAL_LOGGING_WORKER - No breaking changes - only adds missing functionality - Sync path unchanged - no performance impact
This pull request was closed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Title
Add atexit handlers to flush callbacks for async completions
Relevant issues
Fixes #16486
Pre-Submission checklist
tests/litellm/directory, Adding at least 1 test is a hard requirementmake test-unitType
Bug Fix
Changes
Problem
Callbacks configured via
litellm.success_callbackwere not invoked forlitellm.acompletion()in short-lived scripts. Callbacks worked correctly for sync completions but async completions queued callbacks that were lost when the script exited before GLOBAL_LOGGING_WORKER could process them.Solution
Implemented two-level atexit handler approach:
Files Modified
litellm/litellm_core_utils/logging_worker.py: Added_flush_on_exit()method and atexit registrationlitellm/integrations/posthog.py: Added_flush_on_exit()method and atexit registrationtests/logging_callback_tests/test_posthog.py: Added 3 new tests verifying atexit handlersTesting
All 14 tests in test_posthog.py pass. Manual end-to-end testing confirms async events now arrive in PostHog while sync events continue working without regression.
Screenshot of new related tests passing locally:
Impact
Fixes async callback delivery for all integrations using GLOBAL_LOGGING_WORKER. No breaking changes, sync path unchanged.