Abort when coroutine is cancelled #1020
Merged
+3
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
zhuohan123
reviewed
Sep 12, 2023
| @@ -385,8 +385,8 @@ async def generate( | |||
|
|
|||
| async for request_output in stream: | |||
| yield request_output | |||
| except Exception as e: | |||
| # If there is an exception, abort the request. | |||
| except (Exception, asyncio.CancelledError) as e: | |||
There was a problem hiding this comment.
Why is this needed? I believe Exception should already capture all exceptions?
There was a problem hiding this comment.
Changed in version 3.8: CancelledError is now a subclass of BaseException rather than Exception.
Apparently not, guess I tripped on that too
hongxiayang
pushed a commit
to hongxiayang/vllm
that referenced
this pull request
Feb 13, 2024
pi314ever
pushed a commit
to pi314ever/vllm
that referenced
this pull request
Apr 12, 2025
…vllm-project#1020) Twin PR: [135](HabanaAI/vllm-hpu-extension#135) APC will use our implementation of attn --------- Co-authored-by: Michał Kuligowski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a scene, while running
async for result in AsyncLLMEngine.generate(...)，if caller cancel the coroutine at high level (ex. Sanic Framework cancel the coroutine which handling request when transport connection is disconnected), whether to need catchingasyncio.CancelledErrorand aborting token generating in background?