[BugFix] Overhaul async request cancellation

[njhill]

There are a number of problems currently with how request cancellation works upon client disconnection in the openai api server front-end and AsyncLLMEngine:

• Abort logic is missing for streaming chat requests - these continue to run after the client disconnects
• For multi-prompt completion requests, only one of the n running sequences actually gets aborted
• Cancelled requests that are queued will still be pre-filled before getting aborted
• Some of the existing async generator logic may be contributing to clean-shutdown garbage collection issues

This is a problem for production resilience and has a compounding affect when the server is overloaded and client requests time-out, with the server continuing to do useless work.

This PR reworks how the cancellation is propagated to make it more robust and consistent:

• Use native asyncio task/generator cancellation propagation as much as possible. This includes making use of explicitly cancellable async generators (via asyncio.aclose()) rather than async iterators in most cases
• Re-implement merge_async_iterators to encapsulate polling for disconnection, even before any results have been produced (while the request is queued)
• Add equivalent iterate_with_cancellation function used for the single-prompt request cases
• In AsyncLLMEngine differentiate between cancelled requests and those that finish normally (generator returned from the generate() will now raise a CancelledError in the former case)
• Don't abort requests that have already completed normally. This avoids some redundant work in the engine to look up sequence groups that have already gone

I also plan to add some new tests to cover these various cases.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

• Comment /ready on the PR
• Add ready label to the PR
• Enable auto-merge.

🚀

[robertgshaw2-redhat]

Nice

[DarkLight1337]

LGTM, sorry for keeping you waiting.