[Core] Use uvloop with zmq-decoupled front-end #7570
Conversation
It's faster than the native python asyncio event loop implementation. uvicorn already uses this but we need to enable it explicitly when starting the zmq RPC server based engine.
|
👋 Hi! Thank you for contributing to the vLLM project. Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge). To run full CI, you can do one of these:
🚀 |
| from typing import Any, Coroutine | ||
|
|
||
| import cloudpickle | ||
| import uvloop |
There was a problem hiding this comment.
Is it already required by another package? It wouldn't hurt to explicitly add it to requirements-common.txt
There was a problem hiding this comment.
It's already installed via uvloop which is a dependency of fastapi i beleive
There was a problem hiding this comment.
fastapi recently excluded uvicorn as a dependency but uvicorn[standard] (which depends on uvloop) is already in vllm's requirements-common.txt so it is safe for now.
Signed-off-by: Alvant <[email protected]>
Signed-off-by: LeiWang1999 <[email protected]>
4 participants
It's faster than the native python asyncio event loop implementation. uvicorn already uses this but we need to enable it explicitly when starting the zmq RPC server based engine.