Skip to content

Commit 0005bbd

Browse files
njhillAlvant
authored andcommitted
[BugFix] Ensure worker model loop is always stopped at the right time (vllm-project#5987)
Signed-off-by: Alvant <[email protected]>
1 parent 95f3845 commit 0005bbd

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/engine/llm_engine.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -882,7 +882,7 @@ def step(self) -> List[Union[RequestOutput, EmbeddingRequestOutput]]:
882882
# Tracing
883883
self.do_tracing(scheduler_outputs)
884884

885-
if not request_outputs:
885+
if not self.has_unfinished_requests():
886886
# Stop the execute model loop in parallel workers until there are
887887
# more requests to process. This avoids waiting indefinitely in
888888
# torch.distributed ops which may otherwise timeout, and unblocks

0 commit comments

Comments
 (0)