Skip to content

Commit 6f0348c

Browse files
njhilljimpang
authored andcommitted
[BugFix] Ensure worker model loop is always stopped at the right time (vllm-project#5987)
1 parent 951df78 commit 6f0348c

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/engine/llm_engine.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -838,7 +838,7 @@ def step(self) -> List[Union[RequestOutput, EmbeddingRequestOutput]]:
838838
# Tracing
839839
self.do_tracing(scheduler_outputs)
840840

841-
if not request_outputs:
841+
if not self.has_unfinished_requests():
842842
# Stop the execute model loop in parallel workers until there are
843843
# more requests to process. This avoids waiting indefinitely in
844844
# torch.distributed ops which may otherwise timeout, and unblocks

0 commit comments

Comments
 (0)