Skip to content

Commit a6f2b75

Browse files
robertgshaw2-redhatLeiWang1999
authored andcommitted
[BugFix][Core] Multistep Fix Crash on Request Cancellation (vllm-project#8059)
Signed-off-by: LeiWang1999 <[email protected]>
1 parent b6bf7ba commit a6f2b75

File tree

1 file changed

+7
-1
lines changed

1 file changed

+7
-1
lines changed

vllm/engine/output_processor/multi_step.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,9 +88,15 @@ def process_outputs(self,
8888
# TODO: Add support for async if necessary
8989
assert not is_async
9090

91+
# Sequences can be in RUNNING or FINISHED_ABORTED state
92+
# once scheduled, as a sequence is moved to FINSIHED_ABORTED
93+
# if a client disconnects from the api server.
9194
seqs = sequence_group.get_seqs(status=SequenceStatus.RUNNING)
95+
if seqs is None:
96+
seqs = sequence_group.get_seqs(
97+
status=SequenceStatus.FINISHED_ABORTED)
9298

93-
assert seqs, "expected running sequences"
99+
assert seqs, "Expected RUNNING or FINISHED_ABORTED sequences"
94100
assert len(seqs) == 1, (
95101
"Beam search not supported in multi-step decoding.")
96102
seq = seqs[0]

0 commit comments

Comments
 (0)