Skip to content

Commit 7cf1a78

Browse files
committed
fix the prepare_model_input issue during decoding.
1 parent 676fe3c commit 7cf1a78

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/worker/neuron_model_runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -247,7 +247,7 @@ def prepare_model_input(
247247
else:
248248
(input_tokens, input_positions,
249249
input_block_ids) = self._prepare_decode(seq_group_metadata_list)
250-
seq_lens = []
250+
seq_lens = None
251251
sampling_metadata = SamplingMetadata.prepare(
252252
seq_group_metadata_list,
253253
seq_lens,

0 commit comments

Comments
 (0)