Skip to content

Commit ce74366

Browse files
jikunshangcharlifu
authored andcommitted
[XPU] Fix xpu model runner call torch.cuda APIs (vllm-project#25011)
Signed-off-by: Kunshang Ji <[email protected]> Signed-off-by: charlifu <[email protected]>
1 parent 0f3fc8e commit ce74366

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

vllm/v1/worker/xpu_model_runner.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,12 @@ def __init__(self, *args, **kwargs) -> None:
4545
self.synchronize = lambda: None
4646

4747
try:
48-
# replace cuda Event with xpu Event, this should work by default
48+
# replace cuda APIs with xpu APIs, this should work by default
4949
torch.cuda.Event = torch.xpu.Event
50+
torch.cuda.Stream = torch.xpu.Stream
51+
torch.cuda.default_stream = torch.xpu.current_stream
52+
torch.cuda.current_stream = torch.xpu.current_stream
53+
torch.cuda.stream = torch.xpu.stream
5054
yield
5155
finally:
5256
# if anything goes wrong, just patch it with a placeholder

0 commit comments

Comments
 (0)