-
-
Notifications
You must be signed in to change notification settings - Fork 10.7k
Description
Your current environment
macOS cpu torch.ops._C.silu_and_mul does not exist
stack trace:
[rank0]: Traceback (most recent call last):
[rank0]: File "/Users/xuji/programs/vllm_projects_2/example.py", line 10, in
[rank0]: llm = LLM(model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",trust_remote_code=True)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 1096, in inner
[rank0]: return fn(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/entrypoints/llm.py", line 243, in init
[rank0]: self.llm_engine = LLMEngine.from_engine_args(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 521, in from_engine_args
[rank0]: return engine_cls.from_vllm_config(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 497, in from_vllm_config
[rank0]: return cls(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 281, in init
[rank0]: self.model_executor = executor_class(vllm_config=vllm_config, )
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 286, in init
[rank0]: super().init(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 52, in init
[rank0]: self._init_executor()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 125, in _init_executor
[rank0]: self._run_workers("load_model",
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 185, in _run_workers
[rank0]: driver_worker_output = run_method(self.driver_worker, sent_method,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 2347, in run_method
[rank0]: return func(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_worker.py", line 226, in load_model
[rank0]: self.model_runner.load_model()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_model_runner.py", line 491, in load_model
[rank0]: self.model = get_model(vllm_config=self.vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/init.py", line 14, in get_model
[rank0]: return loader.load_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 441, in load_model
[rank0]: model = _initialize_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 127, in _initialize_model
[rank0]: return model_class(vllm_config=vllm_config, prefix=prefix)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 431, in init
[rank0]: self.model = Qwen2Model(vllm_config=vllm_config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/compilation/decorators.py", line 151, in init
[rank0]: old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 300, in init
[rank0]: self.start_layer, self.end_layer, self.layers = make_layers(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 609, in make_layers
[rank0]: [PPMissingLayer() for _ in range(start_layer)] + [
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 610, in
[rank0]: maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 302, in
[rank0]: lambda prefix: Qwen2DecoderLayer(config=config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 218, in init
[rank0]: self.mlp = Qwen2MLP(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 92, in init
[rank0]: self.act_fn = SiluAndMul()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/layers/activation.py", line 68, in init
[rank0]: self.op = torch.ops._C.silu_and_mul
[rank0]: File "/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/torch/_ops.py", line 1232, in getattr
[rank0]: raise AttributeError(
[rank0]: AttributeError: '_OpNamespace' '_C' object has no attribute 'silu_and_mul'
Your output of `python collect_env.py` here
INFO 04-02 17:38:05 [init.py:239] Automatically detected platform cpu.
Collecting environment information...
PyTorch version: 2.6.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 15.3.2 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.0.13.3)
CMake version: version 4.0.0
Libc version: N/A
Python version: 3.9.6 (default, Apr 1 2025, 20:27:55) [Clang 17.0.0 (clang-1700.0.13.3)] (64-bit runtime)
Python platform: macOS-15.3.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M1 Pro
Versions of relevant libraries:
[pip3] numpy==2.0.2
[pip3] optree==0.14.1
[pip3] pyzmq==26.3.0
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.50.3
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.3.dev205+g25293780
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect
VLLM_USE_MODELSCOPE=True
VLLM_TARGET_DEVICE=cpu
LD_LIBRARY_PATH=/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/cv2/../../lib:
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1
[rank0]: File "/Users/xuji/programs/vllm_projects_2/example.py", line 10, in
[rank0]: llm = LLM(model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",trust_remote_code=True)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 1096, in inner
[rank0]: return fn(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/entrypoints/llm.py", line 243, in init
[rank0]: self.llm_engine = LLMEngine.from_engine_args(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 521, in from_engine_args
[rank0]: return engine_cls.from_vllm_config(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 497, in from_vllm_config
[rank0]: return cls(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 281, in init
[rank0]: self.model_executor = executor_class(vllm_config=vllm_config, )
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 286, in init
[rank0]: super().init(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 52, in init
[rank0]: self._init_executor()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 125, in _init_executor
[rank0]: self._run_workers("load_model",
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 185, in _run_workers
[rank0]: driver_worker_output = run_method(self.driver_worker, sent_method,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 2347, in run_method
[rank0]: return func(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_worker.py", line 226, in load_model
[rank0]: self.model_runner.load_model()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_model_runner.py", line 491, in load_model
[rank0]: self.model = get_model(vllm_config=self.vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/init.py", line 14, in get_model
[rank0]: return loader.load_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 441, in load_model
[rank0]: model = _initialize_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 127, in _initialize_model
[rank0]: return model_class(vllm_config=vllm_config, prefix=prefix)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 431, in init
[rank0]: self.model = Qwen2Model(vllm_config=vllm_config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/compilation/decorators.py", line 151, in init
[rank0]: old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 300, in init
[rank0]: self.start_layer, self.end_layer, self.layers = make_layers(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 609, in make_layers
[rank0]: [PPMissingLayer() for _ in range(start_layer)] + [
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 610, in
[rank0]: maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 302, in
[rank0]: lambda prefix: Qwen2DecoderLayer(config=config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 218, in init
[rank0]: self.mlp = Qwen2MLP(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 92, in init
[rank0]: self.act_fn = SiluAndMul()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/layers/activation.py", line 68, in init
[rank0]: self.op = torch.ops._C.silu_and_mul
[rank0]: File "/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/torch/_ops.py", line 1232, in getattr
[rank0]: raise AttributeError(
[rank0]: AttributeError: '_OpNamespace' '_C' object has no attribute 'silu_and_mul'
Your output of `python collect_env.py` here
Collecting environment information...
PyTorch version: 2.6.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.0.13.3)
CMake version: version 4.0.0
Libc version: N/A
Python platform: macOS-15.3.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Apple M1 Pro
[pip3] numpy==2.0.2
[pip3] optree==0.14.1
[pip3] pyzmq==26.3.0
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.50.3
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.3.dev205+g25293780
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect
VLLM_TARGET_DEVICE=cpu
LD_LIBRARY_PATH=/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/cv2/../../lib:
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1
🐛 Describe the bug
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.