Skip to content

[Bug]: torch.ops._C.silu_and_mul does not exist #15941

@xuji1122

Description

@xuji1122

Your current environment

macOS cpu torch.ops._C.silu_and_mul does not exist

Image

stack trace:

[rank0]: Traceback (most recent call last):
[rank0]: File "/Users/xuji/programs/vllm_projects_2/example.py", line 10, in
[rank0]: llm = LLM(model="deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",trust_remote_code=True)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 1096, in inner
[rank0]: return fn(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/entrypoints/llm.py", line 243, in init
[rank0]: self.llm_engine = LLMEngine.from_engine_args(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 521, in from_engine_args
[rank0]: return engine_cls.from_vllm_config(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 497, in from_vllm_config
[rank0]: return cls(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/engine/llm_engine.py", line 281, in init
[rank0]: self.model_executor = executor_class(vllm_config=vllm_config, )
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 286, in init
[rank0]: super().init(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/executor_base.py", line 52, in init
[rank0]: self._init_executor()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 125, in _init_executor
[rank0]: self._run_workers("load_model",
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/executor/mp_distributed_executor.py", line 185, in _run_workers
[rank0]: driver_worker_output = run_method(self.driver_worker, sent_method,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/utils.py", line 2347, in run_method
[rank0]: return func(*args, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_worker.py", line 226, in load_model
[rank0]: self.model_runner.load_model()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/worker/cpu_model_runner.py", line 491, in load_model
[rank0]: self.model = get_model(vllm_config=self.vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/init.py", line 14, in get_model
[rank0]: return loader.load_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 441, in load_model
[rank0]: model = _initialize_model(vllm_config=vllm_config)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/model_loader/loader.py", line 127, in _initialize_model
[rank0]: return model_class(vllm_config=vllm_config, prefix=prefix)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 431, in init
[rank0]: self.model = Qwen2Model(vllm_config=vllm_config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/compilation/decorators.py", line 151, in init
[rank0]: old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 300, in init
[rank0]: self.start_layer, self.end_layer, self.layers = make_layers(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 609, in make_layers
[rank0]: [PPMissingLayer() for _ in range(start_layer)] + [
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/utils.py", line 610, in
[rank0]: maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}"))
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 302, in
[rank0]: lambda prefix: Qwen2DecoderLayer(config=config,
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 218, in init
[rank0]: self.mlp = Qwen2MLP(
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/models/qwen2.py", line 92, in init
[rank0]: self.act_fn = SiluAndMul()
[rank0]: File "/Users/xuji/programs_2/vllm/vllm/model_executor/layers/activation.py", line 68, in init
[rank0]: self.op = torch.ops._C.silu_and_mul
[rank0]: File "/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/torch/_ops.py", line 1232, in getattr
[rank0]: raise AttributeError(
[rank0]: AttributeError: '_OpNamespace' '_C' object has no attribute 'silu_and_mul'

Your output of `python collect_env.py` here

INFO 04-02 17:38:05 [init.py:239] Automatically detected platform cpu.
Collecting environment information...
PyTorch version: 2.6.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 15.3.2 (arm64)
GCC version: Could not collect
Clang version: 17.0.0 (clang-1700.0.13.3)
CMake version: version 4.0.0
Libc version: N/A

Python version: 3.9.6 (default, Apr 1 2025, 20:27:55) [Clang 17.0.0 (clang-1700.0.13.3)] (64-bit runtime)
Python platform: macOS-15.3.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==2.0.2
[pip3] optree==0.14.1
[pip3] pyzmq==26.3.0
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.50.3
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.3.dev205+g25293780
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect

VLLM_USE_MODELSCOPE=True
VLLM_TARGET_DEVICE=cpu
LD_LIBRARY_PATH=/Users/xuji/.pyenv/versions/3.9.6/lib/python3.9/site-packages/cv2/../../lib:
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1

🐛 Describe the bug

Image

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions