[Feature]: Can it support avx cpu's like xeon E5-2670v2 #13662
Description
🚀 The feature, motivation and pitch
[Hardware][Intel] Support CPU inference with AVX2 ISA #5452
In issue #5452, the expert DamonFool has implemented the ability to run vllm on CPUs with the AVX2 instruction set. However, my machine is an older Xeon E5-2670v2, which only supports AVX. Could anyone help modify the code to make it compatible with AVX? Thank you!
4.499 running build_ext
5.040 -- The CXX compiler identification is GNU 12.3.0
5.067 -- Detecting CXX compiler ABI info
5.270 -- Detecting CXX compiler ABI info - done
5.309 -- Check for working CXX compiler: /usr/bin/c++ - skipped
5.310 -- Detecting CXX compile features
5.311 -- Detecting CXX compile features - done
5.368 -- Build type: RelWithDebInfo
5.368 -- Target device: cpu
5.672 -- Found Python: /usr/bin/python3 (found version "3.10.12") found components: Interpreter Development.Module Development.SABIModule
5.672 -- Found python matching: /usr/bin/python3.
8.417 CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
8.417 static library kineto_LIBRARY-NOTFOUND not found.
8.417 Call Stack (most recent call first):
8.417 /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)
8.417 CMakeLists.txt:81 (find_package)
8.417
8.417
8.418 -- Found Torch: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch.so
8.441 CMake Warning at cmake/cpu_extension.cmake:133 (message):
8.441 vLLM CPU backend requires AVX512, AVX2, Power9+ ISA or ARMv8 support.
8.441 Call Stack (most recent call first):
8.441 CMakeLists.txt:89 (include)
8.441
8.441
8.441 -- CPU extension compile flags: -mf16c;-fopenmp;-DVLLM_CPU_EXTENSION
8.442 -- Enabling C extension.
8.442 -- Configuring done (3.6s)
8.479 -- Generating done (0.0s)
8.480 -- Build files have been written to: /workspace/vllm/build/temp.linux-x86_64-cpython-310
46.75 [1/8] Building CXX object CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o
46.75 FAILED: CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o
46.75 ccache /usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/workspace/vllm/csrc -isystem /usr/include/python3.10 -isystem /usr/local/lib/python3.10/dist-packages/torch/include -isystem /usr/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++17 -fPIC -mf16c -fopenmp -DVLLM_CPU_EXTENSION -D_GLIBCXX_USE_CXX11_ABI=0 -MD -MT CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o -c /workspace/vllm/csrc/cpu/layernorm.cpp
46.75 In file included from /workspace/vllm/csrc/cpu/cpu_types.hpp:6,
46.75 from /workspace/vllm/csrc/cpu/layernorm.cpp:1:
46.75 /workspace/vllm/csrc/cpu/cpu_types_x86.hpp:9:15: error: static assertion failed: AVX2 must be supported for the current implementation.
46.75 9 | static_assert(false, "AVX2 must be supported for the current implementation.");
46.75 | ^~~~~
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.