[Installation]: cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’

[Liguoz]

Your current environment

The output of `python collect_env.py`

-- CPU extension compile flags: -fopenmp;-DVLLM_CPU_EXTENSION;-march=armv8.2-a+dotprod+fp16
-- Enabling C extension.
-- Configuring done (3.5s)
-- Generating done (0.0s)
-- Build files have been written to: /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/build/temp.linux-aarch64-cpython-310
[1/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/activation.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/activation.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/activation.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/activation.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/activation.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/activation.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[2/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/cache.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/cache.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/cache.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/cache.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/cache.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/cache.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[3/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/utils.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/utils.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/utils.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/utils.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/utils.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/utils.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[4/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/mla_decode.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/mla_decode.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/mla_decode.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/mla_decode.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/mla_decode.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/mla_decode.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[5/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/attention.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/attention.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/attention.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/attention.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/attention.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/attention.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[6/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/torch_bindings.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/torch_bindings.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/torch_bindings.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/torch_bindings.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/torch_bindings.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/torch_bindings.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[7/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/pos_encoding.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
[8/9] Building CXX object CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o
FAILED: CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o
/usr/bin/c++ -DPy_LIMITED_API=3 -DTORCH_EXTENSION_NAME=_C -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_RPC -DUSE_TENSORPIPE -D_C_EXPORTS -I/tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc -isystem /home/ma-user/anaconda3/envs/py310/include/python3.10 -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include -isystem /tmp/pip-build-env-mwllr031/overlay/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -O2 -g -DNDEBUG -std=gnu++1z -fPIC -fopenmp -DVLLM_CPU_EXTENSION -march=armv8.2-a+dotprod+fp16 -D_GLIBCXX_USE_CXX11_ABI=1 -MD -MT CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o -MF CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o.d -o CMakeFiles/_C.dir/csrc/cpu/layernorm.cpp.o -c /tmp/pip-install-b261gxij/vllm_e99d65c51e5c4cea948384f47668e6c3/csrc/cpu/layernorm.cpp
cc1plus: error: invalid feature modifier in ‘-march=armv8.2-a+dotprod+fp16’
ninja: build stopped: subcommand failed.
Traceback (most recent call last):

How you are installing vllm

pip install -vvv vllm

pip install vllm
packages/vllm/0.8.5.post1/vllm-0.8.5.post1.tar.gz

ARMv8平台安装报错

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[Liguoz]

ARM平台，

[Minamoto25]

same problem, is there a solution?

[Minamoto25]

可以查看一下你的g++版本，切换更高版本的g++后，在我们的环境下通过编译