Build failure due to CUDA version mismatch

[WoosukKwon]

I failed to build the system with the latest NVIDIA PyTorch docker image. The reason is PyTorch installed by pip is built with CUDA 11.7 while the container uses CUDA 12.1.

RuntimeError:
The detected CUDA version (12.1) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.

[Joejoequ]

Same Issue Here. It looks like it did not use CUDA 11.8 in the conda environmnent.
CUDA 11.8 Python 3.8.16 Nvidia A100 80G ubuntu

File "/tmp/pip-build-env-_5k66uxz/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 387, in _check_cuda_version
raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
RuntimeError:
The detected CUDA version (12.0) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.

(vllm) x@x:~/xx$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

[WoosukKwon]

@Joejoequ Thanks for reporting it! I think in your case, the problem can be easily solved by installing CUDA 11.8 version of PyTorch:

pip3 install torch --index-url https://download.pytorch.org/whl/cu118

[Joejoequ]

@WoosukKwon Thanks for the quick response! But the issue still exists after using the command above. It is weird that the detected version is 12.0 since the cudatookit installed in the env is 11.8. Looks like it will access Cuda outside the env.

[Joejoequ]

@WoosukKwon Thanks for the quick response! But the issue still exists after using the command above. It is weird that the detected version is 12.0 since the cudatookit installed in the env is 11.8. Looks like it will access Cuda outside the env.

Finally i installed successfully by changing module env and use "module load" on linux server.

[mosheduminer]

I have the same problem. Pytorch using cuda 11.8 installed, yet I get this error when installing.

The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.

[DavidPeleg6]

im also having the same issue:
nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

and i have installed nightly version of pytorch with cuda 12.1 support

[antonpolishko]

nvcr.io/nvidia/pytorch:22.12-py3 image is the last one with CUDA 11.8 according to compatibility matrix. Then images switched to CUDA 12+

[Ikkyu321]

@WoosukKwon Thanks for the quick response! But the issue still exists after using the command above. It is weird that the detected version is 12.0 since the cudatookit installed in the env is 11.8. Looks like it will access Cuda outside the env.

Finally i installed successfully by changing module env and use "module load" on linux server.

@Joejoequ I got the same problem. Can you show me how to solve this？

[valentin-fngr]

I have the same problem. Pytorch using cuda 11.8 installed, yet I get this error when installing.

The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.

Option 1 : You have cuda 12.1 therefore you should simply uninstall the current binaries of pytorch that you have and then reinstall it using :

pip3 install torch torchvision torchaudio

Option 2 : If you do not want to use the cuda 12.1 that you have installed, you can use another version of cuda (11.7, 11.8, ...).
First you need to uninstall your current cuda (https://stackoverflow.com/a/56827564) and then select the one you want on the cuda install website

[xunfeng1980]

same problem:
The detected CUDA version (12.3) mismatches the version that was used to compile
PyTorch (11.7). Please make sure to use the same CUDA versions.

[Kkkassini]

Possible to use cuda 11.7? there's other services that require 11.7

[Lumingous]

I'm having the same problem. I've re-installed my pytorch to support cuda 11.8. Don't know why still shows this error