Can't build vLLM from Docker due to AWQ's minimum architecture requirements - TORCH_CUDA_ARCH_LIST does not help

[TheBloke]

@casper-hansen @WoosukKwon

I'm trying to build a test vLLM Docker container with the latest vLLM commit.

My Docker container has this:

ARG CUDA_VERSION="11.8.0"
ARG CUDNN_VERSION="8"
ARG UBUNTU_VERSION="22.04"

# Base NVidia CUDA Ubuntu image
FROM nvidia/cuda:$CUDA_VERSION-cudnn$CUDNN_VERSION-devel-ubuntu$UBUNTU_VERSION AS base

ENV PATH="/usr/local/cuda/bin:${PATH}"

ENV TORCH_CUDA_ARCH_LIST="8.0;8.6+PTX;8.9;9.0"
RUN pip3 install torch --index-url https://download.pytorch.org/whl/cu118

RUN git clone https://github.com/vllm-project/vllm && \
    cd vllm && \
    git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 && \
    pip3 install -r requirements.txt && \
    pip3 install -e . && \
    pip3 install --no-cache-dir huggingface-hub hf_transfer && \
    pip3 cache purge

This structure worked fine to build vLLM before, and to build other servers/apps that use Torch.

The line ENV TORCH_CUDA_ARCH_LIST="8.0;8.6+PTX;8.9;9.0" usually works fine to ensure that an app can built in a Docker even though Docker cannot see any GPU.

But trying to build vLLM fails with this:

#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 892; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 892; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 900; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 900; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 908; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 908; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 916; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 916; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 924; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 924; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 928; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 932; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 936; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 940; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 944; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 948; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 952; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 956; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 964; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 964; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 972; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 972; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 980; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 980; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 988; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 988; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 996; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 996; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1000; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1004; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1008; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1012; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1016; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1020; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1024; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1028; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1834; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1834; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1842; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1842; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1850; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1850; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1854; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1858; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1862; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1866; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1874; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1874; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1882; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1882; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1890; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1890; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1894; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1898; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1902; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1906; error   : Feature '.m16n8k16' requires .target sm_80 or higher

Any ideas how I can avoid this error in the Docker? No GPU is available at the time of building, but the GPU will be available at runtime. I guess I need some flag to tell the build to build for sm_80 and higher, like TORCH_CUDA_ARCH_LIST?

Thanks

[TheBloke]

Here's the full Docker build output:

#11 [ 6/20] RUN git clone https://github.com/vllm-project/vllm &&     cd vllm &&     git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 &&     pip3 install -r requirements.txt &&     pip3 install -e . &&     pip3 install --no-cache-dir huggingface-hub hf_transfer &&     pip3 cache purge
#11 0.374 Cloning into 'vllm'...
#11 1.241 Note: switching to 'ff36139ffc66294c19b503c1e52dc42c2cd265f6'.
#11 1.241 
#11 1.241 You are in 'detached HEAD' state. You can look around, make experimental
#11 1.241 changes and commit them, and you can discard any commits you make in this
#11 1.241 state without impacting any branches by switching back to a branch.
#11 1.241 
#11 1.241 If you want to create a new branch to retain commits you create, you may
#11 1.241 do so (now or later) by using -c with the switch command. Example:
#11 1.241 
#11 1.241   git switch -c <new-branch-name>
#11 1.241 
#11 1.241 Or undo this operation with:
#11 1.241 
#11 1.241   git switch -
#11 1.241 
#11 1.241 Turn off this advice by setting config variable advice.detachedHead to false
#11 1.241 
#11 1.241 HEAD is now at ff36139 Remove AsyncLLMEngine busy loop, shield background task (#1059)
#11 1.494 Defaulting to user installation because normal site-packages is not writeable
#11 1.661 Collecting ninja (from -r requirements.txt (line 1))
#11 1.730   Downloading ninja-1.11.1-py2.py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (145 kB)
#11 1.771      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 146.0/146.0 kB 3.6 MB/s eta 0:00:00
#11 1.921 Collecting psutil (from -r requirements.txt (line 2))
#11 1.925   Downloading psutil-5.9.5-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
#11 1.943      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 282.1/282.1 kB 17.1 MB/s eta 0:00:00
#11 2.101 Collecting ray>=2.5.1 (from -r requirements.txt (line 3))
#11 2.101   Obtaining dependency information for ray>=2.5.1 from https://files.pythonhosted.org/packages/d5/d1/50c97d98a8ead5660b832136b7cfcab0b4ed02099bf0fa6e7b18beaf3ba8/ray-2.6.3-cp310-cp310-manylinux2014_x86_64.whl.metadata
#11 2.106   Downloading ray-2.6.3-cp310-cp310-manylinux2014_x86_64.whl.metadata (12 kB)
#11 2.330 Collecting pandas (from -r requirements.txt (line 4))
#11 2.330   Obtaining dependency information for pandas from https://files.pythonhosted.org/packages/fb/4f/4a4372b2e24439f559b73318683486831d75e59544ae02bf8dec8dd6f48b/pandas-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 2.333   Downloading pandas-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
#11 2.436 Collecting sentencepiece (from -r requirements.txt (line 5))
#11 2.440   Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
#11 2.453      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 122.6 MB/s eta 0:00:00
#11 2.777 Collecting numpy (from -r requirements.txt (line 6))
#11 2.777   Obtaining dependency information for numpy from https://files.pythonhosted.org/packages/9b/5a/f265a1ba3641d16b5480a217a6aed08cceef09cd173b568cd5351053472a/numpy-1.26.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 2.780   Downloading numpy-1.26.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (58 kB)
#11 2.785      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.5/58.5 kB 17.1 MB/s eta 0:00:00
#11 2.788 Requirement already satisfied: torch>=2.0.0 in /home/vllm/.local/lib/python3.10/site-packages (from -r requirements.txt (line 7)) (2.0.1+cu118)
#11 2.843 Collecting transformers>=4.33.1 (from -r requirements.txt (line 8))
#11 2.843   Obtaining dependency information for transformers>=4.33.1 from https://files.pythonhosted.org/packages/1a/06/3817f9bb923437ead9a794f0ac0d03b8b5e0478ab112db4c413dd37c09da/transformers-4.33.2-py3-none-any.whl.metadata
#11 2.846   Downloading transformers-4.33.2-py3-none-any.whl.metadata (119 kB)
#11 2.851      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 119.9/119.9 kB 39.1 MB/s eta 0:00:00
#11 2.886 Collecting xformers>=0.0.21 (from -r requirements.txt (line 9))
#11 2.886   Obtaining dependency information for xformers>=0.0.21 from https://files.pythonhosted.org/packages/c7/b4/9f8bea4204f8482c9c9c64bcf86bd209ccfb2ebdb27e3590ef4c1e87b743/xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl.metadata
#11 2.889   Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl.metadata (1.0 kB)
#11 2.970 Collecting fastapi (from -r requirements.txt (line 10))
#11 2.970   Obtaining dependency information for fastapi from https://files.pythonhosted.org/packages/76/e5/ca411b260caa4e72f9ac5482f331fe74fd4eb5b97aa74d1d2806ccf07e2c/fastapi-0.103.1-py3-none-any.whl.metadata
#11 2.973   Downloading fastapi-0.103.1-py3-none-any.whl.metadata (24 kB)
#11 3.010 Collecting uvicorn (from -r requirements.txt (line 11))
#11 3.010   Obtaining dependency information for uvicorn from https://files.pythonhosted.org/packages/79/96/b0882a1c3f7ef3dd86879e041212ae5b62b4bd352320889231cc735a8e8f/uvicorn-0.23.2-py3-none-any.whl.metadata
#11 3.013   Downloading uvicorn-0.23.2-py3-none-any.whl.metadata (6.2 kB)
#11 3.233 Collecting pydantic<2 (from -r requirements.txt (line 12))
#11 3.233   Obtaining dependency information for pydantic<2 from https://files.pythonhosted.org/packages/bc/e0/0371e9b6c910afe502e5fe18cc94562bfd9399617c7b4f5b6e13c29115b3/pydantic-1.10.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 3.236   Downloading pydantic-1.10.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (149 kB)
#11 3.242      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.3/149.3 kB 46.1 MB/s eta 0:00:00
#11 3.383 Collecting click>=7.0 (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.384   Obtaining dependency information for click>=7.0 from https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
#11 3.387   Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
#11 3.390 Requirement already satisfied: filelock in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->-r requirements.txt (line 3)) (3.9.0)
#11 3.431 Collecting jsonschema (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.431   Obtaining dependency information for jsonschema from https://files.pythonhosted.org/packages/2b/ff/af59fd34bc4d7ac3e6e0cd1f3c10317d329b6c1aee179e8b24ad9a79fbac/jsonschema-4.19.0-py3-none-any.whl.metadata
#11 3.434   Downloading jsonschema-4.19.0-py3-none-any.whl.metadata (8.2 kB)
#11 3.528 Collecting msgpack<2.0.0,>=1.0.0 (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.532   Downloading msgpack-1.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (316 kB)
#11 3.538      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.8/316.8 kB 80.9 MB/s eta 0:00:00
#11 3.571 Collecting packaging (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.573   Downloading packaging-23.1-py3-none-any.whl (48 kB)
#11 3.577      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.9/48.9 kB 18.1 MB/s eta 0:00:00
#11 3.888 Collecting protobuf!=3.19.5,>=3.15.3 (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.888   Obtaining dependency information for protobuf!=3.19.5,>=3.15.3 from https://files.pythonhosted.org/packages/bb/c3/6a06208ecf0934ecaf509b51c52a6cf688586f54ae81ac65c56124571494/protobuf-4.24.3-cp37-abi3-manylinux2014_x86_64.whl.metadata
#11 3.891   Downloading protobuf-4.24.3-cp37-abi3-manylinux2014_x86_64.whl.metadata (540 bytes)
#11 3.959 Collecting pyyaml (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.959   Obtaining dependency information for pyyaml from https://files.pythonhosted.org/packages/29/61/bf33c6c85c55bc45a29eee3195848ff2d518d84735eb0e2d8cb42e0d285e/PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 3.963   Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
#11 3.980 Collecting aiosignal (from ray>=2.5.1->-r requirements.txt (line 3))
#11 3.983   Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
#11 4.075 Collecting frozenlist (from ray>=2.5.1->-r requirements.txt (line 3))
#11 4.075   Obtaining dependency information for frozenlist from https://files.pythonhosted.org/packages/1e/28/74b8b6451c89c070d34e753d8b65a1e4ce508a6808b18529f36e8c0e2184/frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 4.078   Downloading frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.2 kB)
#11 4.122 Collecting requests (from ray>=2.5.1->-r requirements.txt (line 3))
#11 4.122   Obtaining dependency information for requests from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
#11 4.125   Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
#11 4.993 Collecting grpcio>=1.42.0 (from ray>=2.5.1->-r requirements.txt (line 3))
#11 4.993   Obtaining dependency information for grpcio>=1.42.0 from https://files.pythonhosted.org/packages/28/94/63bef715d0afea96662c66075d72249bf9b55b018b91b6937045bdc1470c/grpcio-1.58.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 4.997   Downloading grpcio-1.58.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.0 kB)
#11 5.106 Collecting python-dateutil>=2.8.2 (from pandas->-r requirements.txt (line 4))
#11 5.108   Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#11 5.114      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 75.1 MB/s eta 0:00:00
#11 5.174 Collecting pytz>=2020.1 (from pandas->-r requirements.txt (line 4))
#11 5.174   Obtaining dependency information for pytz>=2020.1 from https://files.pythonhosted.org/packages/32/4d/aaf7eff5deb402fd9a24a1449a8119f00d74ae9c2efa79f8ef9994261fc2/pytz-2023.3.post1-py2.py3-none-any.whl.metadata
#11 5.178   Downloading pytz-2023.3.post1-py2.py3-none-any.whl.metadata (22 kB)
#11 5.202 Collecting tzdata>=2022.1 (from pandas->-r requirements.txt (line 4))
#11 5.204   Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
#11 5.211      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 341.8/341.8 kB 81.3 MB/s eta 0:00:00
#11 5.228 Requirement already satisfied: typing-extensions in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->-r requirements.txt (line 7)) (4.4.0)
#11 5.228 Requirement already satisfied: sympy in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->-r requirements.txt (line 7)) (1.11.1)
#11 5.229 Requirement already satisfied: networkx in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->-r requirements.txt (line 7)) (3.0)
#11 5.229 Requirement already satisfied: jinja2 in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->-r requirements.txt (line 7)) (3.1.2)
#11 5.231 Requirement already satisfied: triton==2.0.0 in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->-r requirements.txt (line 7)) (2.0.0)
#11 5.239 Requirement already satisfied: cmake in /home/vllm/.local/lib/python3.10/site-packages (from triton==2.0.0->torch>=2.0.0->-r requirements.txt (line 7)) (3.25.0)
#11 5.240 Requirement already satisfied: lit in /home/vllm/.local/lib/python3.10/site-packages (from triton==2.0.0->torch>=2.0.0->-r requirements.txt (line 7)) (15.0.7)
#11 5.701 Collecting huggingface-hub<1.0,>=0.15.1 (from transformers>=4.33.1->-r requirements.txt (line 8))
#11 5.701   Obtaining dependency information for huggingface-hub<1.0,>=0.15.1 from https://files.pythonhosted.org/packages/50/9d/5eac2733606df7d164b951b14cd76b056e530af96c881aaec89383bdbe45/huggingface_hub-0.17.1-py3-none-any.whl.metadata
#11 5.704   Downloading huggingface_hub-0.17.1-py3-none-any.whl.metadata (13 kB)
#11 6.400 Collecting regex!=2019.12.17 (from transformers>=4.33.1->-r requirements.txt (line 8))
#11 6.400   Obtaining dependency information for regex!=2019.12.17 from https://files.pythonhosted.org/packages/d1/df/460ca6171a8494fcf37af43f52f6fac23e38784bb4a26563f6fa01ef6faf/regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 6.404   Downloading regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
#11 6.409      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 12.4 MB/s eta 0:00:00
#11 6.738 Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers>=4.33.1->-r requirements.txt (line 8))
#11 6.743   Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
#11 6.797      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 149.6 MB/s eta 0:00:00
#11 6.922 Collecting safetensors>=0.3.1 (from transformers>=4.33.1->-r requirements.txt (line 8))
#11 6.922   Obtaining dependency information for safetensors>=0.3.1 from https://files.pythonhosted.org/packages/6c/f0/c17bbdb1e5f9dab29d44cade445135789f75f8f08ea2728d04493ea8412b/safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 6.926   Downloading safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.7 kB)
#11 7.001 Collecting tqdm>=4.27 (from transformers>=4.33.1->-r requirements.txt (line 8))
#11 7.001   Obtaining dependency information for tqdm>=4.27 from https://files.pythonhosted.org/packages/00/e5/f12a80907d0884e6dff9c16d0c0114d81b8cd07dc3ae54c5e962cc83037e/tqdm-4.66.1-py3-none-any.whl.metadata
#11 7.004   Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
#11 7.008      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 kB 20.0 MB/s eta 0:00:00
#11 7.057 Collecting anyio<4.0.0,>=3.7.1 (from fastapi->-r requirements.txt (line 10))
#11 7.057   Obtaining dependency information for anyio<4.0.0,>=3.7.1 from https://files.pythonhosted.org/packages/19/24/44299477fe7dcc9cb58d0a57d5a7588d6af2ff403fdd2d47a246c91a3246/anyio-3.7.1-py3-none-any.whl.metadata
#11 7.060   Downloading anyio-3.7.1-py3-none-any.whl.metadata (4.7 kB)
#11 7.139 Collecting starlette<0.28.0,>=0.27.0 (from fastapi->-r requirements.txt (line 10))
#11 7.139   Obtaining dependency information for starlette<0.28.0,>=0.27.0 from https://files.pythonhosted.org/packages/58/f8/e2cca22387965584a409795913b774235752be4176d276714e15e1a58884/starlette-0.27.0-py3-none-any.whl.metadata
#11 7.141   Downloading starlette-0.27.0-py3-none-any.whl.metadata (5.8 kB)
#11 7.165 Collecting typing-extensions (from torch>=2.0.0->-r requirements.txt (line 7))
#11 7.166   Obtaining dependency information for typing-extensions from https://files.pythonhosted.org/packages/ec/6b/63cc3df74987c36fe26157ee12e09e8f9db4de771e0f3404263117e75b95/typing_extensions-4.7.1-py3-none-any.whl.metadata
#11 7.169   Downloading typing_extensions-4.7.1-py3-none-any.whl.metadata (3.1 kB)
#11 7.199 Collecting h11>=0.8 (from uvicorn->-r requirements.txt (line 11))
#11 7.205   Downloading h11-0.14.0-py3-none-any.whl (58 kB)
#11 7.210      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 21.8 MB/s eta 0:00:00
#11 7.261 Collecting idna>=2.8 (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 10))
#11 7.264   Downloading idna-3.4-py3-none-any.whl (61 kB)
#11 7.269      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 15.0 MB/s eta 0:00:00
#11 7.284 Collecting sniffio>=1.1 (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 10))
#11 7.286   Downloading sniffio-1.3.0-py3-none-any.whl (10 kB)
#11 7.307 Collecting exceptiongroup (from anyio<4.0.0,>=3.7.1->fastapi->-r requirements.txt (line 10))
#11 7.307   Obtaining dependency information for exceptiongroup from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
#11 7.309   Downloading exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
#11 7.454 Collecting fsspec (from huggingface-hub<1.0,>=0.15.1->transformers>=4.33.1->-r requirements.txt (line 8))
#11 7.454   Obtaining dependency information for fsspec from https://files.pythonhosted.org/packages/6a/af/c673e8c663e17bd4fb201a6f029153ad5d7023aa4442d81c7987743db379/fsspec-2023.9.1-py3-none-any.whl.metadata
#11 7.457   Downloading fsspec-2023.9.1-py3-none-any.whl.metadata (6.7 kB)
#11 7.514 Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas->-r requirements.txt (line 4))
#11 7.517   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
#11 7.655 Requirement already satisfied: MarkupSafe>=2.0 in /home/vllm/.local/lib/python3.10/site-packages (from jinja2->torch>=2.0.0->-r requirements.txt (line 7)) (2.1.2)
#11 7.697 Collecting attrs>=22.2.0 (from jsonschema->ray>=2.5.1->-r requirements.txt (line 3))
#11 7.699   Downloading attrs-23.1.0-py3-none-any.whl (61 kB)
#11 7.704      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.2/61.2 kB 22.5 MB/s eta 0:00:00
#11 7.723 Collecting jsonschema-specifications>=2023.03.6 (from jsonschema->ray>=2.5.1->-r requirements.txt (line 3))
#11 7.724   Obtaining dependency information for jsonschema-specifications>=2023.03.6 from https://files.pythonhosted.org/packages/1c/24/83349ac2189cc2435e84da3f69ba3c97314d3c0622628e55171c6798ed80/jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata
#11 7.726   Downloading jsonschema_specifications-2023.7.1-py3-none-any.whl.metadata (2.8 kB)
#11 7.773 Collecting referencing>=0.28.4 (from jsonschema->ray>=2.5.1->-r requirements.txt (line 3))
#11 7.773   Obtaining dependency information for referencing>=0.28.4 from https://files.pythonhosted.org/packages/be/8e/56d6f1e2d591f4d6cbcba446cac4a1b0dc4f584537e2071d9bcee8eeab6b/referencing-0.30.2-py3-none-any.whl.metadata
#11 7.775   Downloading referencing-0.30.2-py3-none-any.whl.metadata (2.6 kB)
#11 7.986 Collecting rpds-py>=0.7.1 (from jsonschema->ray>=2.5.1->-r requirements.txt (line 3))
#11 7.986   Obtaining dependency information for rpds-py>=0.7.1 from https://files.pythonhosted.org/packages/2e/b5/bb6d90878ef8df7df279af68379266285eff135dfa7fe9985e19e10b1a43/rpds_py-0.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 7.988   Downloading rpds_py-0.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.7 kB)
#11 8.121 Collecting charset-normalizer<4,>=2 (from requests->ray>=2.5.1->-r requirements.txt (line 3))
#11 8.121   Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/a4/65/057bf29660aae6ade0816457f8db4e749e5c0bfa2366eb5f67db9912fa4c/charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
#11 8.124   Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (31 kB)
#11 8.178 Collecting urllib3<3,>=1.21.1 (from requests->ray>=2.5.1->-r requirements.txt (line 3))
#11 8.179   Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/9b/81/62fd61001fa4b9d0df6e31d47ff49cfa9de4af03adecf339c7bc30656b37/urllib3-2.0.4-py3-none-any.whl.metadata
#11 8.181   Downloading urllib3-2.0.4-py3-none-any.whl.metadata (6.6 kB)
#11 8.210 Collecting certifi>=2017.4.17 (from requests->ray>=2.5.1->-r requirements.txt (line 3))
#11 8.210   Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
#11 8.213   Downloading certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
#11 8.222 Requirement already satisfied: mpmath>=0.19 in /home/vllm/.local/lib/python3.10/site-packages (from sympy->torch>=2.0.0->-r requirements.txt (line 7)) (1.2.1)
#11 8.419 Downloading ray-2.6.3-cp310-cp310-manylinux2014_x86_64.whl (56.9 MB)
#11 8.918    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.9/56.9 MB 48.2 MB/s eta 0:00:00
#11 8.923 Downloading pandas-2.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.7 MB)
#11 8.999    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.7/12.7 MB 156.1 MB/s eta 0:00:00
#11 9.005 Downloading numpy-1.26.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
#11 9.116    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.2/18.2 MB 122.1 MB/s eta 0:00:00
#11 9.120 Downloading transformers-4.33.2-py3-none-any.whl (7.6 MB)
#11 9.165    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.6/7.6 MB 178.8 MB/s eta 0:00:00
#11 9.173 Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl (167.0 MB)
#11 10.89    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 167.0/167.0 MB 18.0 MB/s eta 0:00:00
#11 10.90 Downloading fastapi-0.103.1-py3-none-any.whl (66 kB)
#11 10.90    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.2/66.2 kB 20.2 MB/s eta 0:00:00
#11 10.90 Downloading uvicorn-0.23.2-py3-none-any.whl (59 kB)
#11 10.91    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.5/59.5 kB 22.8 MB/s eta 0:00:00
#11 10.91 Downloading pydantic-1.10.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
#11 10.93    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 175.1 MB/s eta 0:00:00
#11 10.93 Downloading anyio-3.7.1-py3-none-any.whl (80 kB)
#11 10.94    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.9/80.9 kB 32.0 MB/s eta 0:00:00
#11 10.94 Downloading click-8.1.7-py3-none-any.whl (97 kB)
#11 10.95    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 38.4 MB/s eta 0:00:00
#11 10.95 Downloading grpcio-1.58.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.3 MB)
#11 10.98    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 193.7 MB/s eta 0:00:00
#11 10.99 Downloading huggingface_hub-0.17.1-py3-none-any.whl (294 kB)
#11 10.99    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 294.8/294.8 kB 83.1 MB/s eta 0:00:00
#11 10.99 Downloading protobuf-4.24.3-cp37-abi3-manylinux2014_x86_64.whl (311 kB)
#11 11.00    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 311.6/311.6 kB 97.3 MB/s eta 0:00:00
#11 11.00 Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
#11 11.01    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 502.5/502.5 kB 112.1 MB/s eta 0:00:00
#11 11.02 Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
#11 11.02    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 136.0 MB/s eta 0:00:00
#11 11.03 Downloading regex-2023.8.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (771 kB)
#11 11.04    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 771.9/771.9 kB 128.9 MB/s eta 0:00:00
#11 11.05 Downloading safetensors-0.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
#11 11.06    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 156.1 MB/s eta 0:00:00
#11 11.07 Downloading starlette-0.27.0-py3-none-any.whl (66 kB)
#11 11.07    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.0/67.0 kB 26.0 MB/s eta 0:00:00
#11 11.07 Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)
#11 11.08    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 24.2 MB/s eta 0:00:00
#11 11.08 Downloading typing_extensions-4.7.1-py3-none-any.whl (33 kB)
#11 11.09 Downloading frozenlist-1.4.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (225 kB)
#11 11.09    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.7/225.7 kB 63.2 MB/s eta 0:00:00
#11 11.09 Downloading jsonschema-4.19.0-py3-none-any.whl (83 kB)
#11 11.10    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 83.4/83.4 kB 35.2 MB/s eta 0:00:00
#11 11.10 Downloading requests-2.31.0-py3-none-any.whl (62 kB)
#11 11.10    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB 33.2 MB/s eta 0:00:00
#11 11.11 Downloading certifi-2023.7.22-py3-none-any.whl (158 kB)
#11 11.11    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.3/158.3 kB 58.9 MB/s eta 0:00:00
#11 11.12 Downloading charset_normalizer-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (201 kB)
#11 11.12    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 201.8/201.8 kB 69.1 MB/s eta 0:00:00
#11 11.12 Downloading jsonschema_specifications-2023.7.1-py3-none-any.whl (17 kB)
#11 11.13 Downloading referencing-0.30.2-py3-none-any.whl (25 kB)
#11 11.13 Downloading rpds_py-0.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
#11 11.14    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 156.0 MB/s eta 0:00:00
#11 11.15 Downloading urllib3-2.0.4-py3-none-any.whl (123 kB)
#11 11.15    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 123.9/123.9 kB 50.4 MB/s eta 0:00:00
#11 11.15 Downloading exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
#11 11.16 Downloading fsspec-2023.9.1-py3-none-any.whl (173 kB)
#11 11.16    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 173.4/173.4 kB 51.0 MB/s eta 0:00:00
#11 12.48 Installing collected packages: tokenizers, sentencepiece, safetensors, pytz, ninja, msgpack, urllib3, tzdata, typing-extensions, tqdm, sniffio, six, rpds-py, regex, pyyaml, psutil, protobuf, packaging, numpy, idna, h11, grpcio, fsspec, frozenlist, exceptiongroup, click, charset-normalizer, certifi, attrs, uvicorn, requests, referencing, python-dateutil, pydantic, anyio, aiosignal, starlette, pandas, jsonschema-specifications, huggingface-hub, transformers, jsonschema, fastapi, ray, xformers
#11 12.93   Attempting uninstall: typing-extensions
#11 12.93     Found existing installation: typing_extensions 4.4.0
#11 12.93     Uninstalling typing_extensions-4.4.0:
#11 13.01       Successfully uninstalled typing_extensions-4.4.0
#11 27.19 Successfully installed aiosignal-1.3.1 anyio-3.7.1 attrs-23.1.0 certifi-2023.7.22 charset-normalizer-3.2.0 click-8.1.7 exceptiongroup-1.1.3 fastapi-0.103.1 frozenlist-1.4.0 fsspec-2023.9.1 grpcio-1.58.0 h11-0.14.0 huggingface-hub-0.17.1 idna-3.4 jsonschema-4.19.0 jsonschema-specifications-2023.7.1 msgpack-1.0.5 ninja-1.11.1 numpy-1.26.0 packaging-23.1 pandas-2.1.0 protobuf-4.24.3 psutil-5.9.5 pydantic-1.10.12 python-dateutil-2.8.2 pytz-2023.3.post1 pyyaml-6.0.1 ray-2.6.3 referencing-0.30.2 regex-2023.8.8 requests-2.31.0 rpds-py-0.10.3 safetensors-0.3.3 sentencepiece-0.1.99 six-1.16.0 sniffio-1.3.0 starlette-0.27.0 tokenizers-0.13.3 tqdm-4.66.1 transformers-4.33.2 typing-extensions-4.7.1 tzdata-2023.3 urllib3-2.0.4 uvicorn-0.23.2 xformers-0.0.21
#11 27.88 Defaulting to user installation because normal site-packages is not writeable
#11 27.91 Obtaining file:///home/vllm/vllm
#11 27.92   Installing build dependencies: started
#11 90.27   Installing build dependencies: still running...
#11 90.70   Installing build dependencies: finished with status 'done'
#11 90.70   Checking if build backend supports build_editable: started
#11 90.81   Checking if build backend supports build_editable: finished with status 'done'
#11 90.81   Getting requirements to build editable: started
#11 92.37   Getting requirements to build editable: finished with status 'done'
#11 92.38   Preparing editable metadata (pyproject.toml): started
#11 93.99   Preparing editable metadata (pyproject.toml): finished with status 'done'
#11 94.01 Requirement already satisfied: ninja in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (1.11.1)
#11 94.01 Requirement already satisfied: psutil in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (5.9.5)
#11 94.01 Requirement already satisfied: ray>=2.5.1 in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (2.6.3)
#11 94.01 Requirement already satisfied: pandas in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (2.1.0)
#11 94.01 Requirement already satisfied: sentencepiece in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (0.1.99)
#11 94.01 Requirement already satisfied: numpy in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (1.26.0)
#11 94.01 Requirement already satisfied: torch>=2.0.0 in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (2.0.1+cu118)
#11 94.01 Requirement already satisfied: transformers>=4.33.1 in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (4.33.2)
#11 94.01 Requirement already satisfied: xformers>=0.0.21 in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (0.0.21)
#11 94.01 Requirement already satisfied: fastapi in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (0.103.1)
#11 94.01 Requirement already satisfied: uvicorn in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (0.23.2)
#11 94.01 Requirement already satisfied: pydantic<2 in /home/vllm/.local/lib/python3.10/site-packages (from vllm==0.1.7) (1.10.12)
#11 94.02 Requirement already satisfied: typing-extensions>=4.2.0 in /home/vllm/.local/lib/python3.10/site-packages (from pydantic<2->vllm==0.1.7) (4.7.1)
#11 94.11 Requirement already satisfied: click>=7.0 in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (8.1.7)
#11 94.11 Requirement already satisfied: filelock in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (3.9.0)
#11 94.11 Requirement already satisfied: jsonschema in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (4.19.0)
#11 94.12 Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (1.0.5)
#11 94.12 Requirement already satisfied: packaging in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (23.1)
#11 94.12 Requirement already satisfied: protobuf!=3.19.5,>=3.15.3 in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (4.24.3)
#11 94.12 Requirement already satisfied: pyyaml in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (6.0.1)
#11 94.12 Requirement already satisfied: aiosignal in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (1.3.1)
#11 94.12 Requirement already satisfied: frozenlist in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (1.4.0)
#11 94.12 Requirement already satisfied: requests in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (2.31.0)
#11 94.12 Requirement already satisfied: grpcio>=1.42.0 in /home/vllm/.local/lib/python3.10/site-packages (from ray>=2.5.1->vllm==0.1.7) (1.58.0)
#11 94.13 Requirement already satisfied: sympy in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->vllm==0.1.7) (1.11.1)
#11 94.13 Requirement already satisfied: networkx in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->vllm==0.1.7) (3.0)
#11 94.13 Requirement already satisfied: jinja2 in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->vllm==0.1.7) (3.1.2)
#11 94.13 Requirement already satisfied: triton==2.0.0 in /home/vllm/.local/lib/python3.10/site-packages (from torch>=2.0.0->vllm==0.1.7) (2.0.0)
#11 94.14 Requirement already satisfied: cmake in /home/vllm/.local/lib/python3.10/site-packages (from triton==2.0.0->torch>=2.0.0->vllm==0.1.7) (3.25.0)
#11 94.14 Requirement already satisfied: lit in /home/vllm/.local/lib/python3.10/site-packages (from triton==2.0.0->torch>=2.0.0->vllm==0.1.7) (15.0.7)
#11 94.53 Requirement already satisfied: huggingface-hub<1.0,>=0.15.1 in /home/vllm/.local/lib/python3.10/site-packages (from transformers>=4.33.1->vllm==0.1.7) (0.17.1)
#11 94.53 Requirement already satisfied: regex!=2019.12.17 in /home/vllm/.local/lib/python3.10/site-packages (from transformers>=4.33.1->vllm==0.1.7) (2023.8.8)
#11 94.53 Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /home/vllm/.local/lib/python3.10/site-packages (from transformers>=4.33.1->vllm==0.1.7) (0.13.3)
#11 94.53 Requirement already satisfied: safetensors>=0.3.1 in /home/vllm/.local/lib/python3.10/site-packages (from transformers>=4.33.1->vllm==0.1.7) (0.3.3)
#11 94.53 Requirement already satisfied: tqdm>=4.27 in /home/vllm/.local/lib/python3.10/site-packages (from transformers>=4.33.1->vllm==0.1.7) (4.66.1)
#11 94.55 Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /home/vllm/.local/lib/python3.10/site-packages (from fastapi->vllm==0.1.7) (3.7.1)
#11 94.55 Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /home/vllm/.local/lib/python3.10/site-packages (from fastapi->vllm==0.1.7) (0.27.0)
#11 94.63 Requirement already satisfied: python-dateutil>=2.8.2 in /home/vllm/.local/lib/python3.10/site-packages (from pandas->vllm==0.1.7) (2.8.2)
#11 94.63 Requirement already satisfied: pytz>=2020.1 in /home/vllm/.local/lib/python3.10/site-packages (from pandas->vllm==0.1.7) (2023.3.post1)
#11 94.63 Requirement already satisfied: tzdata>=2022.1 in /home/vllm/.local/lib/python3.10/site-packages (from pandas->vllm==0.1.7) (2023.3)
#11 94.65 Requirement already satisfied: h11>=0.8 in /home/vllm/.local/lib/python3.10/site-packages (from uvicorn->vllm==0.1.7) (0.14.0)
#11 94.67 Requirement already satisfied: idna>=2.8 in /home/vllm/.local/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->vllm==0.1.7) (3.4)
#11 94.67 Requirement already satisfied: sniffio>=1.1 in /home/vllm/.local/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->vllm==0.1.7) (1.3.0)
#11 94.67 Requirement already satisfied: exceptiongroup in /home/vllm/.local/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi->vllm==0.1.7) (1.1.3)
#11 94.76 Requirement already satisfied: fsspec in /home/vllm/.local/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.15.1->transformers>=4.33.1->vllm==0.1.7) (2023.9.1)
#11 94.77 Requirement already satisfied: six>=1.5 in /home/vllm/.local/lib/python3.10/site-packages (from python-dateutil>=2.8.2->pandas->vllm==0.1.7) (1.16.0)
#11 94.88 Requirement already satisfied: MarkupSafe>=2.0 in /home/vllm/.local/lib/python3.10/site-packages (from jinja2->torch>=2.0.0->vllm==0.1.7) (2.1.2)
#11 94.90 Requirement already satisfied: attrs>=22.2.0 in /home/vllm/.local/lib/python3.10/site-packages (from jsonschema->ray>=2.5.1->vllm==0.1.7) (23.1.0)
#11 94.90 Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /home/vllm/.local/lib/python3.10/site-packages (from jsonschema->ray>=2.5.1->vllm==0.1.7) (2023.7.1)
#11 94.90 Requirement already satisfied: referencing>=0.28.4 in /home/vllm/.local/lib/python3.10/site-packages (from jsonschema->ray>=2.5.1->vllm==0.1.7) (0.30.2)
#11 94.90 Requirement already satisfied: rpds-py>=0.7.1 in /home/vllm/.local/lib/python3.10/site-packages (from jsonschema->ray>=2.5.1->vllm==0.1.7) (0.10.3)
#11 94.92 Requirement already satisfied: charset-normalizer<4,>=2 in /home/vllm/.local/lib/python3.10/site-packages (from requests->ray>=2.5.1->vllm==0.1.7) (3.2.0)
#11 94.93 Requirement already satisfied: urllib3<3,>=1.21.1 in /home/vllm/.local/lib/python3.10/site-packages (from requests->ray>=2.5.1->vllm==0.1.7) (2.0.4)
#11 94.93 Requirement already satisfied: certifi>=2017.4.17 in /home/vllm/.local/lib/python3.10/site-packages (from requests->ray>=2.5.1->vllm==0.1.7) (2023.7.22)
#11 94.93 Requirement already satisfied: mpmath>=0.19 in /home/vllm/.local/lib/python3.10/site-packages (from sympy->torch>=2.0.0->vllm==0.1.7) (1.2.1)
#11 95.08 Building wheels for collected packages: vllm
#11 95.08   Building editable for vllm (pyproject.toml): started
#11 166.7   Building editable for vllm (pyproject.toml): still running...
#11 257.2   Building editable for vllm (pyproject.toml): still running...
#11 324.8   Building editable for vllm (pyproject.toml): still running...
#11 392.8   Building editable for vllm (pyproject.toml): still running...
#11 460.9   Building editable for vllm (pyproject.toml): still running...
#11 509.5   Building editable for vllm (pyproject.toml): finished with status 'error'
#11 509.5   error: subprocess-exited-with-error
#11 509.5   
#11 509.5   × Building editable for vllm (pyproject.toml) did not run successfully.
#11 509.5   │ exit code: 1
#11 509.5   ╰─> [941 lines of output]
#11 509.5       No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
#11 509.5       running editable_wheel
#11 509.5       creating /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info
#11 509.5       writing /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/PKG-INFO
#11 509.5       writing dependency_links to /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/dependency_links.txt
#11 509.5       writing requirements to /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/requires.txt
#11 509.5       writing top-level names to /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/top_level.txt
#11 509.5       writing manifest file '/tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/SOURCES.txt'
#11 509.5       reading manifest file '/tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/SOURCES.txt'
#11 509.5       reading manifest template 'MANIFEST.in'
#11 509.5       adding license file 'LICENSE'
#11 509.5       writing manifest file '/tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm.egg-info/SOURCES.txt'
#11 509.5       creating '/tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm-0.1.7.dist-info'
#11 509.5       creating /tmp/pip-wheel-2xu96y8z/.tmp-_j5486qf/vllm-0.1.7.dist-info/WHEEL
#11 509.5       running build_py
#11 509.5       running build_ext
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.8) has a minor version mismatch with the version that was used to compile PyTorch (11.7). Most likely this shouldn't be a problem.
#11 509.5         warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py:398: UserWarning: There are no x86_64-linux-gnu-g++ version bounds defined for CUDA version 11.8
#11 509.5         warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
#11 509.5       building 'vllm.cache_ops' extension
#11 509.5       creating /tmp/tmp7x0_einq.build-temp/csrc
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/cache.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/cache.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/cache.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/cache_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/cache_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=cache_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(298): warning #550-D: variable "src_key_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(299): warning #550-D: variable "src_value_indices" was set but never used
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(43): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu(44): warning #1143-D: arithmetic on pointer to void or function type
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu: In function ‘void swap_blocks(at::Tensor&, at::Tensor&, const std::map<long int, long int>&)’:
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu:42:25: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
#11 509.5          42 |     cudaMemcpyAsync(
#11 509.5             |                 ~~~~    ^
#11 509.5       /home/vllm/vllm/csrc/cache_kernels.cu:42:47: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith]
#11 509.5          42 |     cudaMemcpyAsync(
#11 509.5             |                                               ^
#11 509.5       creating /tmp/tmpz7htqag9.build-lib/vllm
#11 509.5       x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /tmp/tmp7x0_einq.build-temp/csrc/cache.o /tmp/tmp7x0_einq.build-temp/csrc/cache_kernels.o -L/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o /tmp/tmpz7htqag9.build-lib/vllm/cache_ops.cpython-310-x86_64-linux-gnu.so
#11 509.5       building 'vllm.attention_ops' extension
#11 509.5       creating /tmp/tmp7x0_einq.build-temp/csrc/attention
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/attention.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/attention.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/attention.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=attention_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/attention/attention_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/attention/attention_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=attention_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /tmp/tmp7x0_einq.build-temp/csrc/attention.o /tmp/tmp7x0_einq.build-temp/csrc/attention/attention_kernels.o -L/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o /tmp/tmpz7htqag9.build-lib/vllm/attention_ops.cpython-310-x86_64-linux-gnu.so
#11 509.5       building 'vllm.pos_encoding_ops' extension
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/pos_encoding.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/pos_encoding.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/pos_encoding.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pos_encoding_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/pos_encoding_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/pos_encoding_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=pos_encoding_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /tmp/tmp7x0_einq.build-temp/csrc/pos_encoding.o /tmp/tmp7x0_einq.build-temp/csrc/pos_encoding_kernels.o -L/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o /tmp/tmpz7htqag9.build-lib/vllm/pos_encoding_ops.cpython-310-x86_64-linux-gnu.so
#11 509.5       building 'vllm.layernorm_ops' extension
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/layernorm.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/layernorm.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/layernorm.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=layernorm_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/layernorm_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/layernorm_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=layernorm_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /tmp/tmp7x0_einq.build-temp/csrc/layernorm.o /tmp/tmp7x0_einq.build-temp/csrc/layernorm_kernels.o -L/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o /tmp/tmpz7htqag9.build-lib/vllm/layernorm_ops.cpython-310-x86_64-linux-gnu.so
#11 509.5       building 'vllm.activation_ops' extension
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/activation.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/activation.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/activation.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=activation_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/activation_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/activation_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=activation_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       x86_64-linux-gnu-g++ -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 /tmp/tmp7x0_einq.build-temp/csrc/activation.o /tmp/tmp7x0_einq.build-temp/csrc/activation_kernels.o -L/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/lib -L/usr/local/cuda/lib64 -L/usr/lib/x86_64-linux-gnu -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o /tmp/tmpz7htqag9.build-lib/vllm/activation_ops.cpython-310-x86_64-linux-gnu.so
#11 509.5       building 'vllm.quantization_ops' extension
#11 509.5       creating /tmp/tmp7x0_einq.build-temp/csrc/quantization
#11 509.5       creating /tmp/tmp7x0_einq.build-temp/csrc/quantization/awq
#11 509.5       Emitting ninja build file /tmp/tmp7x0_einq.build-temp/build.ninja...
#11 509.5       Compiling objects...
#11 509.5       Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
#11 509.5       [1/2] c++ -MMD -MF /tmp/tmp7x0_einq.build-temp/csrc/quantization.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/quantization.cpp -o /tmp/tmp7x0_einq.build-temp/csrc/quantization.o -g -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quantization_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       [2/2] /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/quantization/awq/gemm_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quantization_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       FAILED: /tmp/tmp7x0_einq.build-temp/csrc/quantization/awq/gemm_kernels.o
#11 509.5       /usr/local/cuda/bin/nvcc  -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/torch/csrc/api/include -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/TH -I/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.10 -c -c /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu -o /tmp/tmp7x0_einq.build-temp/csrc/quantization/awq/gemm_kernels.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O2 -std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 --threads 8 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quantization_ops -D_GLIBCXX_USE_CXX11_ABI=0
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 892; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 892; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 900; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 900; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 908; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 908; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 916; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 916; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 924; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 924; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 928; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 932; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 936; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 940; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 944; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 948; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 952; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 956; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 964; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 964; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 972; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 972; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 980; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 980; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 988; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 988; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 996; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 996; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1000; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1004; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1008; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1012; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1016; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1020; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1024; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1028; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1834; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1834; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1842; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1842; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1850; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1850; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1854; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1858; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1862; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1866; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1874; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1874; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1882; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1882; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1890; error   : Feature 'ldmatrix' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1890; error   : Modifier '.m8n8' requires .target sm_75 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1894; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1898; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1902; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx, line 1906; error   : Feature '.m16n8k16' requires .target sm_80 or higher
#11 509.5       ptxas fatal   : Ptx assembly aborted due to errors
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(34): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(35): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(38): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(52): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(227): warning #177-D: variable "scaling_factors_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(228): warning #177-D: variable "zeros_shared" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(232): warning #177-D: variable "blockIdx_x" was declared but never referenced
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(246): warning #177-D: variable "ld_zero_flag" was declared but never referenced
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=size_t, one_sided=false, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/core/TensorImpl.h(77): here
#11 509.5       
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
#11 509.5                 detected during:
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       (61): here
#11 509.5                   instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/include/ATen/core/qualified_name.h(73): here
#11 509.5       
#11 509.5       /home/vllm/vllm/csrc/quantization/awq/gemm_kernels.cu(21): warning #177-D: function "__pack_half2" was declared but never referenced
#11 509.5       
#11 509.5       ninja: build stopped: subcommand failed.
#11 509.5       Traceback (most recent call last):
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
#11 509.5           subprocess.run(
#11 509.5         File "/usr/lib/python3.10/subprocess.py", line 526, in run
#11 509.5           raise CalledProcessError(retcode, process.args,
#11 509.5       subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
#11 509.5       
#11 509.5       The above exception was the direct cause of the following exception:
#11 509.5       
#11 509.5       Traceback (most recent call last):
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 156, in run
#11 509.5           self._create_wheel_file(bdist_wheel)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
#11 509.5           files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
#11 509.5           self._run_build_subcommands()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
#11 509.5           self.run_command(name)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
#11 509.5           self.distribution.run_command(command)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/dist.py", line 989, in run_command
#11 509.5           super().run_command(command)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
#11 509.5           cmd_obj.run()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 88, in run
#11 509.5           _build_ext.run(self)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
#11 509.5           self.build_extensions()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
#11 509.5           build_ext.build_extensions(self)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
#11 509.5           self._build_extensions_serial()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
#11 509.5           self.build_extension(ext)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 249, in build_extension
#11 509.5           _build_ext.build_extension(self, ext)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
#11 509.5           objects = self.compiler.compile(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
#11 509.5           _write_ninja_file_and_compile_objects(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
#11 509.5           _run_ninja_build(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
#11 509.5           raise RuntimeError(message) from e
#11 509.5       RuntimeError: Error compiling objects for extension
#11 509.5       /tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py:988: _DebuggingTips: Problem in editable installation.
#11 509.5       !!
#11 509.5       
#11 509.5               ********************************************************************************
#11 509.5               An error happened while installing `vllm` in editable mode.
#11 509.5       
#11 509.5               The following steps are recommended to help debug this problem:
#11 509.5       
#11 509.5               - Try to install the project normally, without using the editable mode.
#11 509.5                 Does the error still persist?
#11 509.5                 (If it does, try fixing the problem before attempting the editable mode).
#11 509.5               - If you are using binary extensions, make sure you have all OS-level
#11 509.5                 dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
#11 509.5               - Try the latest version of setuptools (maybe the error was already fixed).
#11 509.5               - If you (or your project dependencies) are using any setuptools extension
#11 509.5                 or customization, make sure they support the editable mode.
#11 509.5       
#11 509.5               After following the steps above, if the problem still persists and
#11 509.5               you think this is related to how setuptools handles editable installations,
#11 509.5               please submit a reproducible example
#11 509.5               (see https://stackoverflow.com/help/minimal-reproducible-example) to:
#11 509.5       
#11 509.5                   https://github.com/pypa/setuptools/issues
#11 509.5       
#11 509.5               See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.
#11 509.5               ********************************************************************************
#11 509.5       
#11 509.5       !!
#11 509.5         cmd_obj.run()
#11 509.5       Traceback (most recent call last):
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
#11 509.5           subprocess.run(
#11 509.5         File "/usr/lib/python3.10/subprocess.py", line 526, in run
#11 509.5           raise CalledProcessError(retcode, process.args,
#11 509.5       subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
#11 509.5       
#11 509.5       The above exception was the direct cause of the following exception:
#11 509.5       
#11 509.5       Traceback (most recent call last):
#11 509.5         File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#11 509.5           main()
#11 509.5         File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#11 509.5           json_out['return_val'] = hook(**hook_input['kwargs'])
#11 509.5         File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 273, in build_editable
#11 509.5           return hook(wheel_directory, config_settings, metadata_directory)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 463, in build_editable
#11 509.5           return self._build_with_temp_dir(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir
#11 509.5           self.run_setup()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/build_meta.py", line 341, in run_setup
#11 509.5           exec(code, locals())
#11 509.5         File "<string>", line 193, in <module>
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/__init__.py", line 103, in setup
#11 509.5           return distutils.core.setup(**attrs)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 185, in setup
#11 509.5           return run_commands(dist)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
#11 509.5           dist.run_commands()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
#11 509.5           self.run_command(cmd)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/dist.py", line 989, in run_command
#11 509.5           super().run_command(command)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
#11 509.5           cmd_obj.run()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 156, in run
#11 509.5           self._create_wheel_file(bdist_wheel)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
#11 509.5           files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
#11 509.5           self._run_build_subcommands()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
#11 509.5           self.run_command(name)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
#11 509.5           self.distribution.run_command(command)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/dist.py", line 989, in run_command
#11 509.5           super().run_command(command)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
#11 509.5           cmd_obj.run()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 88, in run
#11 509.5           _build_ext.run(self)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
#11 509.5           self.build_extensions()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
#11 509.5           build_ext.build_extensions(self)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
#11 509.5           self._build_extensions_serial()
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
#11 509.5           self.build_extension(ext)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/command/build_ext.py", line 249, in build_extension
#11 509.5           _build_ext.build_extension(self, ext)
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
#11 509.5           objects = self.compiler.compile(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
#11 509.5           _write_ninja_file_and_compile_objects(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
#11 509.5           _run_ninja_build(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
#11 509.5           raise RuntimeError(message) from e
#11 509.5       RuntimeError: Error compiling objects for extension
#11 509.5       [end of output]
#11 509.5   
#11 509.5   note: This error originates from a subprocess, and is likely not a problem with pip.
#11 509.5   ERROR: Failed building editable for vllm
#11 509.5 Failed to build vllm
#11 509.5 ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects
#11 ERROR: process "/bin/sh -c git clone https://github.com/vllm-project/vllm &&     cd vllm &&     git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 &&     pip3 install -r requirements.txt &&     pip3 install -e . &&     pip3 install --no-cache-dir huggingface-hub hf_transfer &&     pip3 cache purge" did not complete successfully: exit code: 1
------
 > [ 6/20] RUN git clone https://github.com/vllm-project/vllm &&     cd vllm &&     git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 &&     pip3 install -r requirements.txt &&     pip3 install -e . &&     pip3 install --no-cache-dir huggingface-hub hf_transfer &&     pip3 cache purge:
#11 509.5           _run_ninja_build(
#11 509.5         File "/tmp/pip-build-env-rqvw25jb/overlay/local/lib/python3.10/dist-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
#11 509.5           raise RuntimeError(message) from e
#11 509.5       RuntimeError: Error compiling objects for extension
#11 509.5       [end of output]
#11 509.5   
#11 509.5   note: This error originates from a subprocess, and is likely not a problem with pip.
#11 509.5   ERROR: Failed building editable for vllm
#11 509.5 Failed to build vllm
#11 509.5 ERROR: Could not build wheels for vllm, which is required to install pyproject.toml-based projects
------
Dockerfile:50
--------------------
  49 |     # Date: September 16th, 2023
  50 | >>> RUN git clone https://github.com/vllm-project/vllm && \
  51 | >>>     cd vllm && \
  52 | >>>     git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 && \
  53 | >>>     pip3 install -r requirements.txt && \
  54 | >>>     pip3 install -e . && \
  55 | >>>     pip3 install --no-cache-dir huggingface-hub hf_transfer && \
  56 | >>>     pip3 cache purge
  57 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c git clone https://github.com/vllm-project/vllm &&     cd vllm &&     git checkout ff36139ffc66294c19b503c1e52dc42c2cd265f6 &&     pip3 install -r requirements.txt &&     pip3 install -e . &&     pip3 install --no-cache-dir huggingface-hub hf_transfer &&     pip3 cache purge" did not complete successfully: exit code: 1

[casper-hansen]

You cannot build the GEMM kernel without a GPU, at least not as far as I am aware. You would have to do this when you initialize your container, i.e. when a GPU is available or use a GPU for building

[TheBloke]

That's going to be a major deployment problem for a lot of people then - but also sounds unlikely. All other kernels I know of can be built without GPU.

[TheBloke]

AutoAWQ for example builds fine in a Docker - I'm just double checking now that it works fine. But it certainly built fine.

Does AutoAWQ not build the GEMM kernel?

[casper-hansen]

It does seem like the architecture list is not being picked up correctly. It explicitly says compute 70.

ptxas /tmp/tmpxft_00000226_00000000-11_gemm_kernels.compute_70.ptx

Yeah, AutoAWQ builds the GEMM kernel. The problem here is probably setting the architecture correctly as the setup.py in AutoAWQ certainly does it (I tested this so many times by now).

[casper-hansen]

Can you try with a space between the arcs instead of semicolon?

[TheBloke]

Ah yeah I see you're doing this:

def get_compute_capabilities():
    # Collect the compute capabilities of all available GPUs.
    compute_capabilities = set()
    for i in range(torch.cuda.device_count()):
        major, minor = torch.cuda.get_device_capability(i)
        if major < 8:
            raise RuntimeError("GPUs with compute capability less than 8.0 are not supported.")
        compute_capabilities.add(major * 10 + minor)

    # figure out compute capability
    compute_capabilities = {80, 86, 89, 90}

    capability_flags = []
    for cap in compute_capabilities:
        capability_flags += ["-gencode", f"arch=compute_{cap},code=sm_{cap}"]

    return capability_flags

Which I guess would interact with TORCH_CUDA_ARCH_LIST. Maybe vLLM doesn't have an equivalent?

Can you try with a space between the arcs instead of semicolon?

Sure, will report back

[TheBloke]

Exactly the same problem unfortunately.

I can see in the nvcc commands that it appears to be trying to build for every single platform:

-gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 

So it's not filtered this list according to TORCH_CUDA_ARCH_LIST, and presumably the compilations for 70 and 75 are what's causing the problem.

[casper-hansen]

Exactly the same problem unfortunately.

I can see in the nvcc commands that it appears to be trying to build for every single platform:

-gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_89,code=sm_89 -gencode arch=compute_90,code=sm_90 

So it's not filtered this list according to TORCH_CUDA_ARCH_LIST, and presumably the compilations for 70 and 75 are what's causing the problem.

Well, that’s the problem then. You would need to remove 70 and 75 to build.

[TheBloke]

Yeah the problem is here: https://raw.githubusercontent.com/vllm-project/vllm/main/setup.py - hardcoded support for 70 and 75 if no GPU is available to check actual capabilities. TORCH_CUDA_ARCH_LIST is not used.

I've made a local edit to that as part of my Docker, removing 70 and 75, and it builds OK now.

@WoosukKwon

I see three separate issues here:

1. The code linked above needs to have compute_70 and compute_75 removed, else because of AWQ support, a build will always fail whenever no GPU is available, eg in a Docker.

2. Ideally there should be a way to specify compute capabilities during build. With other software, TORCH_CUDA_ARCH_LIST handles this, so maybe that would be one good way to check which architectures the user wants to build for.

3. By adding AWQ, your minimum supported compute has now changed to compute_80. Even if a user has no intention of using AWQ, they can no longer build on an older GPU. I would think there needs to be a way to build vLLM without AWQ for people on older GPUs (or who just don't want to use it anyway). Otherwise a number of people are now suddenly locked out of vLLM usage.

[WoosukKwon]

Hi @TheBloke thanks for letting us know the issue and providing suggestions. I've addressed the GPU compatibility issue in #1064. For an immediate solution, you can use the awq-warning branch for now. Sorry for the inconvenience.

I will also update setup.py to use TORCH_ARCH_LIST to filter out the target device list. Thanks for the suggestion.

[mckbrchill]

Hi! @WoosukKwon @casper-hansen

I guess AWQ models won't work on Tesla V100?
Tried to build inside a docker, and got error that my GPU compute capability is 70, but AWQ requires at least 80.

[casper-hansen]

Yeah, AWQ kernels will not build on CC less than 80