[Installation]: `ModuleNotFoundError: No module named 'numpy.lib.function_base'` due to NumPy 2.0 release

[glibg10b]

Anything you want to discuss about vllm.

Users may see the following error when trying to run vllm:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 26, in <module>
    from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 27, in <module>
    from vllm.model_executor.guided_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/__init__.py", line 6, in <module>
    from vllm.model_executor.guided_decoding.lm_format_enforcer_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/lm_format_enforcer_decoding.py", line 15, in <module>
    from vllm.model_executor.guided_decoding.outlines_decoding import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_decoding.py", line 13, in <module>
    from vllm.model_executor.guided_decoding.outlines_logits_processors import (
  File "/home/waldo/venv/lib/python3.11/site-packages/vllm/model_executor/guided_decoding/outlines_logits_processors.py", line 24, in <module>
    from outlines.fsm.guide import CFGGuide, Generate, Guide, RegexGuide, Write
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/__init__.py", line 2, in <module>
    import outlines.generate
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/__init__.py", line 2, in <module>
    from .cfg import cfg
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/generate/cfg.py", line 5, in <module>
    from outlines.models import OpenAI
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/__init__.py", line 14, in <module>
    from .openai import OpenAI, azure_openai, openai
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/models/openai.py", line 9, in <module>
    from outlines.base import vectorize
  File "/home/waldo/venv/lib/python3.11/site-packages/outlines/base.py", line 8, in <module>
    from numpy.lib.function_base import (
ModuleNotFoundError: No module named 'numpy.lib.function_base'

This is because numpy recently released a breaking change (2.0.0), and for some reason, vllm's dependencies don't restrict the version based on the major number (which should have been frozen at 1).

#5582 works around this, but this hasn't reached me -- I suspect it's because I'm using pip install instead of building from source. A workaround for users like me is to run pip install numpy==1.26.4.

The purpose of this issue is to show up when users search for the above output. Please do not close it until the fix is available to users of pip install.

[WoosukKwon]

@glibg10b Thanks for reporting the bug! We are aware of the bug and are currently waiting for either outlines or tensorizer to update its Numpy dependency (e.g., by adding numpy < 2.0.0 in its dependency). If they don't, we will make a new release with the fixed Numpy dependency.

[DarkLight1337]

Fixed by #5582.

Edit: Sorry, just saw your last sentence. Let's rename the title to also include the cause of the problem.

[kzos]

still issue present for numpy >= 2.0.0

[DarkLight1337]

still issue present for numpy >= 2.0.0

You should downgrade your numpy version to < 2.0.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[lassevk]

Attempting to install vllm and I still get

Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.)

pip install shows:

Package Version
------- -------
numpy   1.26.4
pip     24.0

This is a fresh Ubuntu vm, all packages updated, only installed pip, set up a venv, and then installed numpy, vllm then fails to install.

Any tips on how to proceed?

[DarkLight1337]

Which version of pytorch are you using?