Could not find setuptools>=40.8.0 (with Setuptools installed)

[jaraco]

Description

In jaraco/pip-run#73, I was investigating an issue surfaced by the removal of setuptools/wheel by default from virtualenv on Python 3.12. While troubleshooting, I added Setuptools as a dependency (jaraco/pip-run@d6783ea), but the error persisted:

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

The test that's failing is simply building a legacy Setuptools project (setup.py, no pyproject.toml). It's failing to find Setuptools presumably because the tests are passing --no-index.

This error is confusing in a couple of ways:

• It's the implicit "wheel" that's missing (adding wheel works around the issue; Add setuptools and wheel as test dependencies jaraco/pip-run#74), but there's no mention of wheel.
• The error indicates that it's not a problem with pip because it's a subprocess, but the error message comes directly from pip.

Expected behavior

Ideally, pip shouldn't try to install setuptools if it's installed and satisfied. If it needs wheel and/or setuptools to do the build but it doesn't have one, it should state as much ("installing implied {setuptools or wheel} for legacy build"), and then only attempt to install the missing package.

Separately, it's apparent that the subprocess boundary isn't a good indicator of which project might be responsible. Better would be for pip to indicate which builder was used and maybe which project might be responsible (maybe).

pip version

23.3.1

Python version

3.12.0a7

OS

any

How to Reproduce

With setuptools/wheel uninstalled:

 $ cat > setup.py
import setuptools; setuptools.setup()
 $ pip-run setuptools -- -m pip install --no-index .
Processing /Users/jaraco/draft
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Note the build succeeds if wheel is present:

 $ pip-run setuptools wheel -- -m pip-run -v --no-index . -- -c pass
Processing /Users/jaraco/draft
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (setup.py) ... done
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=920 sha256=675c0cd23b1d6de46af0e8377682867186cedb68b275787323ea61584caaed0a
  Stored in directory: /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-ephem-wheel-cache-yv5_0l1v/wheels/85/87/4e/90d5ac6e5f68ec20eda74f6d3622a8e87f2c519c5b71ef64a8
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

$ pip-run setuptools -- -m pip install --no-index .

So, you're installing setuptools in your current environment. Then you're installing the current project with --no-index. But the current project needs setuptools (and wheel) installed. Since pip 23.1 we don't use setup.py install for legacy projects, we do a build using an isolated environment. It's that build (which is a subprocess) that needs setuptools and can't get it because of the --no-index.

You can avoid this by using --no-build-isolation, or you can make setuptools and wheel available (via --find-links or similar) for the isolated enviornment creation.

I agree that the error reporting isn't ideal. PRs to improve this would, of course, be welcome.

[tilktilk5]

When running
~/stable-diffusion-webui/venv/bin/pip install --no-cache-dir torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
I am encountering the same error with the addition of
Looking in indexes: https://download.pytorch.org/whl/cu118
before
ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)

In addition, running
./venv/bin/pip install setuptools wheel
returns
Requirement already satisfied: setuptools in ./venv/lib/python3.11/site-packages (65.5.0)
Requirement already satisfied: wheel in ./venv/lib/python3.11/site-packages (0.40.0)

Arch, pip 23.1.2, and python 3.11.3.

[notatallshaw]

@tilktilk5 pleas provide the full error, you've left out the important information to which package is causing this error.

[ruiyuanlu]

pip 22.3.1 encountered this issue when installing imgviz 1.7.3 in offline mode, too. Using win-10 python 3.10.9.
Interestingly, python setup.py install works fine.

[jaraco]

Since pip 23.1 we don't use setup.py install for legacy projects, we do a build using an isolated environment. It's that build (which is a subprocess) that needs setuptools and can't get it because of the --no-index.

That statement doesn't match with what's going on with the workaround. The only difference between the execution that fails and the one that succeeds is additionally supplying wheel.

I realize the usage of pip-run is probably confusing things, so let me illustrate without it. First, create a venv.

 draft @ py -V
Python 3.12.0rc1
 draft @ py -m venv --clear .venv
 draft @ py -m pip freeze --all
pip==23.2.1
 draft @ cat > setup.py
__import__('setuptools').setup()

Attempting to install without setuptools results in the error.

 draft @ py -m pip install --no-index .
Processing /Users/jaraco/draft
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Even installing Setuptools doesn't work around the error and results in the same error message.

 draft @ py -m pip install setuptools
Collecting setuptools
  Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/4f/ab/0bcfebdfc3bfa8554b2b2c97a555569c4c1ebc74ea288741ea8326c51906/setuptools-68.1.2-py3-none-any.whl.metadata
  Using cached setuptools-68.1.2-py3-none-any.whl.metadata (6.2 kB)
Using cached setuptools-68.1.2-py3-none-any.whl (805 kB)
Installing collected packages: setuptools
Successfully installed setuptools-68.1.2
 draft @ py -m pip install --no-index .
Processing /Users/jaraco/draft
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

But, installing wheel works around the error message.

 draft @ py -m pip install wheel
Collecting wheel
  Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/b8/8b/31273bf66016be6ad22bb7345c37ff350276cfd46e389a0c2ac5da9d9073/wheel-0.41.2-py3-none-any.whl.metadata
  Using cached wheel-0.41.2-py3-none-any.whl.metadata (2.2 kB)
Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
Installing collected packages: wheel
Successfully installed wheel-0.41.2
 draft @ py -m pip install --no-index .
Processing /Users/jaraco/draft
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (setup.py) ... done
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=923 sha256=b4d83c723b4e72f618dcc22229aeb17e5d1f52c9fe15e5882854458fb5d2bc36
  Stored in directory: /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-ephem-wheel-cache-dzens2lp/wheels/af/49/6b/4feeb3d319be8bf02986d3c092b59a17ae5a88757cbf49cf5b
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0

My guess is that pip has some logic that falls back from isolated builds when both setuptools and wheel are present.

You can avoid this by using --no-build-isolation, or you can make setuptools and wheel available (via --find-links or similar) for the isolated enviornment creation.

Indeed, uninstalling wheel and then running with --no-build-isolation does provide an error message that hints that wheel isn't installed.

 draft @ py -m pip uninstall wheel
Found existing installation: wheel 0.41.2
Uninstalling wheel-0.41.2:
  Would remove:
    /Users/jaraco/draft/.venv/bin/wheel
    /Users/jaraco/draft/.venv/lib/python3.12/site-packages/wheel-0.41.2.dist-info/*
    /Users/jaraco/draft/.venv/lib/python3.12/site-packages/wheel/*
Proceed (Y/n)? y
  Successfully uninstalled wheel-0.41.2
 draft @ py -m pip install --no-index --no-build-isolation .
Processing /Users/jaraco/draft
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [10 lines of output]
      running dist_info
      creating /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info
      writing /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/dependency_links.txt
      writing top-level names to /private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/top_level.txt
      writing manifest file '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/SOURCES.txt'
      reading manifest file '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/SOURCES.txt'
      writing manifest file '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN.egg-info/SOURCES.txt'
      creating '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pip-modern-metadata-einzd_n9/UNKNOWN-0.0.0.dist-info'
      error: invalid command 'bdist_wheel'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

I acknowledge that I'm holding it wrong, but I'm a seasoned user in the packaging ecosystem and I'm struggling to understand how to hold it right.

Maybe it would help if pip were to emit which build technique it's using ("attempting isolated build (requires dependencies to be installable)..." and "attempting fallback non-isolated build (detected setuptools and wheel available)" and "not attempting fallback non-isolated build (one or more of setuptools and wheel unavailable)"), possibly also with reference to directions as to what that technique does and how to address common issues with that technique or to aid pip in using a preferable technique.

Note also that the "fix" for the pip-run issue was to add setuptools and wheel to the test environment. Perhaps a more robust fix would be to update the package under test to define a "pyproject.toml" instead of a setup.py.

[mohd-akram]

You can avoid this by using --no-build-isolation, or you can make setuptools and wheel available (via --find-links or similar) for the isolated enviornment creation.

This does not work. It's clear that pip does not need setuptools since installing wheel alone is sufficient. It's also not clear that it even needs wheel, since it can somehow manage "Building wheels for collected packages: wheel" without having an existential crisis. See here for more information. I don't think it's too much to ask that pip install my-dependency-free-package.tar.gz should work without an index, cache or internet connection.

[pradyunsg]

I don't think it's too much to ask that pip install my-dependency-free-package.tar.gz should work without an index, cache or internet connection.

It is.

Build time dependencies need to be pulled in from somewhere to be able to build the package.

[mohd-akram]

This is not a build time dependency. This is a core functionality of a package manager. I can do npm install, go install, dnf install, and before 23.1, pip install for dependency free packages without requiring any additional files. pip should contain everything it needs to install a local package and the current behavior is a regression.

[pradyunsg]

pip should contain everything it needs to install a local package and the current behavior is a regression.

No, it is not.

pip install for dependency free packages without requiring any additional files.

You can still do that, as long as you have a built distribution file (i.e. a .whl file). See https://pradyunsg.me/blog/2022/12/31/wheels-are-faster-pure-python/.

[dreirund]

@tilktilk5 pleas provide the full error, you've left out the important information to which package is causing this error.

I do this on @tilktilk5's behalf, since I have the same issue:

/opt/stable-diffusion-web-ui/venv/bin/python3 -m pip install torch==2.0.1+rocm5.4.2 torchvision==0.15.2+rocm5.4.2 --index-url https://download.pytorch.org/whl/rocm5.4.2:

Looking in indexes: https://download.pytorch.org/whl/rocm5.4.2
Collecting torch==2.0.1+rocm5.4.2
  Using cached https://download.pytorch.org/whl/rocm5.4.2/torch-2.0.1%2Brocm5.4.2-cp311-cp311-linux_x86_64.whl (1536.4 MB)
Collecting torchvision==0.15.2+rocm5.4.2
  Using cached https://download.pytorch.org/whl/rocm5.4.2/torchvision-0.15.2%2Brocm5.4.2-cp311-cp311-linux_x86_64.whl (62.4 MB)
Collecting filelock (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/filelock-3.9.0-py3-none-any.whl (9.7 kB)
Collecting typing-extensions (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Collecting sympy (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/sympy-1.12-py3-none-any.whl (5.7 MB)
Collecting networkx (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/networkx-3.2.1-py3-none-any.whl (1.6 MB)
Collecting jinja2 (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting pytorch-triton-rocm<2.1,>=2.0.0 (from torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/pytorch_triton_rocm-2.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (78.4 MB)
Collecting numpy (from torchvision==0.15.2+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/numpy-1.26.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.3 MB)
Collecting requests (from torchvision==0.15.2+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/requests-2.28.1-py3-none-any.whl (62 kB)
Collecting pillow!=8.3.*,>=5.3.0 (from torchvision==0.15.2+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/pillow-10.2.0-cp311-cp311-manylinux_2_28_x86_64.whl (4.5 MB)
Collecting cmake (from pytorch-triton-rocm<2.1,>=2.0.0->torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/cmake-3.25.0-py2.py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23.7 MB)
Collecting lit (from pytorch-triton-rocm<2.1,>=2.0.0->torch==2.0.1+rocm5.4.2)
  Using cached https://download.pytorch.org/whl/lit-15.0.7.tar.gz (132 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [3 lines of output]
      Looking in indexes: https://download.pytorch.org/whl/rocm5.4.2
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I am not familiar with Python's own packaging; I use Arch Linux packages whenever possible.

I have the Arch Linux package python-setuptools, version 1:69.0.3-1 (setuptools 69.0.3) installed system-wide.

Regards!

[pfmoore]

The lit package doesn't appear to have a binary for your platform on https://download.pytorch.org/whl/rocm5.4.2 (which you specified in --index-url). So pip is trying to build it from source for you, and to do that it needs setuptools, which also appears not to be available on https://download.pytorch.org/whl/rocm5.4.2.

I suspect you are using the PyTorch index incorrectly - you should probably re-read their installation instructions, and maybe ask for help on their forums. Maybe you were meant to use --extra-index-url rather than --index-url?

I have the Arch Linux package python-setuptools, version 1:69.0.3-1 (setuptools 69.0.3) installed system-wide.

This won't help, as pip builds packages in an isolated build environment, and so needs to download and install the build backend (setuptools in this case) regardless of whether you have it available in your system environment.