Use pip --use-feature=in-tree-build, replace use of sdh_setup_bdist_wheel by sdh_pip_install

[mkoeppe]

This is a feature added in pip 21.1.

We can now build wheels for all packages using pip wheel, even those with symlinks or that need in-tree configuration files. Direct invocation of setup.py bdist_wheel is no longer necessary.

CC: @jhpalmieri @kliem @isuruf

Component: build

Author: Matthias Koeppe

Branch/Commit: 290fa2f

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/32046

[mkoeppe]

Branch: u/mkoeppe/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a1e78f2	build/pkgs/pillow/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install, fix quoting of CONDA_PREFIX with whitespace
6166a8c	build/bin/sage-dist-helpers: Generalize sdh_globaloptionify to sdh_prefix_args
2dae827	build/pkgs/jupyter_jsmol/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install
f358345	build/pkgs/python_igraph/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install

[sagetrac-git]

Commit: f358345

[sagetrac-git]

Changed commit from f358345 to 413f6b5

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a1d33bc	build/pkgs/pillow/spkg-install.in, build/bin/sage-dist-helpers: Remove debugging code
413f6b5	build/pkgs/{jupyter_jsmol,python_igraph,pillow}/spkg-install.in: Use --build-option, not --global-option

[mkoeppe]

comment:4

A similar change can be done with numpy, but to avoid conflicts, that's best done in #32021 or #31565

[mkoeppe]

Author: Matthias Koeppe

[jhpalmieri]

comment:5

Do we need to change build/pkgs/pip/install-requires.txt to require at least version 21.1?

[mkoeppe]

comment:6

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

8c5ee65

build/pkgs/pip/install-requires.txt: Update lower bound

[sagetrac-git]

Changed commit from 413f6b5 to 8c5ee65

[jhpalmieri]

comment:8

Replying to @mkoeppe:

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

So what happens if someone has a system version of pip installed that doesn't support --use-feature=in-tree-build?

[mkoeppe]

comment:9

We don't use any system Python packages, even when we use system Python.

[jhpalmieri]

comment:10

Are you saying that we always build and use our own pip? Are the entries in pip/distros ignored?

[mkoeppe]

comment:11

Replying to @jhpalmieri:

Are you saying that we always build and use our own pip?

That's right. See build/bin/sage-venv, which is the script that we use to set up our venv over the system Python. https://docs.python.org/3/library/venv.html#venv.EnvBuilder with_pip defaults to False; and we also use system_site_packages=False.
We then bootstrap setuptools, pip, and wheel in our empty virtual environment -- same as we do when we build our own python.

Are the entries in pip/distros ignored?

Yes, they are only decoration. There is currently only one use-case for Python system packages, and that is a conda development environment (see src/environment.yml).