-
Notifications
You must be signed in to change notification settings - Fork 48
Move to uv as a main tool for dependency and Python management #496
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
2d22f14
to
040d142
Compare
Calling setup.py from command line is deprecated by Python for a very long time now.
This is a new standardized way to have dev dependencies.
We use this in CI, and it is generally useful, so lets not require manually adding it.
This reduces amount of files in root and move towards centralizing configuration.
Those were removed some time ago in test-requirements.txt, but not here.
They make it impossible to resolve dependencies on Python 3.13 because gevent 23.9.0 doesn't have wheels for it. This special case was, as far as I can tell, somehow ignored in our CI. By this I mean that even on windows build it used newer gevent. Because of that, removing those cases shouldn't break anything.
Tablets are now available on Scylla version that we run CI with, so I see no reason to keep it separate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR transitions the project to use uv
and pyproject.toml
for Python version and dependency management, removing legacy requirements.txt
, setup scripts, and manual venv setups.
- Replace legacy test requirements files and scripts with a single source of truth in
pyproject.toml
anduv
tooling - Consolidate pytest, cibuildwheel, and build settings into
pyproject.toml
and update CI workflows to useuv
- Update documentation and CI pipelines to invoke
uv
instead of directsetup.py
,pip
, or shell scripts
Reviewed Changes
Copilot reviewed 12 out of 15 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
test-requirements.txt | Removed in favor of pyproject.toml dependency-groups. |
setup.py | Defaulted build_concurrency to None and adjusted parallel build condition. |
scripts/run_integration_test.sh | Removed legacy integration test script, replaced by CI uv steps. |
pytest.ini | Deleted; pytest options moved into pyproject.toml . |
pyproject.toml | Centralized project config, dependencies, uv cache-keys, pytest and cibuildwheel settings. |
docs/pyproject.toml | Updated docs build dependencies and linked local driver. |
docs/installation.rst | Switched docs instructions to uv and added default concurrency note. |
README-dev.rst | Added uv tool recommendation and updated test commands. |
.github/workflows/lib-build-and-push.yml | Swapped actions/setup-python for astral-sh/setup-uv , updated build and test commands. |
.github/workflows/integration-tests.yml | Removed shell script, added uv install, sync, and test steps. |
.github/workflows/docs-pr.yaml | Dropped setup.py develop step before docs build. |
.github/workflows/docs-pages.yaml | Dropped setup.py develop step before docs multiversion build. |
Comments suppressed due to low confidence (2)
.github/workflows/docs-pr.yaml:29
- Docs build step now omits installing the driver, which may cause Sphinx to fail importing modules. Consider adding a 'uv sync' or an equivalent install step before building docs.
- name: Build docs
.github/workflows/docs-pages.yaml:29
- Docs pages pipeline no longer installs the driver before building docs, which could break the documentation build. Add a step such as 'uv sync' or 'pip install -e .' prior to 'make multiversion'.
run: make -C docs setupenv
No longer necessary for anything.
No longer needed for anything, and we should not have 2 places to keep dependencies in. It will, and already has, resulted in those places being out of sync.
Because why not? It will be faster - on my machine ~4x faster.
When a file or variable used by setup.py changes, we should rebuild the driver. If we don't we risk very unpleasant errors caused by source files and binary files differing.
The goal of this is to utilize
pyproject.toml
anduv
tool for management of Python versions and dependencies, instead of older methods like manually creating venvs and installing stuff fromrequirements.txt
.This will ultimately result in tests / tools being easier to run, and less error prone (I can't even count how many hours I wasted because of some misconfigurations related to that....), and faster.
First commits change the docs to not call
setup.py
anymore. Without this, many changes would cause docs to break, even something simple like using license field in pyproject.toml.After this first necessary fix, few commits do some minor improvements in pyproject.toml, like formatting, specifying Python version, or moving pytest config there.
After that, I fix the dependencies list in pyproject.toml, becuse it contained some errors making it impossible to run tests using it.
Then, the integration test pipeline is moved to uv. To make it easier, I remove the experiments section first in a separate commit - tablets are definitely not an experiment anymore.
After that, I move the other part: lib-build-and-push.yml. This allowed me to remove legacy requirements files in next commit.
Last 2 commits adjust documentation, and make one final improvement: building extensions in parallel by default.
To reviewers: Please verify that you are able to build the project locally, run unit and integration tests.
I recommend having ccache (or sccache) set up. With it, the overhead of having to compile extensions basically disappears.
TODO:
lib-build-and-push.yml
, so I may have broken something. @dkropachev you wrote them, so please review my changes.Pre-review checklist
I added relevant tests for new features and bug fixes.- no new features or bug fixesI have provided docstrings for the public items that I want to introduce.- no new items./docs/source/
.I added appropriateFixes:
annotations to PR description.