[DO NOT MERGE] 2.9, Inductor partition, standalone compile, monkeypatch fix(es), spawn #26882
Conversation
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: Huy Do <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: Luka Govedič <[email protected]>
commit a4ee300 Author: angelayi <[email protected]> Date: Tue Oct 14 19:19:25 2025 -0700 test moving import Signed-off-by: angelayi <[email protected]> commit 0ba846b Author: angelayi <[email protected]> Date: Mon Oct 13 13:36:43 2025 -0700 [BugFix] Patch inductor partitioning logic Signed-off-by: angelayi <[email protected]> Signed-off-by: ProExpertProg <[email protected]>
commit 6b0c3c3 Author: Boyuan Feng <[email protected]> Date: Tue Oct 14 21:30:29 2025 -0700 nit Signed-off-by: Boyuan Feng <[email protected]> commit 1016467 Author: Boyuan Feng <[email protected]> Date: Tue Oct 14 21:21:47 2025 -0700 fix multi-graph test Signed-off-by: Boyuan Feng <[email protected]> Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
Signed-off-by: ProExpertProg <[email protected]>
ProExpertProg
requested review from
LucasWilkinson,
WoosukKwon,
bigPYJ1151,
hmellor,
houseroad,
mgoin,
robertgshaw2-redhat,
simon-mo,
tlrmchlsmth,
yewentao256 and
youkaichao
as code owners
|
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a significant number of changes to support PyTorch 2.9, including updates to dependencies, CI/CD configurations, and Dockerfiles. It also enables Inductor graph partitioning by default for PyTorch 2.9+ and switches the default multiprocessing method for workers to spawn. Notably, it includes monkeypatches for PyTorch 2.9 to work around upstream issues, which are well-documented. The overall changes seem to be moving in the right direction for PyTorch 2.9 support. However, I've found some temporary testing code that should be removed before this PR is merged.
| # TESTING, TO BE REMOVED | ||
| VLLM_TEST_USE_PRECOMPILED_NIGHTLY_WHEEL=1 VLLM_USE_PRECOMPILED=1 pip3 install -vvv -e . \ | ||
| --extra-index-url https://download.pytorch.org/whl/test/cu128 |
There was a problem hiding this comment.
This change appears to be for temporary testing, as indicated by the comment # TESTING, TO BE REMOVED. Hardcoding the --extra-index-url and leaving such comments can lead to future issues if not cleaned up. Please remove this temporary code before this pull request is merged.
| # TESTING, TO BE REMOVED | |
| VLLM_TEST_USE_PRECOMPILED_NIGHTLY_WHEEL=1 VLLM_USE_PRECOMPILED=1 pip3 install -vvv -e . \ | |
| --extra-index-url https://download.pytorch.org/whl/test/cu128 | |
| VLLM_TEST_USE_PRECOMPILED_NIGHTLY_WHEEL=1 VLLM_USE_PRECOMPILED=1 pip3 install -vvv -e . |
|
这是来自QQ邮箱的假期自动回复邮件。
您好，我最近正在休假中，无法亲自回复您的邮件。我将在假期结束后，尽快给您回复。
|
ProExpertProg
added
the
ready
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
| pytest.skip("inductor graph partition is only available in PyTorch 2.9+") | ||
|
|
||
| model = "nvidia/Llama-4-Scout-17B-16E-Instruct-FP8" | ||
| if current_platform.get_device_capability()[0] < 10: | ||
| pytest.skip(f"{model} can only be loaded by B200 or above") |
There was a problem hiding this comment.
The new B200 gate in test_inductor_graph_partition_attn_fusion assumes current_platform.get_device_capability() always returns a tuple, but on CPU builds get_device_capability() returns None. The added check current_platform.get_device_capability()[0] < 10 will raise a TypeError before the test can skip, causing the entire test run to crash on environments without CUDA rather than being skipped. Consider storing the capability in a variable and skipping when it is None or has a major version below 10.
Useful? React with 👍 / 👎.
|
Seems like spawn is resolved, closing for now |
4 participants
This is just #26738 but with spawn forced so we can try that in CI as well.