[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) #24542

ProExpertProg · 2025-09-09T22:50:10Z

Purpose

Cleanup compilation tests and custom passes and address #23091 and part of #24376. This PR is a prerequisite for matching torch implementations of custom ops in #24604.

Cleanup:

RMSNorm + QuantFP8 fusion:
- remove custom multi-output matching workaround
- remove .instance() workaround
- remove _seen_patterns.clear() workaround
- rename FusionPass to RMSNormQuantFusionPass
reduce test_fusion.py running time
add cleanup pass that topo-sorts and does DCE
- remove DCE in attention fusion and fix [Bug]: graph.eliminate_dead_code() break the fx graph with enable_fi_allreduce_fusion when TP == 2 #23091
add LazyInitPass test utility
add env var for pattern match debugging (TODO: docs)
fix test_sequence_parallelism.py
add assert to test_silu_mul_quant_fusion for easier diagnostics of fp4 failures.
add init check for custom_ops field syntax and warnings for specifying ops that don't exist
add timing and graph dumping decorator for VllmInductorPass
add VllmPatternMatchPass common pass superclass with pattern tracking and dumping utilities

Test Plan

Unit tests all pass.

mergify · 2025-09-09T22:50:49Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @ProExpertProg.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

tests/compile/test_fusion_attn.py

Signed-off-by: Luka Govedič <[email protected]>

…ass) Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

…eanup Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Signed-off-by: Luka Govedič <[email protected]>

vllm/config/compilation.py

vllm/envs.py

Signed-off-by: Luka Govedič <[email protected]>

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

ZJY0516 · 2025-09-25T15:33:43Z

vllm/compilation/vllm_inductor_pass.py

+        debug_dump_path = Path(debug_dump_path) / f"rank_{rank}"
+        debug_dump_path.mkdir(parents=True, exist_ok=True)
+
+        from vllm.utils import unique_filepath


It seems that vllm.utils does not have unique_filepath.

Yep, my bad, was working on a newer branch. I'll fix in #24604 unless you want to extract the change from there (and pull the import back up) in a separate PR to fix earlier

I will fix it in a separate PR

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

…g utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) (#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: yewentao256 <[email protected]>

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: gaojc <[email protected]>

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

…g utils, fix DCE bug (vllm-project#23091), fix test (vllm-project#24376), and prep for custom op matching (vllm-project#24604) (vllm-project#24542) Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>

mergify bot added the needs-rebase label Sep 9, 2025

mergify bot mentioned this pull request Sep 9, 2025

[torch.compile] Custom op matching #24188

Closed

5 tasks

ProExpertProg force-pushed the luka/custom-pass-cleanup branch from ce05cd7 to 18c8d75 Compare September 9, 2025 22:52

mergify bot removed the needs-rebase label Sep 9, 2025

ProExpertProg force-pushed the luka/custom-pass-cleanup branch from 4bb59cd to 1778b80 Compare September 10, 2025 16:29

ProExpertProg marked this pull request as ready for review September 10, 2025 16:44

ProExpertProg requested review from WoosukKwon, hmellor, houseroad, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners September 10, 2025 16:44

ProExpertProg commented Sep 10, 2025

View reviewed changes

tests/compile/test_fusion_attn.py Outdated Show resolved Hide resolved

ProExpertProg added 12 commits September 10, 2025 14:14

pass mgr util

955030c

Signed-off-by: Luka Govedič <[email protected]>

remove old multi_output_match workaround, add new workaround (tests p…

d97634d

…ass) Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Improve workaround to properly handle inputs of inputs

c34cbdf

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Remove prints, add debug statement

37446e5

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Reduce fusion test running time

8d22102

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Rename FusionPass, remove instance() workaround, other fusion pass cl…

7765aa8

…eanup Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Remove old QuantMultiOutputMatch

e8fae3d

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Remove MultiOutputMatch workaround, add cleanup pass

4bdb73c

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Add LazyInitPass

4f4cf01

Signed-off-by: luka <[email protected]> Signed-off-by: Luka Govedič <[email protected]>

Add pattern match debugging to tests

e584fcd

Signed-off-by: Luka Govedič <[email protected]>

Fix seq-par test, add assert to silu-fp4 for clearer error

a5c2e3b

Signed-off-by: Luka Govedič <[email protected]>

more fusion.py cleanup

5fa85ea

Signed-off-by: Luka Govedič <[email protected]>

ProExpertProg added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 20, 2025

Merge branch 'main' into luka/custom-pass-cleanup

3dc0e04

ProExpertProg requested review from angelayi and ilmarkov September 22, 2025 12:25

angelayi approved these changes Sep 22, 2025

View reviewed changes

vllm/config/compilation.py Show resolved Hide resolved

vllm/envs.py Outdated Show resolved Hide resolved

ProExpertProg commented Sep 22, 2025

View reviewed changes

vllm/envs.py Outdated Show resolved Hide resolved

ProExpertProg commented Sep 22, 2025

View reviewed changes

vllm/envs.py Outdated Show resolved Hide resolved

ProExpertProg added 2 commits September 22, 2025 12:38

Remove env

dba5449

Signed-off-by: Luka Govedič <[email protected]>

Remove env

7fcbe74

Signed-off-by: Luka Govedič <[email protected]>

ProExpertProg enabled auto-merge (squash) September 22, 2025 16:48

simon-mo disabled auto-merge September 22, 2025 19:30

simon-mo merged commit d5e0fca into vllm-project:main Sep 22, 2025
41 of 43 checks passed

github-project-automation bot moved this from In review to Done in torch.compile integration Sep 22, 2025

ProExpertProg deleted the luka/custom-pass-cleanup branch September 22, 2025 19:50

ProExpertProg mentioned this pull request Sep 22, 2025

[Bugfix] fix custom op test #25429

Merged

mgoin mentioned this pull request Sep 23, 2025

[CI Failure] Fix test_enabled_custom_ops with empty ops #25435

Closed

5 tasks

ProExpertProg mentioned this pull request Sep 23, 2025

[Bug]: tests/compile failures #24376

Closed

1 task

ProExpertProg added this to the vllm==v0.12.0/torch==2.9.0 compilation improvements milestone Sep 24, 2025

ProExpertProg mentioned this pull request Sep 24, 2025

[Feature][torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable #25612

Closed

1 task

ZJY0516 reviewed Sep 25, 2025

View reviewed changes

ZJY0516 mentioned this pull request Sep 26, 2025

[Fix][torch.compile] fix unique_filepath #25732

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) #24542

[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) #24542

Uh oh!

ProExpertProg commented Sep 9, 2025 •

edited by github-actions bot

Loading

Uh oh!

mergify bot commented Sep 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZJY0516 Sep 25, 2025

Uh oh!

ProExpertProg Sep 25, 2025

Uh oh!

ZJY0516 Sep 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Uh oh!

[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) #24542

[torch.compile] Cleanup compilation tests and custom passes, add debug utils, fix DCE bug (#23091), fix test (#24376), and prep for custom op matching (#24604) #24542

Uh oh!

Conversation

ProExpertProg commented Sep 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Uh oh!

mergify bot commented Sep 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ZJY0516 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

ProExpertProg Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

ZJY0516 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

ProExpertProg commented Sep 9, 2025 •

edited by github-actions bot

Loading