[BugFix] Fix de-functionalization pass for rotary_embedding #23953
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@ProExpertProg it seems like the existing test_functionalization does not run on master and is not tested on ci. Do you have any suggestions on how to verify this change? I've manually verified the graph for Qwen3-4B has been modified after the pass, and the test case's TinyLlama-1.1B-Chat-v1.0-FP8-e2e graph has also been modified the the same way as before. |
angelayi
requested review from
ProExpertProg,
youkaichao and
zou3519
as code owners
|
Could you add the test to CI? I'll take a closer look later |
|
The test is failing because "'LLMEngine' object has no attribute 'model_executor'" -- sorry, do you know what's the proper way to get the model from this config? |
|
@ProExpertProg I updated this PR, also fixed the test_functionalization test case in (#24376) |
ProExpertProg
approved these changes
Oct 1, 2025
ProExpertProg
added
the
ready
angelayi
force-pushed
the
fix_func
branch
2 times, most recently
from
734275f to
d2c827c
Compare
Signed-off-by: angelayi <[email protected]>
tomeras91
pushed a commit
to tomeras91/vllm
that referenced
this pull request
Oct 6, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]> Signed-off-by: Tomer Asida <[email protected]>
karan
pushed a commit
to karan/vllm
that referenced
this pull request
Oct 6, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]> Signed-off-by: Karan Goel <[email protected]>
southfreebird
pushed a commit
to southfreebird/vllm
that referenced
this pull request
Oct 7, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]>
1 task
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 10, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
lywa1998
pushed a commit
to lywa1998/vllm
that referenced
this pull request
Oct 20, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]>
alhridoy
pushed a commit
to alhridoy/vllm
that referenced
this pull request
Oct 24, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
…ject#23953) Signed-off-by: angelayi <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Fixes #21101
Previously the fix_functionalization pass for rotary_embedding was written assuming the graph would have a specific pattern, where the rotary_embedding surrounded with slice scatters. This fails in the Qwen3-4B model where the graph no longer has this pattern. This PR adds a check for the slice_scatter pattern above, otherwise it will directly replace the auto_functionalized op with the op.
Test Plan
The issue runs successfully
Test Result