Skip to content

[Feature]: Inductor partitioning should decide what ops to partition on dynamically #25691

@ProExpertProg

Description

@ProExpertProg

🚀 The feature, motivation and pitch

Currently, if use_inductor_graph_partition is set, we always partition on attention by making it cudagraph unsafe - instead we should respect splitting_ops (or likely make a new field). Use register_should_partition_rule from pytorch/pytorch#163310 and pytorch/pytorch#163395.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Metadata

Metadata

Type

No type

Projects

Status

Done

Relationships

None yet

Development

No branches or pull requests

Issue actions