[TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs #9689
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mbs-octoml
requested review from
Hzfengsy,
ZihengJiang,
areusch,
comaniac,
icemelon,
jroesch,
junrushao,
kparzysz-quic,
masahi,
merrymercy,
tqchen,
vinx13 and
yzhliu
as code owners
Contributor
|
Have you consider also pass how much mem which primfunc can use? |
Contributor
Author
|
Hi @xqdan, thx for the comment. That takes us more into memory planning and what @manupa-arm is doing with USMP (including a representation of available memory pools, analysis to determine the conflict set for every abstract buffer, and a realization pass to resolve abstract buffers to physical within a pool). Currently the notion of 'memory/storage scope' we are using here is not connected to the USMP memory pools, or indeed anything at all! It's just a label we push around for use downstream. I see us eventually reconciling the 'flow memory scope constraints' aspect I'm working on here with the 'account for memory scope constraints during scheduling' work, but we'll need to get there gradually. |
…imFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery.
mbs-octoml
force-pushed
the
mbs-primfunc-constraints
branch
from
3398250 to
4468dd8
Compare
Contributor
Author
|
PTAL |
junrushao
approved these changes
Dec 9, 2021
Contributor
|
Thanks @mbs-octoml @xqdan @junrushao1994 |
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 7, 2022
…imFuncs (apache#9689) * [TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery. * [checkpoint] Junru's comments.
yangulei
pushed a commit
to yangulei/tvm
that referenced
this pull request
Jan 11, 2022
…imFuncs (apache#9689) * [TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery. * [checkpoint] Junru's comments.
yangulei
pushed a commit
to yangulei/tvm
that referenced
this pull request
Jan 12, 2022
…imFuncs (apache#9689) * [TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery. * [checkpoint] Junru's comments.
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 13, 2022
…imFuncs (apache#9689) * [TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery. * [checkpoint] Junru's comments.
qsqqsqqsq-intellif
pushed a commit
to qsqqsqqsq-intellif/tvm
that referenced
this pull request
Apr 29, 2022
…imFuncs (apache#9689) * [TIR] Allow memory (aka storage) scopes to be retrieved/applied to PrimFuncs This is in support of apache#9613 which allows memory scopes to flow out of already-lowered PrimFuncs into the rest of the Relay program. This means scope choices made during lowering can be accounted for in the rest of the program, with device_copies inserted as required. Somewhat more speculatively we also allow memory scopes to flow in to PrimFuncs. This is in preparation for when we can split lowering into two phases: i) lower "primitive" fused Relay functions to TensorIR in a schedulable form roughly isomorphic to TE, and ii) actual scheduling down to traditional TIR. Once that split is made it will be possible to flow memory scopes out of one PrimFunc and into another so as to avoid unnecessary device_copies being necessary due to independently chosen memory scopes. I also suspect we'll want to put our focus on layouts rather than memory scopes, but this at least sets up some of the machinery. * [checkpoint] Junru's comments.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is in support of #9613 which allows memory scopes to flow
out of already-lowered PrimFuncs into the rest of the Relay
program. This means scope choices made during lowering can
be accounted for in the rest of the program, with device_copies
inserted as required.
Somewhat more speculatively we also allow memory scopes to flow
in to PrimFuncs. This is in preparation for when we can split
lowering into two phases: i) lower "primitive" fused Relay
functions to TensorIR in a schedulable form roughly isomorphic
to TE, and ii) actual scheduling down to traditional TIR. Once
that split is made it will be possible to flow memory scopes
out of one PrimFunc and into another so as to avoid unnecessary
device_copies due to arbitrary independently chosen memory scopes.
I also suspect we'll want to put our focus on layouts rather
than memory scopes, but this at least sets up some of the
machinery.