[USMP] Adding support for U4 usecase #10785
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
cc : @Mousius @areusch @grant-arm |
manupak
force-pushed
the
reusing_io
branch
2 times, most recently
from
f370571 to
1da8837
Compare
21 tasks
areusch
reviewed
Mar 31, 2022
Contributor
areusch
left a comment
areusch
left a comment
There was a problem hiding this comment.
here are some early comments on this one
Contributor
Author
|
a friendly ping for reviews... |
This commit adds support for placing I/O tensors within the workspace buffer. This is enabled using PassConfig option tir.usmp.use_workspace_io. Once it is enabled, it will remove the I/O tensors from the TIR main PrimFunc and replace them with Allocate nodes that is annotated to contain Input and Output tensors. The USMP will plan memory for them accordingly. (i.e. it will re-use space used by them for intermediaries depending on the liveness). This will only be supported with C Interface API. Thus, this commit produces two functions to the metadata sources to obtain input and output structs that points to location inside the workspace struct. Change-Id: I4c7e750ead9a880ba900602c17f53a125f97dbf9
Change-Id: I78f03d36b12b4a5e8eae8d11701f51019489defc
Change-Id: I857f3d0ba7bc192d56d750c44b232998b2876e7a
Contributor
Author
|
Thanks @areusch! I've addressed the comments. |
areusch
approved these changes
Apr 26, 2022
shtinsa
pushed a commit
to Deelvin/tvm
that referenced
this pull request
May 17, 2022
* [USMP] Adding support for U4 usecase This commit adds support for placing I/O tensors within the workspace buffer. This is enabled using PassConfig option tir.usmp.use_workspace_io. Once it is enabled, it will remove the I/O tensors from the TIR main PrimFunc and replace them with Allocate nodes that is annotated to contain Input and Output tensors. The USMP will plan memory for them accordingly. (i.e. it will re-use space used by them for intermediaries depending on the liveness). This will only be supported with C Interface API. Thus, this commit produces two functions to the metadata sources to obtain input and output structs that points to location inside the workspace struct. Change-Id: I4c7e750ead9a880ba900602c17f53a125f97dbf9 * fixup! [USMP] Adding support for U4 usecase Change-Id: I78f03d36b12b4a5e8eae8d11701f51019489defc * fixup! [USMP] Adding support for U4 usecase Change-Id: I857f3d0ba7bc192d56d750c44b232998b2876e7a
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds support for placing I/O
tensors within the workspace buffer.
This is enabled using PassConfig option
tir.usmp.use_workspace_io. Once it is enabled,
it will remove the I/O tensors from the TIR
main PrimFunc and replace them with Allocate
nodes that is annotated to contain Input and
Output tensors.
The USMP will plan memory for them accordingly.
(i.e. it will re-use space used by them for
intermediaries depending on the liveness).
This will only be supported with C Interface API.
Thus, this commit produces two functions to the
metadata sources to obtain input and output structs
that points to location inside the workspace struct.
RFC PR : apache/tvm-rfcs#65