[Torch] Various updates for PyTorch frontend #7348
Merged
+150
−14
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t-vi
reviewed
Jan 27, 2021
Contributor
t-vi
left a comment
t-vi
left a comment
There was a problem hiding this comment.
Looks good!
Since torch.sort returns both sorted values and indices while the Relay one doesn't, torch.sort conversion is not efficient, especially for multidimensional input (currently does both sort and argsort!). Suggestions for a better implementation are welcome.
I think gather is the direct equivalent of how you use take on 1d. Maybe it's worth fixing this in this PR.
Member
Author
|
@t-vi Thanks, I think I tried |
Member
Author
|
Thanks @siju-samuel @t-vi |
alexwong
pushed a commit
to alexwong/tvm
that referenced
this pull request
Feb 11, 2021
* add conversion for detr * remove explicit broadcast_to before batched matmul * use take with wrap mode * add test for transformer and negative indices * add sort and argsort * add logical_and * support masked_select * add gpu targets to masked_select test * improve sort conversion
electriclilies
pushed a commit
to electriclilies/tvm
that referenced
this pull request
Feb 18, 2021
* add conversion for detr * remove explicit broadcast_to before batched matmul * use take with wrap mode * add test for transformer and negative indices * add sort and argsort * add logical_and * support masked_select * add gpu targets to masked_select test * improve sort conversion
Lokiiiiii
pushed a commit
to Lokiiiiii/tvm
that referenced
this pull request
Mar 2, 2021
* add conversion for detr * remove explicit broadcast_to before batched matmul * use take with wrap mode * add test for transformer and negative indices * add sort and argsort * add logical_and * support masked_select * add gpu targets to masked_select test * improve sort conversion
trevor-m
pushed a commit
to neo-ai/tvm
that referenced
this pull request
Mar 2, 2021
* add conversion for detr * remove explicit broadcast_to before batched matmul * use take with wrap mode * add test for transformer and negative indices * add sort and argsort * add logical_and * support masked_select * add gpu targets to masked_select test * improve sort conversion
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds various updates to the PyTorch frontend, to fully support the recent transformer based object detection model from facebook, DETR https://github.com/facebookresearch/detr. After this PR, DETR runs on TVM and gets the correct results. TVM with auto scheduled GPU conv2d and batched matmul is 1.4x faster than PyTorch in my environment.
Also added various missing ops reported by hummingbird projects. @interesaaat
logical_and,masked_select,sortandargsort.masked_selectrequires VM to run.cumsumandmasked_fillop, to enable importing DETR https://github.com/facebookresearch/detr.cumsumis also requested by hummingbird.mode="wrap"in Relaytakeop. Without this, the result doesn't match with DETR.broadcast_tobeforebatch_matmul. Without it, memory usage blows up during constant evaluation of DETR. This is the same issue as [Topi] Allow batch_matmul to broadcast along batch dimension. #6616, please see the explanation there.torch.nn.Transformerto the tests.Since
torch.sortreturns both sorted values and indices while the Relay one doesn't,torch.sortconversion is not efficient,especially for multidimensional input (currently does both sort and argsort!). Suggestions for a better implementation are welcome.UPDATE: fixedplease review @siju-samuel @jwfromm @kevinthesun @t-vi