[CUTLASS] More robust support for pattern matching and alignment #9698
+139
−54
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
masahi
requested review from
Huyuwei,
Laurawly,
MarisaKirisame,
ZihengJiang,
anijain2305,
areusch,
comaniac,
icemelon,
jcf94,
jroesch,
junrushao,
jwfromm,
kevinthesun,
mbrookhart,
merrymercy,
slyubomirsky,
tqchen,
vinx13,
wweic,
yzhliu and
zhiics
as code owners
masahi
force-pushed
the
cutlass-more-robust
branch
from
5db9b52 to
7f258d3
Compare
masahi
force-pushed
the
cutlass-more-robust
branch
3 times, most recently
from
2714e30 to
cd83677
Compare
Member
Author
masahi
force-pushed
the
cutlass-more-robust
branch
3 times, most recently
from
807ea11 to
5b439d1
Compare
masahi
force-pushed
the
cutlass-more-robust
branch
from
5b439d1 to
1d5ce30
Compare
masahi
force-pushed
the
cutlass-more-robust
branch
from
1d5ce30 to
73740c4
Compare
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 7, 2022
…che#9698) * bug fix in im2col encoding * skip legalize when batch size is dynamic * add sm75 kernels to sm80 profilings * add dtype and layout check in parttern match * use align1 kernel for unusual channel cases (IC = 3 etc) * test IC=3 convolution * fixed check functions for fused cases, run infer type before mergecomposite * check align on N dim * add comment on IC == 3 case * lint fix * do not offload depthwise conv2d * lint * trigger CI
yangulei
pushed a commit
to yangulei/tvm
that referenced
this pull request
Jan 11, 2022
…che#9698) * bug fix in im2col encoding * skip legalize when batch size is dynamic * add sm75 kernels to sm80 profilings * add dtype and layout check in parttern match * use align1 kernel for unusual channel cases (IC = 3 etc) * test IC=3 convolution * fixed check functions for fused cases, run infer type before mergecomposite * check align on N dim * add comment on IC == 3 case * lint fix * do not offload depthwise conv2d * lint * trigger CI
yangulei
pushed a commit
to yangulei/tvm
that referenced
this pull request
Jan 12, 2022
…che#9698) * bug fix in im2col encoding * skip legalize when batch size is dynamic * add sm75 kernels to sm80 profilings * add dtype and layout check in parttern match * use align1 kernel for unusual channel cases (IC = 3 etc) * test IC=3 convolution * fixed check functions for fused cases, run infer type before mergecomposite * check align on N dim * add comment on IC == 3 case * lint fix * do not offload depthwise conv2d * lint * trigger CI
ylc
pushed a commit
to ylc/tvm
that referenced
this pull request
Jan 13, 2022
…che#9698) * bug fix in im2col encoding * skip legalize when batch size is dynamic * add sm75 kernels to sm80 profilings * add dtype and layout check in parttern match * use align1 kernel for unusual channel cases (IC = 3 etc) * test IC=3 convolution * fixed check functions for fused cases, run infer type before mergecomposite * check align on N dim * add comment on IC == 3 case * lint fix * do not offload depthwise conv2d * lint * trigger CI
qsqqsqqsq-intellif
pushed a commit
to qsqqsqqsq-intellif/tvm
that referenced
this pull request
Apr 29, 2022
…che#9698) * bug fix in im2col encoding * skip legalize when batch size is dynamic * add sm75 kernels to sm80 profilings * add dtype and layout check in parttern match * use align1 kernel for unusual channel cases (IC = 3 etc) * test IC=3 convolution * fixed check functions for fused cases, run infer type before mergecomposite * check align on N dim * add comment on IC == 3 case * lint fix * do not offload depthwise conv2d * lint * trigger CI
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
N, Kdimensions are not divisible by 8, 4 etc, i.e.align = 1is mandatory. This came up in MaskRCNN where there is a workload(M, N, K) = (?, 91, 1024). Since the output shape is(?, 91), we cannot do vectorized memory access on the output tensor. Hence, onlyalign = 1variants are valid candidates. The same fix also enabled offloading conv2d withIC = 3case.align = 1variants are valid candidates.conv2d(see the change ingen_conv2d.py). But this code will be removed soon anyway when I introduce the dedicated conv2d profiler and kernel selection.Since we need to use
align = 1kernels for cases above and CUTLASS sm80 gemm kernels do not seem to supportalign=1, I had to add sm75 kernels to sm80 kernel selection (see the change ingen_tensor_op.py). It turned out sm75 kernels are faster than sm80 on RTX 3070 on some workloads, so mixing sm75 and 80 seems to be a good idea.cc @comaniac @Laurawly