Add quantized embedding kernels to torchao #1018
Merged
+609
−18
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1018
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 2169c32 with merge base 7aaf0ff ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 4, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
cceed9e to
d596bcb
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 4, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
d596bcb to
10ff165
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 7, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
10ff165 to
99ca201
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 7, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
99ca201 to
49363f4
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 8, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
49363f4 to
aabe6db
Compare
jerryzh168
approved these changes
Oct 8, 2024
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 8, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
aabe6db to
17e2deb
Compare
|
@jerryzh168 if things look good to you, can you approve the diff too D63839255 |
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 11, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
17e2deb to
7ef09aa
Compare
|
@metascroy I'm mostly just stamping the ao/experimental PR, do you need a proper review or just stamps for the diff? |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 15, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
7ef09aa to
8b62d71
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 15, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
8b62d71 to
3b92449
Compare
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 15, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
3b92449 to
8361f53
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
1 similar comment
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 16, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
8361f53 to
a719b34
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 16, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
a719b34 to
4aed4c2
Compare
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 16, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
4aed4c2 to
e5368f2
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
digantdesai
approved these changes
Oct 17, 2024
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 17, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Reviewed By: digantdesai Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
e5368f2 to
cd0f40f
Compare
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
added a commit
to metascroy/ao
that referenced
this pull request
Oct 17, 2024
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Reviewed By: digantdesai Differential Revision: D63839255
metascroy
force-pushed
the
export-D63839255
branch
from
cd0f40f to
7af0756
Compare
Summary: Pull Request resolved: pytorch#1018 This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels. Reviewed By: digantdesai Differential Revision: D63839255
|
This pull request was exported from Phabricator. Differential Revision: D63839255 |
metascroy
force-pushed
the
export-D63839255
branch
from
7af0756 to
2169c32
Compare
yanbing-j
pushed a commit
to yanbing-j/ao
that referenced
this pull request
Dec 9, 2024
Summary: This improves best tokens/sec from 73 to 85. Co-authored-by: Jack-Khuu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: This diff adds lowbit embedding kernels to torchao. These reuse the same bitpacking code as the linear kernels.
Differential Revision: D63839255