chore: Mass integration of release/0.20. #4732
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
) * Restore per-channel pre-quant Signed-off-by: Barry Kang <[email protected]> * Update TRT test script Signed-off-by: Barry Kang <[email protected]> * Fix pre-commit Signed-off-by: Barry Kang <[email protected]> --------- Signed-off-by: Barry Kang <[email protected]>
Signed-off-by: Ivy Zhang <[email protected]>
Signed-off-by: Yiqing Yan <[email protected]>
…e memory and log more memory information (NVIDIA#4660) Signed-off-by: Hui Gao <[email protected]>
Signed-off-by: nv-guomingz <[email protected]>
…d weight loading in fused moe. (NVIDIA#4699) Signed-off-by: Yuxian Qiu <[email protected]>
Signed-off-by: Balaram Buddharaju <[email protected]>
|
/bot run |
|
PR_Github #6768 [ run ] triggered by Bot |
shaharmor98
requested review from
crazydemo,
nv-guomingz,
yuxianq,
brb-nv,
yiqingy0 and
HuiGao-NV
shaharmor98
changed the title
nv-guomingz
approved these changes
May 28, 2025
crazydemo
approved these changes
May 28, 2025
|
PR_Github #6768 [ run ] completed with state |
yiqingy0
approved these changes
May 29, 2025
Barry-Delaney
approved these changes
May 29, 2025
amirkl94
reviewed
May 29, 2025
| # expert_idx is the local slot index of current rank | ||
| expert_idx = local_slot_id | ||
| max_workers = min( | ||
| (self.expert_end - self.expert_start) * 2, |
There was a problem hiding this comment.
Please use self.expert_size_per_partition instead of (self.expert_end - self.expert_start)
There was a problem hiding this comment.
@hlu1 My PR #4699 may conflict with your PR #4790 in this mass integration.
@amirkl94 If Hao's PR merged first, we should cherry-pick this change to tensorrt_llm/_torch/modules/fused_moe/quantization.py in https://github.com/NVIDIA/TensorRT-LLM/pull/4790/files#diff-19b05de4a4dd136814f3e04d4ed51c2e4f2389c7b0b2a6bca49195150ebadd66R87 instead.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Mass integration of release/0.20 to main.