Skip to content

Fix torch version check for SM100 mxfp4 #22535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 12, 2025
Merged

Fix torch version check for SM100 mxfp4 #22535

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
29032dd
Fix torch version check for mxfp4
zifeitong Aug 8, 2025
7efd224
Fix format
zifeitong Aug 8, 2025
e36c83c
not current_platform.is_device_capability(100)
mgoin Aug 12, 2025
f26e9f0
Merge branch 'main' into fix_mxfp4_hopper
mgoin Aug 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions vllm/model_executor/layers/fused_moe/layer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -741,12 +741,14 @@ def __init__(

# we padding globally so EP buffer allocation works
if quant_config and quant_config.get_name() == "mxfp4":
if not is_torch_equal_or_newer("2.8.0"):
raise RuntimeError("Mxfp4 on hopper requires torch >= 2.8.0")
if current_platform.is_device_capability(
90) and not has_triton_kernels():
raise NotImplementedError(
"Triton kernels must be installed for mxfp4 on hopper")
if not current_platform.is_device_capability(100):
if not is_torch_equal_or_newer("2.8.0"):
raise RuntimeError(
"Mxfp4 on non-blackwell requires torch >= 2.8.0")
if not has_triton_kernels():
raise NotImplementedError(
"triton_kernels must be installed for "
"mxfp4 on non-blackwell")
if (current_platform.is_rocm()
or envs.VLLM_USE_FLASHINFER_MOE_MXFP4_MXFP8
or envs.VLLM_USE_FLASHINFER_MOE_MXFP4_BF16):
Expand Down