Skip to content

Conversation

@luccafong
Copy link

@luccafong luccafong commented Sep 29, 2025

fix mtp config with latest code and HF config

ran gsm8k num_fewshot=20 with TP=8 in eager mode:

VLLM_USE_DEEP_GEMM=0  VLLM_MLA_SPARSE_ENABLED=1 vllm serve "deepseek-ai/DeepSeek-V3.2-Exp" --max_model_len=20000 --gpu_memory_utilization=0.9 --tensor_parallel_size 8 --max_num_seqs=256 --speculative_config '{"num_speculative_tokens":1}' --enforce-eager
lm_eval --model local-completio
ns --tasks gsm8k     --model_args model=deepseek-ai/DeepSeek-V3
.2-Exp,base_url=http://127.0.0.1:8000/v1/completions,num_concur
rent=32,max_retries=3,tokenized_requests=False --batch_size 32 
--num_fewshot 20
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|    20|exact_match|↑  |0.9538|±  |0.0058|
|     |       |strict-match    |    20|exact_match|↑  |0.9538|±  |0.0058|

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

@luccafong luccafong force-pushed the mtp_config_enablement branch 6 times, most recently from de922e9 to 9c3d901 Compare September 29, 2025 21:00
@luccafong luccafong changed the title fix mtp config fix mtp config and padding Sep 29, 2025
@luccafong luccafong force-pushed the mtp_config_enablement branch from 9c3d901 to 4273a15 Compare September 29, 2025 21:12
Copy link
Owner

@heheda12345 heheda12345 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix!

@heheda12345 heheda12345 merged commit 618d877 into heheda12345:dsv32 Sep 29, 2025
heheda12345 pushed a commit that referenced this pull request Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants