Skip to content

Upstream merge 25 03 10 #471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 167 commits into from
Mar 10, 2025
Merged

Upstream merge 25 03 10 #471

merged 167 commits into from
Mar 10, 2025

Conversation

@gshtras
Copy link
Collaborator

@gshtras gshtras commented Mar 10, 2025 •
edited
Loading

Including early access to vllm-project#14316

hmellor and others added 30 commits March 3, 2025 17:48
@hmellor
Fix head_dim not existing in all model configs (Transformers backen…
91373a0 
…d) (vllm-project#14141)

Signed-off-by: Harry Mellor <[email protected]>
@markmc
[V0][Metrics] Remove unimplemented vllm:tokens_total (vllm-project#…
c41d271 
…14134)

Signed-off-by: Mark McLoughlin <[email protected]>
@markmc
[V0][Metrics] Deprecate some KV/prefix cache metrics (vllm-project#14136
2dfdfed 
)

Signed-off-by: Mark McLoughlin <[email protected]>
@njhill
[V1] Simplify stats logging (vllm-project#14082)
872db2b 
Signed-off-by: Nick Hill <[email protected]>
@markmc
[WIP][[V1][Metrics] Implement max_num_generation_tokens, request_para…
ae122b1 
…ms_n, and request_params_max_tokens metrics (vllm-project#14055)

Signed-off-by: Mark McLoughlin <[email protected]>
@mgoin
[Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (vl…
2b04c20 
…lm-project#14100)

Signed-off-by: mgoin <[email protected]>
@mgoin
[Kernel] Optimize moe intermediate_cache usage (vllm-project#13625)
19d98e0 
Signed-off-by: mgoin <[email protected]>
@Qubitium @mgoin
[Docs] Add GPTQModel (vllm-project#14056)
cd1d3c3 
Signed-off-by: mgoin <[email protected]>
Co-authored-by: mgoin <[email protected]>
@vanbasten23 @mgoin
[v1] Add comments to the new ragged paged attention Pallas kernel (vl…
79e4937 
…lm-project#14155)

Signed-off-by: Xiongfei Wei <[email protected]>
Co-authored-by: Michael Goin <[email protected]>
@tjohnson31415 @DarkLight1337
[Model] Add support for GraniteMoeShared models (vllm-project#13313)
c060b71 
Signed-off-by: Travis Johnson <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@divakar-amd
[core] moe fp8 block quant tuning support (vllm-project#14068)
bb5b640 
Signed-off-by: Divakar Verma <[email protected]>
@comaniac
[Misc] Remove lru_cache in NvmlCudaPlatform (vllm-project#14156)
989f4f4 
Signed-off-by: Cody Yu <[email protected]>
@ruisearch42
[core] Pass all driver env vars to ray workers unless excluded (vllm-…
bf13d40 
…project#14099)

Signed-off-by: Rui Qiao <[email protected]>
@zhanwenchen
Use math.prod instead of np.prod for trivial ops (vllm-project#14142)
66233af
@mgoin
Fix benchmark_moe.py tuning for CUDA devices (vllm-project#14164)
f78c0be
@youkaichao
[platform] add debug logging during inferring the device type (vllm-p…
ac65bc9 
…roject#14195)

Signed-off-by: youkaichao <[email protected]>
@youkaichao
[sleep mode] error out with expandable_segments (vllm-project#14189)
71c4b40 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[doc] add "Failed to infer device type" to faq (vllm-project#14200)
3610fb4 
Signed-off-by: youkaichao <[email protected]>
@mgoin
[Bugfix] Restrict MacOS CPU detection (vllm-project#14210)
6247bae 
Signed-off-by: mgoin <[email protected]>
@njhill
[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (v…
5db6b2c 
…llm-project#13869)

Signed-off-by: Nick Hill <[email protected]>
@markmc
[V0][Metrics] Deprecate some questionable request time metrics (vllm-…
c8525f0 
…project#14135)

Signed-off-by: Mark McLoughlin <[email protected]>
@lk-chen
[V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (vllm-project…
b3cf368 
…#14161)
@kushanam
add cutlass support for blackwell fp8 gemm (vllm-project#13798)
f89978a
@lsy323 @mgoin
[TPU][Profiler] Support start_profile/stop_profile in TPU worker (vll…
beebf47 
…m-project#13988)

Signed-off-by: Siyuan Liu <[email protected]>
Co-authored-by: mgoin <[email protected]>
@hmellor
Fix performance when --generation-config is not None (vllm-projec…
9badee5 
…t#14223)

Signed-off-by: Harry Mellor <[email protected]>
@hmellor
[Frontend] Do prompt_logprobs clamping for chat as well as completi…
e5b2f16 
…ons (vllm-project#14225)

Signed-off-by: Harry Mellor <[email protected]>
@mgoin
[Docs] Update Dockerfile dependency image (vllm-project#14215)
550c7ba 
Signed-off-by: mgoin <[email protected]>
@markmc @hmellor @comaniac
[v1][Metrics] Add design doc (vllm-project#12745)
c2bd219 
Signed-off-by: Mark McLoughlin <[email protected]>
Signed-off-by: Harry Mellor <[email protected]>
Co-authored-by: Harry Mellor <[email protected]>
Co-authored-by: Cody Yu <[email protected]>
@KuntaiDu
[Security] Serialize using safetensors instead of pickle in Mooncake …
288ca11 
…Pipe (vllm-project#14228)

Signed-off-by: KuntaiDu <[email protected]>
@tlrmchlsmth
Clean up unused padding_idx variables across many model definitions (v…
4f5b059 
…llm-project#13240)

Signed-off-by: Tyler Michael Smith <[email protected]>
russellb and others added 19 commits March 8, 2025 17:19
@russellb
Update CODEOWNERS for structured output (vllm-project#14496)
0d5e73d 
Signed-off-by: Russell Bryant <[email protected]>
@DarkLight1337
[Misc] Upgrade to Python 3.9 typing for additional directories (vllm-…
9513290 
…project#14492)

Signed-off-by: DarkLight1337 <[email protected]>
@22quinn @njhill
[V1] Support bad_words in sampler (vllm-project#13376)
eb8b5eb 
Signed-off-by: 22quinn <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
@robertgshaw2-redhat @ywang96
Revert "[V1][Core] Fix memory issue with logits & sampling" (vllm-pro…
5f0b53c 
…ject#14504)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@LucasWilkinson @tlrmchlsmth
[Attention] Default to FlashMLA backend for MLA (vllm-project#14451)
b0d5419 
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: Tyler Michael Smith <[email protected]>
Co-authored-by: Tyler Michael Smith <[email protected]>
@vanbasten23
[V1][TPU] Remove unnecessary padding for running on TPU. (vllm-projec…
10f7552 
…t#14467)
@YaoJiayi
[Feat] Support chunked prefill for LMCache connector (vllm-project#14505
6d7f037 
)

Signed-off-by: YaoJiayi <[email protected]>
@yanyc428
[Bugfix] Fix tqdm progress bar when SamplingParams.n > 1 (vllm-projec…
73ae0b4 
…t#12428)

Signed-off-by: Yuchen Yan <[email protected]>
@Isotr0py
[Bugfix] Revert QKVCrossParallelLinear usage in Mllama to keep BNB qu…
fb16eea 
…antization work (vllm-project#14498)

Signed-off-by: Isotr0py <[email protected]>
@yaochengji
[Hardware][TPU] Fix the recompiling issue in logits processor after w…
212007b 
…armup (vllm-project#14510)

Signed-off-by: Chengji Yao <[email protected]>
@liuyanyi
[Misc] Ensure out-of-tree quantization method recognize by cli args (v…
a21076e 
…llm-project#14328)

Signed-off-by: liuyanyi <[email protected]>
@martinhoyer
[Bugfix] Wrong requirements path - rocm (vllm-project#14527)
dc74613 
Signed-off-by: Martin Hoyer <[email protected]>
@JenZhao @ywang96
[Feature] Consolidate performance benchmark datasets (vllm-project#14036
1253b15 
)

Signed-off-by: Jennifer Zhao <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Jennifer Zhao <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@chaunceyjiang
[Misc] Add log information for handle_process_request. (vllm-project#…
460f553 
…14130)

Signed-off-by: chaunceyjiang <[email protected]>
@hmellor
[Docs] Mention model_impl arg when explaining Transformers fallback (…
60a98b2 
…vllm-project#14552)

Signed-off-by: Harry Mellor <[email protected]>
@chaunceyjiang
[Frontend] support image embeds (vllm-project#13955)
b0746fa 
Signed-off-by: chaunceyjiang <[email protected]>
@SzymonOzog
[Kernel] Add more dtype support for GGUF kernels (vllm-project#14043)
89cdaa8 
Signed-off-by: SzymonOzog <[email protected]>
Signed-off-by: SzymonOzog <[email protected]>
@DarkLight1337
[Doc] Update PaliGemma note to a warning (vllm-project#14565)
001a9c7 
Signed-off-by: DarkLight1337 <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main'
9ef3d37
@gshtras gshtras merged commit 1095cff into main Mar 10, 2025
3 of 5 checks passed
@gshtras gshtras deleted the upstream_merge_25_03_10 branch March 10, 2025 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shajrawi shajrawi shajrawi approved these changes

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD

@maleksan85 maleksan85 Awaiting requested review from maleksan85

@sunway513 sunway513 Awaiting requested review from sunway513

@divakar-amd divakar-amd Awaiting requested review from divakar-amd

@charlifu charlifu Awaiting requested review from charlifu

@mawong-amd mawong-amd Awaiting requested review from mawong-amd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.