Skip to content

Upstream merge 25 03 03 #458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 95 commits into from
Mar 4, 2025
Merged

Upstream merge 25 03 03 #458

merged 95 commits into from
Mar 4, 2025

Conversation

@gshtras
Copy link
Collaborator

@gshtras gshtras commented Mar 4, 2025

Also reverting vllm-project#13718. It fails correctness tests in combination with custom all reduce. @qli88 to investigate

liangfu and others added 30 commits February 25, 2025 11:47
@liangfu @gnovack @AoyuQC
[Neuron] Add custom_ops for neuron backend (vllm-project#13246)
f75aa72 
Signed-off-by: Liangfu Chen <[email protected]>
Co-authored-by: George Novack <[email protected]>
Co-authored-by: Aoyu Zhang <[email protected]>
@hmellor
Fix failing MyGemma2Embedding test (vllm-project#13820)
34e3494 
Signed-off-by: Harry Mellor <[email protected]>
@mgoin
[Model] Support Grok1 (vllm-project#13795)
07c4353 
Signed-off-by: mgoin <[email protected]>
@hmellor
DeepSeek V2/V3/R1 only place lm_head on last pp rank (vllm-project#…
2467978 
…13833)

Signed-off-by: Harry Mellor <[email protected]>
@ruisearch42
[misc] Show driver IP info when Ray fails to allocate driver worker (v…
9ba2804 
…llm-project#13858)

Signed-off-by: Rui Qiao <[email protected]>
@LiuXiaoxuanPKU
[V1][Spec Decode] Change Spec Decode Rejection Sampling API (vllm-pro…
5629f26 
…ject#13729)
@noemotiovon
[Misc]Code Cleanup (vllm-project#13859)
e1fe759 
Signed-off-by: noemotiovon <[email protected]>
Co-authored-by: noemotiovon <[email protected]>
@henrylhtsang
[Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutla…
094b7d9 
…ss epilogues (vllm-project#13797)
@hmellor
Improve pipeline partitioning (vllm-project#13839)
145944c
@tianyuzhou95
[Doc] fix the incorrect module path of tensorize_vllm_model (vllm-pro…
e656f63 
…ject#13863)
@SageMoore
[ROCm] Disable chunked prefill/prefix caching when running MLA on non…
1d35662 
…-cuda platforms (vllm-project#13844)

Signed-off-by: Sage Moore <[email protected]>
@sethkimmel3
[v0][Core] Use xgrammar shared context to avoid copy overhead for off…
e206b54 
…line engine (vllm-project#13837)

Signed-off-by: Seth Kimmel <[email protected]>
@jeejeelee
[Misc] Improve LoRA spelling (vllm-project#13831)
5157338
@ywang96
[Misc] Fix input processing for Ultravox (vllm-project#13871)
7ca1da0
@DarkLight1337
[Bugfix] Add test example for Ultravox v0.5 (vllm-project#13890)
7b700ec
@hmellor
Add comments on accessing kv_cache and attn_metadata (vllm-projec…
0ecdd98 
…t#13887)

Signed-off-by: Harry Mellor <[email protected]>
@fgreinacher
[Bugfix] Handle None parameters in Mistral function calls. (vllm-proj…
215bf15 
…ect#13786)
@b8zhong
[Misc]: Add support for goodput on guided benchmarking + TPOT calcula…
ec8a5e5 
…tion refactor (vllm-project#13736)

Signed-off-by: Brayden Zhong <[email protected]>
@joerunde
[Bugfix] Do not crash V0 engine on input errors (vllm-project#13101)
3f808cc 
Signed-off-by: Joe Runde <[email protected]>
@DarkLight1337
[Bugfix] Update expected token counts for Ultravox tests (vllm-projec…
934bb99 
…t#13895)
@Chenyaaang
[TPU] use torch2.6 with whl package (vllm-project#13860)
b27122a 
Signed-off-by: Chenyaaang <[email protected]>
@chaunceyjiang
[Misc] fixed qwen_vl_utils parameter error (vllm-project#13906)
d08b285
@wallashss @joerunde
[Bugfix] Backend option to disable xgrammar any_whitespace (vllm-proj…
4cb6fa0 
…ect#12744)

Signed-off-by: Wallas Santos <[email protected]>
Signed-off-by: Joe Runde <[email protected]>
Co-authored-by: Joe Runde <[email protected]>
@WoosukKwon
[BugFix] Make FP8 Linear compatible with torch.compile (vllm-project#…
b382a7f 
…13918)

Signed-off-by: Woosuk Kwon <[email protected]>
@LucasWilkinson
[Kernel] FlashMLA integration (vllm-project#13747)
f959039 
Signed-off-by: Lucas Wilkinson <[email protected]>
Signed-off-by: Lucas Wilkinson <[email protected]>
@HollowMan6
[ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undef…
a31614e 
…ined (vllm-project#13851)

Signed-off-by: Hollow Man <[email protected]>
@mgoin
Use CUDA 12.4 as default for release and nightly wheels (vllm-project…
ca377cf 
…#12098)
@ruisearch42
[misc] Rename Ray ADAG to Compiled Graph (vllm-project#13928)
c9944ac
@SageMoore
[ROCm][V1] Update reshape_and_cache to properly work with CUDA graph …
378b3ef 
…padding (vllm-project#13922)
@markmc
[V1][Metrics] Handle preemptions (vllm-project#13169)
cd711c4
heheda12345 and others added 15 commits March 1, 2025 20:46
@heheda12345
[v1] Add __repr__ to KVCacheBlock to avoid recursive print (vllm-pr…
d54990d 
…oject#14081)
@jeejeelee
[Model] Add LoRA support for TransformersModel (vllm-project#13770)
cc5e8f6 
Signed-off-by: Jee Jee Li <[email protected]>
@waltforme
[Misc] Accurately capture the time of loading weights (vllm-project#1…
82fbeae 
…4063)

Signed-off-by: Jun Duan <[email protected]>
@qux-bbb
[Doc] Source building add clone step (vllm-project#14086)
bc6ccb9 
Signed-off-by: qux-bbb <[email protected]>
@gaocegege
[v0][structured output] Support reasoning output (vllm-project#12955)
bf33700 
Signed-off-by: Ce Gao <[email protected]>
@hmellor
Update deprecated Python 3.8 typing (vllm-project#13971)
cf069aa
@realShengYao
[Bugfix] Explicitly include "omp.h" for MacOS to avoid installation f…
09e56f9 
…ailure (vllm-project#14051)
@noooop
[Misc] duplicate code in deepseek_v2 (vllm-project#14106)
e584b85
@MengqingCao
[Misc][Platform] Move use allgather to platform (vllm-project#14010)
b87c21f 
Signed-off-by: Mengqing Cao <[email protected]>
@comaniac
[Build] Make sure local main branch is synced when VLLM_USE_PRECOMPIL…
f35f8e2 
…ED=1 (vllm-project#13921)

Signed-off-by: Cody Yu <[email protected]>
@markmc
[V1] Refactor parallel sampling support (vllm-project#13774)
4167252 
Signed-off-by: Mark McLoughlin <[email protected]>
@hmellor
Improve the docs for TransformersModel (vllm-project#14147)
98175b2 
Signed-off-by: Harry Mellor <[email protected]>
@tjtanaa
[ROCm] Faster Custom Paged Attention kernels (vllm-project#12348)
848a643
@gshtras
Merge remote-tracking branch 'upstream/main' into upstream_merge_25_0…
24c6283 
…3_03
@gshtras
Revert "[core] Perf improvement for DSv3 on AMD GPUs (vllm-project#13718
87bf00a 
)"

This reverts commit 8294773.
shajrawi
shajrawi previously approved these changes Mar 4, 2025
View reviewed changes
@gshtras gshtras merged commit caa2810 into main Mar 4, 2025
13 of 15 checks passed
@gshtras gshtras deleted the upstream_merge_25_03_03 branch March 4, 2025 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD

@maleksan85 maleksan85 Awaiting requested review from maleksan85

@sunway513 sunway513 Awaiting requested review from sunway513

@divakar-amd divakar-amd Awaiting requested review from divakar-amd

@charlifu charlifu Awaiting requested review from charlifu

@mawong-amd mawong-amd Awaiting requested review from mawong-amd

@AdrianAbeyta AdrianAbeyta Awaiting requested review from AdrianAbeyta

@dllehr-amd dllehr-amd Awaiting requested review from dllehr-amd

1 more reviewer

@shajrawi shajrawi shajrawi approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.