Skip to content

Upstream merge 2025 04 07 #503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 70 commits into from
Apr 7, 2025
Merged

Upstream merge 2025 04 07 #503

merged 70 commits into from
Apr 7, 2025

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Apr 7, 2025 •
edited by github-actions bot
Loading

Compatible with LLama 4 in V1
For V0 need to have vllm-project#16198 when it's ready

yihong0618 and others added 30 commits April 3, 2025 15:18
@yihong0618
fix: tiny fix make format.sh excutable (vllm-project#16015)
84884cd 
Signed-off-by: yihong0618 <[email protected]>
@kylesayrs
[SupportsQuant] Bert, Blip, Blip2, Bloom (vllm-project#15573)
421c462 
Signed-off-by: Kyle Sayers <[email protected]>
@kylesayrs
[SupportsQuant] Chameleon, Chatglm, Commandr (vllm-project#15952)
82e7e19 
Signed-off-by: Kyle Sayers <[email protected]>
@liangfu
[Neuron][kernel] Fuse kv cache into a single tensor (vllm-project#15911)
d2b58ca 
Signed-off-by: Liangfu Chen <[email protected]>
@bnellnm
[Minor] Fused experts refactor (vllm-project#15914)
15ba07e 
Signed-off-by: Bill Nell <[email protected]>
@yarongmu-google
[Misc][Performance] Advance tpu.txt to the most recent nightly torch … (
45b1ff7 
vllm-project#16024)
@Alexei-V-Ivanov-AMD
Re-enable the AMD Testing for the passing tests. (vllm-project#15586)
03a70ea 
Signed-off-by: Alexei V. Ivanov <[email protected]>
@vanbasten23
[TPU] Support sliding window and logit soft capping in the paged atte…
b6be6f8 
…ntion kernel for TPU. (vllm-project#15732)

Signed-off-by: Xiongfei Wei <[email protected]>
@robertgshaw2-redhat
[TPU] Switch Test to Non-Sliding Window (vllm-project#15981)
f15e70d 
Signed-off-by: Robert Shaw <[email protected]>
Co-authored-by: Robert Shaw <[email protected]>
@bnellnm
[Bugfix] Fix function names in test_block_fp8.py (vllm-project#16033)
dcc56d6 
Signed-off-by: Bill Nell <[email protected]>
@huydhn
[ROCm] Tweak the benchmark script to run on ROCm (vllm-project#14252)
092475f
@reidliu41
[Misc] improve gguf check (vllm-project#15974)
86cbd2e 
Signed-off-by: reidliu41 <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
@yaochengji
[TPU][V1] Remove ragged attention kernel parameter hard coding (vllm-…
fadc59c 
…project#16041)

Signed-off-by: Chengji Yao <[email protected]>
@yihong0618
doc: add info for macos clang errors (vllm-project#16049)
4ef0bb1 
Signed-off-by: yihong0618 <[email protected]>
@markmc
[V1][Spec Decode] Avoid logging useless nan metrics (vllm-project#16023)
a35a8a8 
Signed-off-by: Mark McLoughlin <[email protected]>
@jonghyunchoe
[Model] use AutoWeightsLoader for baichuan, gpt-neox, mpt (vllm-proje…
bf7e3c5 
…ct#15939)

Signed-off-by: Jonghyun Choe <[email protected]>
@zhenwei-intel
[Hardware][Gaudi][BugFix] fix arguments of hpu fused moe (vllm-projec…
0812d8d 
…t#15945)

Signed-off-by: zhenwei <[email protected]>
@Isotr0py
[Bugfix][kernels] Fix half2float conversion in gguf kernels (vllm-pro…
230b131 
…ject#15995)

Signed-off-by: Isotr0py <[email protected]>
@StevenShi-23 @ywang96
[Benchmark][Doc] Update throughput benchmark and README (vllm-project…
95862f7 
…#15998)

Signed-off-by: StevenShi-23 <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@bigPYJ1151
[CPU] Change default block_size for CPU backend (vllm-project#16002)
2386803 
Signed-off-by: jiang1.li <[email protected]>
@ilmarkov
[Distributed] [ROCM] Fix custom allreduce enable checks (vllm-project…
ef608c3 
…#16010)

Signed-off-by: ilmarkov <[email protected]>
Co-authored-by: ilmarkov <[email protected]>
@gshtras
[ROCm][Bugfix] Use platform specific FP8 dtype (vllm-project#15717)
40a36cc 
Signed-off-by: Gregory Shtrasberg <[email protected]>
@gshtras
[ROCm][Bugfix] Bring back fallback to eager mode removed in vllm-proj…
a6d042d 
…ect#14917, but for ROCm only (vllm-project#15413)

Signed-off-by: Gregory Shtrasberg <[email protected]>
@mgoin
[Bugfix] Fix default behavior/fallback for pp in v1 (vllm-project#16057)
4708f13 
Signed-off-by: mgoin <[email protected]>
@khluu
[CI] Reorganize .buildkite directory (vllm-project#16001)
4dc52e1 
Signed-off-by: kevin <[email protected]>
@njhill
[V1] DP scale-out (1/N): Use zmq ROUTER/DEALER sockets for input queue (
651cf0f 
vllm-project#15906)

Signed-off-by: Nick Hill <[email protected]>
@DarkLight1337 @mgoin @ywang96
[V1] Scatter and gather placeholders in the model runner (vllm-projec…
f5722a5 
…t#15712)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: mgoin <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: mgoin <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
@ywang96
Revert "[V1] Scatter and gather placeholders in the model runner" (vl…
af51d80 
…lm-project#16075)
@bnellnm
[Kernel][Minor] Re-fuse triton moe weight application (vllm-project#1…
d6fc629 
…6071)

Signed-off-by: Bill Nell <[email protected]>
@mgoin
[Bugfix][TPU] Fix V1 TPU worker for sliding window (vllm-project#16059)
70ad3f9 
Signed-off-by: Michael Goin <[email protected]>
reidliu41 and others added 19 commits April 6, 2025 14:56
@reidliu41
[Misc] update requires-python in pyproject.toml (vllm-project#16116)
72c8f1a 
Signed-off-by: reidliu41 <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
@yaochengji
[TPU] Update PyTorch/XLA (vllm-project#16130)
9bde5ba 
Signed-off-by: Chengji Yao <[email protected]>
@WoosukKwon
[V1][Minor] Optimize get_cached_block (vllm-project#16135)
b10e519
@martinhoyer
Fix requires-python (vllm-project#16132)
2549c0d
@yankay
[Metrics] Add bucket for request_latency, time_to_first_token and…
86fc232 
… `time_per_output_token` (vllm-project#15202)

Signed-off-by: Kay Yan <[email protected]>
@WoosukKwon
[V1][Minor] Minor simplification for get_computed_blocks (vllm-projec…
3749e28 
…t#16139)

Signed-off-by: Woosuk Kwon <[email protected]>
@DarkLight1337
[Misc] Update Mistral-3.1 example (vllm-project#16147)
0a57386 
Signed-off-by: DarkLight1337 <[email protected]>
@Isotr0py
[Bugfix] Make dummy encoder prompt padding alternative and add missin…
fc0f877 
…g warnings (vllm-project#16129)

Signed-off-by: Isotr0py <[email protected]>
@ywang96
[CI] Set max transformers version for Ultravox model test (vllm-proje…
bb8dab8 
…ct#16149)

Signed-off-by: Roger Wang <[email protected]>
@yihong0618
doc: fix some typos in doc (vllm-project#16154)
95d63f3 
Signed-off-by: yihong0618 <[email protected]>
@Isotr0py
[VLM] Florence-2 supports online serving (vllm-project#16164)
7c80368 
Signed-off-by: Isotr0py <[email protected]>
@shen-shanshan
[V1][Structured Output] Add supports_structured_output() method to …
e9ba99f 
…Platform (vllm-project#16148)

Signed-off-by: shen-shanshan <[email protected]>
@YamPengLi @DarkLight1337
[Model] Add Qwen3 and Qwen3MoE (vllm-project#15289)
7699258 
Signed-off-by: YamPengLi <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@reidliu41
[Misc] improve example mlpspeculator and llm_engine_example (vllm-pro…
dc3529d 
…ject#16175)

Signed-off-by: reidliu41 <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
@WangErXiao
[Doc]Update image to latest version (vllm-project#16186)
8017c8d 
Signed-off-by: WangErXiao <[email protected]>
@houseroad @luccafong
@ywang96 @DarkLight1337
Upstream Llama4 Support to Main (vllm-project#16113)
55dcce9 
Signed-off-by: Aston Zhang <[email protected]>
Signed-off-by: Chris Thi <[email protected]>
Signed-off-by: drisspg <[email protected]>
Signed-off-by: Jon Swenson <[email protected]>
Signed-off-by: Keyun Tong <[email protected]>
Signed-off-by: Lu Fang <[email protected]>
Signed-off-by: Xiaodong Wang <[email protected]>
Signed-off-by: Yang Chen <[email protected]>
Signed-off-by: Ye (Charlotte) Qi <[email protected]>
Signed-off-by: Yong Hoon Shin <[email protected]>
Signed-off-by: Zijing Liu <[email protected]>
Signed-off-by: Lu Fang <[email protected]>
Signed-off-by: Lu Fang <[email protected]>
Signed-off-by: Lucia Fang <[email protected]>
Signed-off-by: Roger Wang <[email protected]>
Signed-off-by: DarkLight1337 <[email protected]>
Co-authored-by: Lu Fang <[email protected]>
Co-authored-by: Roger Wang <[email protected]>
Co-authored-by: DarkLight1337 <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main'
f657987
@DarkLight1337
[Bugfix] Re-enable support for ChatGLMForConditionalGeneration (vll…
027b204 
…m-project#16187)

Signed-off-by: DarkLight1337 <[email protected]>
@gshtras
Merge remote-tracking branch 'upstream/main' into upstream_merge_2025…
2b6e9c9 
…_04_07
@gshtras gshtras merged commit d17d4df into main Apr 7, 2025
3 of 5 checks passed
@gshtras gshtras deleted the upstream_merge_2025_04_07 branch April 7, 2025 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shajrawi shajrawi shajrawi approved these changes

@Alexei-V-Ivanov-AMD Alexei-V-Ivanov-AMD Awaiting requested review from Alexei-V-Ivanov-AMD

@maleksan85 maleksan85 Awaiting requested review from maleksan85

@sunway513 sunway513 Awaiting requested review from sunway513

@hongxiayang hongxiayang Awaiting requested review from hongxiayang

@divakar-amd divakar-amd Awaiting requested review from divakar-amd

@charlifu charlifu Awaiting requested review from charlifu

@mawong-amd mawong-amd Awaiting requested review from mawong-amd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.