[Chore] Cleanup guided namespace, move to structured outputs config #22772
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aarnphm
requested review from
DarkLight1337,
WoosukKwon,
alexm-redhat,
comaniac,
hmellor,
mgoin,
njhill,
patrickvonplaten,
robertgshaw2-redhat,
russellb,
simon-mo,
youkaichao,
ywang96 and
zhuohan123
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
mergify
bot
added
documentation
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Aaron Pham <[email protected]>
aarnphm
force-pushed
the
feat/decoding-args-rename-all
branch
from
d3ac885 to
69068cd
Compare
aarnphm
requested review from
houseroad,
tlrmchlsmth and
yewentao256
as code owners
weijinqian0
pushed a commit
to weijinqian0/vllm-ascend
that referenced
this pull request
Sep 22, 2025
…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]>
mansiag05
pushed a commit
to mansiag05/pytorch
that referenced
this pull request
Sep 22, 2025
They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: pytorch#163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet
Mercykid-bash
pushed a commit
to Mercykid-bash/vllm-ascend
that referenced
this pull request
Sep 22, 2025
…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]> Signed-off-by: Che Ruan <[email protected]>
Mercykid-bash
pushed a commit
to Mercykid-bash/vllm-ascend
that referenced
this pull request
Sep 22, 2025
…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]> Signed-off-by: Che Ruan <[email protected]>
cleonard530
pushed a commit
to cleonard530/pytorch
that referenced
this pull request
Sep 22, 2025
They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: pytorch#163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet
|
@aarnphm can we add backward compatibility for one version so people know how to migrate? |
|
BC for BC for CLI was already maintained in this PR The onlt other area we may want to consider is BC in the server API (not sure how we want to handle that) |
|
I think backwards compatibility in the server API is critical. https://docs.vllm.ai/en/latest/contributing/deprecation_policy.html#overview |
|
I can make a similar PR ensuring BC for the server API. However I'm not sure how we can warn a user of the deprecated API that it's deprecated? |
Only in docs for the first step. The next step for the HTTP API would be to turn it off by default, but make an option for turning it back on. Then finally, remove it completely. |
russellb
added a commit
to russellb/vllm
that referenced
this pull request
Sep 23, 2025
This was a mistake introduced by vllm-project#22772. Structured output requests were not actually working because the format spec was not placed in the proper new location in the request body. Signed-off-by: Russell Bryant <[email protected]>
slokesha
pushed a commit
to slokesha/vllm-gaudi
that referenced
this pull request
Sep 24, 2025
Culprit commit: vllm-project/vllm#22772 --------- Signed-off-by: Agata Dobrzyniewicz <[email protected]> Signed-off-by: slokesha <[email protected]>
This was referenced Sep 24, 2025
|
I've just created #25615 to add BC for the server API too |
FeiDaLI
pushed a commit
to FeiDaLI/vllm
that referenced
this pull request
Sep 25, 2025
…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]>
charlifu
pushed a commit
to ROCm/vllm
that referenced
this pull request
Sep 25, 2025
…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]> Signed-off-by: charlifu <[email protected]>
dsashidh
pushed a commit
to dsashidh/pytorch
that referenced
this pull request
Sep 26, 2025
They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: pytorch#163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet
pytorchbot
pushed a commit
to pytorch/pytorch
that referenced
this pull request
Sep 30, 2025
They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: #163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet (cherry picked from commit a31acf3)
Camyll
pushed a commit
to pytorch/pytorch
that referenced
this pull request
Sep 30, 2025
Clean up obsoleted vLLM tests (#163383) They have been removed in vllm-project/vllm#25117 and vllm-project/vllm#22772, thus failing in trunk at the moment after the latest pin commit update Pull Request resolved: #163383 Approved by: https://github.com/wdvr, https://github.com/seemethere, https://github.com/malfet (cherry picked from commit a31acf3) Co-authored-by: Huy Do <[email protected]>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 10, 2025
…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
choprahetarth
pushed a commit
to Tandemn-Labs/vllm
that referenced
this pull request
Oct 11, 2025
…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]>
Angazenn
pushed a commit
to Angazenn/vllm-ascend
that referenced
this pull request
Oct 21, 2025
…llm-project#2907) ### What this PR does / why we need it? 1. This pr bump vllm commit to vllm-project/vllm@6d8246a 2. fix upstream changes vllm-project/vllm#24548 abort multi-modal kwargs, make vllm main and `v0.10.2` both adaptable 3. fix metadata_builder changes introduced by vllm-project/vllm#23693 4. fix `structured_outputs_config` changes introduced by vllm-project/vllm#22772 5. fix `moe_config` changes introduced by vllm-project/vllm#22537 Co-authored-by: MengqingCao <[email protected]> Co-authored-by: Yikun Jiang <[email protected]> - vLLM version: v0.10.2 - vLLM main: vllm-project/vllm@c60e613 --------- Signed-off-by: wangli <[email protected]> Signed-off-by: MengqingCao <[email protected]> Co-authored-by: MengqingCao <[email protected]>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
…llm-project#22772) Signed-off-by: Aaron Pham <[email protected]> Signed-off-by: Harry Mellor <[email protected]> Co-authored-by: Harry Mellor <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Continuation of #17420
This PR introduces the args --structured-output-config as a way to unify all related structured outputs config in one CLI field.
This would help simplify general UX for specifying custom options with backends.
I also remove all previous guided_decoding options
This would also be considered breaking. There will be no
--guided-decoding-*option anymore. Instead, you should use--structured-outputs-config '{...}'or--structured-outputs-config.backend outlinesSigned-off-by: Aaron Pham [email protected]
Signed-off-by: Harry Mellor [email protected]
Co-authored-by: Nick Hill [email protected]
Co-authored-by: Harry Mellor [email protected]