-
Couldn't load subscription status.
- Fork 6.8k
[serve.llm] release tests for 1p1d #53190
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
393835d to
09b5523
Compare
b7a6517 to
1191f64
Compare
87dbbdc to
a0c2f84
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some minor comments, I'll review once more when tests pass and this is rebased.
release/llm_tests/serve/configs/model_config/qwen_3_8B_fp8_1replica.yaml
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@lk-chen conflict? |
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
Signed-off-by: Linkun Chen <[email protected]>
9f66469 to
9c273df
Compare
Signed-off-by: Linkun Chen <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
approval for the byod script. nice job!
Why are these changes needed?
Enable release test for P/D disagg. implemented in #53092. This PR includes:
update of ray-llm image, that installs UCX/NIXL, and set proper env varsmoved to [ray-llm] Update ray-llm docker to install UCX/NIXL #53377Related issue number
RFC #53257
Ray-LLM Roadmap #51313
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.