Skip to content

Add tritonrelease container #4455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
May 22, 2025
Merged

Add tritonrelease container #4455

merged 3 commits into from
May 22, 2025

Conversation

Tabrizian
Copy link
Member

@Tabrizian Tabrizian commented May 19, 2025 •
edited
Loading

Add a Triton container that contains TensorRT-LLM and Triton backend

This replaces the old docker file that existed in the TRT-LLM backend repo. The build can be optimized further just having something that works for now.

@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch 2 times, most recently from 4cd1570 to 7a35e9f Compare May 19, 2025 21:17
@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch from 7a35e9f to 5b46366 Compare May 19, 2025 22:15
@Tabrizian
Copy link
Member Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5778 [ run ] triggered by Bot

@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch from 5b46366 to 5e2e7bc Compare May 20, 2025 01:12
@Tabrizian
Copy link
Member Author

/bot skip

@github-actions GitHub Actions
Copy link

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]

Launch build/test pipelines. All previously running jobs will be killed.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-[Post-Merge]-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@Tabrizian
Copy link
Member Author

/bot skip --comment "The build and related tests were successful"

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5790 [ skip ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5778 [ run ] completed with state ABORTED

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5790 [ skip ] completed with state SUCCESS
Skipping testing for commit 5e2e7bc

MartinMarciniszyn
MartinMarciniszyn requested changes May 20, 2025
View reviewed changes
@Tabrizian
Copy link
Member Author

/bot --skip-test

@github-actions GitHub Actions
Copy link

GitHub Bot Help

/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...

Provide a user friendly way for developers to interact with a Jenkins server.

Run /bot [-h|--help] to print this help message.

See details below for each supported subcommand.

run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]

Launch build/test pipelines. All previously running jobs will be killed.

--disable-fail-fast (OPTIONAL) : Disable fail fast on build/tests/infra failures.

--skip-test (OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.

--stage-list "A10-1, xxx" (OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.

--gpu-type "A30, H100_PCIe" (OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.

--only-multi-gpu-test (OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.

--disable-multi-gpu-test (OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.

--add-multi-gpu-test (OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.

--post-merge (OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.

--extra-stage "H100_PCIe-[Post-Merge]-1, xxx" (OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".

kill

kill

Kill all running builds associated with pull request.

skip

skip --comment COMMENT

Skip testing for latest commit on pull request. --comment "Reason for skipping build/test" is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

reuse-pipeline

reuse-pipeline

Reuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.

@Tabrizian
Copy link
Member Author

/bot run --skip-test

@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch from f51a23c to 97bfa6b Compare May 20, 2025 17:23
@tensorrt-cicd
Copy link
Collaborator

PR_Github #5901 [ run ] triggered by Bot

@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch 2 times, most recently from 6c31869 to 61b52cd Compare May 20, 2025 18:54
@tensorrt-cicd
Copy link
Collaborator

PR_Github #5901 [ run ] completed with state FAILURE
/LLM/main/L0_MergeRequest_PR pipeline #4325 (Partly Tested) completed with status: 'FAILURE'

@Tabrizian
Copy link
Member Author

/bot run --skip-test

@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch 2 times, most recently from aa3546f to 90b1e3b Compare May 20, 2025 20:24
@tensorrt-cicd
Copy link
Collaborator

PR_Github #5911 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5911 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4332 (Partly Tested) completed with status: 'FAILURE'

@mc-nv mc-nv mentioned this pull request May 20, 2025
Merged
@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch from 90b1e3b to d342a32 Compare May 21, 2025 04:34
@Tabrizian
Copy link
Member Author

/bot run --skip-test

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5955 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5955 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #4356 (Partly Tested) completed with status: 'SUCCESS'

@Tabrizian Tabrizian changed the base branch from main to release/0.20 May 21, 2025 14:16
@Tabrizian Tabrizian requested review from a team as code owners May 21, 2025 14:16
@Tabrizian Tabrizian changed the base branch from release/0.20 to main May 21, 2025 14:18
MartinMarciniszyn
MartinMarciniszyn approved these changes May 21, 2025
View reviewed changes
@MartinMarciniszyn MartinMarciniszyn mentioned this pull request May 21, 2025
Merged
@Tabrizian Tabrizian closed this May 21, 2025
@Tabrizian
Copy link
Member Author

Closing in favor of: #4544

@Tabrizian Tabrizian reopened this May 22, 2025
Tabrizian and others added 3 commits May 21, 2025 22:01
@Tabrizian
Add tritonrelease container
2bbe439 
Signed-off-by: Iman Tabrizian <[email protected]>
@Tabrizian
Review comments
383f9ec 
Signed-off-by: Iman Tabrizian <[email protected]>
@Tabrizian @MartinMarciniszyn
Update docker/Makefile
5945729 
Co-authored-by: Martin Marciniszyn Mehringer <[email protected]>
Signed-off-by: Iman Tabrizian <[email protected]>
@Tabrizian Tabrizian force-pushed the user/imant/tritonrelease branch from 338d708 to 5945729 Compare May 22, 2025 02:01
@Tabrizian
Copy link
Member Author

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #6081 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #6081 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #5955 (Partly Tested) for commit 5945729

@Tabrizian Tabrizian merged commit e741d2b into NVIDIA:main May 22, 2025
3 checks passed
@tongyuantongyu tongyuantongyu mentioned this pull request May 28, 2025
Merged
darraghdog pushed a commit to darraghdog/TensorRT-LLM that referenced this pull request Jun 3, 2025
@Tabrizian @MartinMarciniszyn @darraghdog
Add tritonrelease container (NVIDIA#4455)
cb4f56f 
* Add tritonrelease container

Signed-off-by: Iman Tabrizian <[email protected]>

* Review comments

Signed-off-by: Iman Tabrizian <[email protected]>

* Update docker/Makefile

Co-authored-by: Martin Marciniszyn Mehringer <[email protected]>
Signed-off-by: Iman Tabrizian <[email protected]>

---------

Signed-off-by: Iman Tabrizian <[email protected]>
Signed-off-by: Iman Tabrizian <[email protected]>
Co-authored-by: Martin Marciniszyn Mehringer <[email protected]>
Signed-off-by: darraghdog <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MartinMarciniszyn MartinMarciniszyn MartinMarciniszyn approved these changes

@schetlur-nv schetlur-nv Awaiting requested review from schetlur-nv

@chzblych chzblych Awaiting requested review from chzblych

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Tabrizian @tensorrt-cicd @MartinMarciniszyn