Skip to content

[Hardware][TPU] Implement tensor parallelism with Ray #5871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 72 commits into from
Jul 27, 2024
Merged

[Hardware][TPU] Implement tensor parallelism with Ray #5871

merged 72 commits into from
Jul 27, 2024

Conversation

WoosukKwon
Copy link
Collaborator

@WoosukKwon WoosukKwon commented Jun 26, 2024 •
edited
Loading

This PR implements Ray TPU executor for distributed inference support on TPU.

@WoosukKwon WoosukKwon added the tpu Related to Google TPUs label Jun 26, 2024
@WoosukKwon WoosukKwon changed the title [Hardware][TPU] Support tensor parallelism with Ray [Hardware][TPU] Implement tensor parallelism with Ray Jun 26, 2024
@WoosukKwon
Copy link
Collaborator Author

WoosukKwon commented Jun 26, 2024 •
edited
Loading

For this PR, I will merge it after getting reviews. :)

The changes outside the TPU backend was reviewed in #6812 and #6813.

@WoosukKwon WoosukKwon added the ready ONLY add when PR is ready to merge/full CI is needed label Jul 21, 2024
This was referenced Jul 26, 2024
Merged
Merged
@WoosukKwon WoosukKwon merged commit 52f07e3 into main Jul 27, 2024
@WoosukKwon WoosukKwon deleted the tpu-n branch July 27, 2024 03:54
@dtrifiro dtrifiro mentioned this pull request Aug 5, 2024
Closed
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
@WoosukKwon @Alvant
[Hardware][TPU] Implement tensor parallelism with Ray (vllm-project#5871
08963df 
)

Signed-off-by: Alvant <[email protected]>
LeiWang1999 pushed a commit to LeiWang1999/vllm-bitblas that referenced this pull request Mar 26, 2025
@WoosukKwon @LeiWang1999
[Hardware][TPU] Implement tensor parallelism with Ray (vllm-project#5871
eac7762 
)

Signed-off-by: LeiWang1999 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed tpu Related to Google TPUs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@WoosukKwon