[Feature]: max-num-partial-prefills in V1

[eransh777]

🚀 The feature, motivation and pitch

#10235 Add support to concurrently calculate prefills of multiple requests, which allows to reduce the TTFT of shorter requests, while other longer requests also run on the same vLLM server.

This feature is currently only supported in V0, but it would be very useful for me in V1, as V0 is about to deprecated.

This feature should also solve #21495.

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

cc @WoosukKwon @heheda12345

[chaunceyjiang]

I’d like to study this issue and try to reproduce the work on V1.

[chaunceyjiang]

It seems there's already a related PR.
#21651

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!