Does vllm support KV-cache between multi-turn conversation

[junleiz]

Hello, in many cases, we need to take multi-turn conversation. That is, the output of the first step will be added to the input of the second step. Could you please tell me whether the common part of the multi-turn conversation will be cached?

[jameswu2014]

I have the same question.

[kongjiellx]

+1

[HouYueJie]

+1

[hmellor]

Yes, in the form of prefix caching #2762