Abstract mooncake store connector to kv store connector

[maobaolong]

After this abstraction, kvstore connector will support

• mooncakestore
• filestore
• any other store, will be implemented by contributors future.

How to test

For the MooncakeStoreConnector, everything remain as same as before.

The following is a test for FileStoreConnector.

• Start vllm

VLLM_MLA_DISABLE=0 VLLM_USE_V1=0 \
vllm serve /disc/data1/deepseek/DeepSeek-V2-Lite-Chat/ \
           --trust-remote-code \
           --served-model-name vllm_cpu_offload \
           --max-model-len 32768 \
           --max-seq-len-to-capture 10000 \
           --max-num-seqs 64 \
           --gpu-memory-utilization 0.9 \
           --host 0.0.0.0 \
           -tp 1 \
           --enforce-eager --kv-transfer-config '{"kv_connector":"FileStoreConnector","kv_role":"kv_both","kv_connector_extra_config":{"fs_storage_path":"/disc/data1/baoloongmao/file_store/data/"}}'

• Start a test by curl

curl http://localhost:8000/v1/chat/completions     -H "Content-Type: application/json"     -d '{
    "model": "vllm_cpu_offload",
    "messages": [{"role": "user", "content": "Hello, how are you?"}],
    "max_tokens": 10,
    "temperature": 0,
    "top_p": 0.95
    }'

• Check the file

ll /disc/data1/baoloongmao/file_store/data/
total 848
drwxr-xr-x 2 root root    105 May 30 20:13 ./
drwxr-xr-x 3 root root     26 May 30 20:01 ../
-rw-r--r-- 1 root root 808860 May 30 20:15 412762975381691241_0.safetensors
-rw-r--r-- 1 root root  53396 May 30 20:15 412762975381691241_hidden_0.safetensors

• Check the log

INFO 05-30 05:15:12 [logger.py:39] Received request chatcmpl-49c705d748c74ecb96c510f05bb4f94c: prompt: '<｜begin▁of▁sentence｜>User: Hello, how are you?\n\nAssistant:', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.0, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=10, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None, extra_args=None), prompt_token_ids: None, lora_request: None, prompt_adapter_request: None.
INFO 05-30 05:15:12 [engine.py:310] Added request chatcmpl-49c705d748c74ecb96c510f05bb4f94c.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[maobaolong]

@ShangmingCai @KuntaiDu Thanks for the review about #18721 , this PR is a forward step base on #18721.

This abstraction make kvstoreConnector can support FileStore easily.

[ShangmingCai]

Since vllm is deprecating v0, I think maybe we don't need to add new features to v0 or optimize the codebase anymore. Can you try MultiConnector with v1? Will it fit your use case?

[maobaolong]

@ShangmingCai Thanks for your reminder. As v0 is deprecated, close this PR.