Skip to content

Conversation

@Gong-xxx
Copy link

@Gong-xxx Gong-xxx commented May 27, 2025

…7B Thinker module

What this PR does / why we need it?

Synchronize vLLM mrope modifications, support Qwen2.5-OMni-7B thinker module.

Does this PR introduce any user-facing change?

support audio mrope

How was this patch tested?

Related: #967

@Gong-xxx Gong-xxx changed the title [Feature] Synchronize vLLM mrope modifications, support Qwen2.5-OMni-… [Feature] Synchronize vLLM mrope modifications, support Qwen2.5-OMni-7B Thinker. May 27, 2025
Copy link
Collaborator

@Yikun Yikun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gong-xxx Thanks for your contributions!

@wangxiyuan
Copy link
Collaborator

wangxiyuan commented May 28, 2025

++ for V1 engine. V0 will not be supported in the future.

@Yikun Yikun requested a review from shen-shanshan June 3, 2025 15:15
@wangxiyuan wangxiyuan mentioned this pull request Jun 4, 2025
76 tasks
@wangxiyuan wangxiyuan added ready read for review and removed ready read for review labels Jun 6, 2025
@Yikun
Copy link
Collaborator

Yikun commented Jul 12, 2025

We don't need this anymore.

pip install qwen-omni-utils -U
vllm serve Qwen/Qwen2.5-Omni-7B --enforce-eager

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "https://modelscope.oss-cn-beijing.aliyuncs.com/resource/qwen.png"}},
        {"type": "audio_url", "audio_url": {"url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5-Omni/cough.wav"}},
        {"type": "text", "text": "What is the text in the illustrate ans what it the sound in the audio?"}
    ]}
    ]
    }'
{"id":"chatcmpl-9ddb4f86a49343f989fcffeb8ac28241","object":"chat.completion","created":1752334669,"model":"Qwen/Qwen2.5-Omni-7B","choices":[{"index":0,"message":{"role":"assistant","reasoning_content":null,"content":"The text in the image is \"TONGYI Qwen.\" The sound in the audio is a cough.","tool_calls":[]},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":156,"total_tokens":180,"completion_tokens":24,"prompt_tokens_details":null},"prompt_logprobs":null,"kv_transfer_params":null

I test it in v0.9.2rc1, so close this.

@Yikun Yikun closed this Jul 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants