[BugFix] enable DOTALL to match multi-line tool_call parameters in extract_tool_call_required_streaming

[shijun-yin]

Purpose

The original regex in extract_tool_call_required_streaming will meet trouble if there is \n between two tool calls in current_text.

param_match = re.search(r'.*"parameters":\s*(.*)',
                                           current_text)

In that case, the first tool call parameters will always be captured so that the delta messages will be incorrect starting from the second tool call.

It can be simply fixed by adding re.DOTALL as parameter.

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

The pull request enables the re.DOTALL flag in the extract_tool_call_required_streaming function to correctly match multi-line tool call parameters. This fixes an issue where newline characters within the parameters would cause incorrect parsing. I have added a review comment to highlight the importance of this fix.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[shijun-yin]

@aarnphm @chaunceyjiang

[chaunceyjiang]

It looks reasonable—could you please post the code for the reproduction steps?

[shijun-yin]

It looks reasonable—could you please post the code for the reproduction steps?

Prerequisites: #23312 (to avoid duplicate tokens, which is another bug)
Server: vllm serve models/Qwen3-8B-AWQ/ --quantization awq --gpu-memory-utilization 0.9 --max-model-len 8192 --tool-call-parser hermes --enable-auto-tool-choice --chat-template-content-format string
Client: python examples/online_serving/openai_chat_completion_client_with_tools_required.py

Log from the client:

[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='f', name=None), type=None)]
[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='ahrenheit', name=None), type=None)]
[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='"}', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id='chatcmpl-tool-4943cbb6af0d4b70b49c71fd1de1740e', function=ChoiceDeltaToolCallFunction(arguments='{"city": "Dallas", "state": "TX", "unit": "fahrenheit"}', name='get_forecast'), type='function')]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments='city', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments='":', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)]

The arguments in the first Delta message for the second tool call should be {". But it turns out to be {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}, which is the full argument for the first tool call.

And if the following debug log is added in extract_tool_call_required_streaming

                    # get partly generated arguments from the latest tool call
                    param_match = re.search(r'.*"parameters":\s*(.*)',
                                            current_text)
                    arguments = param_match.group(1) if param_match else ""
                    print(f"[TEST] arguments: {arguments}")
                    arguments, _ = OpenAIServingChat._filter_delta_text(
                        arguments, previous_text)

Then the log from the server:

# the regex match for the first tool call
[TEST] arguments: {"
......
# the regex match for the second tool call, which is incorrect
[TEST] arguments: {"city": "Dallas", "state": "TX", "unit": "fahrenheit"}},

The BUG can be fixed after adding re.DOTALL:

[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='f', name=None), type=None)]
[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='ahrenheit', name=None), type=None)]
[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='"}', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id='chatcmpl-tool-877ea29107904b8ebe7f93bc249dd98b', function=ChoiceDeltaToolCallFunction(arguments='{"', name='get_forecast'), type='function')]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments='city', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments='":', name=None), type=None)]
[ChoiceDeltaToolCall(index=1, id=None, function=ChoiceDeltaToolCallFunction(arguments=' "', name=None), type=None)]

# the regex match for the first tool call
[TEST] arguments: {"
......
# the regex match for the second tool call
[TEST] arguments: {"

@chaunceyjiang

[shijun-yin]

@chaunceyjiang any comment?

[chaunceyjiang]

LGTM

[shijun-yin]

LGTM

@chaunceyjiang seems like the workflows still need approval from a maintainer lol