[Bug]: vllm v0.6.2/v0.6.3 is easy to generate random output if there are many symbols(not words) in prompt #9409
Description
Your current environment
The output of `python collect_env.py`
Your output of `python collect_env.py` here
Model Input Dumps
No response
🐛 Describe the bug
I have an observation:
If there are many symbols(non-words) in prompt, like following cases, v0.6.x is much easier to generate random output:
- markdown table symbo "|"
- yaml in json value
- json in json
One output is as following:
I have been changed to be | Mandatory. | | | | | | | | | | | | Mandatory (SIP...-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP-SIP | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. | Mandatory. |"
If I switch back to 0.5.4, this issue will be gone.
Could you please help to guide me the possible reason and how to avoid it?
Thanks in advance!
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.