Fix empty output when temp is too low

[CatherineSue]

Hi, when the temp is too low, we found the generated text would be empty in v0.3.0:

# python3 -m vllm.entrypoints.openai.api_server --port 8080 --model /models/Llama-2-70b-chat-hf --tensor-parallel-size 2
# curl http://localhost:8080/v1/completions     -H "Content-Type: application/json"     -d '{
       "model": "/models/Llama-2-70b-chat-hf",
       "prompt": "The capital of France is",
       "max_tokens": 100,
       "n": 1,
       "temperature": 1e-4,
       "ignore_eos": 1
    }'
{"id":"cmpl-1e09b4828b154423b9b147897f49b599","object":"text_completion","created":1203634,"model":"/models/Llama-2-70b-chat-hf","choices":[{"index":0,"text":"","logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":6,"total_tokens":106,"completion_tokens":100}}

And it would raise RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 in v0.2.4.

Here are some findings after debugging:

• When temp = 1e-4, there is nan or inf in probs at this line: https://github.com/vllm-project/vllm/blob/v0.3.0/vllm/model_executor/layers/sampler.py#L102, and it is caused by that logits are divided by temperatures(here).
• The reason v0.3.0 doesn't raise RuntimeError is because vllm has implemented its own _multinomial method in v0.3.0 in this PR, but v0.2.4 still uses torch.multinomial, which checks if there is Nan in tensors and throws RuntimeError.

We think it would be confusing if the output kept being empty when users set the temperature too low. It would be nice to have the temperature max out the temperature to 1e-2 or 1e-3 to avoid the empty output. This PR:

• max out the temperature to 1e-2, and log a warning to users
• Use float32 for logits to avoid numerical errors.

[DarkLight1337]

Thanks for the QoL change! (And sorry for the long wait) I'll help get this merged.

[DarkLight1337]

Finally merged, sorry for the long delay!