Fix multi-turn chat-style prompt formatting/tokenization

[cebtenzzre]

There are issues with the way we are formatting chat-style prompt templates that have been causing strange output from models that use them.

Most importantly, we are probably not preserving the EOS after the assistant's turn, as required by Llama-2 chat here (Mistral Instruct uses this format), and implemented in llama.cpp's server example here.

If we're not preserving the standard </s> EOS token, we are probably also failing to preserve the <|im_end|> token used by ChatML models such as Mistral OpenOrca, since that becomes the EOS token in those models (except for MPT Chat, which has an incorrect upstream config.json IMO).

The other potential issue is the way we are processing special tokens, especially </s>, in the prompt template. TinyLlama-Chat has been seen printing a literal </s>, which might be a sign that we are not tokenizing correctly (screenshots from @ThiloteE):

[cebtenzzre]

capybarahermes-2.5-mistral-7b.Q4_K_M is an example of a ChatML model that is printing <|im_end|> at the end of its responses. two theories:

• we tokenize <|im_end|> in the prompt template and/or system prompt as text instead of the special token
• we are correctly preserving <|im_end|> in context, but also trying to print it, and somehow EOS is special-cased to not be printed in the output.

[cebtenzzre]

Partially fixed by #1970

[cebtenzzre]

#1970 as merged fully fixes these issues. Included in the v2.7.1 release.