Skip to content

Conversation

martindevans
Copy link
Member

Removed (marked as obsolete) prompting with a string for Conversation. Tokenization requires extra parameters (e.g. addBos, special) which require special considersation. For now it's better to tokenize using other tools and pass the tokens directly. Once templating and tokenization are more complete we can come back and look at how best to integrate them into Conversation (if at all).

…n`. Tokenization requires extra parameters (e.g. addBos, special) which require special considersation. For now it's better to tokenize using other tools and pass the tokens directly.
@martindevans martindevans merged commit 0510018 into SciSharp:master May 6, 2024
@martindevans martindevans deleted the remove-batched-conversation-prompt-with-string branch May 6, 2024 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant