Normalize text before checking for missing characters in the tokenizer

[mmartin9684-sil]

SentencePiece normalizes some decomposed characters (e.g., "B" and underscore) into their canonical form (e.g., "Ḇ). When we are checking the source text for characters that are not known to the tokenizer, we are checking before this normalization occurs. It would be better to check for characters that are unknown to the tokenizer using the normalized version of the text.

[mshannon-sil]

To provide further detail, although we were comparing normalized characters to the tokenizer's vocab list, we were applying the normalization individually to each character in the set of all characters in the corpus, rather than applying the normalization to each line in the corpus before collecting the character set.