Normalise Lakkhangyao #673
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for contributing! |
wannaphong
added a commit
that referenced
this pull request
Sep 1, 2022
See [3.1 Milestone](https://github.com/PyThaiNLP/pythainlp/milestone/16). ## What is new? ### Deprecation and other API changes #687 Remove deprecated function - pythainlp.word_vector; doesnt_match, get_model, most_similar_cosmul, sentence_vectorizer, similarity. use WordVector class instead - pythainlp.util.delete_tone. use pythainlp.util.remove_tonemark instead - Remove pythainlp.util.time_time. use pythainlp.util.time_to_thaiword instead - pythainlp.tokenize.syllable_tokenize. use pythainlp.tokenize.subword_tokenize instead ### Name Entity Tagging - #665 Add Thai-NNER `pythainlp.tag.NNER` - #658 Add LST20NER onnx model. It is LST20NER model to onnx model from fine-turning by [WangchanBERTa model](https://huggingface.co/airesearch/wangchanberta-base-att-spm-uncased). ### Transliteration - #659 Add ISO 11940 transliteration - #660 Add Thai W2P v0.2 - #686 Add wunsen - #694 Wunsen Mandarin and Japanese update ### PyThaiNLP Corpus downloader - #656 Add support zip/tar.gz to download corpus ### Text normalization - #673 Add a normalising rule for Lakkhangyao ๅ ### Translate - #674 add gpu option ### Text summarize - #679 Add mt5 cpe kmutt thai sentence sum ### Util - #682 Add live-dead syllable classification - #684 Add live dead syllable classify - #690 Add tone detector ### Other - #689 map NG tag to PART - #691 Remove TinyDB as a dependency - #692 Fix notifications that newer versions of corpora are available
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this change do?
Add a normalising rule for Lakkhangyao
ๅAccording to this, it should only follow,
ฤandฦ. Apart from that it should be normalised toา.How this fixes it
Add the following regex in reorder_vowels function
Your checklist for this pull request
🚨Please review the guidelines for contributing to this repository.