Skip to content

Conversation

bact
Copy link
Member

@bact bact commented Jun 24, 2021

What does this changes

Fix safe mode algorithm that choose the cut-off position.

What was wrong

token_max_len = 0 was inside for loop, makes it always reset to zero.

This make the cut-off position may not be the one that covers longest token.

How this fixes it

Move token_max_len = 0 outside of for loop.

Your checklist for this pull request

🚨Please review the guidelines for contributing to this repository.

  • Passed code styles and structures
  • Passed code linting checks and unit test

@bact bact added the bug bugs in the library label Jun 24, 2021
@bact bact modified the milestones: 2.4, 2.3 Jun 24, 2021
@bact bact self-assigned this Jun 24, 2021
@coveralls
Copy link

coveralls commented Jun 24, 2021

Coverage Status

Coverage decreased (-0.03%) to 95.934% when pulling ca25598 on bact:fix-newmm-safemode-token_max_len into d51d5a5 on PyThaiNLP:dev.

@bact bact merged commit 215b266 into PyThaiNLP:dev Jun 24, 2021
@bact bact deleted the fix-newmm-safemode-token_max_len branch June 24, 2021 00:57
@bact bact restored the fix-newmm-safemode-token_max_len branch June 24, 2021 00:58
bact added a commit that referenced this pull request Jun 24, 2021
bact added a commit that referenced this pull request Jun 24, 2021
@wannaphong wannaphong modified the milestones: 2.3, 2.4 Jul 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug bugs in the library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants