Skip to content

Conversation

@amkCha
Copy link
Contributor

@amkCha amkCha commented Aug 21, 2025

maxOffset can be 2³² (MXPX_THRESHOLD)
in row i + 8, when divided by 32, ie 2⁵, res can be 2²⁷
in row i + 9, previous res is squared and divided by 512, ie 2^9, so resA can be 2^45
We need to replace the definition of resA as an i64 i/o i32

For resB,
in row i + 9, previous res is squared and divided by 512, ie 2^9, so resA can be 2^45
in the case where the remainder is 0, the ceiling == the quotient, ie resB = resA so resB can also be 2^45

@amkCha amkCha changed the title fix: resA and resB can be > i32 fix: resA can be > i32 Aug 21, 2025
@amkCha amkCha force-pushed the fix/resA-resB-column-size-in-mxp branch from cbfc3de to 89f4c45 Compare August 25, 2025 09:46
@amkCha amkCha changed the title fix: resA can be > i32 fix: resA and resB can be > i32 Aug 25, 2025
@amkCha amkCha merged commit 3ad0ad3 into master Aug 25, 2025
6 checks passed
@amkCha amkCha deleted the fix/resA-resB-column-size-in-mxp branch August 25, 2025 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants