Skip to content

Floating point round function gives some inaccurate results #7306

@raphlinus

Description

@raphlinus

This bug is branched from rust-lang/rust#55107.

The heart of the implementation of round() in src/library.js is:

   return d >= +0 ? +Math_floor(d + +0.5) : +Math_ceil(d - +0.5);

This produces incorrect values for 0.49999999999999994 (correct 0.0, actual 1.0) and 4503599627370497.0 (correct same as input, actual 4503599627370498.0) among others. The latter is disturbing because it means the function does not reliably preserve integers.

I worked out a fix, which is basically this in the positive branch:

        Math_floor((d + (0.25 - 0.5 * F64_EPSILON)) + (0.25 + 0.5 * F64_EPSILON))

where F64_EPSILON is is the difference between 1.0 and the next largest representable number, or 2.2204460492503131e-16. At least this gives correct results on my x86_64; I'm not sure to what extent correctness is guaranteed across all target platforms.

Also, the ternary might profitably be replaced by (this is C syntax, because I know the libm names of things but not necessarily the relevant library functions/intrinsics in Emscripten):

    copysign(floor(((abs(d) + (0.25 - 0.5 * F64_EPSILON)) + (0.25 + 0.5 * F64_EPSILON)), d)

This will probably be faster in wasm because there is a copysign intrinsic. But if the intrinsic doesn't get compiled properly it's probably an overall lose.

I'm willing to work on this, but wanted to give people a heads-up, and maybe could get some guidance on the copysign question.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions