is Float16 a real computational type or not?

[StefanKarpinski]

At this point, Float16 is in a kind of weird limbo where it is sort of a storage type but we keep haphazardly adding more and more functionality to it. I think we really need to figure out where to draw the line here. Either we severely limit the scope of Float16's functionality and stop using it in tests for anything but converting to and from other numeric types, or we just go full throttle and make it a full computation type with as much functionality as Float32 or Float64 – albeit slower, since many operations will not be implemented as native machine ops.

[jiahao]

I've wondered about this as well. Until extremely recently, we've had some tests for linear algebra over Float16s, which have turned out to be very stressful on our test criteria.

[andreasnoack]

I saw your last comment about this and I am preparing to remove the Float16 linalg tests. They are also overfloating all the time.

[andreasnoack]

@jiahao Almost the same comment at the same time. In #5902 I have removed Float16 from the tests.

[JeffBezanson]

I think it should have the same status as other numeric types that lack
extensive hardware and library support, like rational. Probably all we need
to do is generalize the definitions that make libm functions applicable to
ints and rationals.
On Feb 24, 2014 5:48 PM, "Andreas Noack Jensen" notifications@github.com
wrote:

I saw your last comment about this and I am preparing to remove the
Float16 linalg tests. They are also overfloating all the time.

Reply to this email directly or view it on GitHubhttps://github.com//issues/5942#issuecomment-35949938
.

[JeffBezanson]

Oh and fixed point would be another type in this category, though it is
more plausible to compute with.
On Feb 24, 2014 5:55 PM, "Jeff Bezanson" jeff.bezanson@gmail.com wrote:

I think it should have the same status as other numeric types that lack
extensive hardware and library support, like rational. Probably all we need
to do is generalize the definitions that make libm functions applicable to
ints and rationals.
On Feb 24, 2014 5:48 PM, "Andreas Noack Jensen" notifications@github.com
wrote:

I saw your last comment about this and I am preparing to remove the
Float16 linalg tests. They are also overfloating all the time.

Reply to this email directly or view it on GitHubhttps://github.com//issues/5942#issuecomment-35949938
.

[pao]

fixed point would be another type in this category, though it is more plausible to compute with

It's very plausible in some domains, but these are also places you are unlikely to get the Julia runtime to work.

[JeffBezanson]

Is that a challenge? :-)
On Feb 24, 2014 6:37 PM, "pao" notifications@github.com wrote:

fixed point would be another type in this category, though it is more
plausible to compute with

It's very plausible in some domains, but these are also places you are
unlikely to get the Julia runtime to work.

Reply to this email directly or view it on GitHubhttps://github.com//issues/5942#issuecomment-35954405
.

[JeffBezanson]

Currently the fallbacks for transcendental functions look like

        ($f)(x::Real) = ($f)(float(x))

How about we use instead

        ($f)(x::Integer) = ($f)(float(x))
        ($f)(x::Real) = oftype(x, ($f)(float64(x)))

Then we could remove most of the definitions in float16.jl, and it wouldn't seem haphazard anymore.

[jiahao]

This seems to work.

• It looks like you can also remove the ($f)(x::FloatingPoint) fallback methods in math.jl, which just complain that the method is not defined.
• The complex definitions seem to be redundant with the generic implementations in complex.jl, modulo the fact that some type instability results from the current behavior that float16(1) * im is not a Complex32, but rather a Complex64.

[jiahao]

julia> typeof(float16(1.0)*true)
Float32

julia> promote_type(Bool, Float16)
Float32

[jiahao]

MathConsts also need special handling.

With the proposed fallbacks:

julia> sqrt(pi)
ERROR: no method convert(Type{MathConst{:π}}, Float64)
 in sqrt at math.jl:274

[JeffBezanson]

I guess there is a case to be made that any operation on Float16 should return a Float32, since that way there's no roundoff and subsequent operations will actually be faster. That's probably what it means to be a "storage type". There is a strong similarity to how we do e.g. Int8 + Int8.

[jiahao]

Fair enough, but having to explicitly recast the complex elementary functions strikes me as really quite inelegant, given the generic implementations in complex.jl:

julia> for f in (sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, exp, log, sqrt)
          x=float16(rand()+rand()*im); @show f, typeof(f(x))
       end
(f,typeof(f(x))) => (sin,Complex{Float16})
(f,typeof(f(x))) => (cos,Complex{Float16})
(f,typeof(f(x))) => (tan,Complex{Float32})
(f,typeof(f(x))) => (asin,Complex{Float32})
(f,typeof(f(x))) => (acos,Complex{Float32})
(f,typeof(f(x))) => (atan,Complex{Float32})
(f,typeof(f(x))) => (sinh,Complex{Float16})
(f,typeof(f(x))) => (cosh,Complex{Float16})
(f,typeof(f(x))) => (tanh,Complex{Float32})
(f,typeof(f(x))) => (asinh,Complex{Float32})
(f,typeof(f(x))) => (acosh,Complex{Float32})
(f,typeof(f(x))) => (atanh,Complex{Float32})
(f,typeof(f(x))) => (exp,Complex{Float16})
(f,typeof(f(x))) => (log,Complex{Float32})
(f,typeof(f(x))) => (sqrt,Complex{Float32})