Element-wise call syntax does not always lower to standard broadcast()

[nalimilan]

It appears that some simple uses of the element-wise call syntax .( are not lowered to a standard broadcast call. This is a problem for NullableArray where we would like to overload broadcast to apply lifting semantics (JuliaStats/NullableArrays.jl#166).

For example, the following code doesn't call broadcast(get, X, 0). Therefore it fails since my custom broadcast function has a special behavior for get which doesn't get triggered here.

Why it this the case?

julia> expand(:(get.([Nullable()], 0)))
:($(Expr(:thunk, CodeInfo(:(begin 
        $(Expr(:thunk, CodeInfo(:(begin 
        global ##3#4
        const ##3#4
        $(Expr(:composite_type, Symbol("##3#4"), :((Core.svec)()), :((Core.svec)()), :(Core.Function), :((Core.svec)()), false, 0))
        return
    end))))
        $(Expr(:method, false, :((Core.svec)((Core.svec)(##3#4,Any),(Core.svec)())), CodeInfo(:(begin 
        return get(#temp#,0)
    end)), false))
        #3 = $(Expr(:new, Symbol("##3#4")))
        SSAValue(0) = #3
        SSAValue(1) = (Base.vect)(Nullable())
        return (Base.broadcast)(SSAValue(0),SSAValue(1))
    end)))))

[yuyichao]

This is an optimization since broadcast(get, X, 0) shouldn't do anything other than what the special lowering does.

[nalimilan]

So that means one cannot specialize broadcast for a custom array type? Sounds quite restrictive to me. Is the expected behavior documented somewhere?

[nalimilan]

To clarify, IIUC, get.(X, 0) gets lowered to broadcast(x -> get(x, 0), X) instead of broadcast(get, X, 0) for performance reasons. Since that's done at parse time, no dispatch on the input types is possible. I see the point of this optimization, but unfortunately it hardcodes the behavior of broadcast for any type in the parse, which is unusual for a Julia method which by definition can be overridden.

Maybe there should be a type check, and the fast code would only be used for Array and Base scalar types?

[martinholters]

If you need to specialize broadcast for get, that is a bad sign anyway since it wouldn't work with fusion. E.g. neither get.(foo.(X), 0) nor get.(foo.(X), i) would end up calling your specialization.

[nalimilan]

Yes, I've noticed that too. Whatever we try to improve the Nullable workflow, we bump against lack of support in Base. I'm starting to get tired of these repeated dead ends. See JuliaStats/NullableArrays.jl#166.

[stevengj]

Yes, as I commented in #17623, the existence of parse-time fusion by itself (even without constant-folding optimizations) makes defining specialized broadcast methods for particular functions effectively useless.

On the plus side, this sometimes forces us to pursue more general solutions, e.g. for preserving sparsity for broadcast on sparse matrices.

[nalimilan]

Right. Unfortunately, we can't just call the function to check whether it is zero-preserving or looks at its return type to decide whether to return a BitArray. Here, non-lifting functions really need to be identified and transformed before being fused with others. The generalization that this encourages is to lift all function calls involving Nullable by default, even outside broadcast.

[stevengj]

@nalimilan, what is a "non-lifting" function and how would you identify one at parse-time?

[nalimilan]

I don't think hard-coding them at parse time would be a good idea, even though their list should be pretty limited. Basically, these are functions which operate on the Nullable object itself rather than the value it wraps: isnull, get, plus logical operators isequal, isless, & and | (for three-valued logic). See JuliaStats/NullableArrays.jl#166 and JuliaStats/NullableArrays.jl#144.

[stevengj]

I guess in principle we could transform:

f.(g.(x...)) ---> isfusing(g,x...) ? broadcast((_...) -> f(g(_...)), x...) : broadcast(f, broadcast(g, x...))

where isfusing would default to true. i.e. you allow the caller to define "fusion barriers" for particular functions. Or isfusing could automatically return false if a specialized broadcast method exists for g? I'm not sure if this is a good direction to pursue, however.

[stevengj]

Note that if you want a non-fusing broadcast-like operation, you can just define a method without dots that acts on arrays. e.g. & and | will never be fusing. Similarly, you could just define get(a::NullableArray, ...) that acts like a get.(a, ...) but doesn't fuse.

[nalimilan]

That would work. Though strictly speaking loops can fuse, it's just that we need to be able to transform the function if needed before fusing it. That is, something like:

f.(g.(x...)) --> broadcast(_ -> maybe_lift(f, maybe_lift(g, _...)), x...)

with

maybe_lift(f, _...) = f(_...)

for non-lifted functions (the current case).

Of course we can always define special methods instead of using the dot syntax. But I prefer trying to push the logic as far as possible to see all implications and (in)consistency of the design choice first.