global variables are unpredictably slow (codegen inference issue)

[bvdmitri]

Consider two functions:

function foo(::Type{T}) where T
    return (v) -> T(v)
end

function bar(::Type{T}) where T
    R = T
    return (v) -> R(v)
end

while these functions semantically should do the same, their performance is drastically different:

julia> f = foo(Int)
#9 (generic function with 1 method)

julia> b = bar(Int)
#11 (generic function with 1 method)

julia> @btime f(1)
  13.685 ns (0 allocations: 0 bytes)
1

julia> @btime b(1)
  98.902 ns (0 allocations: 0 bytes)
1

This happens because Julia cannot infer correctly the type in the second case, which results in bad performance overall.

julia> dump(f)
#9 (function of type var"#9#10"{Int64})

julia> dump(b)
#11 (function of type var"#11#12"{DataType})
  R: Int64 <: Signed

Provided MWE seems to be very simple, but we hit this performance issue in a very complex code involving a lot of lambda functions. @cscherrer faced similar issue in his MeasureBase.jl package.

julia> versioninfo()
Julia Version 1.6.3
Commit ae8452a9e0 (2021-09-23 17:34 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  CPU: Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

Reproducible on master 146de38 as well (same hardware).

[KristofferC]

Dup of notifications #23618?

[bvdmitri]

Fixed by #40985 (but not merged). Performance is same after the fix, however inference is still a bit different, not sure if it is a problem:

julia> dump(f)
#1 (function of type var"#1#2"{Int64})

julia> dump(b)
#3 (function of type var"#3#4"{Type{Int64}})
  R: Int64 <: Signed

[JeffBezanson]

Thanks for reducing this to such a simple example; that saves us a lot of time!

[adienes]

1.11

julia> @btime f(1)
  11.094 ns (0 allocations: 0 bytes)

julia> @btime b(1)
  87.191 ns (0 allocations: 0 bytes)

1.12 & nightly

julia> @btime f(1)
  15.406 ns (0 allocations: 0 bytes)
1

julia> @btime b(1)
  16.032 ns (0 allocations: 0 bytes)

so the discrepancy was fixed, but f got 50% slower

[vtjnash]

It is entirely dispatch time now, and dynamic symbol lookup (f) used to be zero cost, and now is not, which seems to be an implementation mistake (since it looks like it should have been zero cost still). The function f itself actually appears to have gotten a bit faster.

[vtjnash]

This is the MWE of the new codegen failure:

julia> f = 1
1

julia> code_llvm(()) do; f; end
; Function Signature: var"#5"()
;  @ REPL[2]:1 within `#5`
define nonnull ptr @"julia_#5_1009"() local_unnamed_addr #0 {
top:
  %0 = call ptr @jl_get_binding_value_seqcst(ptr nonnull @"*Main.f#1011.jit")
  %.not = icmp eq ptr %0, null
  br i1 %.not, label %err, label %ok

err:                                              ; preds = %top
  call void @ijl_undefined_var_error(ptr nonnull @"jl_sym#f#1012.jit", ptr nonnull @"jl_global#1013.jit")
  unreachable

ok:                                               ; preds = %top
  ret ptr %0
}

That function call is not supposed to be present, as it makes accessing normal global variables much slower than they are supposed to be.

And also very confusing and unreliable too

Compare access to f vs h, since one gets correctly optimized and one does not:

julia> f = 1
1

julia> g() = 1
g (generic function with 1 method)

julia> h = 2
2

julia> code_llvm(()) do; g() + f + h; end
; Function Signature: var"#8"()
;  @ REPL[6]:1 within `#8`
define nonnull ptr @"julia_#8_1176"() local_unnamed_addr #0 {
top:
  %jlcallframe1 = alloca [2 x ptr], align 8
  %gcframe2 = alloca [4 x ptr], align 16
  call void @llvm.memset.p0.i64(ptr align 16 %gcframe2, i8 0, i64 32, i1 true)
  %thread_ptr = call ptr asm "movq %fs:0, $0", "=r"() #7
  %tls_ppgcstack = getelementptr inbounds i8, ptr %thread_ptr, i64 -8
  %tls_pgcstack = load ptr, ptr %tls_ppgcstack, align 8
  store i64 8, ptr %gcframe2, align 8
  %frame.prev = getelementptr inbounds nuw i8, ptr %gcframe2, i64 8
  %task.gcstack = load ptr, ptr %tls_pgcstack, align 8
  store ptr %task.gcstack, ptr %frame.prev, align 8
  store ptr %gcframe2, ptr %tls_pgcstack, align 8
  %f.checked = load atomic ptr, ptr getelementptr inbounds nuw (i8, ptr @"*Main.f#1178.jit", i64 8) unordered, align 8
  %.not = icmp eq ptr %f.checked, null
  br i1 %.not, label %err, label %ok

err:                                              ; preds = %top
  call void @ijl_undefined_var_error(ptr nonnull @"jl_sym#f#1179.jit", ptr nonnull @"jl_global#1180.jit")
  unreachable

ok:                                               ; preds = %top
  %gc_slot_addr_0 = getelementptr inbounds nuw i8, ptr %gcframe2, i64 16
  store ptr %f.checked, ptr %gc_slot_addr_0, align 8
  %0 = call ptr @jl_get_binding_value_seqcst(ptr nonnull @"*Main.h#1181.jit")
  %.not5 = icmp eq ptr %0, null
  br i1 %.not5, label %err1, label %ok3

err1:                                             ; preds = %ok
  store ptr null, ptr %gc_slot_addr_0, align 8
  call void @ijl_undefined_var_error(ptr nonnull @"jl_sym#h#1182.jit", ptr nonnull @"jl_global#1180.jit")
  unreachable

ok3:                                              ; preds = %ok
  %gc_slot_addr_1 = getelementptr inbounds nuw i8, ptr %gcframe2, i64 24
  store ptr %0, ptr %gc_slot_addr_1, align 8
; ┌ @ operators.jl:650 within `+`
   store ptr @"jl_global#1184.jit", ptr %jlcallframe1, align 8
   %1 = getelementptr inbounds nuw i8, ptr %jlcallframe1, i64 8
   store ptr %f.checked, ptr %1, align 8
   %2 = call nonnull ptr @ijl_apply_generic(ptr nonnull @"jl_global#1183.jit", ptr nonnull %jlcallframe1, i32 2)
   store ptr %2, ptr %gc_slot_addr_0, align 8
   store ptr %2, ptr %jlcallframe1, align 8
   store ptr %0, ptr %1, align 8
   %3 = call nonnull ptr @ijl_apply_generic(ptr nonnull @"jl_global#1183.jit", ptr nonnull %jlcallframe1, i32 2)
   %frame.prev16 = load ptr, ptr %frame.prev, align 8
   store ptr %frame.prev16, ptr %tls_pgcstack, align 8
   ret ptr %3
; └
}

[KristofferC]

Would be good to fix up the performance but doesn't have to block 1.12.