Wrong call of `DataType` method with `UnionAll` argument causes segfault

[Keluaa]

Currently on the latest Julia release: 1.10.0

I observed this difference with 1.9:

struct TupleHolder
    tuple::Type{<:Tuple}
end

do_thing(t::DataType) = println("DataType")
do_thing(t::UnionAll) = println("UnionAll")

call_thing(t::TupleHolder) = do_thing(t.tuple)

call_thing(TupleHolder(Tuple{}))

Outputs "UnionAll", which is unexpected since Tuple{} isa DataType. In 1.9, it outputs "DataType".

By changing call_thing to call_thing(t::TupleHolder) = (println(@which do_thing(t.tuple)); do_thing(t.tuple)) we get:

do_thing(t::DataType) @ Main REPL[2]:1
UnionAll

Meaning that there is a discrepancy between @which and the method Julia chooses to call.
Adding println(typeof(t)) to do_thing(t::UnionAll) outputs DataType, which makes me think this is definitly a bug.

Interestingly, removing the ::Type{<:Tuple} from the TupleHolder declaration fixes the issue.

[Keluaa]

I initially thought I found another workaround, where removing the ::UnionAll type constaint on the do_thing method would allow calling the correct method.

This doesn't fix anything however, since calling call_thing using a type with a real UnionAll simply crashes Julia:

struct TupleHolder
    tuple::Type{<:Tuple}
end

do_thing(t::DataType) = t.parameters[1]  # Julia crashes here
do_thing(t) = do_thing(t.body)  # Assume that `t isa UnionAll`

call_sig(t::TupleHolder) = do_thing(t.tuple)

call_sig(TupleHolder(Tuple{V} where {V}))

Stacktrace

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7fffae6a7763 -- jl_f__svec_ref at C:/workdir/src\builtins.c:1661
in expression starting at C:\Users\lucbr\bug_sig_func.jl:12
jl_f__svec_ref at C:/workdir/src\builtins.c:1661
getindex at .\essentials.jl:764 [inlined]
do_thing at C:\Users\lucbr\bug_sig_func.jl:6
unknown function (ip: 000001dbfd9202eb)
call_sig at C:\Users\lucbr\bug_sig_func.jl:10
unknown function (ip: 000001dbfd92023b)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
include_string at .\loading.jl:2070
_include at .\loading.jl:2130
include at .\Base.jl:495
jfptr_include_49873.1 at C:\Users\lucbr\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
exec_options at .\client.jl:318
_start at .\client.jl:552
jfptr__start_86719.1 at C:\Users\lucbr\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2907 (Pool: 2899; Big: 8); GC: 0

By replacing the faulty line by do_thing(t::DataType) = t.parameters, we get this error instead:

ERROR: LoadError: type UnionAll has no field parameters
Stacktrace:
 [1] getproperty
   @ Base .\Base.jl:32 [inlined]
 [2] do_thing
   @ Main C:\Users\lucbr\.julia\dev\GenericKernels.jl\bug_sig_func.jl:7 [inlined]
 [3] call_sig(t::TupleHolder)
   @ Main C:\Users\lucbr\.julia\dev\GenericKernels.jl\bug_sig_func.jl:10
 [4] top-level scope
   @ C:\Users\lucbr\.julia\dev\GenericKernels.jl\bug_sig_func.jl:12
in expression starting at C:\Users\lucbr\.julia\dev\GenericKernels.jl\bug_sig_func.jl:12

My guess is that the faulty method somehow bypasses the type check for DataType (maybe because of inlining?). Then the crash occurs when jl_f__svec_ref is called with a pointer taken from the UnionAll type with the offset of the parameters field of a DataType.

While the initial issue only affects 1.10, this crash also concerns Julia 1.9 (but not 1.8). I was also able to replicate it on the nightly build.

[Keluaa]

The crash can be avoided by using inferencebarrier:

struct TupleHolder
    tuple::Type{<:Tuple}
end

do_thing(t::DataType) = Base.inferencebarrier(t).parameters[1]
do_thing(t) = do_thing(t.body)  # Assume that `t isa UnionAll`

call_sig(t::TupleHolder) = do_thing(t.tuple)

call_sig(TupleHolder(Tuple{V} where {V}))

In 1.9 and above, I "correctly" get a type UnionAll has no field parameters exception, while in 1.8 and below it works without any issues.

[inkydragon]

Reduced test code by maleadt/creduce_julia (with variable renaming)

struct TupleHolder52644
    t
end
f(d) = d.parameters
g(d) = f(d.t)

g(TupleHolder52644(Tuple{f} where f))

But those code will crash on v1.10/1.9/1.8/1.6

Long creash long

PS D:\jl\creduce_julia> julia +1.10  main.jl

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff914a97763 -- jl_f__svec_ref at C:/workdir/src\builtins.c:1661
in expression starting at D:\jl\creduce_julia\main.jl:7
jl_f__svec_ref at C:/workdir/src\builtins.c:1661
getindex at .\essentials.jl:764 [inlined]
f at D:\jl\creduce_julia\main.jl:4
unknown function (ip: 00000152a8e602eb)
g at D:\jl\creduce_julia\main.jl:5
unknown function (ip: 00000152a8e6023b)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:635
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:775
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:934
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:877
ijl_toplevel_eval at C:/workdir/src\toplevel.c:943 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:985
eval at .\boot.jl:385 [inlined]
include_string at .\loading.jl:2070
_include at .\loading.jl:2130
include at .\Base.jl:495
jfptr_include_49873.1 at C:\Users\inkyd\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
exec_options at .\client.jl:318
_start at .\client.jl:552
jfptr__start_86719.1 at C:\Users\inkyd\.julia\juliaup\julia-1.10.0+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1982 [inlined]
true_main at C:/workdir/src\jlapi.c:582
jl_repl_entrypoint at C:/workdir/src\jlapi.c:731
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2907 (Pool: 2899; Big: 8); GC: 0

PS D:\jl\creduce_julia> julia +1.9  main.jl 

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff914b2567c -- jl_f__svec_ref at C:/workdir/src\builtins.c:1614
in expression starting at D:\jl\creduce_julia\main.jl:7
jl_f__svec_ref at C:/workdir/src\builtins.c:1614
getindex at .\essentials.jl:697 [inlined]
f at D:\jl\creduce_julia\main.jl:4
unknown function (ip: 000001b386e101d6)
g at D:\jl\creduce_julia\main.jl:5
unknown function (ip: 000001b386e10146)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:226
eval_stmt_value at C:/workdir/src\interpreter.c:177 [inlined]
eval_body at C:/workdir/src\interpreter.c:624
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:762
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:912
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:856
ijl_toplevel_eval at C:/workdir/src\toplevel.c:921 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:971
eval at .\boot.jl:370 [inlined]
include_string at .\loading.jl:1903
_include at .\loading.jl:1963
include at .\Base.jl:457
jfptr_include_38014.clone_1 at C:\Users\inkyd\.julia\juliaup\julia-1.9.4+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
exec_options at .\client.jl:307
_start at .\client.jl:522
jfptr__start_42472.clone_1 at C:\Users\inkyd\.julia\juliaup\julia-1.9.4+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
true_main at C:/workdir/src\jlapi.c:573
jl_repl_entrypoint at C:/workdir/src\jlapi.c:717
mainCRTStartup at C:/workdir/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2999 (Pool: 2987; Big: 12); GC: 0

PS D:\jl\creduce_julia> julia +1.8  main.jl

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x18407df018c -- <= at .\int.jl:481 [inlined]
getindex at .\essentials.jl:609 [inlined]
f at D:\jl\creduce_julia\main.jl:4
in expression starting at D:\jl\creduce_julia\main.jl:7
<= at .\int.jl:481 [inlined]
getindex at .\essentials.jl:609 [inlined]
f at D:\jl\creduce_julia\main.jl:4
unknown function (ip: 0000018407df0216)
g at D:\jl\creduce_julia\main.jl:5
unknown function (ip: 0000018407df0155)
jl_apply at C:/workdir/src\julia.h:1843 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:215
eval_stmt_value at C:/workdir/src\interpreter.c:166 [inlined]
eval_body at C:/workdir/src\interpreter.c:612
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:750
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:906
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:850
ijl_toplevel_eval at C:/workdir/src\toplevel.c:915 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:965
eval at .\boot.jl:368 [inlined]
include_string at .\loading.jl:1428
_include at .\loading.jl:1488
include at .\Base.jl:419
jfptr_include_51599.clone_1 at C:\Users\inkyd\.julia\juliaup\julia-1.8.5+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
exec_options at .\client.jl:303
_start at .\client.jl:522
jfptr__start_54247.clone_1 at C:\Users\inkyd\.julia\juliaup\julia-1.8.5+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1843 [inlined]
true_main at C:/workdir/src\jlapi.c:575
jl_repl_entrypoint at C:/workdir/src\jlapi.c:719
mainCRTStartup at C:/workdir/cli\loader_exe.c:59
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2907 (Pool: 2895; Big: 12); GC: 0

PS D:\jl\creduce_julia> julia +1.6  main.jl

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x568d01b5 -- <= at .\int.jl:442 [inlined]
getindex at .\essentials.jl:588 [inlined]
f at D:\jl\creduce_julia\main.jl:4
in expression starting at D:\jl\creduce_julia\main.jl:7
<= at .\int.jl:442 [inlined]
getindex at .\essentials.jl:588 [inlined]
f at D:\jl\creduce_julia\main.jl:4
g at D:\jl\creduce_julia\main.jl:5
unknown function (ip: 00000000568d0140)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
do_call at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:115
eval_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:204
eval_stmt_value at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:155 [inlined]
eval_body at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:576
jl_interpret_toplevel_thunk at /cygdrive/c/buildbot/worker/package_win64/build/src\interpreter.c:670
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:877
jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:825
jl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:886 [inlined]
jl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win64/build/src\toplevel.c:929
eval at .\boot.jl:360 [inlined]
include_string at .\loading.jl:1116
_include at .\loading.jl:1170
include at .\Base.jl:384
exec_options at .\client.jl:285
_start at .\client.jl:485
jfptr__start_28700.clone_1 at C:\Users\inkyd\.julia\juliaup\julia-1.6.7+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at /cygdrive/c/buildbot/worker/package_win64/build/src\julia.h:1703 [inlined]
true_main at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:560
repl_entrypoint at /cygdrive/c/buildbot/worker/package_win64/build/src\jlapi.c:702
mainCRTStartup at /cygdrive/c/buildbot/worker/package_win64/build/cli\loader_exe.c:51
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 2648 (Pool: 2638; Big: 10); GC: 0

PS D:\jl\creduce_julia> julia +1.0  main.jl
ERROR: LoadError: MethodError: no method matching f(::Type{Tuple{f} where f})
Closest candidates are:
  f(!Matched::DataType) at D:\jl\creduce_julia\main.jl:4
Stacktrace:
 [1] g(::TupleHolder52644) at D:\jl\creduce_julia\main.jl:5
 [2] top-level scope at none:0
 [3] include at .\boot.jl:317 [inlined]
 [4] include_relative(::Module, ::String) at .\loading.jl:1044
 [5] include(::Module, ::String) at .\sysimg.jl:29
 [6] exec_options(::Base.JLOptions) at .\client.jl:266
 [7] _start() at .\client.jl:425
in expression starting at D:\jl\creduce_julia\main.jl:7

[vtjnash]

Looks like an inlining bug, where inlining adds an assertion that the field is a DataType that was not present earlier:

julia> Base.code_ircode(call_sig, typeof((TupleHolder(Tuple{V} where {V}),)), optimize_until="compact 1")
1-element Vector{Any}:
 1 1 ─ %1 = Base.getproperty(_2, :tuple)::Type{<:Tuple}
  │   %2 = Main.do_thing(%1)::Any
  └──      return %2
   => Any

julia> Base.code_ircode(call_sig, typeof((TupleHolder(Tuple{V} where {V}),)), optimize_until="compact 2")
1-element Vector{Any}:
 1 1 ─ %1 = Base.getfield(_2, :tuple)::Type{<:Tuple}
  │   %2 = invoke Main.do_thing(%1::DataType)::Any
  └──      return %2
   => Any

[vtjnash]

Traced back to this unsound assumption made by the comment here. If we see duplicated dispatch signatures, it does not mean that this method is impossible to call, rather it means that type-intersection, since it is not guaranteed to precisely and accurate represent all type-intersections, could not precisely describe the domain over which the call will be applicable. @aviatesk (from #44512)

diff --git a/base/compiler/ssair/inlining.jl b/base/compiler/ssair/inlining.jl
index 639a7fcc09..33eb14596b 100644
--- a/base/compiler/ssair/inlining.jl
+++ b/base/compiler/ssair/inlining.jl
@@ -1463,7 +1465,7 @@ function handle_match!(cases::Vector{InliningCase},
     # We may see duplicated dispatch signatures here when a signature gets widened
     # during abstract interpretation: for the purpose of inlining, we can just skip
     # processing this dispatch candidate (unless unmatched type parameters are present)
-    !allow_typevars && _any(case->case.sig === spec_types, cases) && return true
+    #!allow_typevars && _any(case->case.sig === spec_types, cases) && return true
     item = analyze_method!(match, argtypes, info, flag, state; allow_typevars)
     item === nothing && return false
     push!(cases, InliningCase(spec_types, item))

and a few other drive-by bugs:

diff --git a/base/compiler/ssair/inlining.jl b/base/compiler/ssair/inlining.jl
index 639a7fcc09..33eb14596b 100644
--- a/base/compiler/ssair/inlining.jl
+++ b/base/compiler/ssair/inlining.jl
@@ -1320,6 +1320,8 @@ function handle_any_const_result!(cases::Vector{InliningCase},
         result = inlining_policy(state.interp, result, info, flag, result.mi, argtypes)
         if isa(result, SemiConcreteResult)
             return handle_semi_concrete_result!(cases, result, info, flag, state; allow_abstract)
+        else
+            return false
         end
     end
     if isa(result, ConstPropResult)
@@ -1433,9 +1435,9 @@ function compute_inlining_cases(@nospecialize(info::CallInfo), flag::UInt8, sig:
             match = meth[1]
             result = getresult(info, 1)
         end
-        handle_any_const_result!(cases,
-            result, match, argtypes, info, flag, state; allow_abstract=true, allow_typevars=true)
-        fully_covered = handled_all_cases = match.fully_covers
+        if handle_any_const_result!(cases, result, match, argtypes, info, flag, state; allow_abstract=true, allow_typevars=true)
+            fully_covered = handled_all_cases = match.fully_covers
+        end
     elseif !handled_all_cases
         # if we've not seen all candidates, union split is valid only for dispatch tuples
         filter!(case::InliningCase->isdispatchtuple(case.sig), cases)