CI test failure due to ambiguous method in ccall test

[Keno]

https://travis-ci.org/JuliaLang/julia/jobs/418783280#L2142-L2145

Some tests did not pass: 597 passed, 1 failed, 0 errored, 0 broken.LinearAlgebra/matmul: Test Failed at /tmp/julia/share/julia/stdlib/v1.1/LinearAlgebra/test/matmul.jl:449
  Expression: detect_ambiguities(LinearAlgebra, Base; imported=true, recursive=true) == []
   Evaluated: Tuple{Method,Method}[(unsafe_convert(::Type{Main.Test97Main_ccall.SpillPint}, vr) in Main.Test97Main_ccall at /tmp/julia/share/julia/test/ccall.jl:1268, unsafe_convert(::Type{T}, x::T) where T in Base at essentials.jl:346)] == Any[]

Looks like this may happen if the ccall test and the LinearAlgebra tests are run on the same worker? May need a unsafe_convert(::Type{SpillPint}, x::SpillPint) = x method.

[Keno]

Another one with Furlongs:
https://travis-ci.org/JuliaLang/julia/jobs/418836430#L2340

 Evaluated: Tuple{Method,Method}[(promote_type(::Type{Main.Furlongs.Furlong{p,T}}, ::Type{Main.Furlongs.Furlong{p,S}}) where {p, T, S} in Main.Furlongs at /tmp/julia/share/julia/test/testhelpers/Furlongs.jl:21, promote_type(::Type{T}, ::Type{T}) where T in Base at promotion.jl:210), (^(x::Number, y::Main.Test84Main_math.Float22716) in Main.Test84Main_math at /tmp/julia/share/julia/test/math.jl:619, ^(z::Complex{T}, p::S) where {T<:Real, S<:Real} in Base at complex.jl:751), (convert(::Type{Main.Test81Main_core.type_1{T}}, x::S) where {T<:Number, S<:Number} in Main.Test81Main_core at /tmp/julia/share/julia/test/core.jl:2051, convert(::Type{T}, x::T) where T<:Number in Base at number.jl:6), (//(x::Main.Furlongs.Furlong{p,T} where T<:Number, y::S) where {p, S<:Number} in Main.Furlongs at /tmp/julia/share/julia/test/testhelpers/Furlongs.jl:69, //(x::Number, y::Complex) in Base at rational.jl:58), (^(x::Number, y::Main.Test84Main_math.Float22716) in Main.Test84Main_math at /tmp/julia/share/julia/test/math.jl:619, ^(::Irrational{:ℯ}, x::Number) in Base.MathConstants at mathconstants.jl:91), (^(x::Number, y::Main.Test84Main_math.Float22716) in Main.Test84Main_math at /tmp/julia/share/julia/test/math.jl:619, ^(z::Complex{T}, p::T) where T<:Real in Base at complex.jl:737)] == Any[]

[StefanKarpinski]

Could we exclude methods for test-only types from the ambiguity test? Or maybe always run that first?

[Keno]

Another one:

   Evaluated: Tuple{Method,Method}[(isequal(x, ::Main.Test80Main_arrayops.totally_not_five26034) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2459, isequal(::Missing, ::Any) in Base at missing.jl:60), (==(x, ::Main.Test80Main_arrayops.totally_not_five26034) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2464, ==(w::WeakRef, v) in Base at gcutils.jl:4), (==(::Main.Test80Main_arrayops.totally_not_five26034, x) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2463, ==(::Any, ::Missing) in Base at missing.jl:55), (==(x, ::Main.Test80Main_arrayops.totally_not_five26034) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2464, ==(::Missing, ::Any) in Base at missing.jl:54), (==(::Main.Test80Main_arrayops.totally_not_five26034, x) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2463, ==(w, v::WeakRef) in Base at gcutils.jl:5), (isequal(::Main.Test80Main_arrayops.totally_not_five26034, x) in Main.Test80Main_arrayops at /tmp/julia/share/julia/test/arrayops.jl:2458, isequal(::Any, ::Missing) in Base at missing.jl:61)] == Any[]

Disabling ambiguities that contain types only defined in the test suite seems reasonable to me.

[StefanKarpinski]

Running the ambiguity test first would have the same effect and be much easier to implement :)

[tkf]

There are still two more ambiguities in SparseArrays #28665. Fixing it probably adds detect_ambiguities to its test. If you are to shuffle test order, can you move it to the very end so that similar failure won't show up?

[Keno]

Easiest thing to do is just to run the ambiguity test in a clean process. We do that for a bunch of other things also (search for _exec).

[tkf]

It looks like that test/ambiguous.jl is moved to the beginning manually?:

julia/test/choosetests.jl

Lines 127 to 133 in 81b04cf

	# do ambiguous first to avoid failing if ambiguities are introduced by other tests
	if "ambiguous" in skip_tests
	filter!(x -> x != "ambiguous", tests)
	elseif "ambiguous" in tests
	filter!(x -> x != "ambiguous", tests)
	prepend!(tests, ["ambiguous"])
	end

So maybe the easiest solution would be to do the global check in test/ambiguous.jl (#28665) and remove detect_ambiguities from LinearAlgebra?

[tkf]

Actually, I changed my mind. Running the ambiguity test in a separated process sounds like the cleanest solution. I made a PR to do this.