1.12 - 2x+ TTFX Regressions

[chris-b1]

Not sure if this is worth investigating/reporting more at the package level, but so far 1.12 has felt generally significantly more sluggish to me on TTFX and a few things I've tested have all confirmed, mostly seeing it packages but also base

setup & versioninfo()

cat > temp.csv <<EOF
a,b,c
1,2,3
4,5,6
EOF

julia> versioninfo()
Julia Version 1.12.0
Commit b907bd0600f (2025-10-07 15:42 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × 12th Gen Intel(R) Core(TM) i7-1270P
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, alderlake)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 16 virtual cores)

1.11

julia> @time show([1, 2, 3])
[1, 2, 3]  0.072002 seconds (78.65 k allocations: 4.288 MiB, 97.32% compilation time)

julia> @time df = DataFrame(:a => [1, 2, 3], :b => ["a", "b", "c"])
  0.015479 seconds (607 allocations: 27.250 KiB, 97.67% compilation time)

julia> @time CSV.File("temp.csv")
  1.183245 seconds (2.28 M allocations: 138.655 MiB, 14.13% gc time, 99.57% compilation time: 98% of which was recompilation)

1.12

# 3x slower
julia> @time show([1, 2, 3])  
[1, 2, 3]  0.211916 seconds (530.79 k allocations: 28.203 MiB, 99.89% compilation time)

# 4.5x slower
julia> @time df = DataFrame(:a => [1, 2, 3], :b => ["a", "b", "c"])  
  0.068731 seconds (794.13 k allocations: 41.691 MiB, 33.44% gc time, 95.99% compilation time)

# 2.5x slower
julia> @time CSV.File("temp.csv")
  2.761471 seconds (10.10 M allocations: 523.286 MiB, 12.35% gc time, 99.72% compilation time: 99% of which was recompilation)

[bclyons12]

I've also been seeing slowdowns as well in v1.12 on my MacOS. Following this for now and I'll post examples if/when I find MWEs,

[bvdmitri]

We also see a consistent increase in TTFX in RxInfer since 1.10

time julia +1.10 --startup-file=no -e '@time include("testr.jl"); @time foo()'
  2.696834 seconds (5.36 M allocations: 372.804 MiB, 7.28% gc time, 47.55% compilation time: 2% of which was recompilation)
  3.395323 seconds (10.81 M allocations: 733.528 MiB, 6.06% gc time, 99.67% compilation time)
julia +1.10 --startup-file=no -e '@time include("testr.jl"); @time foo()'  7.97s user 0.21s system 114% cpu 7.128 total

time julia +1.11 --startup-file=no -e '@time include("testr.jl"); @time foo()'
  3.237015 seconds (7.17 M allocations: 407.736 MiB, 6.11% gc time, 47.44% compilation time: 2% of which was recompilation)
  3.801961 seconds (15.46 M allocations: 774.450 MiB, 3.23% gc time, 99.75% compilation time)
julia +1.11 --startup-file=no -e '@time include("testr.jl"); @time foo()'  9.08s user 0.33s system 112% cpu 8.345 total

time julia +1.12 --startup-file=no -e '@time include("testr.jl"); @time foo()'
  3.613156 seconds (7.96 M allocations: 446.449 MiB, 8.58% gc time, 42.87% compilation time: 1% of which was recompilation)
  4.442799 seconds (20.87 M allocations: 1.040 GiB, 4.54% gc time, 99.70% compilation time)
julia +1.12 --startup-file=no -e '@time include("testr.jl"); @time foo()'  10.68s user 0.54s system 103% cpu 10.810 total

Script content

using RxInfer, Random

n = 500  # Number of coin flips
p = 0.75 # Bias of a coin

distribution = Bernoulli(p)
dataset      = rand(distribution, n)

# GraphPPL.jl export `@model` macro for model specification
# It accepts a regular Julia function and builds a factor graph under the hood
@model function coin_model(y, a, b)
    # We endow θ parameter of our model with some prior
    θ ~ Beta(a, b)
    # We assume that outcome of each coin flip
    # is governed by the Bernoulli distribution
    for i in eachindex(y)
        y[i] ~ Bernoulli(θ)
    end
end

foo() = infer(
    model = coin_model(a = 2.0, b = 7.0),
    data  = (y = dataset, )
)

[KristofferC]

This thread will become very noisy if everyone dumps their latency regressions in it. At least here, let's focus on the examples given by the OP.

[KristofferC]

Regarding show, in 1.12 we now compile:

#=  134.5 ms =# precompile(Tuple{typeof(Base.show), Base.TTY, Array{Int64, 1}})

which we don't on 1.11. It is possible that #59850 fixes this.

Update: With #59850 we still have the precompile signature being emitted but it is cheaper:

#=   60.4 ms =# precompile(Tuple{typeof(Base.show), Base.TTY, Array{Int64, 1}})

This shows some sort of regression I guess:

julia +1.11 -e '@time precompile(Tuple{typeof(Base.show), Base.TTY, Array{Int64, 1}})'
  0.071955 seconds (175.17 k allocations: 9.552 MiB, 99.97% compilation time)

julia +1.12 -e '@time precompile(Tuple{typeof(Base.show), Base.TTY, Array{Int64, 1}})'
  0.122043 seconds (551.51 k allocations: 29.385 MiB, 99.98% compilation time)

[KristofferC]

Regarding the CSV, I think this is a bit better repro just because it is easy to copy paste:

# repro.jl
@time using DataFrames
@time using CSV
write("temp.csv", """
Username; Identifier;First name;Last name
booker12;9012;Rachel;Booker
grey07;2070;Laura;Grey
johnson81;4081;Craig;Johnson
jenkins46;9346;Mary;Jenkins
smith79;5079;Jamie;Smith
""")
@time @eval df = DataFrame(:a => [1, 2, 3], :b => ["a", "b", "c"])
@time @eval CSV.File("temp.csv")

1.11:

❯ julia +1.11 --project repro.jl
  0.566445 seconds (785.63 k allocations: 53.055 MiB, 6.74% gc time, 5.74% compilation time: 31% of which was recompilation)
  0.122995 seconds (188.31 k allocations: 12.114 MiB, 8.02% compilation time)
  0.019919 seconds (105.00 k allocations: 5.244 MiB, 97.88% compilation time)
  0.644914 seconds (2.62 M allocations: 159.916 MiB, 10.90% gc time, 99.78% compilation time: 82% of which was recompilation)

1.12:

❯ julia +1.12 --project repro.jl
  0.464734 seconds (696.39 k allocations: 48.334 MiB, 13.52% gc time, 2.01% compilation time)
  0.100332 seconds (180.85 k allocations: 11.647 MiB, 10.42% compilation time)
  0.022272 seconds (121.16 k allocations: 6.184 MiB, 97.91% compilation time)
  1.471911 seconds (10.49 M allocations: 551.199 MiB, 10.22% gc time, 99.92% compilation time: 92% of which was recompilation)

We can see quite a bit more recompilation for example.

#=  459.3 ms =# precompile(Tuple{typeof(CSV.parsefilechunk!), CSV.Context, Int64, Int64, Int64, Int64, Array{CSV.Column, 1}, Type{Tuple{}}}) # recompile

seems to be a big one.

If we do not load DataFrames:

# repro_nodf.jl
@time using CSV
write("temp.csv", """
Username; Identifier;First name;Last name
booker12;9012;Rachel;Booker
grey07;2070;Laura;Grey
johnson81;4081;Craig;Johnson
jenkins46;9346;Mary;Jenkins
smith79;5079;Jamie;Smith
""")
@time @eval @time CSV.File("temp.csv")

then 1.11 and 1.12 are equally good:

❯ julia +1.11 --project repro_nodf.jl 
  0.207815 seconds (254.40 k allocations: 17.146 MiB, 6.33% compilation time)
  0.125057 seconds (367.94 k allocations: 18.361 MiB, 98.79% compilation time)

❯ julia +1.12 --project repro.jl      
  0.130998 seconds (241.96 k allocations: 15.483 MiB, 10.21% compilation time)
  0.125074 seconds (396.03 k allocations: 20.293 MiB, 99.11% compilation time)

so the crux seems to be DataFrames invalidating code in CSV.

These seem to be some of the big invalidators:

 inserting hash(ocr1::DataFrames.OnColRow) @ DataFrames ~/.julia/packages/DataFrames/rwKZk/src/join/core.jl:37 invalidated:
   backedges: 1: superseding hash(x) @ Base hashing.jl:28 with MethodInstance for hash(::Any) (1142 children)

 inserting merge(m::DataStructures.SortedMultiDict{K, D, Ord}, others...) where {K, D, Ord<:Ordering} @ DataStructures ~/.julia/packages/DataStructures/7NQC7/src/sorted_multi_dict.jl:379 invalidated:
   mt_backedges: 1: signature Tuple{typeof(merge), Any, @NamedTuple{delim_flags::UInt16}} triggered MethodInstance for Base.JuliaSyntax.parse_brackets(::Function, ::Base.JuliaSyntax.ParseState, ::Base.JuliaSyntax.Kind, ::Bool) (1146 children)

This should specialize on the function:

[KristofferC]

The CSV case is fixed by JuliaData/DataFrames.jl#3516

[bvdmitri]

As an idea, would it be possible to detect those common invalidation pattern (like 1-arg hash function) during package registration and block the automatic package registration if package invalidates Base code for example. That would stimulate the ecosystem to fix at least those common patterns

[KristofferC]

I think #31222 (comment) is the language feature to track w.r.t that.

[adienes]

I also think hash should more strongly warn against doing anything but implementing the 2-arg form. I made a PR for this last week #59781

in this case it was ok since isequal was only defined among OnColRows, but in general defining a 1-arg hash is likely to lead to bugs

[KristofferC]

With JuliaData/DataFrames.jl#3516 1.12 is significantly better than 1.11:

❯ julia +1.11 --project repro.jl
  0.401179 seconds (789.82 k allocations: 53.316 MiB, 8.67% gc time, 5.04% compilation time: 51% of which was recompilation)
  0.082140 seconds (189.75 k allocations: 12.121 MiB, 10.04% compilation time)
  0.017577 seconds (105.00 k allocations: 5.244 MiB, 98.42% compilation time)
  0.615735 seconds (2.62 M allocations: 159.915 MiB, 10.45% gc time, 99.84% compilation time: 83% of which was recompilation)

❯ ~/julia1.12/julia --project repro.jl
  0.337246 seconds (702.26 k allocations: 48.437 MiB, 16.33% gc time, 2.52% compilation time)
  0.065399 seconds (185.14 k allocations: 11.876 MiB, 13.57% compilation time)
  0.019137 seconds (121.91 k allocations: 5.935 MiB, 98.38% compilation time)
  0.120706 seconds (397.33 k allocations: 19.386 MiB, 15.62% gc time, 99.21% compilation time)

---

I bisected the show regression to 377643f9848cf88b79e so it isn't strange that this takes longer to compile since we are also compiling the full printstyled code now.

[chris-b1]

Thanks for the quick investigation/fix @KristofferC! I was able to see similar speedups by checking out your DataFrames PR

[KristofferC]

Nightly has:

nightly:
julia> @time @eval show([1,2,3])
[1, 2, 3]  0.061596 seconds (181.74 k allocations: 9.049 MiB, 99.46% compilation time)

1.11:
julia> @time @eval show([1,2,3])
[1, 2, 3]  0.068166 seconds (135.40 k allocations: 7.237 MiB, 38.56% gc time, 99.27% compilation time)

so at this point I don't think there is anything more here to do. This is obviously not saying that every latency regression in 1.12 is fixed.

Feel free to open more issues, the more specific the better. A link to a full CI run that is slower is not super helpful because it only shows things that are more or less known (that there are cases where 1.12 latency is worse).