Hide loading internals in LoadError / precompilation errors #55816

IanButterworth · 2024-09-19T21:19:36Z

Related to #52988

With an error injected into the top level in Dates, and showing the effect deep in LazyArtifacts

This PR

Failed to precompile LazyArtifacts [4af54fe1-eca0-43a8-85a7-787d91b784e3] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/LazyArtifacts/jl_oVLId5".
ERROR: LoadError: Dummy error
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:44
 [2] top-level scope
   @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Dates/src/Dates.jl:35
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Dates/src/Dates.jl:3
ERROR: LoadError: Failed to precompile Dates [ade2ca70-3891-5945-98fb-dc099432e06a] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/Dates/jl_ZPwCf1".
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/TOML/src/TOML.jl:3
ERROR: LoadError: Failed to precompile TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/TOML/jl_CUVhh4".
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Pkg.jl:3
ERROR: LoadError: Failed to precompile Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/Pkg/jl_rhQyB7".
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/LazyArtifacts/src/LazyArtifacts.jl:3

Master

Failed to precompile LazyArtifacts [4af54fe1-eca0-43a8-85a7-787d91b784e3] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/LazyArtifacts/jl_qbAO3Q".
ERROR: LoadError: Dummy error
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:44
  [2] top-level scope
    @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Dates/src/Dates.jl:35
  [3] include
    @ ./Base.jl:582 [inlined]
  [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2848
  [5] top-level scope
    @ stdin:4
  [6] eval
    @ ./boot.jl:439 [inlined]
  [7] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2670
  [8] include_string
    @ ./loading.jl:2680 [inlined]
  [9] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:327
 [10] _start()
    @ Base ./client.jl:558
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Dates/src/Dates.jl:3
in expression starting at stdin:4
ERROR: LoadError: Failed to precompile Dates [ade2ca70-3891-5945-98fb-dc099432e06a] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/Dates/jl_1zLF1g".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:44
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64})
    @ Base ./loading.jl:3117
  [3] (::Base.var"#_require##0#_require##1"{Base.PkgId})()
    @ Base ./loading.jl:2519
  [4] mkpidlock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:93
  [5] #mkpidlock#7
    @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:88 [inlined]
  [6] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:114
  [7] #invokelatest#1
    @ ./essentials.jl:1050 [inlined]
  [8] invokelatest
    @ ./essentials.jl:1045 [inlined]
  [9] maybe_cachefile_lock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:3682
 [10] maybe_cachefile_lock
    @ ./loading.jl:3679 [inlined]
 [11] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:2515
 [12] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2343
 [13] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [14] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [15] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2330
 [16] macro expansion
    @ ./loading.jl:2269 [inlined]
 [17] macro expansion
    @ ./lock.jl:287 [inlined]
 [18] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2226
 [19] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [20] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [21] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2219
 [22] include
    @ ./Base.jl:582 [inlined]
 [23] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2848
 [24] top-level scope
    @ stdin:4
 [25] eval
    @ ./boot.jl:439 [inlined]
 [26] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2670
 [27] include_string
    @ ./loading.jl:2680 [inlined]
 [28] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:327
 [29] _start()
    @ Base ./client.jl:558
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/TOML/src/TOML.jl:3
in expression starting at stdin:4
ERROR: LoadError: Failed to precompile TOML [fa267f1f-6049-4f14-aa54-33bafae1ed76] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/TOML/jl_BKimdq".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:44
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64})
    @ Base ./loading.jl:3117
  [3] (::Base.var"#_require##0#_require##1"{Base.PkgId})()
    @ Base ./loading.jl:2519
  [4] mkpidlock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:93
  [5] #mkpidlock#7
    @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:88 [inlined]
  [6] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:114
  [7] #invokelatest#1
    @ ./essentials.jl:1050 [inlined]
  [8] invokelatest
    @ ./essentials.jl:1045 [inlined]
  [9] maybe_cachefile_lock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:3682
 [10] maybe_cachefile_lock
    @ ./loading.jl:3679 [inlined]
 [11] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:2515
 [12] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2343
 [13] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [14] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [15] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2330
 [16] macro expansion
    @ ./loading.jl:2269 [inlined]
 [17] macro expansion
    @ ./lock.jl:287 [inlined]
 [18] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2226
 [19] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [20] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [21] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2219
 [22] include
    @ ./Base.jl:582 [inlined]
 [23] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
    @ Base ./loading.jl:2848
 [24] top-level scope
    @ stdin:4
 [25] eval
    @ ./boot.jl:439 [inlined]
 [26] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2670
 [27] include_string
    @ ./loading.jl:2680 [inlined]
 [28] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:327
 [29] _start()
    @ Base ./client.jl:558
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Pkg.jl:3
in expression starting at stdin:4
ERROR: LoadError: Failed to precompile Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] to "/Users/ian/Documents/GitHub/julia/usr/share/julia/compiled/v1.12/Pkg/jl_G5UVfl".
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:44
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool; flags::Cmd, cacheflags::Base.CacheFlags, reasons::Dict{String, Int64})
    @ Base ./loading.jl:3117
  [3] (::Base.var"#_require##0#_require##1"{Base.PkgId})()
    @ Base ./loading.jl:2519
  [4] mkpidlock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, at::String, pid::Int32; kwopts::@Kwargs{stale_age::Int64, wait::Bool})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:93
  [5] #mkpidlock#7
    @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:88 [inlined]
  [6] trymkpidlock(::Function, ::Vararg{Any}; kwargs::@Kwargs{stale_age::Int64})
    @ FileWatching.Pidfile ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/FileWatching/src/pidfile.jl:114
  [7] #invokelatest#1
    @ ./essentials.jl:1050 [inlined]
  [8] invokelatest
    @ ./essentials.jl:1045 [inlined]
  [9] maybe_cachefile_lock(f::Base.var"#_require##0#_require##1"{Base.PkgId}, pkg::Base.PkgId, srcpath::String; stale_age::Int64)
    @ Base ./loading.jl:3682
 [10] maybe_cachefile_lock
    @ ./loading.jl:3679 [inlined]
 [11] _require(pkg::Base.PkgId, env::String)
    @ Base ./loading.jl:2515
 [12] __require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2343
 [13] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [14] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [15] _require_prelocked(uuidkey::Base.PkgId, env::String)
    @ Base ./loading.jl:2330
 [16] macro expansion
    @ ./loading.jl:2269 [inlined]
 [17] macro expansion
    @ ./lock.jl:287 [inlined]
 [18] __require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2226
 [19] #invoke_in_world#2
    @ ./essentials.jl:1082 [inlined]
 [20] invoke_in_world
    @ ./essentials.jl:1079 [inlined]
 [21] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:2219
 [22] include
    @ ./Base.jl:582 [inlined]
 [23] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2848
 [24] top-level scope
    @ stdin:4
 [25] eval
    @ ./boot.jl:439 [inlined]
 [26] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2670
 [27] include_string
    @ ./loading.jl:2680 [inlined]
 [28] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:327
 [29] _start()
    @ Base ./client.jl:558
in expression starting at /Users/ian/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/LazyArtifacts/src/LazyArtifacts.jl:3
in expression starting at stdin:4

KristofferC · 2024-09-20T11:42:54Z

Fixes #50161?

topolarity · 2024-09-20T14:31:52Z

base/errorshow.jl

    !isa(ex.error, LoadError) && print(io, "LoadError: ")
+    if bt isa StackTraces.StackTrace
+        # Remove the internals of how precompilation is set up
+        StackTraces.remove_frames!(bt, :include_package_for_output; reverse=true, n=1)


This feels a bit error-prone - in particular, doesn't it require ruling out recursion in the loading code?

I wonder if rethrow(truncate_bt=true) / throw(e; truncate_bt=true) or Base.truncate_catch_backtrace(...) or a similar functionality would be better behaved

Yeah I guess the most certain part of this PR is what the result should look like. How that's achieved I was less clear on a good design.

What would be a good test case for recursion?

@topolarity if you had a particular loading pattern in mind it'd be great to understand it better.

By recursion do you mean an included file that has an include in it?

Sorry for the delay here @IanButterworth, I've been sitting on some changes to our stack trace handling that I wanted to upstream so that we can do this properly (without scanning for function names, etc.)

Let me get those open and come back to this early next week 🙂

Perfect. Thanks

@topolarity I was thinking about getting this in for 1.12. Did that work get completed?

Just to document what I was thinking from the hackathon. I'd like to try this pattern out:

# in loading code... struct UserCodeError err::Any backtrace_size::UInt end function foo() try unreliable_user_code() catch err # Save the backtrace size so that we can trim any "internal" frames that # are added later. rethrow(UserCodeError(err, length(Core.raw_catch_backtrace()))) end end function loading_entrypoint() try # ... (some code that directly / indirectly calls foo() catch err if err isa UserCodeError # Restore the backtrace to its state when returning from the user code # (i.e. remove any frames between `foo()` to `loading_entrypoint()`) resize!(Core.raw_catch_backtrace(), err.backtrace_size) # Forward the error to the user rethrow(err.err) else rethrow(err) end end end

Let me know if you have any thoughts 👍

c42f · 2025-08-07T02:07:01Z

An alternative could be just to delete LoadError entirely? It's a bit useless - it's more a vestige from the days when our backtraces didn't work properly, than anything. I've tried to do this in the past, but ran out of steam (#31882)

In general I don't love rethrow() because it lies to the user about the state of the system when the exception was thrown. For similar reasons, I don't love the idea of truncating backtraces. I do think it can be justified and I've used it occasionally, but I really wish we had a better solution to the problems it tries to solve 😅

IanButterworth added 2 commits September 19, 2024 17:03

hide loading internals in LoadError / precompilation errors

9c229f6

rename PkgPrecompileError -> PrecompilePkgsError

de6a472

IanButterworth added the packages Package management and loading label Sep 19, 2024

fixes

17633ca

IanButterworth requested a review from vtjnash September 20, 2024 12:02

topolarity reviewed Sep 20, 2024

View reviewed changes

Merge branch 'master' into ib/loading_stacktrace_tidy

b04754c

topolarity mentioned this pull request Aug 6, 2025

Allow outside catchers to see full stacktrace information of macro expansion c42f/JuliaLowering.jl#23

Merged

Uh oh!

Uh oh!

Hide loading internals in LoadError / precompilation errors #55816

Are you sure you want to change the base?

Hide loading internals in LoadError / precompilation errors #55816

Uh oh!

Conversation

IanButterworth commented Sep 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented Sep 20, 2024

Uh oh!

topolarity Sep 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

IanButterworth Sep 20, 2024

Choose a reason for hiding this comment

Uh oh!

IanButterworth Oct 1, 2024

Choose a reason for hiding this comment

Uh oh!

IanButterworth Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

IanButterworth Oct 12, 2024

Choose a reason for hiding this comment

Uh oh!

topolarity Oct 12, 2024

Choose a reason for hiding this comment

Uh oh!

IanButterworth Oct 12, 2024

Choose a reason for hiding this comment

Uh oh!

IanButterworth Feb 4, 2025

Choose a reason for hiding this comment

Uh oh!

topolarity Jul 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

c42f commented Aug 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

IanButterworth commented Sep 19, 2024 •

edited

Loading

topolarity Sep 20, 2024 •

edited

Loading

topolarity Jul 27, 2025 •

edited

Loading