Skip to content

Conversation

IanButterworth
Copy link
Member

@IanButterworth IanButterworth commented Aug 20, 2024

Master

┌ Warning: Circular dependency detected. Precompilation will be skipped for:
│ Base.PkgId(Base.UUID("7f7a1694-90dd-40f0-9382-eb1efda571ba"), "Optimization")
│ Base.PkgId(Base.UUID("bca83a33-5cc9-4baa-983d-23429ab6bcbb"), "OptimizationBase")
│ Base.PkgId(Base.UUID("4297ee4d-0239-47d8-ba5d-195ecdf594fe"), "SymbolicAnalysis")
│ Base.PkgId(Base.UUID("04dc0061-c1f3-5338-88cf-aa3b7fdaee92"), "OptimizationForwardDiffExt")
│ Base.PkgId(Base.UUID("4a213a23-c09c-5cde-9712-b631ad2c72df"), "SymbolicsForwardDiffExt")
│ Base.PkgId(Base.UUID("d479e226-fb54-5ebe-a75e-a7af7f39127f"), "SymbolicsPreallocationToolsExt")
└ @ Base.Precompilation precompilation.jl:552

This PR

┌ Info: Circular dependency detected. Loading may still succeed, but parallel precompilation will be skipped for:
│   Optimization [7f7a1694-90dd-40f0-9382-eb1efda571ba]
│   OptimizationBase [bca83a33-5cc9-4baa-983d-23429ab6bcbb]
│   SymbolicAnalysis [4297ee4d-0239-47d8-ba5d-195ecdf594fe]
│   OptimizationForwardDiffExt [04dc0061-c1f3-5338-88cf-aa3b7fdaee92]
│   SymbolicsForwardDiffExt [4a213a23-c09c-5cde-9712-b631ad2c72df]
└   SymbolicsPreallocationToolsExt [d479e226-fb54-5ebe-a75e-a7af7f39127f]

The full output, showing that in this case loading kind of worked but the experience was quite alarming...

(@v1.12) pkg> add Optimization
...
    Updating `~/.julia/environments/v1.12/Project.toml`
  [7f7a1694] + Optimization v3.27.0
    Updating `~/.julia/environments/v1.12/Manifest.toml`
...
┌ Warning: Circular dependency detected. Loading may still succeed, but parallel precompilation will be skipped for:
│   Optimization [7f7a1694-90dd-40f0-9382-eb1efda571ba]
│   OptimizationBase [bca83a33-5cc9-4baa-983d-23429ab6bcbb]
│   SymbolicAnalysis [4297ee4d-0239-47d8-ba5d-195ecdf594fe]
│   OptimizationForwardDiffExt [04dc0061-c1f3-5338-88cf-aa3b7fdaee92]
│   SymbolicsForwardDiffExt [4a213a23-c09c-5cde-9712-b631ad2c72df]
│   SymbolicsPreallocationToolsExt [d479e226-fb54-5ebe-a75e-a7af7f39127f]
└ @ Base.Precompilation precompilation.jl:560
Precompiling all packages...
  174 dependencies successfully precompiled in 56 seconds. 37 already precompiled. 6 skipped due to circular dependency.

julia> using Optimization
┌ Warning: Circular dependency detected. Loading may still succeed, but parallel precompilation will be skipped for:
│   Optimization [7f7a1694-90dd-40f0-9382-eb1efda571ba]
│   OptimizationBase [bca83a33-5cc9-4baa-983d-23429ab6bcbb]
│   SymbolicAnalysis [4297ee4d-0239-47d8-ba5d-195ecdf594fe]
│   OptimizationForwardDiffExt [04dc0061-c1f3-5338-88cf-aa3b7fdaee92]
│   SymbolicsForwardDiffExt [4a213a23-c09c-5cde-9712-b631ad2c72df]
│   SymbolicsPreallocationToolsExt [d479e226-fb54-5ebe-a75e-a7af7f39127f]
└ @ Base.Precompilation precompilation.jl:560
[ Info: Precompiling Optimization [7f7a1694-90dd-40f0-9382-eb1efda571ba] 
┌ Warning: Module SymbolicsPreallocationToolsExt with build ID ffffffff-ffff-ffff-0009-02c06562b1df is missing from the cache.
│ This may mean SymbolicsPreallocationToolsExt [d479e226-fb54-5ebe-a75e-a7af7f39127f] does not support precompilation but is imported by a module that does.
└ @ Base loading.jl:2439
┌ Error: Error during loading of extension SymbolicsPreallocationToolsExt of Symbolics, use `Base.retry_load_extensions()` to retry.
│   exception =
│    1-element ExceptionStack:
│    Error when precompiling module, potentially caused by a __precompile__(false) declaration in the module.
│    Stacktrace:
│      [1] _require(pkg::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:2443
│      [2] __require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│        @ Base ./loading.jl:2291
│      [3] #invoke_in_world#3
│        @ ./essentials.jl:1082 [inlined]
│      [4] invoke_in_world
│        @ ./essentials.jl:1079 [inlined]
│      [5] _require_prelocked
│        @ ./loading.jl:2278 [inlined]
│      [6] _require_prelocked
│        @ ./loading.jl:2277 [inlined]
│      [7] run_extension_callbacks(extid::Base.ExtensionId)
│        @ Base ./loading.jl:1479
│      [8] run_extension_callbacks(pkgid::Base.PkgId)
│        @ Base ./loading.jl:1514
│      [9] run_package_callbacks(modkey::Base.PkgId)
│        @ Base ./loading.jl:1335
│     [10] __require_prelocked(uuidkey::Base.PkgId, env::String)
│        @ Base ./loading.jl:0
│     [11] #invoke_in_world#3
│        @ ./essentials.jl:1082 [inlined]
│     [12] invoke_in_world
│        @ ./essentials.jl:1079 [inlined]
│     [13] _require_prelocked
│        @ ./loading.jl:2278 [inlined]
│     [14] macro expansion
│        @ ./loading.jl:2217 [inlined]
│     [15] macro expansion
│        @ ./lock.jl:273 [inlined]
│     [16] __require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:2174
│     [17] #invoke_in_world#3
│        @ ./essentials.jl:1082 [inlined]
│     [18] invoke_in_world
│        @ ./essentials.jl:1079 [inlined]
│     [19] require(into::Module, mod::Symbol)
│        @ Base ./loading.jl:2167
│     [20] include
│        @ ./Base.jl:582 [inlined]
│     [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│        @ Base ./loading.jl:2796
│     [22] top-level scope
│        @ stdin:4
│     [23] eval
│        @ ./boot.jl:439 [inlined]
│     [24] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│        @ Base ./loading.jl:2618
│     [25] include_string
│        @ ./loading.jl:2628 [inlined]
│     [26] exec_options(opts::Base.JLOptions)
│        @ Base ./client.jl:322
│     [27] _start()
│        @ Base ./client.jl:553
└ @ Base loading.jl:1485

@vtjnash
Copy link
Member

vtjnash commented Aug 20, 2024

The error occurs because it fails to declare __precompile__(false). The circularity is not disallowed, but it is forbidden in code that declared it can be precompiled

@nsajko nsajko added compiler:precompilation Precompilation of modules error messages Better, more actionable error messages labels Aug 21, 2024
@jishnub
Copy link
Member

jishnub commented Aug 21, 2024

This needs to be documented better

@IanButterworth
Copy link
Member Author

@vtjnash also there could be circularity in the dependency tree but conditional loading might mean the circularity isn't complete.

@vtjnash
Copy link
Member

vtjnash commented Aug 21, 2024

That could be true, but as the message there said "SymbolicsPreallocationToolsExt does not support precompile, but packages that depend on it fail to declare __precompile__(false)"

@IanButterworth
Copy link
Member Author

Sorry, yes. I missed that you were talking about that part.

@topolarity
Copy link
Member

topolarity commented Aug 21, 2024

It's very unusual from my perspective that:

  1. I get this warning repeatedly upon both using X and ] precompile
  2. Despite the statements in this PR that this circularity is not allowed, loading seems to work anyway...

Does (1) go away after __precompile__(false) is added?

(2) presumably means that some extensions do not load, but otherwise loading succeeds? The experience is pretty bad right now though due to #55557

@topolarity
Copy link
Member

Also it's worth mentioning that the most common case is that you hit the error regarding the missing __precompile__(...) and then spend a very long time silently doing pre-compilation

As a user, it's hard to know that the error wasn't fatal and that the using X hasn't hung...

@IanButterworth
Copy link
Member Author

IIUC #55589 should remove the need for this warning

@IanButterworth IanButterworth deleted the ib/precompile_circular_warn_soften branch September 5, 2024 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compiler:precompilation Precompilation of modules error messages Better, more actionable error messages

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants