Precompiling packages conflict with Distributed @everywhere

[BioTurboNick]

When I update packages using pkg>update, using Distributed with @everywhere using MyPackage, I will routinely encounter various errors related to precompilation emitted by the various workers.

Some examples here:
https://discourse.julialang.org/t/failed-to-precompile-error-on-worker-processes/40638/8

I don't know if this is a Pkg issue or a Distributed issue.

Sometimes manually using the problematic packages in the REPL allows things to work again.

[BioTurboNick]

It's possible that there may be a condition occurring where the main REPL doesn't think a package needs to be precompiled, but the worker procs do? Or maybe it has something to do with dependencies-of-dependencies, where the top level is fully precompiled but the dependencies aren't?

[zomborid]

I experienced this on a local machine with workers. I think it is caused by the main thread and the workers trying to precompile at the same time.
My workaround is to limit the precompilation to the main thread by omitting the @everywhere. First run @time using/include: the time causes the program to wait until precompilation is done, then run the command again, with @everywhere.

For example:

@time include("lots_of_package_includes.jl")
@everywhere include("lots_of_package_includes.jl")

Edit:
The probelm still remains but usually in REPL with multiple tries (2-3) it starts to work.
From cmd I am unable to start a distributed script because Julia 1.5 still precompiles on every startup and that causes a conflict...

[BioTurboNick]

I think maybe #36416 which is present in 1.6.0-rc1 may be the fix for this? If so, this issue can be closed. I'll try it out a few times first.

[stillyslalom]

Did #36416 address the problem (and can the issue be closed)?

[BioTurboNick]

I suppose I didn't encounter an issue when trying again, so yes.