Incremental sysimage build doesn't support packages with same name but different uuid

[krynju]

This one is similiar to #767, but setup and place where it errors are different.

Setup

Simple two projects that have a top level dependency PackageA, but with different uuids

using PackageCompiler

using Pkg

Pkg.activate("project1") # has Package1 with uuid=1234
create_sysimage(["Package1"]; sysimage_path="sysimage5.so")

Pkg.activate("project2") # has Package1 with uuid=5678
create_sysimage(["Package1"]; sysimage_path="sysimage6.so", base_sysimage="sysimage5.so")

Error

julia> create_sysimage(["Package1"]; sysimage_path="sysimage6.so", base_sysimage="sysimage5.so")
⠙ [00m:07s] PackageCompiler: compiling incremental system imageLoadError
⡆ [00m:08s] PackageCompiler: compiling incremental system imageimporting Package1 into Main conflicts with an existing identifier
in expression starting at /tmp/jl_bOn2oN:7
✖ [00m:08s] PackageCompiler: compiling incremental system image
ERROR: failed process: Process(`/home/krynju/julia-1.7.3/bin/julia --color=yes --startup-file=no --cpu-target=native -O3 --sysimage=sysimage5.so --project=/home/krynju/sameuuidtest/project2 --output-o=/tmp/jl_sKRDQC.o /tmp/jl_bOn2oN`, ProcessExited(1)) [1]

Error appears due to this code:

PackageCompiler.jl/src/PackageCompiler.jl

Lines 377 to 382 in 05f6854

	# Make packages available in Main. It is unclear if this is the right thing to do.
	for pkg in packages
	print(julia_code_buffer, """
	import $pkg
	""")
	end

Solution

I worked around this by commenting out this part of code and everything looks to be functioning just right.

The error didn't appear if the package was the same uuid, but a different version, which is probably something weird to handle as well

A build flag to turn off this import behavior would probably be good enough as I'm not sure what the import to Main is dedicated for

[KristofferC]

A build flag to turn off this import behavior would probably be good enough as I'm not sure what the import to Main is dedicated for

It is there to be able to get a reference to the module (e.g. using .Plots) without having to have a Project file available to lookup the UUID when doing using Plots.

[staticfloat]

Here's a simple way to reproduce this:

julia> versioninfo()
Julia Version 1.9.0-rc1
Commit 3b2e0d8fbc1 (2023-03-07 07:51 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 32 × AMD Ryzen 9 3950X 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, znver2)
  Threads: 1 on 32 virtual cores
Environment:
  JULIA_PKG_PRECOMPILE_AUTO = 1
  LD_LIBRARY_PATH = 
  JULIA_PKG_SERVER = https://pkg.julialang.org

$ cat broken/Project.toml 
[deps]
Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
IntervalSets = "8197267c-284f-5f27-9208-e0e47529a953"

$ cat broken/broken.jl 
using PackageCompiler, Pkg

Pkg.activate(@__DIR__)
create_sysimage(["Accessors", "IntervalSets"]; sysimage_path=joinpath(@__DIR__,"broken.so"))

$ julia +1.9 --project broken/broken.jl 
  Activating project at `~/.julia/dev/PackageCompiler/broken`
Precompiling environment...
  4 dependencies successfully precompiled in 1 seconds. 12 already precompiled.
⠸ [00m:08s] PackageCompiler: compiling incremental system imageinvalid redefinition of constant IntervalSetsExt
Stacktrace:
 [1] top-level scope
   @ none:1
⣠ [00m:08s] PackageCompiler: compiling incremental system imageModule, e::Any)
   @ Core ./boot.jl:370
 [3] top-level scope
   @ /tmp/jl_w3cHYriBwz:61
 [4] eval(m::Module, e::Any)
   @ Core ./boot.jl:370
 [5] top-level scope
   @ /tmp/jl_w3cHYriBwz:56
in expression starting at /tmp/jl_w3cHYriBwz:56
✖ [00m:08s] PackageCompiler: compiling incremental system image

[KristofferC]

There are two different issues being discussed here:

1. The OP which is caused by us trying to make packages available in Main. If two packages have the same name but different UUID this will not work. This will only happen when you are making an incremental sysimage with a package name on top of another sysimage with a different package of the same name.
2. The one described by @staticfloat. This occurs when we are replaying precompile statements we need to have a reference to the module. If there are multiple modules with the same name, this used to error because we tried import all of those into the same module. This is a dup of Precompile stage doesn't support packages with the same names and different uuids #767 and has been fixed by Some fixes to precompile "replay" #805