mx assumes base JDK includes jmods

[jerboaa]

Currently, it's not possible to build GraalVM with configuration --no-jlinking when the base JDK doesn't include JMOD files. Only building native-image works in the --no-jlinking config: mx --primary-suite substratevm --no-jlinking build.

[...]
Duplicate archive entry: 'graalvm-a5374bbf2b-java24-24.2.0-dev/lib/static/linux-amd64/musl/libmanagement_agent.a'
  old provenance: graalvm-a5374bbf2b-java24-24.2.0-dev/lib/<-extracted-dependency:substratevm:SVM_STATIC_LIBRARIES_SUPPORT
  new provenance: graalvm-a5374bbf2b-java24-24.2.0-dev/<-dependency:graalvm-jimage/*
Note: Some input files use or override a deprecated API that is marked for removal.
Note: Recompile with -Xlint:removal for details.
Compiling jdk.graal.compiler.hotspot.jdk21.test with javac-daemon(JDK 24)... [dependency jdk.graal.compiler.test updated]
Compiling jdk.graal.compiler.microbenchmarks with javac-daemon(JDK 24)... [dependency jdk.graal.compiler updated]
Archiving GRAAL_TEST... [dependency jdk.graal.compiler.test updated]
Compiling jdk.graal.compiler.hotspot.jdk23.test with javac-daemon(JDK 24)... [dependency jdk.graal.compiler.test updated]
Compiling jdk.graal.compiler.virtual.bench with javac-daemon(JDK 24)... [dependency jdk.graal.compiler.microbenchmarks updated]
Archiving GRAAL_COMPILER_WHITEBOX_MICRO_BENCHMARKS... [dependency jdk.graal.compiler.virtual.bench updated]
Archiving GRAAL_TEST_PREVIEW_FEATURE... [dependency jdk.graal.compiler.hotspot.jdk21.test updated]
$ /disk/graal/upstream-sources/graal/sdk/mxbuild/linux-amd64/GRAALVM_A5374BBF2B_JAVA24/graalvm-a5374bbf2b-java24-24.2.0-dev/bin/native-image --version
native-image 24 2025-03-18
OpenJDK Runtime Environment (build 24+24-jvmci-b01)
OpenJDK 64-Bit Server VM (build 24+24-jvmci-b01, mixed mode)

However, this breaks when the base JDK doesn't include the jmods directory. The whole point of --no-jlinking is to keep the base JDK untouched. So it seems like a bug to require jmods (which only jlink is using anyway) for such a build.

What's more, JEP 493, included in JDK 24, allows for builds being produced which - if enabled - don't include jmods. This makes for a smaller JDK installation size. Base JDKs with JEP 493 enabled won't work:

mx --primary-suite substratevm --no-jlinking -v build fails with this error:

  File "/usr/lib64/python3.13/threading.py", line 1012, in _bootstrap
    self._bootstrap_inner()
extracting file org/tukaani/xz/XZFormatException.java to org/graalvm/shadowed/org/tukaani/xz/XZFormatException.java
extracting file org/jline/utils/Log.java to org/graalvm/shadowed/org/jline/utils/Log.java
  File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
    self.run()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 395, in run
    super(_DummyProcess, self).run()
  File "/usr/lib64/python3.13/threading.py", line 992, in run
    self._target(*self._args, **self._kwargs)
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 14694, in executeTask
    task.execute()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/build/tasks/build.py", line 165, in execute
    _built = self.build()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 5090, in build
    self.subject.make_archive(getattr(self, 'javac_daemon', None))
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[2024-11-21T17:32:46.639822843Z:127.0.0.1] Result = 0
extracting file org/tukaani/xz/XZIOException.java to org/graalvm/shadowed/org/tukaani/xz/XZIOException.java
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx_jardistribution.py", line 339, in make_archive
    jmd = mx.make_java_module(self, jdk, stager.bin_archive, javac_daemon=javac_daemon)
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx_javamodules.py", line 1093, in make_java_module
    mx.abort('Missing directory containing JMOD files: ' + jdk_jmods)
extracting file org/antlr/v4/runtime/atn/SetTransition.java to org/graalvm/shadowed/org/antlr/v4/runtime/atn/SetTransition.java  File "/disk/graal/upstream-sources/mx/src/mx/_impl/support/logging.py", line 254, in abort
    traceback.print_stack()
Missing directory containing JMOD files: /home/sgehwolf/.mx/jdks/labsjdk-ce-latest-24+24-jvmci-b01-copy/jmods

Reproducer:

$ cp -r ~/.mx/jdks/labsjdk-ce-latest-24+24-jvmci-b01{,-copy}
$ rm -rf ~/.mx/jdks/labsjdk-ce-latest-24+24-jvmci-b01-copy/jmods
$ mx --java-home=$(echo ~/.mx/jdks/labsjdk-ce-latest-24+24-jvmci-b01-copy) --primary-suite substratevm --no-jlinking -v build
[...]
extracting file org/antlr/v4/runtime/atn/SemanticContext.java to org/graalvm/shadowed/org/antlr/v4/runtime/atn/SemanticContext.java
  File "/usr/lib64/python3.13/threading.py", line 1012, in _bootstrap
    self._bootstrap_inner()
extracting file org/tukaani/xz/XZFormatException.java to org/graalvm/shadowed/org/tukaani/xz/XZFormatException.java
extracting file org/jline/utils/Log.java to org/graalvm/shadowed/org/jline/utils/Log.java
  File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
    self.run()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 395, in run
    super(_DummyProcess, self).run()
  File "/usr/lib64/python3.13/threading.py", line 992, in run
    self._target(*self._args, **self._kwargs)
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 14694, in executeTask
    task.execute()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/build/tasks/build.py", line 165, in execute
    _built = self.build()
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx.py", line 5090, in build
    self.subject.make_archive(getattr(self, 'javac_daemon', None))
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
[2024-11-21T17:32:46.639822843Z:127.0.0.1] Result = 0
extracting file org/tukaani/xz/XZIOException.java to org/graalvm/shadowed/org/tukaani/xz/XZIOException.java
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx_jardistribution.py", line 339, in make_archive
    jmd = mx.make_java_module(self, jdk, stager.bin_archive, javac_daemon=javac_daemon)
  File "/disk/graal/upstream-sources/mx/src/mx/_impl/mx_javamodules.py", line 1093, in make_java_module
    mx.abort('Missing directory containing JMOD files: ' + jdk_jmods)
extracting file org/antlr/v4/runtime/atn/SetTransition.java to org/graalvm/shadowed/org/antlr/v4/runtime/atn/SetTransition.java  File "/disk/graal/upstream-sources/mx/src/mx/_impl/support/logging.py", line 254, in abort
    traceback.print_stack()
Missing directory containing JMOD files: /home/sgehwolf/.mx/jdks/labsjdk-ce-latest-24+24-jvmci-b01-copy/jmods

extracting file org/jline/utils/AttributedString.java to org/graalvm/shadowed/org/jline/utils/AttributedString.java
Archiving TRUFFLE_MODULARIZED_TEST_SEPARATE_MODULE_TEST: Failed due to error: 1
[2024-11-21T17:32:46.994795474Z:127.0.0.1] Shutting down

It would be nice if we could reduce the dependency on jmods being present in the base JDK.

[jerboaa]

#287 is a proposal to fix this.

[jerboaa]

oracle/graal#10161 is the corresponding GraalVM change using the abstraction added in #287.

[jerboaa]

Closed with e262358