JDK bundles are not discovered and added to native image with JDK16

[ilopmar]

Describe the issue

Bundles included in JDK 16 are not discovered and can't be included in the native image.

I have an application that declares the following in resource-config.json:

{
  "resources": {
    "includes": []
  },
  "bundles": [
    {"name": "com.sun.org.apache.xerces.internal.impl.xpath.regex.message"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.DatatypeMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.DOMMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.JAXPValidationMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.SAXMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.XIncludeMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.XMLMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.XMLSchemaMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.XMLSerializerMessages"},
    {"name": "com.sun.org.apache.xerces.internal.impl.msg.XPointerMessages"}
  ]
}

When building a native image with GraalVM JDK11 every works and all those bundles are included in the native image.

If I try to build the native image with GraalVM JDK16 (compiled from master branch) there are warnings about those bundles not being found, and thus, not being included in the native image.

The bundles are present in both JDK11 & 16:

JDK 11

JDK 16

Steps to reproduce the issue

I've created a very simple hello world app with the previous resource-config.json file to illustrate the problem.

• git clone https://github.com/ilopmar/graalvm-issue-jdk16-bundles
• cd graalvm-issue-jdk16-bundles
• Use GraaVM JDK16 compiled from master branch
• ./build-native-image.sh

During native image generation we can see the following warnings:

[my-app:169076]    classlist:     533.07 ms,  0.96 GB
[my-app:169076]        (cap):     435.69 ms,  0.96 GB
[my-app:169076]        setup:   1,589.10 ms,  0.96 GB
The bundle named: com.sun.org.apache.xerces.internal.impl.xpath.regex.message, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.DatatypeMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.DOMMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.JAXPValidationMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.SAXMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.XIncludeMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.XMLMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.XMLSchemaMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.XMLSerializerMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
The bundle named: com.sun.org.apache.xerces.internal.impl.msg.XPointerMessages, has not been found. If the bundle is part of a module, verify the bundle name is a fully qualified class name. Otherwise verify the bundle path is accessible in the classpath.
[my-app:169076]     (clinit):     134.85 ms,  1.22 GB
[my-app:169076]   (typeflow):   3,619.45 ms,  1.22 GB
[my-app:169076]    (objects):   3,305.42 ms,  1.22 GB
[my-app:169076]   (features):     235.11 ms,  1.22 GB
[my-app:169076]     analysis:   7,463.60 ms,  1.22 GB
[my-app:169076]     universe:     271.81 ms,  1.22 GB
[my-app:169076]      (parse):     720.00 ms,  1.69 GB
[my-app:169076]     (inline):     863.66 ms,  1.69 GB
[my-app:169076]    (compile):   4,688.34 ms,  2.33 GB
[my-app:169076]      compile:   6,677.85 ms,  2.33 GB
[my-app:169076]        image:   1,210.21 ms,  2.33 GB
[my-app:169076]        write:     140.52 ms,  2.33 GB
# Printing build artifacts to: /home/ivan/workspaces/misc/basic-jdk16-app/my-app.build_artifacts.txt
[my-app:169076]      [total]:  18,110.17 ms,  2.33 GB

Now switch to GraalVM 21.1 JDK11 and create the native image again:

[my-app:169627]    classlist:     726.34 ms,  0.96 GB
[my-app:169627]        (cap):     386.42 ms,  0.96 GB
[my-app:169627]        setup:   1,604.13 ms,  0.96 GB
[my-app:169627]     (clinit):     141.27 ms,  1.68 GB
[my-app:169627]   (typeflow):   4,108.72 ms,  1.68 GB
[my-app:169627]    (objects):   3,917.23 ms,  1.68 GB
[my-app:169627]   (features):     287.73 ms,  1.68 GB
[my-app:169627]     analysis:   8,794.90 ms,  1.68 GB
[my-app:169627]     universe:     325.43 ms,  1.68 GB
[my-app:169627]      (parse):     727.20 ms,  1.68 GB
[my-app:169627]     (inline):     931.23 ms,  1.69 GB
[my-app:169627]    (compile):   5,073.26 ms,  3.23 GB
[my-app:169627]      compile:   7,226.39 ms,  3.23 GB
[my-app:169627]        image:   1,248.52 ms,  3.23 GB
[my-app:169627]        write:     372.31 ms,  3.13 GB
# Printing build artifacts to: my-app.build_artifacts.txt
[my-app:169627]      [total]:  20,505.66 ms,  3.13 GB

You can see that there are no warnings with JDK11.

As mentioned before this a very basic reproducer. This is failing in Micronaut Liquibase integration because Liquibase tries to find some of those message bundles during startup and it fails because it can't find them.

Describe GraalVM and your environment:

• GraalVM version: 21.2.0-dev compiled from d5ec6c8
• JDK major version: 16
• OS: Linux Mint 20.1 Cinnamon
• Architecture: AMD64

[ilopmar]

I've just discovered that if I pass the following flags to native-image command, it works:

-J--add-opens=java.xml/com.sun.org.apache.xerces.internal.impl.msg=ALL-UNNAMED
-J--add-opens=java.xml/com.sun.org.apache.xerces.internal.impl.xpath.regex=ALL-UNNAMED

Another thing I've tried is adding the module name as a prefix for the classes in resource-config.json like:

{
  "resources": {
    "includes": []
  },
  "bundles": [
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.xpath.regex.message"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.DatatypeMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.DOMMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.JAXPValidationMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.SAXMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.XIncludeMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.XMLMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.XMLSchemaMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.XMLSerializerMessages"},
    {"name": "java.xml/com.sun.org.apache.xerces.internal.impl.msg.XPointerMessages"}
  ]
}

But it doesn't work.

I'm not sure if this is just a workaround or the final solution. But I think it would be great if this is handled automatically by GraalVM and it adds the necessary configuration options to include all the bundles detected. Unless there is a way to pass those flags to native-image programmatically from an @AutomaticFeature so we can include it directly in our Micronaut Liquibase integration.

Thoughts?

[oubidar-Abderrahim]

Tracked internally on GR-31418

[pejovica]

@ilopmar See the following in the JDK 16 Release Notes:

JEP 396: Strongly Encapsulate JDK Internals by Default
Strongly encapsulate all internal elements of the JDK by default, except for critical internal APIs such as sun.misc.Unsafe. Allow end users to choose the relaxed strong encapsulation that has been the default since JDK 9.

With this change, the default value of the launcher option --illegal-access is now deny rather than permit. As a consequence, existing code that uses most internal classes, methods, or fields of the JDK will fail to run. Such code can be made to run on JDK 16 by specifying --illegal-access=permit. That option will, however, be removed in a future release.

Due to this change, the java.xml module that contains these bundles is no longer opened to all unnamed modules by default, which is why Java applications can no longer use these bundles without additional configuration options. So, unless the options for building an image are different from ones for running an application, I would say that this is not a bug.

[ilopmar]

@pejovica Thanks for the explanation.

As mentioned, this happens when testing our Micronaut-Liquibase integration. What I don't understand is why the application runs ok in the JVM when using JDK16 but it needs the extra configuration for GraalVM: https://github.com/micronaut-graal-tests/micronaut-liquibase-graal/blob/00dc7d9f2223a8bf68ca6a8bbe44374756a0a0a2/build.gradle#L63-L69

[pejovica]

... why the application runs ok in the JVM when using JDK16 but it needs the extra configuration for GraalVM

Hmm, that shouldn't be the case. I'll try to see if I can reproduce the problem locally using your Micronaut-Liquibase integration: https://github.com/micronaut-graal-tests/micronaut-liquibase-graal/tree/3.0.x_h2.

[ilopmar]

Thank you. Please let me know if you need help with that app that reproduces the problem :-)

[pejovica]

I was able to reproduce the problem and this is indeed a bug on our side. Specifically, in the Micronaut-Liquibase integration test, these bundles are accessed from within the java.xml module, which is why everything works on the JVM. However, during image building, the image builder tries to access these bundles from the outside (i.e., from an unnamed module) and fails because it does not have the required access.

So, until we have a proper solution, you will need additional JVM arguments when building the native image as a workaround. You can use -J--add-opens=... as you already figured out, or you can use -J--illegal-access=permit instead.

[olpaw]

@ivan-ristovic this is the issue we were talking about today. This needs the builder-specific opening up of private packages where a given ResourceBundle is located in (see substratevm/src/com.oracle.svm.util.jdk11/src/com/oracle/svm/util/ModuleSupport.java). Once we have that, also com.oracle.svm.util.ModuleSupport#getResourceBundleFallback should become obsolete (so we can delete it).