Provide GraalVM metadata repository support with Buildpacks

[sdeleuze]

(edit)

Splitting into sub-tasks:

• Provide GraalVM metadata repository files in Spring Boot jar built with Gradle #32408
• Provide maven configuration for native-build-tools reachability metadata goal #32736
• When the GraalVM Native Image Gradle plugin is applied, configure bootJar to contain the output of collectReachabilityMetadata #32737
• Generate native-image argfile containing reachability metadata excludes for buildpacks to use #32738

External issues:

• Include buildtool support for bundling reachability metadata into jar files graalvm/native-build-tools#322
• Allow --exclude-config arguments to be specified inside jar paketo-buildpacks/native-image#196 & Added support for argument file provided... paketo-buildpacks/native-image#201

---

As discussed with @dmikusa-pivotal, @mhalbritter and @scottfrederick, we need to integrate GraalVM metadata repository in Buildpacks in addition to Native Build Tools current support (see #31687) in order to allow compiling Spring Boot application with third-party dependencies to a native executable. 2 implementations have been discussed.

Based on our discussions, integration at Spring Boot Maven and Gradle plugin AOT generation level is the recommended way to implement such integration because:

• It allows a single and consistent way to provide third-party native configuration for Spring Boot applications
• It can leverage graalvm-reachability-metadata Java library and its GraalVMReachabilityMetadataRepository class maintained by GraalVM team.
• Reference implementation in NBT Gradle and Maven plugins can help us for the Spring Boot implementation.
• Conceptually a good fit to put third party hints along to Spring ones
• Better DevXP (all the JSON hints generated at the same location at the same time)
• Better build reproducibility
• Gives more control to Spring Boot on the reachability metadata repository release a certain version of Spring Boot uses

The alternative way is to have 2 distinct implementations : the one at NBT plugins level and another at Buildpacks level, but the lack of consistency, poorer DevXP, the fact that we would likely have to reimplement graalvm-reachability-metadata in Go and maintaining it seems to indicate this should not be the way to move forward.

[wilkinsona]

On the Gradle side, we may be able to use GraalVMReachabilityMetadataService which is a build service that implements GraalVMReachabilityMetadataRepository.

[snicoll]

On the Maven Side, things aren't as easy to reuse I am afraid. We've discussed this and we think that extracting that capability in a dedicated goal of the Native Build Plugins would be a nice option. We don't want to redefine the attributes that this feature need in our own plugin and rather add an extra execution of an existing goal in our parent.

For instance, the existing compile goal could have an additional attribute that points to the directory where the metadata has been processed. And the new goal would write to this directory, supporting the existing attributes that the unique goal has at the moment.

[wilkinsona]

I've been thinking about this some more, and I'm starting to wonder if relying on capabilities of the native build tools' Gradle and Maven plugins is a good idea.

If buildpacks are creating the native image, will the user have even configured/applied the native build tools plugin? For those that haven't, I think it would feel quite strange to have to apply/configure the plugin locally solely for the purpose of providing reachability metadata to the buildpack.

I know it's more work on the buildpacks side, but it feels to me that if the buildpacks are creating the native image, they should be capable of retrieving and using the reachability metadata. We could provide GAV metadata to the buildpacks to facilitate retrieving the reachability metadata without requiring the user to configure anything.

[sdeleuze]

I've been thinking about this some more, and I'm starting to wonder if relying on capabilities of the native build tools' Gradle and Maven plugins is a good idea.

I think the right level of reuse is the graalvm-reachability-metadata dependency. Notice that if you feel GraalVMReachabilityMetadataRepository is not enough and more could be in the shared logic, we can refactor Native Build Tools accordingly but I think we would need a concrete proposal for that.

I know it's more work on the buildpacks side, but it feels to me that if the buildpacks are creating the native image, they should be capable of retrieving and using the reachability metadata. We could provide GAV metadata to the buildpacks to facilitate retrieving the reachability metadata without requiring the user to configure anything.

One of the main issue discussed with @dmikusa-pivotal during our last meeting related to implementing that at Buildpack level is that it will be hard to leverage GraalVMReachabilityMetadataRepository via graalvm-reachability-metadata JVM dependency from Buildpacks infra which is in Go, at least that's what I remember.

I think any solution not based on graalvm-reachability-metadata will be a maintenance nightmare given the fact that the repository format will evolve.

[dmikusa]

I know it's more work on the buildpacks side, but it feels to me that if the buildpacks are creating the native image, they should be capable of retrieving and using the reachability metadata. We could provide GAV metadata to the buildpacks to facilitate retrieving the reachability metadata without requiring the user to configure anything.

One of the main issue discussed with @dmikusa-pivotal during our last meeting related to implementing that at Buildpack level is that it will be hard to leverage GraalVMReachabilityMetadataRepository via graalvm-reachability-metadata JVM dependency from Buildpacks infra which is in Go, at least that's what I remember.

I think any solution not based on graalvm-reachability-metadata will be a maintenance nightmare given the fact that the repository format will evolve.

100%. The big challenge is that we can't reuse any of their library code. The only path forward would be if that project were to release a CLI that we could call from the buildpack, either java -jar foo.jar or a native-image binary. If we can invoke the logic from that repo through a CLI, that would be the first hurdle cleared.

The second hurdle is that buildpacks have to account for two scenarios: 1.) building from source and 2.) building from precompiled assets. I think we could make building from source work (assuming we have the CLI to call) but I'm very skeptical that we could make the second scenario, building from a precompiled asset, work. There's just a lack of information since all we'd have is a JAR file, and we cannot assume it's a Spring Boot JAR. The buildpack has to work with any kind of JAR.

This might seem like a weird use case, who's going to do half the build locally and then half with buildpacks, but that's exactly what the Spring Boot tools do, ./mvnw spring-boot:build-image generates the JAR locally and builds the native-image in the container.

[dmikusa]

We could provide GAV metadata to the buildpacks to facilitate retrieving the reachability metadata without requiring the user to configure anything.

Can you expand on this? What type of information would this provide to buildpacks? Basically, just a full list of all the dependencies installed?

In particular, for the case where ./mvnw spring-boot:build-image so all the buildpack gets is a Spring Boot JAR file. How do you see this working? Something in /META-INF/?

[mhalbritter]

My 2 cents: right now the NBT plugins (Maven & Gradle) download the metadata and invoke the native-image with it. When we add this logic into our plugins or into the buildpack, there's a third place where the repository logic is implemented.

My take would be to talk to the GraalVM team if the metadata repository support could be implemented in native-image itself. We could then invoke the native-image tool like this:

native-image -jar /path/to/spring-boot.jar --dependencies /file/which/lists/dependencies.json --metadata-version=0.1.1

and native-image would fetch the metadata needed for the given dependencies and build a native image with this. With this change the repository fetching could be removed from NBT maven and gradle plugins and we don't need to implement the repository fetching for every build tool out there.

This doesn't solve the problem how exactly the dependencies get passed into the build pack, but maybe we could put it under META-INF somewhere. Maybe even a location which native-image would read by default. This way you could build self-contained JAR files which are executable on the JVM and could then be transformed into a native-image without additional information.

What do you think?

[sdeleuze]

We have already discussed this possibility with the GraalVM team in the past which has been refused, no hope to see this happening short/middle term.

[mhalbritter]

If buildpacks are creating the native image, will the user have even configured/applied the native build tools plugin? For those that haven't, I think it would feel quite strange to have to apply/configure the plugin locally solely for the purpose of providing reachability metadata to the buildpack.

That popped into my mind, too. Right now, the org.graalvm.buildtools.native Gradle plugin serves as a trigger to apply our AOT plugin to generate all the AOT code. When the user is using buildpacks, they may not have the NBT plugin applied.

In terms of UX, what is the signal to apply AOT processing + native? gradle bootBuildImage -Pnative=true? gradle bootBuildNativeImage? My guess is that users want to have the option to build a "normal" OCI image and a native OCI image without fiddling with build.gradle.

[sdeleuze]

+1, on Spring Native we have seen a higher than expected amount of users just using Buildpack support without Native Build Tools configured so that seems to be a strong use case.