Deleted `jdk.internal.loader.NativeLibrary::findEntry0` reachable in JDK 22+25 builds

[zakkak]

Describe the issue
After the upgrade to JDK 22+25 the deleted jdk.internal.loader.NativeLibrary::findEntry0 appears to be reachable.

Steps to reproduce the issue

git clone https://github.com/quarkusio/quarkus.git
cd quarkus
export GRAALVM_HOME=/path/to/latest/graalvm/build
./mvnw install -Dquickly -T4
./mvnw -Dnative -pl integration-tests/awt -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests -Dlog.level=ALL clean package -Dquarkus.native.container-build=false

Describe GraalVM and your environment:

• GraalVM version (latest snapshot builds can be found here), or commit id if built from source: 9805b1b
• JDK major version: 22+25
• OS: Fedora 39
• Architecture: AMD64

More details
Full log:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] /home/runner/work/mandrel/mandrel/graalvm-home/bin/native-image -J-Dlogging.initial-configurator.min-level=500 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --features=io.quarkus.awt.runtime.graal.DarwinAwtFeature,io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+UnlockExperimentalVMOptions -H:BuildOutputJSONFile=quarkus-integration-test-awt-999-SNAPSHOT-runner-build-output-stats.json -H:-UnlockExperimentalVMOptions -H:+UnlockExperimentalVMOptions -H:+AllowFoldMethods -H:-UnlockExperimentalVMOptions -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+UnlockExperimentalVMOptions -H:+ReportExceptionStackTraces -H:-UnlockExperimentalVMOptions -J-Xmx5g -H:-AddAllCharsets --enable-url-protocols=http -H:NativeLinkerOption=-no-pie --enable-monitoring=heapdump -H:+UnlockExperimentalVMOptions -H:-UseServiceLoaderFeature -H:-UnlockExperimentalVMOptions -J--add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-integration-test-awt-999-SNAPSHOT-runner -jar quarkus-integration-test-awt-999-SNAPSHOT-runner.jar
Warning: The option '-H:ReflectionConfigurationResources=META-INF/native-image/io.netty/netty-transport/reflection-config.json' is experimental and must be enabled via '-H:+UnlockExperimentalVMOptions' in the future.
Warning: Please re-evaluate whether any experimental option is required, and either remove or unlock it. The build output lists all active experimental options, including where they come from and possible alternatives. If you think an experimental option should be considered as stable, please file an issue.
========================================================================================================================
GraalVM Native Image: Generating 'quarkus-integration-test-awt-999-SNAPSHOT-runner' (executable)...
========================================================================================================================
For detailed information and explanations on the build output, visit:
https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/BuildOutput.md
------------------------------------------------------------------------------------------------------------------------
[1/8] Initializing...                                                                                    (5.8s @ 0.15GB)
 Java version: 22+25, vendor version: GraalVM CE 22-dev+25.1
 Graal compiler: optimization level: 2, target machine: x86-64-v3
 C compiler: gcc (linux, x86_64, 11.4.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 5 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - io.quarkus.awt.runtime.graal.DarwinAwtFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
 - org.eclipse.angus.activation.nativeimage.AngusActivationFeature
------------------------------------------------------------------------------------------------------------------------
 4 experimental option(s) unlocked:
 - '-H:+AllowFoldMethods' (origin(s): command line)
 - '-H:BuildOutputJSONFile' (origin(s): command line)
 - '-H:-UseServiceLoaderFeature' (origin(s): command line)
 - '-H:ReflectionConfigurationResources': Use a reflect-config.json in your META-INF/native-image/<groupID>/<artifactID> directory instead. (origin(s): 'META-INF/native-image/io.netty/netty-transport/native-image.properties' in 'file:///home/runner/work/mandrel/mandrel/quarkus/integration-tests/awt/target/quarkus-integration-test-awt-999-SNAPSHOT-native-image-source-jar/lib/io.netty.netty-transport-4.1.100.Final.jar')
------------------------------------------------------------------------------------------------------------------------
Build resources:
 - 4.44GB of memory (28.5% of 15.61GB system memory, set via '-Xmx5g')
 - 4 thread(s) (100.0% of 4 available processor(s), determined at start)
[2/8] Performing analysis...  []                                                                        (50.6s @ 1.14GB)
   12,569 reachable types   (84.0% of   14,958 total)
   19,268 reachable fields  (50.7% of   38,041 total)
   61,599 reachable methods (54.6% of  112,885 total)
    3,891 types,   188 fields, and 3,201 methods registered for reflection

Fatal error: com.oracle.graal.pointsto.util.AnalysisError$ParsingError: Error encountered while parsing jdk.internal.loader.RawNativeLibraries$RawNativeLibraryImpl.find(RawNativeLibraries.java:170) 
Parsing context:
   at jdk.internal.loader.NativeLibrary.lookup(NativeLibrary.java:49)
   at jdk.internal.foreign.SystemLookup.lambda$libLookup$8(SystemLookup.java:127)
   at jdk.internal.foreign.SystemLookup.find(SystemLookup.java:154)
   at java.lang.foreign.SymbolLookup.lambda$or$0(SymbolLookup.java:172)
   at java.lang.foreign.SymbolLookup$$Lambda/0x000000006993b5b8.get(Unknown Source)
   at java.util.Optional.orElseGet(Optional.java:364)
   at root method.(Unknown Source)

	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.AnalysisError.parsingError(AnalysisError.java:149)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:184)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:152)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraphInfo(MethodTypeFlow.java:110)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:114)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:628)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.PointsToAnalysis$1.run(PointsToAnalysis.java:541)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:169)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:154)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute(ForkJoinTask.java:1726)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.compute(ForkJoinTask.java:1717)
	at java.base/java.util.concurrent.ForkJoinTask$InterruptibleTask.exec(ForkJoinTask.java:1641)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1486)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:2077)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2028)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)
Caused by: jdk.graal.compiler.java.BytecodeParser$BytecodeParserError: java.lang.InternalError: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Error loading a referenced type: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method jdk.internal.loader.NativeLibrary.findEntry0(long, String) is reachable
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
	at parsing jdk.internal.loader.RawNativeLibraries$RawNativeLibraryImpl.find(RawNativeLibraries.java:170)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.throwParserError(BytecodeParser.java:2564)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.throwParserError(SharedGraphBuilderPhase.java:176)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3451)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.iterateBytecodesForBlock(SharedGraphBuilderPhase.java:727)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3403)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3245)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.build(BytecodeParser.java:1136)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.build(SharedGraphBuilderPhase.java:156)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1028)
	at jdk.graal.compiler/jdk.graal.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:102)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:110)
	at jdk.graal.compiler/jdk.graal.compiler.phases.Phase.run(Phase.java:49)
	at jdk.graal.compiler/jdk.graal.compiler.phases.BasePhase.apply(BasePhase.java:435)
	at jdk.graal.compiler/jdk.graal.compiler.phases.Phase.apply(Phase.java:42)
	at jdk.graal.compiler/jdk.graal.compiler.phases.Phase.apply(Phase.java:38)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:146)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.parseGraph(AnalysisMethod.java:892)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsedHelper(AnalysisMethod.java:857)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:840)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:190)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:610)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:167)
	... 15 more
Caused by: java.lang.InternalError: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Error loading a referenced type: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method jdk.internal.loader.NativeLibrary.findEntry0(long, String) is reachable
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
	at jdk.graal.compiler/jdk.graal.compiler.serviceprovider.GraalServices.lookupMethodWithCaller(GraalServices.java:465)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.lookupMethodInPool(BytecodeParser.java:4366)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.lookupMethodInPool(SharedGraphBuilderPhase.java:204)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.lookupMethod(BytecodeParser.java:4358)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1673)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5453)
	at jdk.graal.compiler/jdk.graal.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3443)
	... 34 more
Caused by: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Error loading a referenced type: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method jdk.internal.loader.NativeLibrary.findEntry0(long, String) is reachable
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.lookupMethod(WrappedConstantPool.java:146)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at jdk.graal.compiler/jdk.graal.compiler.serviceprovider.GraalServices.lookupMethodWithCaller(GraalServices.java:458)
	... 40 more
Caused by: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported method jdk.internal.loader.NativeLibrary.findEntry0(long, String) is reachable
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.lookup(AnnotationSubstitutionProcessor.java:261)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.SubstitutionProcessor$ChainedSubstitutionProcessor.lookup(SubstitutionProcessor.java:140)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:439)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.lookupMethod(WrappedConstantPool.java:138)
	... 43 more
------------------------------------------------------------------------------------------------------------------------
                       8.6s (14.8% of total time) in 145 GCs | Peak RSS: 2.17GB | CPU load: 3.74
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /home/runner/work/mandrel/mandrel/quarkus/integration-tests/awt/target/quarkus-integration-test-awt-999-SNAPSHOT-native-image-source-jar/quarkus-integration-test-awt-999-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'quarkus-integration-test-awt-999-SNAPSHOT-runner' in 57.4s.

[jerboaa]

/cc @zapster

[zapster]

@jerboaa @zakkak I'm not able to follow your instructions:

$ ./mvnw install -Dquickly -T4
...
[INFO] The Gradle Enterprise server (ge.quarkus.io) rejected the request due to authentication being required.
[INFO] If you are able to sign in to this server in a web browser, you can authenticate your environment by running the following goal:

In addition, which revison of quarkus did you use (I tried main and got some other maven issues) and what was JAVA_HOME set to?

[zakkak]

I'm not able to follow your instructions

Sorry for that.

which revision of quarkus did you use?

I can reproduce with latest main, i.e., 3c1d1a07967ed0ce99711c93e0d0b162980c0b44.

what was JAVA_HOME set to?

Please use OpenJDK 17.0.9+9 as JAVA_HOME.

[zapster]

Thanks @zakkak.

Another problem with

./mvnw install -Dquickly -T4

was running into concurrency issues with the maven dependency downloads. Removing the -T4 flag resolved these. While the command then succeeds, it seems to downloads tons of dependencies and runs tasks that do not really seem to be necessary for a simple reproducer. Isn't there something more lightweight?

The next problem was with

./mvnw -Dnative -pl integration-tests/awt -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests -Dlog.level=ALL clean package -Dquarkus.native.container-build=false

It seems that the GraalVM version check does not like dev builds.

[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-awt: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR]         [error]: Build step io.quarkus.awt.deployment.AwtProcessor#setupJava2DClasses threw an exception: java.lang.RuntimeException: Failed to get GraalVM version
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildRunner.getGraalVMVersion(NativeImageBuildRunner.java:40)
[ERROR]         at io.quarkus.awt.deployment.AwtProcessor.setupJava2DClasses(AwtProcessor.java:153)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:568)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR]         at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:833)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] Caused by: java.lang.IllegalArgumentException: Cannot parse version from output:
[ERROR] native-image 22 2024-03-19
[ERROR] Java(TM) SE Runtime Environment Oracle GraalVM 22-dev+25.1 (build 22+25-jvmci-b01)
[ERROR] Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 22-dev+25.1 (build 22+25-jvmci-b01, mixed mode, sharing)
[ERROR]         at io.quarkus.deployment.pkg.steps.GraalVM$Version.of(GraalVM.java:281)
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildRunner.getGraalVMVersion(NativeImageBuildRunner.java:37)
[ERROR]         ... 12 more
[ERROR]
[ERROR]         [error]: Build step io.quarkus.awt.deployment.AwtProcessor#setupAWTInit threw an exception: java.lang.RuntimeException: Failed to get GraalVM version
[ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildRunner.getGraalVMVersion(NativeImageBuildRunner.java:40)
[ERROR]         at io.quarkus.awt.deployment.AwtProcessor.setupAWTInit(AwtProcessor.java:121)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:568)
[ERROR]         at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:849)
[ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
[ERROR]         at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
[ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:833)
[ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] Caused by: java.lang.IllegalArgumentException: Cannot parse version from output:

Is there some workaround for this? If not, can the version check be improved?

Long story short, I was not able to reproduce the problem with the provided instructions.

However, I was able to reproduce the problem locally via some internal benchmarking command and will look for a solution.

[zakkak]

While the command then succeeds, it seems to downloads tons of dependencies and runs tasks that do not really seem to be necessary for a simple reproducer.

Unfortunately I am not aware of any way to build only the necessary modules required for a specific integration test. I would have to create an external reproducer which I found not trivial since this only popped up in the AWT integration test which is quite complex.

Is there some workaround for this?

In the github actions we work around this by not building a native image for native-image:

graal/.github/workflows/quarkus.yml

Lines 116 to 120 in ddb574c

	- name: Build graalvm native-image
	run: |
	export JAVA_HOME=${LABSJDK_HOME}
	cd substratevm
	${MX_PATH}/mx --native=native-image,lib:jvmcicompiler --components="Native Image,LibGraal" build

If not, can the version check be improved?

I will have a look. This should be handled by quarkusio/quarkus#37651. I added the version listed above as a test case as well (see quarkusio/quarkus@aefe8a3)

However, I was able to reproduce the problem locally via some internal benchmarking command and will look for a solution.

Good to hear. Sorry for the trouble with the reproducer.

[zakkak]

Hi @zapster, it looks like this was fixed in #7980 by b659678

I will leave the issue open for you to close in case your reproducer is still triggering it.

[zapster]

Thanks @zakkak for the heads up!

[zapster]

After merging in the latest version, the problem disappeared. Closing.