Build projects that are both multi-module and multi-release.

[desruisseaux]

Before this commit, a project could be either multi-module or multi-release, but not both in same time. As a side effect of the debugging work, this commit also fixes the content of the target/javac.args file: before this commit, the source files for all versions were mixed in the same target/javac.args file. After this commit, separated javac.args, javac-17.args, javac-21.args, etc. files are produced.

Directory layout choice

The directory structure produced in the target directory is like below:

target
├─ foo.bar.module1
│  └─ foo/bar/module1/*.class
├─ foo.bar.module2
│  └─ foo/bar/module2/*.class
└─ META-INF
   └─ versions
      └─ 17
          ├─ foo.bar.module1
          │  └─ foo/bar/module1/*.class
          └─ foo.bar.module2
             └─ foo/bar/module2/*.class

This directory structure will not be immediately usable by the Maven JAR plugin, because that plugin would rather expect the META-INF/versions/17 path to be repeated inside each module. But we cannot easily comply with this expectation, because the Java compiler accepts only one -d argument for all modules. We could move the directories after compilation, but incremental compilation of only a few files would require that we move the directories back to the location shown above before compilation, then move these directories again after the compilation. Furthermore, the users would not be able to compile themeselves on the command-line, unless they perform those moves manually (that would be tedious).

I believe that it is easier, more convenient and less risky to leave the directory structure as produced by javac, and instead modify the Maven JAR plugin for reading the files at the locations shown above. An evolution of the Maven JAR plugin will be needed anyway, as the current version will not understand those org.bar.module1 and org.bar.module2 directories anyway, no matter if multi-release or not.

[desruisseaux]

This pull request partially depends on apache/maven#11118. But it still work without that fix if there is at most one version after the base version.

[desruisseaux]

I'm not sure who to ask for a review. Maybe @slawekjaranowski ? Keeping in mind that the decision about "Directory layout choice" is probably not too important for now, because most developers will probably not use it before other plugins have been updated.

[gnodet]

This should also relate to the work needed on m-sources-p to better support source files for multi-release jars.
See apache/maven-source-plugin#234

[gnodet]

This directory structure will not be immediately usable by the Maven JAR plugin, because that plugin would rather expect the META-INF/versions/17 path to be repeated inside each module. But we cannot easily comply with this expectation, because the Java compiler accepts only one -d argument for all modules. We could move the directories after compilation, but incremental compilation of only a few files would require that we move the directories back to the location shown above before compilation, then move these directories again after the compilation. Furthermore, the users would not be able to compile themeselves on the command-line, unless they perform those moves manually (that would be tedious).

Why does the m-jar-plugin comes into play here ? AFAIK, it's completely agnostic and simply jars whatever is in the target/classes directory. Or you mean, this layout is not directly usable by the JVM ? if so, I'd rather comply with what the JVM expects. Can we instruct javac to output the correct files in the final layout:

├─ module-info.class (base version)
├─ foo/bar/module1/
│  └─ *.class
├─ foo/bar/module2/
│  └─ *.class
└─ META-INF/
   ├─ MANIFEST.MF
   └─ versions/
      └─ 17/
         ├─ module-info.class (Java 17+ version)
         ├─ foo/bar/module1/SomeClass.class (only if different from base)
         └─ foo/bar/module2/AnotherClass.class (JDK-17 specific)

One of the reason is that the target/classes can be added to the classpath when building other subprojects, so I think we need to keep it in sync with JVM's expectations.

[desruisseaux]

When target/classes/ is added to the classpath of other subprojects, the directory layout discussed in this issue does not matter, because META-INF/versions/ will be ignored no matter its location. The META-INF/versions/ sub-directories are scanned only if included in a JAR file. Even in that case, they are scanned only if explicitly requested. This is the topic of apache/maven#11153 (by default, even java.util.jar.JarFile ignores these sub-directories).

If we want target/classes/META-INF/versions/ directories to be taken in account during tests and in other sub-projects, it will require modifications in other plugins in any cases, regardless this issue. We will need to put those directories ourselves on the class-path. This is something that I'm planing to contribute: propose improvement in Surefire plugin for JPMS support, and opportunistically improve the multi-release support in same time. Maybe with a Surefire plugin configuration option such as <releaseToTest> which would control which target/classes/META-INF/versions/ directories to add on the class-path or patch-module paths.

Regarding the JAR plugin, there are two scenarios in the context of JPMS. A key point is that module hierarchy is an optional feature of Java. It is possible to do JPMS with package hierarchy, at the cost of being constrained to a single JPMS module per Maven sub-project. When using package hierarchy, the target/classes/ directory content can be used directly by the JAR plugin, including the META-INF/versions/ sub-directories. But when using the module hierarchy, the content of target/classes/ cannot be used directly by the JAR plugin (it can be used directly by the --module-path option of other sub-projects however). The reason is that the JAR format is not yet multi-module. I saw discussion on the OpenJDK mailing list about potentially creating such format, but I don't know if any work started. In the meantime, the JAR plugin would have to create one JAR file per module only in the specific case where module source hierarchy is used. If we accept to put some intelligence in the plugin for that specific case, it could handle the above-described layout directory in same time.

NOTE: the fact that other plugins such has Surefire and JAR have not yet been updated does not block users from using the JPMS support provided by this pull request and #963. It only blocks the use of module source hierarchy. Other JPMS features are available and can be used now if users stick to package hierarchy.

Can we instruct javac to output the correct files in the final layout

For a sub-project using package hierarchy, regardless if JPMS or not, yes and it is already the case. For module hierarchy, not as far as I know. But as said above, module hierarchy is optional even for a JPMS project, and will require update in other plugins anyway, regardless the directory layout discussed above. Even for a classical class-path project ignoring all the JPMS stuff, improvement of the Surefire plugin is needed anyway if we want to test any version other than the base version.

[desruisseaux]

Documentation about this feature has been added in #976, in particular in the sources.md file. Would it help acceptance if I add a commit for printing a message saying that this feature is experimental when multi-module and multi-release are used in same time?

[Copilot]

Pull Request Overview

This PR extends the Maven compiler plugin to support projects that are both multi-module and multi-release, previously only one or the other was supported. It also fixes the separation of javac.args files by Java release version, creating separate files like javac-17.args, javac-21.args, etc.

• Adds support for multi-module, multi-release projects through new directory structure handling
• Implements WorkaroundForPatchModule class to handle compiler compatibility issues with patch module paths
• Refactors debug file generation to create separate javac.args files per Java release

Reviewed Changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
WorkaroundForPatchModule.java	New workaround class for handling unsupported patch module operations in some compilers
ToolExecutor.java	Major refactoring to support multi-module multi-release compilation with new dependency management
AbstractCompilerMojo.java	Updates debug file writing to generate separate files per Java release
SourcesForRelease.java	Adds dependency snapshot tracking and output directory path for debug file generation
Options.java	Fixes command line formatting to handle -J options properly
SourcePathType.java	Adds equals/hashCode implementation for proper map usage
Integration test files	New test case demonstrating multi-module multi-release functionality

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[gnodet]

The reason is that the JAR format is not yet multi-module.

Are we switching to a mode where we'd have a single project, but it would create multiple jars in one run ? Afaik, most plugins expects a single main artefact as the main output, while others side artifacts can be attached to the project, but on the same groupId/artifactId. So generating multiple main artifacts goes really against the way of doing things in maven. I'm not opposed to a change, but this would require a larger consensus imho.

[desruisseaux]

Yes, compiling a project which uses the module source hierarchy would result in many JAR files, one per module. For each JAR, the dependencies in the pom.xml would be the intersection of the dependencies declared in the pom.xml of the Maven project and the dependencies declared directly or indirectly (including transitive dependencies and service providers) in the module-info.java of each Java module. In other words, Maven <dependencies> section would effectively become a <dependenciesManagement>.

Note 1: this is already the case in Maven 3: any dependency declared in Maven <dependencies> will be ignored (except for some particular things such as annotation processor) by the compiler if it is not implied by some statements in module-info, even in single-module projects using the package hierarchy.

Note 2: this idea is not only mine. I saw it independently expressed somewhere on GitHub, but couldn't find it back. For information, some of the ideas related to module-info support are also found in slightly different forms in the Christian Stein's blog, themselves based on Robert Scholte's emails on OpenJDK mailing list (references in the blog). I think that the same idea appears independently because in a modular world, the fact that <dependencies> would effectively behave as <dependenciesManagement> is not in our control (unless we choose to generate --add-modules options).

Note 3: about 10 years ago, the maven-jar-plugin was just zipping whatever exist in target/classes/ with the addition of a MANIFEST.MF file. But now, there are some options such as --hash-modules which require a module-path. So creating a JAR file is already becoming more <dependencies>-dependent than before.

Note 4: while Maven is still quite JAR-oriented, JAR files are not necessarily the main artifacts anymore. The main artifact could be the jlink output, in which case a multi-module project can still produce a single artifact (the executable application), and *.jar files are only intermediate outputs like the *.class files.