HADOOP-19402. [JDK11] JDiff Support JDK11.

[zhtttylz]

Description of PR

JIRA:HADOOP-19402. [JDK11] JDiff Support JDK11.

This PR restores the ability to generate JDiff API XML on JDK 17 by porting Hadoop’s in‑tree doclets to the modern JDK 9+ Doclet API (jdk.javadoc.doclet), while preserving the legacy command‑line surface and XML output contract used in our release process.

Background. The original JDiff tool and its legacy doclet APIs predate JPMS and JDK 9; the upstream artifact (jdiff:1.0.9) is from 2005 and does not support JDK 9+ doclet APIs. Hadoop’s release process, however, still depends on JDiff XML to publish API diffs between releases.

Why now. Since JEP 221 (JDK 9), the standard com.sun.javadoc doclet path was superseded by jdk.javadoc.doclet.*, and the old doclet was eventually removed/replaced in modern JDKs. To run on JDK 17, we need doclets implemented against the new API.

How was this patch tested?

Built with JDK 17 and the docs profile to ensure JDiff XML is emitted.

mvn -T 1C clean package \
  -Pnative -Pdocs \
  -Drequire.snappy -Drequire.zstd -Drequire.openssl \
  -Djdiff.version=1.1.1-jdk17 \
  -DskipTests -Denforcer.skip=true

This produces JDiff XML for each module. For example, HDFS:
Apache_Hadoop_HDFS_3.5.0-SNAPSHOT.xml

For code changes:

All changes are scoped to the hadoop-annotations module and only affect build-time doclet code.

1. Add a JDK‑specific source set and Maven profile
   • Introduce src/main/java17 under hadoop-common-project/hadoop-annotations.
   • Add a Maven profile jdk17 (activated on JDK ≥ 17) that:
     • Adds the java17 sources via build-helper-maven-plugin.
     • Compiles them with maven-compiler-plugin at -release 17 and -add-modules jdk.javadoc.
   • This keeps JDK 8/11 builds unchanged and makes JDK 17 builds self‑contained.
2. Port / re‑implement the doclets on the new Doclet API (under java17)
   • ExcludePrivateAnnotationsJDiffDoclet
   • IncludePublicAnnotationsJDiffDoclet
   • ExcludePrivateAnnotationsStandardDoclet
   • IncludePublicAnnotationsStandardDoclet
   • The JDiff doclets delegate to the (forked) JDiff implementation and run via Doclet#run(DocletEnvironment).
   • The Standard doclets wrap jdk.javadoc.doclet.StandardDoclet to keep the same inclusion/exclusion semantics when producing regular Javadoc.
3. Port the processing utilities to JDK 17
   • Provide RootDocProcessor and StabilityOptions variants that operate on DocletEnvironment, preserving:
     • audience filtering (exclude @InterfaceAudience.Private / LimitedPrivate)
     • “public‑only” mode (IncludePublic* doclets)
     • stability flags such as unstable, evolving, etc.
4. No changes to runtime or public APIs
   • Everything here runs at doc generation time only.
   • No new third‑party runtime dependencies are introduced.

Related JDiff changes (temporary local test build; JDK 17 adaptation implemented)

I implemented the JDK 17 adaptation in JDiff (port to the jdk.javadoc.doclet API) and validated Hadoop’s docs build using a temporary, locally installed artifact: jdiff:1.1.1-jdk17. This build is not published to Maven Central and is provided only so reviewers can reproduce JDiff XML generation on JDK 17. The implementation is available in zhtttylz/jdiff#4.

[slfan1989]

@steveloughran @pan3793 This PR has been in preparation for quite some time by Hualong, and it consists of two main

parts:

• Refactoring JDiff to make it compatible and functional under JDK 17.
• Updating Hadoop’s custom doc-related annotations to ensure they are compatible with JDK 17 as well.

I’d appreciate your help reviewing this PR.
For reference, the JDiff-related changes are available in zhtttylz/jdiff#4

cc: @cnauroth @ayushtkn @Hexiaoqiao

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	21m 3s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	8m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 31s		trunk passed
+1 💚	compile	15m 14s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	15m 35s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	2m 58s		trunk passed
+1 💚	mvnsite	2m 8s		trunk passed
+1 💚	javadoc	2m 8s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	2m 0s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+0 🆗	spotbugs	0m 46s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+0 🆗	spotbugs	0m 41s		branch/hadoop-project-dist no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	25m 49s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 14s	/patch-mvninstall-hadoop-common-project_hadoop-annotations.txt	hadoop-annotations in the patch failed.
-1 ❌	compile	0m 18s	/patch-compile-root-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04.
-1 ❌	javac	0m 18s	/patch-compile-root-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04.
-1 ❌	compile	0m 18s	/patch-compile-root-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04.
-1 ❌	javac	0m 18s	/patch-compile-root-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	root in the patch failed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04.
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	0m 15s	/buildtool-patch-checkstyle-root.txt	The patch fails to run checkstyle in root
-1 ❌	mvnsite	0m 14s	/patch-mvnsite-hadoop-common-project_hadoop-annotations.txt	hadoop-annotations in the patch failed.
-1 ❌	javadoc	0m 14s	/patch-javadoc-hadoop-common-project_hadoop-annotations-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-annotations in the patch failed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04.
-1 ❌	javadoc	0m 13s	/patch-javadoc-hadoop-common-project_hadoop-annotations-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-annotations in the patch failed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04.
+0 🆗	spotbugs	0m 16s		hadoop-project has no data from spotbugs
-1 ❌	spotbugs	0m 14s	/patch-spotbugs-hadoop-common-project_hadoop-annotations.txt	hadoop-annotations in the patch failed.
+0 🆗	spotbugs	0m 17s		hadoop-project-dist has no data from spotbugs
-1 ❌	shadedclient	1m 2s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 15s		hadoop-project in the patch passed.
-1 ❌	unit	0m 14s	/patch-unit-hadoop-common-project_hadoop-annotations.txt	hadoop-annotations in the patch failed.
+1 💚	unit	0m 14s		hadoop-project-dist in the patch passed.
+1 💚	asflicense	0m 28s		The patch does not generate ASF License warnings.
		136m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8038/1/artifact/out/Dockerfile
GITHUB PR	#8038
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux a85451735ded 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 8f9c11f
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8038/1/testReport/
Max. process+thread count	768 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-annotations hadoop-project-dist U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8038/1/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.