Skip to content

Conversation

@wks
Copy link
Contributor

@wks wks commented Nov 19, 2025

We merge the branch for OpenJDK 21 into master. From now on, the master branch of the mmtk-openjdk repository will directly target OpenJDK 21. The jdk-21.0.9+3-mmtk branch will be decommissioned, and will remain its current state (i.e. the state just before merging into master), and will only be looked up when any developers are interested in the history of the development of the OpenJDK binding.

The following things work:

  • We can run OpenJDK 21 with MMTk on x86_64, and it supports compressed oops.
  • CI tests for pull requests of the mmtk-openjdk binding work.
  • CI binding tests (including both minimal tests and extended tests) for pull requests of the mmtk-core repo work.

One important thing that doesn't work immediately is the performance regression CI. It requires a change in the ci-perf-kit repo and also needs related changes in the CI scripts in mmtk-core. This PR is for OpenJDK, and will not trigger the performance regression CI. This leaves us a small window to update mmtk-core and ci-perf-kit after this PR is merged.

This PR should not be merged with "squash merge". Instead we should make a proper merge commit so that the jdk-21.0.9+3-mmtk branch will be considered as an ancestor of the master branch from Git's point of view. GitHub will also consider the branch jdk-21.0.9+3-mmtk as "merged".

Major changes compared to the OpenJDK 11 binding

We made the following changes in the mmtk-openjdk repo:

  • We made changes to reflect upstream changes in OpenJDK, including initialization, mutex API, roots, JIT compilers, etc.
  • We now support multiple architectures, including x86_64, ARM64 and RISC-V 64. We changed the directory structure so that architecture-specific parts are held in separate files.
  • We updated the CI scripts to reflect related changes, such as removing the -normal part from the OpenJDK CONF string.

wks added 17 commits August 15, 2025 22:14
OpenJDK 21 no longer has `REF_OTHER` in the enumeration type
`ReferenceType`.
This makes it easier to debug
The MMTkHeap instance will receive calls to `object_iterate` when
connected from VisualVM and panic.  Leaving this function blank will not
properly implement the iteration, but will allow the VM to be profiled
with VisualVM.
The super class method `BarrierSet::on_thread_attach` now does
initializations related to nmethods, particularly the "nmethod disarmed
guard value".
This commit addresses minor issues after merging the SATB barrier and
the barrier refactoring to the OpenJDK 21 binding.

In x86 barrier assembler, we use `testptr` for testing if a register is
zero.

OpenJDK 21 now gives 3 tmp registers to assemblers.  We use them
instead.  We also removed the workaround for OpenJDK 11 where the given
`tmp1` and `tmp2` registers overlap with operand registers.

We updated `MMTkSATBBarrierSetC2::load_at_resolved` by copying some code
from `G1BarrierSetC2`.
We revert to our old repository and directory name `openjdk` instead of
`jdk`.

-   The upstream OpenJDK project creates one repository for every major
    release, such as `jdk21u`, and updates to that major versions are
    pushed into that repository.  The `jdk` repository is for the main
    line development.  Since our `openjdk` repo actually follows update
    repos (such as `jdk21u`), it doesn't make sense to use `jdk` as the
    repository name.
-   There are many other JVM implementations out there, such as JikesRVM
    and OpenJ9.  Using the repository name `openjdk` emphasizes that it
    is OpenJDK.
@wks wks marked this pull request as draft November 19, 2025 09:31
@qinsoon
Copy link
Member

qinsoon commented Nov 20, 2025

Would it be cleaner if we rename master to jdk-11.x.x+x-mmtk, and keep the branch jdk-21.0.9+3-mmtk and make it the default branch? We would change places where we assume master is the default branch in our CI.

@wks
Copy link
Contributor Author

wks commented Nov 20, 2025

Would it be cleaner if we rename master to jdk-11.x.x+x-mmtk, and keep the branch jdk-21.0.9+3-mmtk and make it the default branch? We would change places where we assume master is the default branch in our CI.

This approach has the advantage that each branch maps uniquely to an openjdk repo branch, and we can still let the mmtk-core CI choose which branch to test against.

But there is an obvious disadvantage. We may switch the default OpenJDK branch from time to time, such as from jdk-21+35-mmtk to jdk-21.0.9+3-mmtk for reasons mentioned here. If we don't have a canonical master branch, some of us may switch the default branch while other developers are not aware of it. They may still have jdk-21+35-mmtk in their local worktrees. git pull won't fix that, and they need to explicitly run git checkout, and someone needs to tell them to do so. This complicates things.

Another disadvantage is that we will need to change the mmtk-core CI every time we make this change, even if the CI scripts in the new branch is 100% compatible with the mmtk-core CI scripts.

So I suggest we still keep one unique master branch which is pointing to OpenJDK11 now, and will point to the revision of this PR in the future. Other developers on the master branch can simply run git pull and it will be updated to the binding for OpenJDK 21.

Another advantage of keeping the master branch is that when we start developing a branch of a new OpenJDK version, such as jdk-25+36-mmtk (where jdk-25+36 is the latest released OpenJDK LTS version now), we can develop it in parallel with the master branch, and we always know that master is the branch which jdk-25+36-mmtk should merge with (or rebase onto) from time to time. When it is the time to merge that into master, we can still use the regular pull-request workflow for code review, as we are doing now (and previously in #307).

@qinsoon
Copy link
Member

qinsoon commented Nov 20, 2025

We may switch the default OpenJDK branch from time to time, such as from jdk-21+35-mmtk to jdk-21.0.9+3-mmtk for reasons mentioned #307 (comment). If we don't have a canonical master branch, some of us may switch the default branch while other developers are not aware of it.

We could have different default branches for each JDK version, like jdk-11, jdk-21 -- those are kept updated to the latest version we support.

Another disadvantage is that we will need to change the mmtk-core CI every time we make this change, even if the CI scripts in the new branch is 100% compatible with the mmtk-core CI scripts.

I don't see this as a disadvantage. We can easily change what branch mmtk-core tests with: https://github.com/mmtk/mmtk-core/blob/e25ad8bb969f5d650de3f1946582075923bafcc9/.github/workflows/pr-binding-refs.yml#L125

Explicitly switching to a different branch is not a bad idea.

So I suggest we still keep one unique master branch which is pointing to OpenJDK11 now, and will point to the revision of this PR in the future.

Consider this scenario: someone is using master with JDK 11. They did a pull, and suddenly there is a ton of conflicts and the new commit also no longer works for 11.

@qinsoon
Copy link
Member

qinsoon commented Nov 20, 2025

My intuition is that this pull request does not really look like a normal pull request. So maybe it should not be a pull request, and we should consider other approaches to get the changes. I am happy to discuss more on this.

@wks
Copy link
Contributor Author

wks commented Nov 20, 2025

We could have different default branches for each JDK version, like jdk-11, jdk-21 -- those are kept updated to the latest version we support.

This sounds good to me, too. We can use jdk-21 as the default branch and update it as we update the mmtk/openjdk repo, and it is relatively stable. We will switch to jdk-25 if we feel the time is appropriate, and we will announce in our group meeting or on Zulip.

I don't see this as a disadvantage. We can easily change what branch mmtk-core tests with: https://github.com/mmtk/mmtk-core/blob/e25ad8bb969f5d650de3f1946582075923bafcc9/.github/workflows/pr-binding-refs.yml#L125

Explicitly switching to a different branch is not a bad idea.

OK. It's OK as long as it is easy to do so.

Consider this scenario: someone is using master with JDK 11. They did a pull, and suddenly there is a ton of conflicts and the new commit also no longer works for 11.

Yes. This is indeed a concern. But to support multiple parallel branches like jdk-11 and jdk-21, we need to phase out the master branch in a graceful way. In Git, it feels abruptive to just let a branch disappear. People usually don't expect a branch to suddenly disappear on the server.

I think one way to solve it is to update master to JDK 21, and introduce a branch jdk-11 which keeps the state of master before the merge. This has two implications:

  • jdk-11 is a legacy branch, and people should generally not work on legacy JDK versions.
  • If some developers do need to work on legacy versions for some reasons, they should explicitly base on the jdk-XX branches. The large number of merge conflicts is an indication that some non-trivial changes are happening, and they should switch branches locally.

Git also has a feature called "symbolic refs". As suggested in this page, we can use it to make aliases of branches. We can make an actual jdk-21 branch, and make master a symbolic ref to jdk-21. I'll do some experiments locally on temporary repositories to see if this idea can work for mmtk-openjdk.

@wks
Copy link
Contributor Author

wks commented Nov 20, 2025

My intuition is that this pull request does not really look like a normal pull request. So maybe it should not be a pull request, and we should consider other approaches to get the changes. I am happy to discuss more on this.

One way to make the transition is to keep an explicit jdk-21 branch (and maybe also using git symbolic-ref to make master an alias to it), and make a PR in mmtk-core so that it tests against mmtk-openjdk branch jdk-21. As for development,

  • We announce in the meeting when we formally make the transition, and/or we make master point to the revision (be an alias of) jdk-21 so that we will make new changes there by default.
  • Development on mmtk-core will continue as usual, but new PRs on mmtk-core will be tested against jdk-21.

We should probably discuss this in our next meeting.

The benchmark name defined in base.yml is simply "dacapochopin", and we
reference that in normal-heap.yml that.

We also mention the MR2 version of the DaCapo Chopin JAR file wherever
appropriate.
@qinsoon
Copy link
Member

qinsoon commented Nov 21, 2025

Kunshan and I discussed more on this.

There are a few things we agreed on:

  • We will have different branches for each JDK version, such as jdk-11, jdk-21, etc.
  • We usually start developing for new versions, based on the current version (e.g. developing jdk-25 based on jdk-21). Because of this, we will need to merge from a old version (current trunk) to a new version. But we should avoid merging from a new version back to a old version -- we backport necessary changes by cherry-picking.

We haven't agreed on whether we should keep using master.

I am favoring deprecating master. We do not use master any more, and set jdk-21 as the default branch for GitHub. The argument is that between versions, there are significant changes, and people care about the version they are using and don't want to get changes about new versions by just using git pull. It will be disruptive for now to deprecate master, as users all track our master. But once we deprecate master and ask users to track their own version branch (e.g. jdk-21), in the future version there is no problem.

Kunshan favors keeping master. Once the jdk-21 branch is ready, we should make master an alias to jdk-21. master represents where our development is happening. Users will always get new changes from master when they do git pull. In future versions, we will keep changing master to point to the latest version we are developing.

@wks If I missed any point from our discussion, just add to this thread.

@qinsoon
Copy link
Member

qinsoon commented Nov 21, 2025

Kunshan favors keeping master. Once the jdk-21 branch is ready, we should make master an alias to jdk-21. master represents where our development is happening. Users will always get new changes from master when they do git pull. In future versions, we will keep changing master to point to the latest version we are developing.

I feel this is okay to me if:

  • Git alias works (git symbolic-ref) so we don't have to push new commits to two separate branches.
  • We advise users to track their corresponding version branch (jdk-11, jdk-21, etc.) if they only care about a specific version.

@wks
Copy link
Contributor Author

wks commented Nov 21, 2025

@qinsoon has summarized what we discussed today. I'll clarify, in terms of Git commit graph, what will happen after we make the transition

This is the current status. master is for OpenJDK 11. master is also a proper ancestor of the jdk-21 branch because I have merged master into jdk-21 from time to time and resolved conflicts.

mmtk-openjdk-21-branching-1

The figure below is the state right after merging. Before merging, we will create a branch jdk-11 to point to the commit of the current master. By merging, we create a new merge commit node, and the jdk-21 branch will point to it. We will also change master so that it is a symbolic reference (alias) of jdk-21 so they always point to the same commit. The only difference between my proposal and @qinsoon's proposal is that in @qinsoon's proposal, the master branch will disappear rather than becoming the alias of jdk-21.

mmtk-openjdk-21-branching-2

The figure below is the state in the future. Major development efforts will be on master (alias of jdk-21).

We will also have a jdk-25 branch when OpenJDK 25 is stable. It will start from a commit in master (alias of jdk-21), and we will make JDK-25-specific commits on the jdk-25 branch. From time to time, we will synchronize jdk-25 with master by running git merge master on the jdk-25 branch. Eventually, master will become an alias of jdk-25 (if we decide to keep master), and we will start the development cycle (for example, of JDK 29, or whatever will become the LTS version) on a new branch, making jdk-21 legacy, too.

We will also have a jdk-11 branch for the legacy OpenJDK 11 version. We will keep it running by back-porting important features or fixes from the master branch to jdk-11. But jdk-11 will never merge with master again.

mmtk-openjdk-21-branching-3

@wks
Copy link
Contributor Author

wks commented Nov 21, 2025

Here is a summary of some opposing ideas.

Do people stick to a particular JDK version, or do they move to the newest JDK version?

  • stay: People tend to stay at a particular JDK version. This is especially true if
    • They made significant changes to the OpenJDK runtime. For example, they changed OpenJDK 11 to add support for publishing to support Iso.
    • They made changes to mmtk-core and have not merged, yet. For example, they changed the reference processing API, and also changed a large portion of the OpenJDK 11 binding. It takes time to port the change to OpenJDK 21.
  • move: People tend to follow the latest JDK version we support.
    • For GC algorithm developers, they need a stable test ground for the GC algorithm, but they don't care whether it is OpenJDK 11 or OpenJDK 21, as long as it is maintained.
    • We mmtk-core team may develop new features that are only supported on OpenJDK 21 (and we deem it unworthy to backport to OpenJDK 11), which greatly improves the performance of any GC algorithm.

Do we use GitHub's default branch to point to the active branch, or do we have a master that moves from 11 to 21 to 25 to ...?

  • GitHub default: We set the default branch of the https://github.com/mmtk/mmtk-openjdk repo to the most actively maintained branch.
    • Good: Whenever a developer does git pull, it always pulls a revision for the same OpenJDK version.
    • Bad: Developer may stay at a branch (e.g. jdk-11) for a long time (e.g. 6 months) without noticing that we changed the default branch. (Note that git pull doesn't notify them about default branch changes.) They may notice the development becomes less active (because it is legacy), and they will only come to our web site or Zulip platform to find out something changed if they are curious enough.
  • Moving master branch: Have a master branch that moves as we change the default branch.
    • Good: Whenever a developer does git pull, it automatically moves on to the most actively developed OpenJDK version.
    • Bad: Developers that made changes to the mmtk-openjdk repository may suddenly see lots and lots of merge conflicts when master changes from one OpenJDK version to the next.
      • Not that bad: The merge conflicts are a signal that they should either switch the remote branch to a legacy branch (e.g. jdk-11) or move on to the next OpenJDK version by resolving the merge conflicts.

Note that we can have both versioned branches such as jdk-11, jdk-21, jdk-25, etc. and a master, and allow the user to select whether they want to follow one particular OpenJDK version or the most actively developed version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants