Skip to content

Conversation

breskeby
Copy link
Contributor

@breskeby breskeby commented Aug 14, 2025

Updating to the latest Gradle GA version to keep our build uptodate

@breskeby breskeby requested a review from a team as a code owner August 14, 2025 14:54
@breskeby breskeby added >non-issue :Delivery/Build Build or test infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts Team:Delivery Meta label for Delivery team auto-backport Automatically create backport pull requests when merged v9.2.0 v8.17.11 v8.18.6 v9.0.6 v9.1.3 v8.19.3 labels Aug 14, 2025
@breskeby breskeby self-assigned this Aug 14, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

jackson = "2.15.0"
junit5 = "5.12.1"
spock = "2.1-groovy-3.0"
spock = "2.3-groovy-4.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

required due to groovy update within gradle

@elasticsearchmachine elasticsearchmachine added the serverless-linked Added by automation, don't add manually label Aug 14, 2025
include ':consumer'
include ':producer-tar'
"""
subProject("consumer")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gradle throws error if the project folder does not exist yet. therefore tweaks to our tests in that area.

"""

subProject(":distribution:archives:integ-test-zip") << "configurations.create('extracted')\n"
subProject(":distribution:archives:integ-test-zip") << "configurations.create('default')\n"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manual creating default configuration is deprecated now

from fileTree("archiveRoot")
into('config') {
dirMode 0750
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that APi has been removed


package org.elasticsearch.gradle.internal

import groovy.ant.AntBuilder
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

packages for some groovy util classes have moved

* @param value the value used in the replacement. For example "bar"
*/
public void replaceValueInMatch(String subKey, Object value) {
getTransformations().add(new ReplaceValueInMatch(subKey, MAPPER.convertValue(value, JsonNode.class)));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JsonNode cannot be serialized anymore due to restrictions to the serializer. (details in commit messages). therefore we wrap this in a serializable class

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it simply because JsonNode doesn't implement java.io.Serializable?
Could you use BaseJsonNode? As far as I can see this is the only subclass of the JsonNode and implement Serializable. This way we could potentially avoid having SerializableJsonNode class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No its not that. its the fact that NodeSerialisation relies on ObjectInputStream#.readFully which is unsupported by the gradle serialization. Usually serializing any more complex class often results in a hassle in Gradle therefore should be avoided as it causes this kind of unexpected behaviour

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I misinterpreted the change. I thought SerializableJsonNode is just a wrapper on JsonNode, but it actually stores the original value and only change it to JsonNode when toJsonNode() is called.
Is there any risk related to the value not being Serializable? Would it make sense to changed if from Object to Serializable?

httpclient5 = "org.apache.httpcomponents.client5:httpclient5:5.4.2"
idea-ext = "gradle.plugin.org.jetbrains.gradle.plugin.idea-ext:gradle-idea-ext:1.1.4"
javaparser = "com.github.javaparser:javaparser-core:3.18.0"
javaparser = "com.github.javaparser:javaparser-core:3.27.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

saw some flakyness around this with the update. so pushed a new version that has fixed this. My assumption is there is some caching /serialization issue

@breskeby breskeby force-pushed the update-gradle-wrapper-900 branch from bf41c69 to e4db346 Compare August 18, 2025 08:25
@breskeby breskeby requested a review from jozala August 18, 2025 08:26
Copy link
Contributor

@jozala jozala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding the comments. This is super helpful.
In general it looks good.
I've left two comments in the code for the potential minor improvements and one question.

Comment on lines +40 to +41
YAMLFactory YAML_FACTORY = new YAMLFactory();
ObjectMapper MAPPER = new ObjectMapper(YAML_FACTORY);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you decide to keep using JsonNode (see previous comment).

I would consider moving the MAPPER to the static field of the class. I assume the construction of the object is not very costly, but since it is called multiple times it it still beneficial.
ObjectMapper is thread-safe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah there'is a overhead with this. but should only be triggered by execution time which means we don't need to deal with this class at configuration time at all. I'll take another look on using a static field usually I try to avoid those for avoiding unpredicted state in the gradle daemon / classloading issues

* @param value the value used in the replacement. For example "bar"
*/
public void replaceValueInMatch(String subKey, Object value) {
getTransformations().add(new ReplaceValueInMatch(subKey, MAPPER.convertValue(value, JsonNode.class)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it simply because JsonNode doesn't implement java.io.Serializable?
Could you use BaseJsonNode? As far as I can see this is the only subclass of the JsonNode and implement Serializable. This way we could potentially avoid having SerializableJsonNode class.

import javax.inject.Inject;

public class ExtractCurrentVersionsTask extends AbstractVersionsTask {
public abstract class ExtractCurrentVersionsTask extends AbstractVersionsTask {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that you've changed the tasks classes to be abstract, to make them compatible with lazy configuration?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i don't remember why i did that here. it should not be required to be lazy task compatible. Let me change that back or find out why that was required

@breskeby breskeby force-pushed the update-gradle-wrapper-900 branch from e4db346 to 9f74701 Compare August 19, 2025 13:54
@breskeby breskeby merged commit 8e2c948 into elastic:main Aug 19, 2025
33 of 35 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.17 Commit could not be cherrypicked due to conflicts
8.18 Commit could not be cherrypicked due to conflicts
9.0 Commit could not be cherrypicked due to conflicts
9.1
8.19 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 132932

breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 19, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 19, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 19, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 19, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml
@breskeby
Copy link
Contributor Author

💔 Some backports could not be created

Status Branch Result
9.0
8.19
8.18
8.17 Conflict resolution was aborted by the user

Manual backport

To create the backport manually run:

backport --pr 132932

Questions ?

Please refer to the Backport tool documentation

@breskeby breskeby removed the v8.17.11 label Aug 19, 2025
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 20, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 20, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 20, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 20, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml
breskeby added a commit that referenced this pull request Aug 21, 2025
* Update Gradle wrapper to 9.0.0 (#132932)

Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml

* Fix verification data
breskeby added a commit that referenced this pull request Aug 21, 2025
* Update Gradle wrapper to 9.0.0 (#132932)

Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java
#	gradle/build.versions.toml
#	gradle/verification-metadata.xml

* Fix merge issues

* More fixes lost in backports

* Fix docker bin context copy
breskeby added a commit that referenced this pull request Aug 21, 2025
* Update Gradle wrapper to 9.0.0 (#132932)

Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser

(cherry picked from commit 8e2c948)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/idea/IdeaXmlUtil.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/RestCompatTestTransformTask.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseTo.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/transform/text/ReplaceIsTrue.java
#	build-tools-internal/src/main/resources/minimumGradleVersion
#	build-tools-internal/src/test/java/org/elasticsearch/gradle/internal/test/rest/transform/close_to/ReplaceValueInCloseToTests.java

* Fix merge conflict

* More fixes lost in backports

* Fix docker bin context copy
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Aug 21, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser
elasticsearchmachine pushed a commit that referenced this pull request Aug 21, 2025
Updating to the latest Gradle GA version to keep our build uptodate

* Rework RestCompatTestTransformTask to CC compatibility
* Fix build integration tests due to api changes
* Updating javaparser
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending :Delivery/Build Build or test infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts >non-issue serverless-linked Added by automation, don't add manually Team:Delivery Meta label for Delivery team v8.18.6 v8.19.3 v9.0.6 v9.1.3 v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants