GH action for updating logstash version #18035

donoghuc · 2025-08-25T22:17:37Z

This commit adds a GH action for bumping the logstash version. It uses the updatescli based on a suggestion for how the beats team is doing file updates. There is a workflow_dispatch trigger that accepts a new logstash version. Both the versions.yml and lock file are updated. If it is a branch where we are not vendoring a lockfile that file is skipped without failing the workflow step.

Implements https://github.com/elastic/ingest-dev/issues/5977

github-actions · 2025-08-25T22:17:47Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify · 2025-08-25T22:18:15Z

This pull request does not have a backport label. Could you fix it @donoghuc? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
If no backport is necessary, please add the backport-skip label

donoghuc · 2025-08-25T22:18:20Z

Use of updatecli for dep bumps was based on this comment #17945 (comment)

donoghuc · 2025-08-25T22:22:47Z

To test locally:

install updatecli

brew install updatecli/updatecli/updatecli

generate config file:

# Create the same config the GitHub Action uses
cat > test-updatecli.yml << 'EOF'
name: Update versions.yml

sources:
  logstash_version:
    kind: shell
    spec:
      command: echo "9.3.0"

targets:
  update_logstash_version:
    name: Update logstash version in versions.yml
    kind: yaml
    spec:
      file: versions.yml
      key: $.logstash
    sourceid: logstash_version
  
  update_logstash_core_version:
    name: Update logstash-core version in versions.yml
    kind: yaml
    spec:
      file: versions.yml
      key: $.logstash-core
    sourceid: logstash_version

---
name: Update lock file

sources:
  logstash_version:
    kind: shell
    spec:
      command: echo "9.3.0"

conditions:
  check_lockfile_exists:
    name: Check if lockfile exists
    kind: shell
    disablesourceinput: true
    spec:
      command: test -f Gemfile.jruby-3.1.lock.release

targets:
  update_gemfile_lock_dependency:
    name: Update logstash-core dependency
    kind: file
    spec:
      file: Gemfile.jruby-3.1.lock.release
      matchpattern: 'logstash-core \(= [0-9]+\.[0-9]+\.[0-9]+'
      replacepattern: 'logstash-core (= 9.3.0'
    sourceid: logstash_version
  
  update_gemfile_lock_spec:
    name: Update logstash-core spec
    kind: file
    spec:
      file: Gemfile.jruby-3.1.lock.release
      matchpattern: 'logstash-core \([0-9]+\.[0-9]+\.[0-9]+-java\)'
      replacepattern: 'logstash-core (9.3.0-java)'
    sourceid: logstash_version
EOF

run

➜  logstash git:(bump-ls-version-action) ✗ updatecli apply --config test-updatecli.yml



+++++++++++
+ PREPARE +
+++++++++++

Loading Pipeline "test-updatecli.yml"

SCM repository retrieved: 0


++++++++++++++++++
+ AUTO DISCOVERY +
++++++++++++++++++



++++++++++++
+ PIPELINE +
++++++++++++



#######################
# UPDATE VERSIONS.YML #
#######################

source: source#logstash_version
-----------------------
The shell 🐚 command "/bin/sh /var/folders/cw/q_xjr4md1wj_w_c1xwfrnxdw0000gn/T/updatecli/bin/33b3af0384bd90f14aef112598cfabdf6d6e77168b475723abbaaf5f269b28f2.sh" ran successfully with the following output:
----
9.3.0
----
✔ shell command executed successfully

target: target#update_logstash_core_version
-----------------------------------
⚠ - change detected:
	* key "$.logstash-core" updated from "9.2.0" to "9.3.0", in file "versions.yml"

target: target#update_logstash_version
------------------------------
⚠ - change detected:
	* key "$.logstash" updated from "9.2.0" to "9.3.0", in file "versions.yml"


####################
# UPDATE LOCK FILE #
####################

source: source#logstash_version
-----------------------
The shell 🐚 command "/bin/sh /var/folders/cw/q_xjr4md1wj_w_c1xwfrnxdw0000gn/T/updatecli/bin/33b3af0384bd90f14aef112598cfabdf6d6e77168b475723abbaaf5f269b28f2.sh" ran successfully with the following output:
----
9.3.0
----
✔ shell command executed successfully

condition: condition#check_lockfile_exists
-------------------------------
The shell 🐚 command "/bin/sh /var/folders/cw/q_xjr4md1wj_w_c1xwfrnxdw0000gn/T/updatecli/bin/c4efa96ffd6665bfcc18724aff4523a9b7e976d0a59d6da437e1263a76b5f8c6.sh" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----
shell command failed. Expected exit code 0 but got 1
✗ shell condition of type "console/output" not passing

target: target#update_gemfile_lock_spec
-------------------------------

target: target#update_gemfile_lock_dependency
-------------------------------------


ACTIONS
========


=============================

SUMMARY:



- Update lock file:
	Source:
		✔ [logstash_version]
	Condition:
		✗ [check_lockfile_exists] Check if lockfile exists
	Target:
		- [update_gemfile_lock_dependency] Update logstash-core dependency
		- [update_gemfile_lock_spec] Update logstash-core spec


⚠ Update versions.yml:
	Source:
		✔ [logstash_version]
	Target:
		⚠ [update_logstash_core_version] Update logstash-core version in versions.yml
		⚠ [update_logstash_version] Update logstash version in versions.yml


Run Summary
===========
Pipeline(s) run:
  * Changed:	1
  * Failed:	0
  * Skipped:	1
  * Succeeded:	0
  * Total:	2

donoghuc · 2025-08-27T23:10:08Z

This was tested with #18050 and produced a PR with a version bump! Ready for review.

.github/workflows/bump-logstash.yml

v1v · 2025-09-09T06:55:31Z

.ci/updatecli/values.d/scm.yml

+  # begin updatecli-compose policy values
+  user: 'github-actions[bot]'
+  email: '41898282+github-actions[bot]@users.noreply.github.com'
+  # end updatecli-compose policy values


You don't need those values, that's when using a shared policy with updatecli, in this case, you only added the .ci/updatecli/bump-logstash-version.yml manifest.

Suggested change

# begin updatecli-compose policy values

user: 'github-actions[bot]'

email: '41898282+github-actions[bot]@users.noreply.github.com'

# end updatecli-compose policy values

v1v · 2025-09-09T06:56:55Z

.github/workflows/bump-java-version.yml

@@ -1,19 +1,34 @@
-name: Stub GH action for devoping new workflows [STUB]
+


duplicates .github/workflows/bump-version.yml?

same comment as #18035 (comment)

v1v · 2025-09-09T06:57:49Z

.github/workflows/bump-java-version.yml

  pull_request:
    types: [opened, synchronize, reopened]


IIUC, this event will not work with forked PRs, since the GH token will never have any elevated access but read-only.

Yeah, the idea here was that this "stub" action is already checked in with a pull_request trigger. I was hoping that by doing that and targeting that workflow file from a branch that pushed to the elastic master copy of logstash would be enough to actually give a valid token for raising a PR. It may not actually be necessary though. Either way I will be removing the "stub" before this is actually ready for review.

This commit adds a GH action for bumping the logstash version. It uses the updatescli based on a suggestion for how the beats team is doing file updates. There is a workflow_dispatch trigger that accepts a new logstash version. Both the versions.yml and lock file are updated. If it is a branch where we are not vendoring a lockfile that file is skipped without failing the workflow step.

use pattern suggested in code review

v1v · 2025-09-16T19:23:09Z

.ci/updatecli/bump-logstash-version.yml

@@ -0,0 +1,81 @@
+---
+name: Update logstash version files
+pipelineid: "logstash/version-updates"


If you run this manifest targeting multiple branches, then the pipelineid should be unique. I'd recommend using LOGSTASH_BRANCH.

Suggested change

pipelineid: "logstash/version-updates"

pipelineid: "logstash/version-updates-{{ requiredEnv "LOGSTASH_BRANCH" }}"

.github/workflows/bump-logstash.yml

v1v · 2025-09-16T19:25:13Z

.ci/updatecli/values.d/scm.yml

+  enabled: true
+  owner: elastic
+  repository: logstash
+  commitusingapi: true


nit: this is hardcoded in the updatecli manifest

Suggested change

commitusingapi: true

v1v · 2025-09-16T19:25:38Z

.ci/updatecli/values.d/scm.yml

@@ -0,0 +1,5 @@
+scm:
+  enabled: true


nit: unused

Suggested change

enabled: true

v1v · 2025-09-16T19:26:11Z

.github/workflows/bump-java-version.yml

  workflow_dispatch:
+    inputs:
+      logstash_version:
+        description: 'Logstash version'


Do you think adding an example might be useful here?

Suggested change

description: 'Logstash version'

description: 'Logstash version (i.e: 1.2.3)'

yep, added one

v1v · 2025-09-16T19:26:34Z

.github/workflows/bump-java-version.yml

+        required: true
+        type: string
+      logstash_branch:
+        description: 'Logstash branch'


Suggested change

description: 'Logstash branch'

description: 'Logstash branch (i.e: 8.19)'

yep, added one

elastic-sonarqube · 2025-09-16T22:26:09Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

elasticmachine · 2025-09-16T22:34:04Z

💚 Build Succeeded

Buildkite Build
Commit: ea5ddcc

History

💚 Build #3509 succeeded 364c831
💚 Build #3498 succeeded fa1538e
💚 Build #3497 succeeded f6306aa
💛 Build #3491 was flaky fe19127
💚 Build #3484 succeeded 83568e4
💛 Build #3463 was flaky 5118e44

donoghuc · 2025-09-22T16:22:59Z

@v1v Could you take another pass at reviewing this? I updated based on your comments, and used the test action to do all the version bumps this cycle!

v1v

LGTM, thanks for this PR, it's wonderful!

github-actions · 2025-09-23T19:04:40Z

@Mergifyio backport 8.18 8.19 9.0 9.1

mergify · 2025-09-23T19:04:49Z

backport 8.18 8.19 9.0 9.1

✅ Backports have been created

#18205 [8.18] (backport #18035) GH action for updating logstash version has been created for branch 8.18
#18206 [8.19] (backport #18035) GH action for updating logstash version has been created for branch 8.19
#18207 [9.0] (backport #18035) GH action for updating logstash version has been created for branch 9.0
#18208 [9.1] (backport #18035) GH action for updating logstash version has been created for branch 9.1

* GH action for updating logstash version This commit adds a GH action for bumping the logstash version. It uses the updatescli based on a suggestion for how the beats team is doing file updates. There is a workflow_dispatch trigger that accepts a new logstash version. Both the versions.yml and lock file are updated. If it is a branch where we are not vendoring a lockfile that file is skipped without failing the workflow step. * use stub workflow to test * Separate step when modifying * dont track temp updatecli file and install action * restore stub * Refactor updatecli workflow use pattern suggested in code review * Use test stub to validate changes * fix inputs section * fix branch * dont fail if file is not present * try with no global conditionals * file condition * name conditional * split into separate pipeline files * remove unused scmid * no explicit pr action? * add back PR * try sources pattern * explicitly connect target with scmid * ensure lock file lookup executes from checkout * Cleanup * test * restore LOGSTASH_BRANCH * codereview feedback (cherry picked from commit ad20032)

* GH action for updating logstash version This commit adds a GH action for bumping the logstash version. It uses the updatescli based on a suggestion for how the beats team is doing file updates. There is a workflow_dispatch trigger that accepts a new logstash version. Both the versions.yml and lock file are updated. If it is a branch where we are not vendoring a lockfile that file is skipped without failing the workflow step. * use stub workflow to test * Separate step when modifying * dont track temp updatecli file and install action * restore stub * Refactor updatecli workflow use pattern suggested in code review * Use test stub to validate changes * fix inputs section * fix branch * dont fail if file is not present * try with no global conditionals * file condition * name conditional * split into separate pipeline files * remove unused scmid * no explicit pr action? * add back PR * try sources pattern * explicitly connect target with scmid * ensure lock file lookup executes from checkout * Cleanup * test * restore LOGSTASH_BRANCH * codereview feedback (cherry picked from commit ad20032) Co-authored-by: Cas Donoghue <[email protected]>

* GH action for updating logstash version This commit adds a GH action for bumping the logstash version. It uses the updatescli based on a suggestion for how the beats team is doing file updates. There is a workflow_dispatch trigger that accepts a new logstash version. Both the versions.yml and lock file are updated. If it is a branch where we are not vendoring a lockfile that file is skipped without failing the workflow step. * use stub workflow to test * Separate step when modifying * dont track temp updatecli file and install action * restore stub * Refactor updatecli workflow use pattern suggested in code review * Use test stub to validate changes * fix inputs section * fix branch * dont fail if file is not present * try with no global conditionals * file condition * name conditional * split into separate pipeline files * remove unused scmid * no explicit pr action? * add back PR * try sources pattern * explicitly connect target with scmid * ensure lock file lookup executes from checkout * Cleanup * test * restore LOGSTASH_BRANCH * codereview feedback

donoghuc mentioned this pull request Aug 26, 2025

Replace buildkite jdk version check w/GH action #17945

Merged

5 tasks

donoghuc force-pushed the bump-ls-version-action branch from 410469e to 2514eec Compare August 27, 2025 22:37

mergify bot mentioned this pull request Aug 27, 2025

TEST: use stub GH action workflow to test new LS version action [NOT FOR MERGE] #18050

Closed

donoghuc force-pushed the bump-ls-version-action branch 3 times, most recently from 0c6def8 to 1292374 Compare August 27, 2025 23:09

donoghuc marked this pull request as ready for review August 27, 2025 23:10

donoghuc requested review from a team as code owners August 27, 2025 23:10

v1v reviewed Sep 2, 2025

View reviewed changes

.github/workflows/bump-logstash.yml Outdated Show resolved Hide resolved

.github/workflows/bump-logstash.yml Outdated Show resolved Hide resolved

donoghuc force-pushed the bump-ls-version-action branch 2 times, most recently from 956fd79 to 38c0131 Compare September 8, 2025 22:35

v1v reviewed Sep 9, 2025

View reviewed changes

donoghuc added 12 commits September 9, 2025 14:56

use stub workflow to test

e75a275

Separate step when modifying

9f2a3bd

dont track temp updatecli file and install action

3d29a0a

restore stub

fafaf94

Refactor updatecli workflow

beab304

use pattern suggested in code review

Use test stub to validate changes

537f00d

fix inputs section

f7bf2c6

fix branch

5eb366e

dont fail if file is not present

c0b7aab

try with no global conditionals

bc2f8da

file condition

76da246

donoghuc added 2 commits September 16, 2025 10:35

test

c7bc793

restore LOGSTASH_BRANCH

364c831

v1v reviewed Sep 16, 2025

View reviewed changes

codereview feedback

ea5ddcc

donoghuc requested a review from v1v September 16, 2025 22:12

v1v approved these changes Sep 23, 2025

View reviewed changes

donoghuc added the backport-active-all Automated backport with mergify to all the active branches label Sep 23, 2025

donoghuc merged commit ad20032 into elastic:main Sep 23, 2025
14 checks passed

mergify bot mentioned this pull request Sep 23, 2025

[8.18] (backport #18035) GH action for updating logstash version #18205

Merged

mergify bot mentioned this pull request Sep 23, 2025

[8.19] (backport #18035) GH action for updating logstash version #18206

Merged

mergify bot mentioned this pull request Sep 23, 2025

[9.0] (backport #18035) GH action for updating logstash version #18207

Merged

mergify bot mentioned this pull request Sep 23, 2025

[9.1] (backport #18035) GH action for updating logstash version #18208

Merged

		@@ -1,19 +1,34 @@
		name: Stub GH action for devoping new workflows [STUB]

	pipelineid: "logstash/version-updates"
	pipelineid: "logstash/version-updates-{{ requiredEnv "LOGSTASH_BRANCH" }}"

	description: 'Logstash version'
	description: 'Logstash version (i.e: 1.2.3)'

	description: 'Logstash branch'
	description: 'Logstash branch (i.e: 8.19)'

GH action for updating logstash version #18035

GH action for updating logstash version #18035

Uh oh!

Conversation

donoghuc commented Aug 25, 2025

Uh oh!

github-actions bot commented Aug 25, 2025

🤖 GitHub comments

Uh oh!

mergify bot commented Aug 25, 2025

Uh oh!

donoghuc commented Aug 25, 2025

Uh oh!

donoghuc commented Aug 25, 2025

Uh oh!

donoghuc commented Aug 27, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elastic-sonarqube bot commented Sep 16, 2025

Quality Gate passed

Uh oh!

elasticmachine commented Sep 16, 2025

💚 Build Succeeded

History

Uh oh!

donoghuc commented Sep 22, 2025

Uh oh!

v1v left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Sep 23, 2025

Uh oh!

mergify bot commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mergify bot commented Sep 23, 2025 •

edited

Loading