Preserve upgrade marker when rolling back and add rollback reason to upgrade details

[pchila]

What does this PR do?

A new reason string field has been added in UpgradeDetails struct. Such field should contain a human-readable description of the cause of the last state transition.
In the specific case this is used to distinguish how elastic-agent got to the UPG_ROLLBACK state: the only possible reason at the moment is an automatic rollback initiated by the elastic-agent watcher, but this could be used for any state transition

Why is it important?

This PR will allow to to distinguish if the cause of an UPG_ROLLBACK status is:

• the new agent misbehaving (automatic rollback)
• a manual rollback (will be introduced with Handle rollback command/action in upgrade flow #6889)

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

1. Upgrade an agent built from this PR to:

• either a lower version agent (9.0.2 for example)
• the same commit repackaged as an IAR, for example

  AGENT_PACKAGE_VERSION="9.1.0+build20250626000000" BEAT_VERSION="9.1.0-SNAPSHOT" EXTERNAL=true PACKAGES=tar.gz  PLATFORMS="linux/amd64" mage clean package

2. Kill/restart the updated agent 3 times during the grace period

3. Check that the agent rolls back to the starting version

4. Verify that now a new reason field is present in upgrade details metadata:

   • with an elastic-agent status command

   root@elastic-agent-dev:~# elastic-agent status
   ┌─ fleet
   │  └─ status: (STARTING)
   ├─ elastic-agent
   │  └─ status: (HEALTHY) Running
   └─ upgrade_details
      ├─ target_version: 9.1.0
      ├─ state: UPG_ROLLBACK
      ├─ action_id: 82b460db-898c-453f-9372-303042605841
      └─ metadata
         └─ reason: automatic rollback

   • with a call to fleet API GET kbn:/api/fleet/agents/7a0ac237-9e82-49ac-bb0d-a2a9edac0ead if it's a managed agent:

   {
   "item": {
       "id": "7a0ac237-9e82-49ac-bb0d-a2a9edac0ead",
       "type": "PERMANENT",
       "active": true,
       "enrolled_at": "2025-06-26T11:35:58Z",
       "upgraded_at": "2025-06-26T11:39:05Z",
       "upgrade_started_at": null,
       "upgrade_details": {
         "metadata": {
           "retry_until": "2025-06-26T13:36:10.184616451Z",
           "reason": "automatic rollback"
         },
         "action_id": "82b460db-898c-453f-9372-303042605841",
         "state": "UPG_ROLLBACK",
         "target_version": "9.1.0"
       },

Related issues

• Closes Add reason in UPG_ROLLBACK upgrade details metadata #6877

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[mergify]

This pull request does not have a backport label. Could you fix it @pchila? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add-rollback-reason upstream/add-rollback-reason
git merge upstream/main
git push upstream add-rollback-reason

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b add-rollback-reason upstream/add-rollback-reason
git merge upstream/main
git push upstream add-rollback-reason

[pchila]

Not sure why lint is still failing since the replace directive in go.mod pointing to my fork of elastic-agent-libs has been removed (I merged my changes there and created a new release) 🤔

Edit:
@v1v @dliappis @pkoutsovasilis it seems that the linting action is working on a commit that contains obsolete code. Could you take a look please ?

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elastic-sonarqube]

Quality Gate passed

Issues
1 New issue
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
83.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6e3e3ca

History

• 💚 Build #23154 succeeded 98d4f2b
• 💔 Build #23114 failed 09286ef
• 💔 Build #23098 failed 09f53c0
• 💔 Build #23090 failed ad6365a
• 💔 Build #23088 failed 23c0f74
• 💔 Build #23078 failed 83c694e

cc @pchila

[pkoutsovasilis]

LGTM thanks for all the testing, this was helpful to me