[8.19] (backport #10412) fix: do not report agent managed otel extensions statuses

[mergify]

What does this PR do?

This PR ensures that agent-managed extensions (such as beatsauth, elastic_diagnostics, and the Agent’s internal healthcheckv2 extension) are not included in the status output, as these are internal implementation details that should not surface to users.

Key changes:

• Added a generic filtering mechanism in handleOtelStatusUpdate to remove known agent-managed extensions (beatsauth/*, elastic_diagnostics, and agent-injected healthcheckv2) from the status tree.
• Introduced logic to remove the agent-injected healthcheck extension from subprocess status reports while preserving user-defined ones.
• Refactored subprocess reporting to go through a new method reportSubprocessCollectorStatus() which safely clones and publishes collector status.
  • Added a deep clone helper (cloneCollectorStatus) to prevent mutation races on shared status references between the collector and reporting goroutines.
• Adjusted unit tests to verify:
  • Managed extensions are hidden from the reported status.
  • User-defined healthcheck extensions remain visible.
  • No race conditions occur when mutating collector status during reporting.

Why is it important?

Previously, all OTel extensions - including agent internal ones - were always visible in elastic-agent status output.
This caused confusion and clutter in the status tree, exposing implementation details irrelevant to users.

By filtering out agent-managed extensions and cloning the status tree before reporting, this change ensures cleaner, more user-focused status output.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

Since a release with otel runtime enabled by default is not yet published there is no disruptive user impact

How to test this PR locally

mage unitTest

Related issues

N/A

---

This is an automatic backport of pull request #10412 done by Mergify.

[mergify]

Cherry-pick of d8f1dae has failed:

On branch mergify/bp/8.19/pr-10412
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit d8f1daeae.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   internal/pkg/otel/manager/manager_test.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/otel/manager/execution_subprocess.go
	both modified:   internal/pkg/otel/manager/manager.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[pkoutsovasilis]

@swiatekm @cmacknz don't we need #10240 in 8.19?

[mergify]

This pull request has not been merged yet. Could you please review and merge it @pkoutsovasilis? 🙏

[pierrehilbert]

According to me, yes we should have it in 8.19 even if it shouldn't be use there.

[swiatekm]

I agree with Pierre. It was blocked by #10232, but now it can go in.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.19/pr-10412 upstream/mergify/bp/8.19/pr-10412
git merge upstream/8.19
git push upstream mergify/bp/8.19/pr-10412

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 37d12f3

History

• 💔 Build #28618 failed b03f5e6
• 💔 Build #28435 failed 16d2efd

cc @pkoutsovasilis

[pkoutsovasilis]

thanks for the assist @ycombinator 🙏