Skip to content

[9.1] (backport #10343) feat: utilise continue_on_err in beatsauthextension #10444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[9.1] (backport #10343) feat: utilise continue_on_err in beatsauthextension #10444

wants to merge 1 commit into from

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Oct 9, 2025

What does this PR do?

This PR improves error handling for Elasticsearch output configurations in the Hybrid Elastic Agent by:

  1. Moving partially configuration translation ownership: Relocates some of the Elasticsearch output translation logic from the beats library (libbeat/otelbeat/oteltranslate/outputs/elasticsearch) into the elastic-agent package (internal/pkg/otel/translate/output_elasticsearch.go). In the future we should do a full transition to elastic-agent repo as this gives elastic-agent full control over the translation.

  2. Enabling graceful error handling: Adds continue_on_error: true to the beatsauth extension configuration in getBeatsAuthExtensionConfig(). This prevents the OpenTelemetry collector from exiting on startup when encountering invalid SSL configurations (e.g., missing certificate files) respective PR.

Why is it important?

When an Elasticsearch output has invalid configuration (like a missing SSL certificate), the collector exits with a vague error message that doesn't identify which output caused the failure:

error found during service initialization: failed to build extensions: failed to create extension "beatsauth": failed unpacking config: open /etc/client/cert.pem: no such file or directory

Benefits of this PR:

  • Collector stays running instead of exiting at startup, thus allowing other pipelines that utilise different exporters with valid config to continue push data
  • Errors are surfaced at the exporter level when requests are made, making it clear which output failed
image

Screenshot shows the intended behavior: collector continues running and errors are properly surfaced at the exporter level.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

No disruptive user impact expected.

How to test this PR locally

build and install elastic-agent from this branch with the following configuration

  id: agent-pernode-debug
  outputs:
    default:
      hosts:
      - ${ES_HOST}
      password: ${ES_PASSWORD}
      type: elasticsearch
      username: ${ES_USERNAME}
      # invalid ssl settings
      ssl.certificate: /etc/client/cert.pem
      ssl.enabled: true
      ssl.key: /etc/client/cert.key
      ssl.key_passphrase: null
      ssl.key_passphrase_path: null
      ssl.verification_mode: none
    system:
      hosts:
      - ${ES_HOST}
      password: ${ES_PASSWORD}
      type: elasticsearch
      username: ${ES_USERNAME}
  secret_references: []
  agent:
    monitoring:
      # enable otel collector for self-monitoring
      _runtime_experimental: otel
      enabled: true
      logs: true
      metrics: true
      namespace: default
      use_output: default
  inputs:
    - data_stream:
        namespace: default
      id: system-logs
      streams:
      - data_stream:
          dataset: system.auth
          type: logs
        exclude_files:
        - \.gz$
        ignore_older: 72h
        multiline:
          match: after
          pattern: ^\s
        paths:
        - /var/log/auth.log*
        - /var/log/secure*
        processors:
        - add_locale: null
        tags:
        - system-auth
      - data_stream:
          dataset: system.syslog
          type: logs
        exclude_files:
        - \.gz$
        ignore_older: 72h
        multiline:
          match: after
          pattern: ^\s
        paths:
        - /var/log/messages*
        - /var/log/syslog*
        - /var/log/system*
        processors:
        - add_locale: null
        tags: null
      type: logfile
      use_output: system
    - data_stream:
        namespace: default
      id: system-metrics
      streams:
      - cpu.metrics:
        - percentages
        - normalized_percentages
        data_stream:
          dataset: system.cpu
          type: metrics
        metricsets:
        - cpu
        period: 10s
      - data_stream:
          dataset: system.diskio
          type: metrics
        diskio.include_devices: null
        metricsets:
        - diskio
        period: 10s
      - data_stream:
          dataset: system.filesystem
          type: metrics
        metricsets:
        - filesystem
        period: 1m
        processors:
        - drop_event.when.regexp:
            system.filesystem.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - data_stream:
          dataset: system.fsstat
          type: metrics
        metricsets:
        - fsstat
        period: 1m
        processors:
        - drop_event.when.regexp:
            system.fsstat.mount_point: ^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)
      - condition: ${host.platform} != 'windows'
        data_stream:
          dataset: system.load
          type: metrics
        metricsets:
        - load
        period: 10s
      - data_stream:
          dataset: system.memory
          type: metrics
        metricsets:
        - memory
        period: 10s
      - data_stream:
          dataset: system.network
          type: metrics
        metricsets:
        - network
        network.interfaces: null
        period: 10s
      - data_stream:
          dataset: system.process
          type: metrics
        metricsets:
        - process
        period: 10s
        process.cgroups.enabled: false
        process.cmdline.cache.enabled: true
        process.include_cpu_ticks: false
        process.include_top_n.by_cpu: 5
        process.include_top_n.by_memory: 5
        processes:
        - .*
      - data_stream:
          dataset: system.process_summary
          type: metrics
        metricsets:
        - process_summary
        period: 10s
      - data_stream:
          dataset: system.socket_summary
          type: metrics
        metricsets:
        - socket_summary
        period: 10s
      - data_stream:
          dataset: system.uptime
          type: metrics
        metricsets:
        - uptime
        period: 10s
      type: system/metrics
      use_output: system

Related issues

N/A

This is an automatic backport of pull request #10343 done by [Mergify](https://mergify.com).

@pkoutsovasilis @mergify
feat: utilise continue_on_err in beatsauthextension (#10343)
5e63000 
* feat: rework elasticsearch output translation to otel config to exclude validation errors

* ci: add integration test

(cherry picked from commit 0c0dada)
@mergify mergify bot added the backport label Oct 9, 2025
@mergify mergify bot requested a review from a team as a code owner October 9, 2025 08:42
@mergify mergify bot requested review from blakerouse and pkoutsovasilis and removed request for a team October 9, 2025 08:42
@github-actions github-actions bot added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team skip-changelog labels Oct 9, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Oct 9, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @pkoutsovasilis

@mergify Mergify
Copy link
Contributor Author

mergify bot commented Oct 13, 2025

This pull request has not been merged yet. Could you please review and merge it @pkoutsovasilis? 🙏

@pkoutsovasilis
Copy link
Contributor

after syncing with the team we decided that this shouldn't be merged in 9.1 branch

@mergify mergify bot deleted the mergify/bp/9.1/pr-10343 branch October 16, 2025 13:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blakerouse blakerouse Awaiting requested review from blakerouse blakerouse is a code owner automatically assigned from elastic/elastic-agent-control-plane

@pkoutsovasilis pkoutsovasilis Awaiting requested review from pkoutsovasilis pkoutsovasilis is a code owner automatically assigned from elastic/elastic-agent-control-plane

Assignees

@pkoutsovasilis pkoutsovasilis

Labels

backport skip-changelog Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@elasticmachine @pkoutsovasilis