Skip to content

Conversation

sbalmos
Copy link
Contributor

@sbalmos sbalmos commented Sep 21, 2025

Summary

This adds proper compression of HEC indexer ack queries, using the sink's compression setting. Usage of the sink's Compressor is adapted from the sink's underlying post-batching request body encoding code.

Vector configuration

data_dir: /tmp
api:
  enabled: true
  address: 127.0.0.1:8686
  playground: false
sources:
  demo_in:
    type: demo_logs
    format: json
    interval: 0.1
transforms:
  parse_msg:
    type: remap
    inputs:
      - demo_in
    source: |-
      .message = parse_json!(.message)
sinks:
  hec_out:
    type: splunk_hec_logs
    inputs:
      - parse_msg
    endpoint: http://localhost:8088
    default_token: *** redacted ***
    encoding:
      codec: json
    compression: gzip

How did you test this PR?

Tested against a local copy of the publicly-available demo Splunk 10 Docker container, ensuring that both events were written to the main index, and Vector increments the event-out count for the sink in vector top. The sink event out counter is only incremented if the indexer ack query validating the event(s) were written by Splunk is successfully sent to, and responded by, the HEC.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@sbalmos sbalmos requested a review from a team as a code owner September 21, 2025 20:30
@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Sep 21, 2025
@sbalmos sbalmos force-pushed the hec_ack_compression branch from 7a1d40b to 64c5006 Compare September 21, 2025 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: sinks Anything related to the Vector's sinks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement splunk_hec sink indexer ack query compression
1 participant