feat: Event-time based aggregation support #23694
Open
+440
−30
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#14015
Summary
Event-time handling (absolute metrics in
Auto
andLatest/Diff
):Processing-time path:
Auto
andLatest/Diff
are still replaced by the last arriving sample (no event-timestamp check yet).Other aggregation modes preserved:
Sum
,Count
for incremental metrics unchanged.Max
,Min
,Mean
,Stdev
continue to aggregate multiple samples per bucket.Diff
emits the difference between the current and previous bucket values.New configuration options:
clock
: choose bucket alignment by Processing (Vector wall clock) or Event (event timestamps).allowed_lateness_ms
: wait time for late or out-of-order samples in event-time mode (default: 120s).emit_ts
: set output timestamp to BucketStart or BucketEnd.Important Notes
Vector configuration
prom_aggregate:
type: aggregate
inputs:
- prometheus
interval_ms: 60000
mode: Latest
clock: Event
allowed_lateness_ms: 60000
How did you test this PR?
I built the changes locally (win11 - Build 26100) and verified the results by sending metrics into InfluxDB and inspecting them in Grafana dashboards.
Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changelog
label to this PR.References
Notes
@vectordotdev/vector
to reach out to us regarding this PR.pre-push
hook, please see this template.cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo nextest run --workspace
(alternatively, you can runcargo test --all
)git merge origin master
andgit push
.Cargo.lock
), pleaserun
cargo vdev build licenses
to regenerate the license inventory and commit the changes (if any). More details here.