Skip to content

Conversation

adam-fowler
Copy link
Collaborator

@adam-fowler adam-fowler commented Sep 25, 2025

See #225

I made an attempt at this a while back but it was decided not to merge as the error type didn't contain enough information. Preferably it would contain a stack of labels to show what you were decoding. We still don't have that but this time the error is extensible such that we can add it later if needed.

@adam-fowler adam-fowler marked this pull request as draft September 25, 2025 19:55
Copy link

❌ Pull request has performance regressions ❌

Summary
===============================================================
Threshold deviations for ValkeyBenchmarks:Client: GET benchmark
===============================================================
Malloc (total) (K, %) main pull_request Difference % Threshold %
p25 81 85 5 5

New baseline 'pull_request' is WORSE than the 'main' baseline thresholds.

Full Benchmark Comparison

Comparing results between 'main' and 'pull_request'

Host 'ca388943be88' with 4 'x86_64' processors with 15 GB memory, running:
#18~24.04.1-Ubuntu SMP Sat Jun 28 04:46:03 UTC 2025

ValkeyBenchmarks

Client: GET benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 81 81 83 85 88 88 88 6
pull_request 83 85 86 87 89 89 89 6
Δ 2 4 3 2 1 1 1 0
Improvement % -2 -5 -4 -2 -1 -1 -1 0

Client: GET benchmark | parallel 20 | 20 concurrent connections metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 94 99 104 107 110 113 113 25
pull_request 96 102 103 106 109 114 114 25
Δ 2 3 -1 -1 -1 1 1 0
Improvement % -2 -3 1 1 1 -1 -1 0

Connection: GET benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 4 4 4 4 4 4 4 8
pull_request 4 4 4 4 4 4 4 8
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

Connection: GET benchmark – NoOpTracer metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 7 8 8 10 10 10 10 8
pull_request 7 8 8 10 10 10 10 8
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

Connection: Pipeline array benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 37 37 38 43 43 43 43 5
pull_request 37 37 38 43 45 45 45 5
Δ 0 0 0 0 2 2 2 0
Improvement % 0 0 0 0 -5 -5 -5 0

Connection: Pipeline benchmark metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 36 37 37 37 37 37 37 5
pull_request 36 37 37 37 37 37 37 5
Δ 0 0 0 0 0 0 0 0
Improvement % 0 0 0 0 0 0 0 0

HashSlot – {user}.whatever metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 18
pull_request 0 0 0 0 0 0 0 19
Δ 0 0 0 0 0 0 0 1
Improvement % 0 0 0 0 0 0 0 1

ValkeyCommandEncoder – Command with 7 words metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 744
pull_request 0 0 0 0 0 0 0 742
Δ 0 0 0 0 0 0 0 -2
Improvement % 0 0 0 0 0 0 0 -2

ValkeyCommandEncoder – Simple GET metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 1896
pull_request 0 0 0 0 0 0 0 1869
Δ 0 0 0 0 0 0 0 -27
Improvement % 0 0 0 0 0 0 0 -27

ValkeyCommandEncoder – Simple MGET 15 keys metrics

Malloc (total): results within specified thresholds, fold down for details.

Malloc (total) * p0 p25 p50 p75 p90 p99 p100 Samples
main 0 0 0 0 0 0 0 360
pull_request 0 0 0 0 0 0 0 356
Δ 0 0 0 0 0 0 0 -4
Improvement % 0 0 0 0 0 0 0 -4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant