-
Notifications
You must be signed in to change notification settings - Fork 162
Open
Description
Looking at the performance report linked here:
| Benchmark & Profile | Scenario | % Change | Significance Factor ? |
|---|---|---|---|
| encoding opt | full | -0.80%? | 2.21x |
| encoding opt | incr-full | -0.72%? | 2.36x |
| coercions check | incr-patched: println | -0.64% | 4.82x |
| helloworld debug | incr-full | -0.53%? | 2.07x |
| helloworld debug | incr-unchanged | -0.52% | 4.79x |
| coercions check | incr-unchanged | -0.51% | 7.92x |
| helloworld debug | full | -0.45%? | 1.83x |
| coercions opt | incr-unchanged | -0.43% | 5.67x |
| ripgrep opt | incr-unchanged | -0.42% | 2.73x |
| cranelift-codegen opt | incr-unchanged | 0.38%? | 1.81x |
| coercions debug | incr-unchanged | -0.36% | 3.87x |
| issue-58319 debug | incr-unchanged | -0.34% | 2.09x |
| token-stream-stress debug | full | -0.31% | 1.74x |
| token-stream-stress debug | incr-unchanged | -0.31% | 2.25x |
| coercions doc | full | -0.25% | 2.89x |
| unicode_normalization check | incr-unchanged | -0.24% | 3.79x |
| coercions check | full | -0.24% | 3.79x |
| issue-58319 debug | incr-full | -0.24% | 2.00x |
| tuple-stress debug | incr-unchanged | -0.24% | 1.94x |
| helloworld opt | incr-patched: println | -0.21% | 2.55x |
| helloworld opt | incr-unchanged | -0.20% | 2.19x |
| coercions check | incr-full | -0.20% | 5.63x |
why isn't this classified as a definite improvement in the performance triage script?
Yes, a lot of them have question marks on them (though I'm not sure the performance triage script includes the question marks in its weighting at this point, based on what I saw elsewhere), but even if you remove the question mark cases, you're still left with e.g.
| Benchmark & Profile | Scenario | % Change | Significance Factor ? |
|---|---|---|---|
| coercions check | incr-patched: println | -0.64% | 4.82x |
| helloworld debug | incr-unchanged | -0.52% | 4.79x |
| coercions check | incr-unchanged | -0.51% | 7.92x |
which seems like it should have been enough for being classified as a performance win, not a "probably" ?
Metadata
Metadata
Assignees
Labels
No labels