[Perf] Linux/x64: 447 Improvements on 3/11/2023 1:35:17 AM #14064
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
| Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
| Diff | Diff |
Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| CreateInstance - Duration of single invocation | 20.85 μs | 17.17 μs | 0.82 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.CreateInstance
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.171230598669624 < 19.5119985362668.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 40.88034086052476 (T) = (0 -17230.601921141508) / Math.Sqrt((219731.44459378414 / (299)) + (65752.96117323592 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.15756098303064664 = (20453.23349709992 - 17230.601921141508) / 20453.23349709992 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b9002422acc79f65a2f1b83f342ce52eb1ce2326 |
| Compare | 2bacbca96ddcf7ae809a0e32a0d5dbffd6354199 |
| Diff | Diff |
Improvements in System.Diagnostics.Perf_Activity
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'Payloads
Histogram
System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsSmall
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 526.7314886004132 < 687.9240937335404.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 91.7162233648849 (T) = (0 -525.2760891628732) / Math.Sqrt((957.6305585800761 / (299)) + (6.681048778942418 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.25292406917172483 = (703.10937280566 - 525.2760891628732) / 703.10937280566 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 539.675132529471 < 699.7038151528635.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 61.14322073376243 (T) = (0 -542.4500134832044) / Math.Sqrt((787.0463550298581 / (299)) + (65.53861350757117 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2428114715224693 = (716.4002003225032 - 542.4500134832044) / 716.4002003225032 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.67462182505709 < 122.76101403548154.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 1/28/2023 8:08:27 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 27.599143285951307 (T) = (0 -92889.30932704662) / Math.Sqrt((11772410.07938605 / (299)) + (16383250.73542031 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2604305322261224 = (125599.16731912384 - 92889.30932704662) / 125599.16731912384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 559.6156008449531 < 611.3997724442542.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 43.807742474734205 (T) = (0 -550.4813747036407) / Math.Sqrt((1579.3069154374211 / (299)) + (54.826481392518325 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.19985443831283872 = (687.9765395972619 - 550.4813747036407) / 687.9765395972619 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinkTagsLarge
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.6148976256983 < 120.49890022347557.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 57.04010084860241 (T) = (0 -89080.66118338755) / Math.Sqrt((12529679.67945038 / (299)) + (3451139.27156117 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2687690580217057 = (121822.88257986736 - 89080.66118338755) / 121822.88257986736 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 542.291080029281 < 697.3112096588249.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 66.42047300678459 (T) = (0 -539.9252094080067) / Math.Sqrt((790.3247438429504 / (299)) + (52.987734785882616 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.24632996024221374 = (716.3946832509454 - 539.9252094080067) / 716.3946832509454 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsLarge
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.35001075619296 < 119.66130044552851.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 54.6635785670049 (T) = (0 -89096.84557456967) / Math.Sqrt((14420675.784917979 / (299)) + (3765552.7883988065 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.26962101139952893 = (121987.14224418504 - 89096.84557456967) / 121987.14224418504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagObjectsSmall
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 523.6626131770176 < 682.2766556176398.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 46.540096648859844 (T) = (0 -524.7926845551136) / Math.Sqrt((831.6921770826424 / (299)) + (137.80529689943776 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.25090862072336295 = (700.5723187762242 - 524.7926845551136) / 700.5723187762242 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksLarge
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 96.39040245132955 < 122.9115599441042.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 50.26593618554598 (T) = (0 -93300.5189911641) / Math.Sqrt((12894089.688763965 / (299)) + (4478718.394223835 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.25795746601634567 = (125734.73179533308 - 93300.5189911641) / 125734.73179533308 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagsLarge
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 74.0586489435229 < 93.12594952233043.
IsChangePoint: Marked as a change because one of 2/1/2023 12:50:52 AM, 2/28/2023 5:58:27 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsImprovementStdDev: Marked as improvement because 42.35026417547688 (T) = (0 -74783.40624518858) / Math.Sqrt((89203892.33010742 / (299)) + (4860314.281795838 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.32200788601919894 = (110301.292158254 - 74783.40624518858) / 110301.292158254 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository