[Perf] Linux/arm64: 3 Improvements on 5/16/2023 12:30:38 AM #17889
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 4c0f2e7bd4cb86917e362052e0642df600984b6c |
| Compare | 522c281d5e2ccbfa9a114018a7309965571ab4b6 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Boolean
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 9.63 ns | 7.88 ns | 0.82 | 0.21 | False | |||||
| Parse - Duration of single invocation | 1.80 ns | 0.43 ns | 0.24 | 0.49 | False | |||||
| Parse - Duration of single invocation | 1.40 ns | 0.00 ns | 0.00 | 0.64 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'Payloads
Histogram
System.Tests.Perf_Boolean.Parse(value: " True ")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.882743301693134 < 8.765882378766976.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 5:11:34 PM, 5/18/2023 3:09:06 AM falls between 5/8/2023 3:52:55 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 16.718637474956395 (T) = (0 -7.316602221123974) / Math.Sqrt((0.25818246118533783 / (299)) + (0.11725927849292223 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2627070422235325 = (9.92360247572339 - 7.316602221123974) / 9.92360247572339 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Boolean.Parse(value: "False")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0.4278836826658589 < 1.685754427469476.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/15/2023 5:11:34 PM, 5/18/2023 3:09:06 AM falls between 5/8/2023 3:52:55 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 24.62653924760108 (T) = (0 -0.4969439148575317) / Math.Sqrt((0.06313032025334635 / (299)) + (0.018227100079501502 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.7547507875809929 = (2.0262813892690725 - 0.4969439148575317) / 2.0262813892690725 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Boolean.Parse(value: "TRUE")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 0 < 1.2993030528233986.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 5/18/2023 3:09:06 AM falls between 5/8/2023 3:52:55 PM and 5/18/2023 3:09:06 AM.
IsImprovementStdDev: Marked as improvement because 22.68193759411296 (T) = (0 -0.17630352722859483) / Math.Sqrt((0.04715082799163767 / (299)) + (0.01422717409332948 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.8757806789880579 = (1.4192923113115836 - 0.17630352722859483) / 1.4192923113115836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository