[Perf] Linux/x64: 3 Regressions on 4/25/2023 8:51:41 AM #17251
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | 620b0db3a987bab2694a0043cacb3e2a86673df5 |
| Compare | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 106.15 ns | 114.73 ns | 1.08 | 0.04 | False | |||||
| Parse - Duration of single invocation | 110.88 ns | 117.34 ns | 1.06 | 0.02 | False | |||||
| TryParse - Duration of single invocation | 103.78 ns | 117.34 ns | 1.13 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'Payloads
Histogram
System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 114.72776452606185 > 112.64234511867348.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -32.884648198755364 (T) = (0 -113.99280178975746) / Math.Sqrt((0.4901504597440322 / (20)) + (0.3947803557275337 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.05930865116405167 = (107.61056436619602 - 113.99280178975746) / 107.61056436619602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.33712036188545 > 116.35218278092339.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/13/2023 2:14:21 PM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -25.09315015042158 (T) = (0 -117.44066061049307) / Math.Sqrt((1.5561279971824895 / (23)) + (0.11046915556988329 / (27))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (27) - 2, .025) and -0.060707276085276124 = (110.71919959286804 - 117.44066061049307) / 110.71919959286804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.33776759573571 > 110.26771743548994.
IsChangePoint: Marked as a change because one of 2/16/2023 1:21:52 PM, 3/13/2023 2:14:21 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsRegressionStdDev: Marked as regression because -36.70681844748822 (T) = (0 -116.9210158086416) / Math.Sqrt((1.6630322126402004 / (20)) + (0.6535415888474102 / (30))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (30) - 2, .025) and -0.11321273031426488 = (105.03025398895167 - 116.9210158086416) / 105.03025398895167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository