Regressions in System.Collections.Sort #90933
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 71a3d36e5cdaf1e2d41957fb1d2cebff8af0e063 |
| Compare | 9d53816dd6d2a69bc0b8592014f3b7a5640685c4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.Sort<IntStruct>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 5.11 μs | 8.79 μs | 1.72 | 0.51 | False | ||||
| 5.11 μs | 8.58 μs | 1.68 | 0.53 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<IntStruct>*'Payloads
System.Collections.Sort<IntStruct>.Array(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 8/14/2023 9:42:18 AM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsRegressionStdDev: Marked as regression because -22.302485870313667 (T) = (0 -8993.261476472246) / Math.Sqrt((58368.38135392599 / (4)) + (235091.61196554397 / (13))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (4) + (13) - 2, .025) and -0.81251891080426 = (4961.747666666667 - 8993.261476472246) / 4961.747666666667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<IntStruct>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 8/14/2023 9:42:18 AM, 8/21/2023 10:38:45 PM falls between 8/13/2023 2:50:54 AM and 8/21/2023 10:38:45 PM.
IsRegressionStdDev: Marked as regression because -23.69456434037528 (T) = (0 -8678.07050220088) / Math.Sqrt((93129.27371496327 / (4)) + (15244.993484834415 / (14))) is less than -2.1199052992212764 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (4) + (14) - 2, .025) and -0.7429194893826071 = (4979.042666666666 - 8678.07050220088) / 4979.042666666666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository