- 
                Notifications
    
You must be signed in to change notification settings  - Fork 4
 
Description
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| Enumerate - Duration of single invocation | 53.20 ns | 27.77 ns | 0.52 | 0.69 | False | |||
| Enumerate - Duration of single invocation | 369.93 ns | 186.82 ns | 0.51 | 0.61 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>*'Payloads
System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 10)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.767009811934383 < 48.51135011827212.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 45.158235243022695 (T) = (0 -23.226634429046513) / Math.Sqrt((27.928587840573435 / (299)) + (5.310728174697654 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5793778477838206 = (55.21971276755093 - 23.226634429046513) / 55.21971276755093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Tests.Perf_PriorityQueue<Guid, Guid>.Enumerate(Size: 100)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 186.81889550078404 < 358.9407251642224.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 51.08247795927214 (T) = (0 -188.47112714334725) / Math.Sqrt((1222.6535736464969 / (299)) + (119.20043343676014 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.49670320271903945 = (374.47313029122074 - 188.47112714334725) / 374.47313029122074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Collections.Sort<BigStruct>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| LinqOrderByExtension - Duration of single invocation | 49.13 μs | 33.83 μs | 0.69 | 0.60 | False | |||
| List - Duration of single invocation | 32.43 μs | 11.79 μs | 0.36 | 0.57 | True | |||
| Array_ComparerClass - Duration of single invocation | 40.65 μs | 19.60 μs | 0.48 | 0.64 | False | |||
| LinqQuery - Duration of single invocation | 59.53 μs | 35.66 μs | 0.60 | 0.66 | False | |||
| Array - Duration of single invocation | 36.42 μs | 12.84 μs | 0.35 | 0.62 | True | |||
| Array_Comparison - Duration of single invocation | 46.64 μs | 28.00 μs | 0.60 | 0.61 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort<BigStruct>*'Payloads
System.Collections.Sort<BigStruct>.LinqOrderByExtension(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 33.8267808 < 50.763701250000004.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/10/2023 7:23:52 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 37.73706296530533 (T) = (0 -32540.17940499671) / Math.Sqrt((106128500.91085872 / (299)) + (10067686.483912067 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5520612499657567 = (72644.26085599679 - 32540.17940499671) / 72644.26085599679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.790741200000001 < 30.931886379999995.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 68.95025499829599 (T) = (0 -11380.989138461537) / Math.Sqrt((7553376.623052363 / (299)) + (1293564.3942190614 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6815247164731878 = (35735.863117626766 - 11380.989138461537) / 35735.863117626766 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.596763428571432 < 40.316406279999995.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 47.51748572018004 (T) = (0 -23978.794817582417) / Math.Sqrt((45881992.41890829 / (299)) + (3204048.293044588 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5561813587789766 = (54028.36336845275 - 23978.794817582417) / 54028.36336845275 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.LinqQuery(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 35.65762240000001 < 53.091612789999985.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 45.207696025557176 (T) = (0 -34679.99218461538) / Math.Sqrt((99723825.59999678 / (299)) + (5110423.497969243 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5263351638171501 = (73216.31148322731 - 34679.99218461538) / 73216.31148322731 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.8355276 < 33.254545560000004.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 56.79024694709898 (T) = (0 -11838.170568421054) / Math.Sqrt((7682216.299255251 / (299)) + (2048564.7530363554 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6725302835500264 = (36150.42849383457 - 11838.170568421054) / 36150.42849383457 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 27.99535 < 41.20473646.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 42.855528529184426 (T) = (0 -27609.41415384615) / Math.Sqrt((43838063.12540266 / (299)) + (2825047.4865321224 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4835751065165613 = (53462.59350050399 - 27609.41415384615) / 53462.59350050399 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| ReadStructAndReverseBE - Duration of single invocation | 8.73 ns | 2.25 ns | 0.26 | 0.81 | False | |||
| ReadStructFieldByFieldLE - Duration of single invocation | 11.52 ns | 3.56 ns | 0.31 | 0.82 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'Payloads
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructAndReverseBE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.2458085313014173 < 9.815373601300951.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 33.22393548525289 (T) = (0 -2.571899950726367) / Math.Sqrt((5.085633528723349 / (299)) + (1.4116683461262365 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.820728933939 = (14.346430839270157 - 2.571899950726367) / 14.346430839270157 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldLE
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.5602361525446375 < 10.981856694896164.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 13.007618312634781 (T) = (0 -5.329956723993892) / Math.Sqrt((3.7093710063698047 / (299)) + (2.653277204786963 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5317371126630062 = (11.382402637768925 - 5.329956723993892) / 11.382402637768925 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in LinqBenchmarks
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| Count00ForX - Duration of single invocation | 263.81 ms | 151.13 ms | 0.57 | 0.65 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'Payloads
LinqBenchmarks.Count00ForX
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 151.126252 < 234.73689364999998.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 28.444742130926024 (T) = (0 -163599386.07692307) / Math.Sqrt((1523147911669066.5 / (299)) + (233762901286332.75 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.45510550730392907 = (300240483.7447584 - 163599386.07692307) / 300240483.7447584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Text.Json.Document.Tests.Perf_EnumerateObject
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| EnumerateProperties - Duration of single invocation | 1.57 μs | 820.04 ns | 0.52 | 0.68 | False | |||
| EnumerateProperties - Duration of single invocation | 1.60 μs | 800.23 ns | 0.50 | 0.72 | False | |||
| EnumerateProperties - Duration of single invocation | 3.19 μs | 1.66 μs | 0.52 | 0.69 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'Payloads
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 820.037332079527 < 1.5815438474093892.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 40.79242829852285 (T) = (0 -809.7720439214077) / Math.Sqrt((28269.36290027673 / (299)) + (5228.73523691844 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5289161862377417 = (1718.955354152913 - 809.7720439214077) / 1718.955354152913 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 800.2268348623853 < 1.505810155765881.
IsChangePoint: Marked as a change because one of 6/12/2023 3:15:58 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 37.840062189340664 (T) = (0 -816.1386072788356) / Math.Sqrt((33660.43205674317 / (299)) + (5415.826363125424 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5161062503527577 = (1686.6070451908074 - 816.1386072788356) / 1686.6070451908074 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.6642048860345047 < 2.4461029176623055.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 44.789943090032004 (T) = (0 -1469.486631125483) / Math.Sqrt((104858.81853433007 / (299)) + (10833.502536159625 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5119130054188462 = (3010.706385214189 - 1469.486631125483) / 3010.706385214189 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Collections.IterateForEach<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| LinkedList - Duration of single invocation | 2.50 μs | 709.13 ns | 0.28 | 0.71 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<Int32>*'Payloads
System.Collections.IterateForEach<Int32>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 709.1274965633277 < 2.2492125599738677.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 65.70315747817274 (T) = (0 -771.5757730823217) / Math.Sqrt((348940.0316351847 / (299)) + (3763.0358446884675 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.7646966534449643 = (3279.068421161854 - 771.5757730823217) / 3279.068421161854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value | 
|---|---|
| Architecture | arm64 | 
| OS | Windows 10.0.25094 | 
| Queue | AmpereWindows | 
| Baseline | cf50b7d86a1456e9aad09ddeafd5ff8701f9afdc | 
| Compare | bba7a9c67e5b6728833bedbb368f732fb499bcc1 | 
| Diff | Diff | 
| Configs | CompilationMode:tiered, RunKind:micro | 
Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | 
|---|---|---|---|---|---|---|---|---|
| IterateForEachMemory - Duration of single invocation | 33.55 ns | 17.65 ns | 0.53 | 0.76 | False | |||
| IterateForEachTenSegments - Duration of single invocation | 73.48 ns | 58.48 ns | 0.80 | 0.67 | False | |||
| IterateForEachSingleSegment - Duration of single invocation | 34.05 ns | 12.24 ns | 0.36 | 0.76 | False | 
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'Payloads
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachMemory
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 17.64661455124211 < 31.7249708996507.
IsChangePoint: Marked as a change because one of 6/9/2023 7:03:36 PM, 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 30.938608315365528 (T) = (0 -16.73415151822059) / Math.Sqrt((29.27623847410969 / (299)) + (7.107057368901237 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.5974854698434845 = (41.574030909427314 - 16.73415151822059) / 41.574030909427314 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.475582577621616 < 77.07180968952534.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 26.348960510619378 (T) = (0 -51.16439704542401) / Math.Sqrt((66.40722882205698 / (299)) + (20.91632732878024 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.4106760331619641 = (86.81879564468068 - 51.16439704542401) / 86.81879564468068 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.236525218479027 < 27.285728486271278.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/5/2023 5:05:14 PM falls between 6/27/2023 3:42:18 AM and 7/5/2023 5:05:14 PM.
IsImprovementStdDev: Marked as improvement because 52.700870352321616 (T) = (0 -12.069378494147855) / Math.Sqrt((14.46062702543302 / (299)) + (1.0846463415857637 / (13))) is greater than 1.9676459287777595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (13) - 2, .975) and 0.6131841665375054 = (31.201872958796795 - 12.069378494147855) / 31.201872958796795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

















