-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Exceptions.Handling
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryAndFinallyDeep - Duration of single invocation | 26.37 μs | 23.67 μs | 0.90 | 0.01 | False | |||||
| TryAndFinallyDeep - Duration of single invocation | 23.79 μs | 22.40 μs | 0.94 | 0.01 | False | |||||
| TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation | 25.53 μs | 23.92 μs | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'Payloads
Histogram
Exceptions.Handling.TryAndFinallyDeep(kind: Hardware)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.667180944055943 < 24.48372064336737.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 43.8479109703613 (T) = (0 -23758.081085433227) / Math.Sqrt((272511.9460629169 / (299)) + (27887.386469907655 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07396702137324568 = (25655.761332242066 - 23758.081085433227) / 25655.761332242066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Exceptions.Handling.TryAndFinallyDeep(kind: Software)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.40473842313179 < 22.63437085798244.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 32.277550726615736 (T) = (0 -22087.12661522816) / Math.Sqrt((245777.03447302387 / (299)) + (43136.21649564222 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06562044863525396 = (23638.28123482359 - 22087.12661522816) / 23638.28123482359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Hardware)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.91791336982441 < 24.27221944330137.
IsChangePoint: Marked as a change because one of 3/5/2023 5:50:49 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 49.28958632238853 (T) = (0 -23509.54896186874) / Math.Sqrt((272303.4330807403 / (299)) + (23447.656416563746 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0799794457037753 = (25553.28666526642 - 23509.54896186874) / 25553.28666526642 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableSortedDictionary - Duration of single invocation | 30.28 μs | 23.78 μs | 0.79 | 0.06 | False | |||||
| IDictionary - Duration of single invocation | 4.91 μs | 3.80 μs | 0.77 | 0.02 | True | |||||
| ImmutableDictionary - Duration of single invocation | 10.25 μs | 9.55 μs | 0.93 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse<Int32, Int32>*'Payloads
Histogram
System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.779260025062655 < 28.47579826520276.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 4/18/2023 7:42:00 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 9.757945067323757 (T) = (0 -24667.401140217615) / Math.Sqrt((1629264.0246366533 / (299)) + (630415.8634918083 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06123194900550584 = (26276.353476330958 - 24667.401140217615) / 26276.353476330958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsKeyFalse<Int32, Int32>.IDictionary(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.7979641365024754 < 4.69967712061654.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 94.3389749486866 (T) = (0 -3888.599183783336) / Math.Sqrt((13282.179108354563 / (299)) + (1190.3208909346256 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.18319616298755712 = (4760.750387763052 - 3888.599183783336) / 4760.750387763052 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.545371070133012 < 9.758154885813289.
IsChangePoint: Marked as a change because one of 3/14/2023 5:06:59 PM, 4/17/2023 1:30:24 AM, 4/19/2023 8:38:27 PM, 5/1/2023 3:42:23 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 22.67721728501366 (T) = (0 -9524.164486876845) / Math.Sqrt((59778.904441200946 / (299)) + (2512.161694511214 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.06411955690919177 = (10176.689295293585 - 9524.164486876845) / 10176.689295293585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.CtorGivenSize<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| HashSet - Duration of single invocation | 266.10 ns | 237.17 ns | 0.89 | 0.07 | False | |||||
| Stack - Duration of single invocation | 64.14 ns | 58.85 ns | 0.92 | 0.04 | False | |||||
| SortedList - Duration of single invocation | 126.30 ns | 119.71 ns | 0.95 | 0.06 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<Int32>*'Payloads
Histogram
System.Collections.CtorGivenSize<Int32>.HashSet(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 237.16766378908096 < 252.6114218710926.
IsChangePoint: Marked as a change because one of 2/27/2023 2:37:44 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 48.384130224951136 (T) = (0 -235.68353612357237) / Math.Sqrt((44.46805965341196 / (299)) + (5.945291695576023 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1088128515325354 = (264.46020516438887 - 235.68353612357237) / 264.46020516438887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorGivenSize<Int32>.Stack(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.84615520241454 < 61.0128692322338.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 28.356121527657564 (T) = (0 -60.18614668582785) / Math.Sqrt((1.826095759403877 / (299)) + (0.5049114306081608 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0673836122346056 = (64.53472990115209 - 60.18614668582785) / 64.53472990115209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorGivenSize<Int32>.SortedList(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.70958294074771 < 120.46053319111562.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 3.7658451986438664 (T) = (0 -120.66427810901405) / Math.Sqrt((8.042645462085032 / (299)) + (87.44800963959754 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.051625919546695265 = (127.23278777435463 - 120.66427810901405) / 127.23278777435463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_GC<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AllocateUninitializedArray - Duration of single invocation | 158.72 ns | 140.51 ns | 0.89 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC<Char>*'Payloads
Histogram
System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: False)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 140.50508176428428 < 153.41142203322082.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 33.096844401607264 (T) = (0 -138.7679227686109) / Math.Sqrt((38.84317622787117 / (299)) + (4.8379256689625505 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11498302190739224 = (156.79690469631905 - 138.7679227686109) / 156.79690469631905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.CtorGivenSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| HashSet - Duration of single invocation | 392.92 ns | 329.03 ns | 0.84 | 0.06 | False | |||||
| Queue - Duration of single invocation | 122.66 ns | 113.06 ns | 0.92 | 0.06 | False | |||||
| List - Duration of single invocation | 123.33 ns | 112.71 ns | 0.91 | 0.05 | False | |||||
| Dictionary - Duration of single invocation | 451.26 ns | 398.80 ns | 0.88 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize<String>*'Payloads
Histogram
System.Collections.CtorGivenSize<String>.HashSet(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 329.03160333797115 < 332.43746172814826.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 14.310216482193265 (T) = (0 -328.96496221041735) / Math.Sqrt((179.97077250228153 / (299)) + (90.46599075113991 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07848041895660136 = (356.98097900203476 - 328.96496221041735) / 356.98097900203476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorGivenSize<String>.Queue(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.06373357643109 < 116.68051716860782.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 5.733217806028427 (T) = (0 -115.62000028708721) / Math.Sqrt((10.219015939156423 / (299)) + (51.995718901952735 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06281669936676806 = (123.36967614442723 - 115.62000028708721) / 123.36967614442723 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorGivenSize<String>.List(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 112.70915879590446 < 117.1153629898588.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 9.819711602422664 (T) = (0 -115.43485506288754) / Math.Sqrt((29.155766334681125 / (299)) + (16.147123527194264 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06438020887722545 = (123.37795347869022 - 115.43485506288754) / 123.37795347869022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.CtorGivenSize<String>.Dictionary(Size: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 398.79892961372383 < 418.5702209471429.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 19.46403303228649 (T) = (0 -405.8547723088656) / Math.Sqrt((148.98911890953946 / (299)) + (58.92340825405066 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0708772635635791 = (436.8150260378844 - 405.8547723088656) / 436.8150260378844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.IO.Tests.StreamReaderReadLineTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ReadLine - Duration of single invocation | 145.82 μs | 107.96 μs | 0.74 | 0.01 | True | |||||
| ReadLine - Duration of single invocation | 23.93 μs | 21.45 μs | 0.90 | 0.02 | False | |||||
| ReadLine - Duration of single invocation | 78.36 μs | 69.33 μs | 0.88 | 0.01 | True | |||||
| ReadLine - Duration of single invocation | 113.62 μs | 101.14 μs | 0.89 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'Payloads
Histogram
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.95675066312998 < 138.5197923309949.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 138.78418400880423 (T) = (0 -108496.89526241974) / Math.Sqrt((4639842.014497902 / (299)) + (1645367.8203059884 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2558634783202876 = (145802.40601215707 - 108496.89526241974) / 145802.40601215707 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.450794127747255 < 22.628731778870474.
IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 66.65626346859676 (T) = (0 -21098.683504396693) / Math.Sqrt((367479.66562281817 / (299)) + (16140.109492701935 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1177754919855381 = (23915.322361517112 - 21098.683504396693) / 23915.322361517112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8])
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.33491203703704 < 74.51960145682338.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 141.89965089286167 (T) = (0 -69010.27594408779) / Math.Sqrt((492764.0536256683 / (299)) + (90635.52036622605 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12565599518921156 = (78928.05985330898 - 69010.27594408779) / 78928.05985330898 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 101.14195564516129 < 107.88267691660697.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 27.674779898138144 (T) = (0 -103396.85691147698) / Math.Sqrt((3462932.4756100858 / (299)) + (4229282.352419844 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09600053169296012 = (114377.12137719824 - 103396.85691147698) / 114377.12137719824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.TryAddGiventSize<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 10.10 μs | 9.15 μs | 0.91 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize<String>*'Payloads
Histogram
System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.147549650025297 < 9.618575179190486.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 7:27:15 AM, 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 30.085473550039353 (T) = (0 -9188.53904361235) / Math.Sqrt((72817.31994984383 / (299)) + (20731.035118788717 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09203495374006408 = (10119.925961314835 - 9188.53904361235) / 10119.925961314835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 2e0033cf25cee5457ee326dc1eaff013b939d00d |
| Compare | 9a7db5556f4ad892c611b2e174594db17fdec3af |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_UInt64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryParseHex - Duration of single invocation | 7.49 ns | 6.36 ns | 0.85 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'Payloads
Histogram
System.Tests.Perf_UInt64.TryParseHex(value: "0")
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.36027501696286 < 7.140182690545437.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 53.52905279320354 (T) = (0 -6.1078690619904705) / Math.Sqrt((0.011185647415314057 / (299)) + (0.011403482112376528 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.15388109026245939 = (7.218688758398135 - 6.1078690619904705) / 7.218688758398135 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository










