Skip to content

[Perf] Windows/x64: 26 Improvements on 4/25/2023 9:59:52 PM #17335

@performanceautofiler

Description

@performanceautofiler

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryAndFinallyDeep - Duration of single invocation 26.37 μs 23.67 μs 0.90 0.01 False
TryAndFinallyDeep - Duration of single invocation 23.79 μs 22.40 μs 0.94 0.01 False
TryAndCatchDeep_CaugtAtTheTop - Duration of single invocation 25.53 μs 23.92 μs 0.94 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'

Payloads

Baseline
Compare

Histogram

Exceptions.Handling.TryAndFinallyDeep(kind: Hardware)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.667180944055943 < 24.48372064336737.
IsChangePoint: Marked as a change because one of 3/5/2023 11:54:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 43.8479109703613 (T) = (0 -23758.081085433227) / Math.Sqrt((272511.9460629169 / (299)) + (27887.386469907655 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07396702137324568 = (25655.761332242066 - 23758.081085433227) / 25655.761332242066 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndFinallyDeep(kind: Software)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 22.40473842313179 < 22.63437085798244.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 32.277550726615736 (T) = (0 -22087.12661522816) / Math.Sqrt((245777.03447302387 / (299)) + (43136.21649564222 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06562044863525396 = (23638.28123482359 - 22087.12661522816) / 23638.28123482359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Exceptions.Handling.TryAndCatchDeep_CaugtAtTheTop(kind: Hardware)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.91791336982441 < 24.27221944330137.
IsChangePoint: Marked as a change because one of 3/5/2023 5:50:49 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 49.28958632238853 (T) = (0 -23509.54896186874) / Math.Sqrt((272303.4330807403 / (299)) + (23447.656416563746 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0799794457037753 = (25553.28666526642 - 23509.54896186874) / 25553.28666526642 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 30.28 μs 23.78 μs 0.79 0.06 False
IDictionary - Duration of single invocation 4.91 μs 3.80 μs 0.77 0.02 True
ImmutableDictionary - Duration of single invocation 10.25 μs 9.55 μs 0.93 0.01 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.779260025062655 < 28.47579826520276.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 4/18/2023 7:42:00 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 9.757945067323757 (T) = (0 -24667.401140217615) / Math.Sqrt((1629264.0246366533 / (299)) + (630415.8634918083 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06123194900550584 = (26276.353476330958 - 24667.401140217615) / 26276.353476330958 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<Int32, Int32>.IDictionary(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.7979641365024754 < 4.69967712061654.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 94.3389749486866 (T) = (0 -3888.599183783336) / Math.Sqrt((13282.179108354563 / (299)) + (1190.3208909346256 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.18319616298755712 = (4760.750387763052 - 3888.599183783336) / 4760.750387763052 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableDictionary(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.545371070133012 < 9.758154885813289.
IsChangePoint: Marked as a change because one of 3/14/2023 5:06:59 PM, 4/17/2023 1:30:24 AM, 4/19/2023 8:38:27 PM, 5/1/2023 3:42:23 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 22.67721728501366 (T) = (0 -9524.164486876845) / Math.Sqrt((59778.904441200946 / (299)) + (2512.161694511214 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.06411955690919177 = (10176.689295293585 - 9524.164486876845) / 10176.689295293585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 266.10 ns 237.17 ns 0.89 0.07 False
Stack - Duration of single invocation 64.14 ns 58.85 ns 0.92 0.04 False
SortedList - Duration of single invocation 126.30 ns 119.71 ns 0.95 0.06 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<Int32>.HashSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 237.16766378908096 < 252.6114218710926.
IsChangePoint: Marked as a change because one of 2/27/2023 2:37:44 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 48.384130224951136 (T) = (0 -235.68353612357237) / Math.Sqrt((44.46805965341196 / (299)) + (5.945291695576023 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1088128515325354 = (264.46020516438887 - 235.68353612357237) / 264.46020516438887 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.Stack(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 58.84615520241454 < 61.0128692322338.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 28.356121527657564 (T) = (0 -60.18614668582785) / Math.Sqrt((1.826095759403877 / (299)) + (0.5049114306081608 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0673836122346056 = (64.53472990115209 - 60.18614668582785) / 64.53472990115209 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<Int32>.SortedList(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.70958294074771 < 120.46053319111562.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 3.7658451986438664 (T) = (0 -120.66427810901405) / Math.Sqrt((8.042645462085032 / (299)) + (87.44800963959754 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.051625919546695265 = (127.23278777435463 - 120.66427810901405) / 127.23278777435463 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AllocateUninitializedArray - Duration of single invocation 158.72 ns 140.51 ns 0.89 0.05 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: False)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 140.50508176428428 < 153.41142203322082.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 33.096844401607264 (T) = (0 -138.7679227686109) / Math.Sqrt((38.84317622787117 / (299)) + (4.8379256689625505 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11498302190739224 = (156.79690469631905 - 138.7679227686109) / 156.79690469631905 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 392.92 ns 329.03 ns 0.84 0.06 False
Queue - Duration of single invocation 122.66 ns 113.06 ns 0.92 0.06 False
List - Duration of single invocation 123.33 ns 112.71 ns 0.91 0.05 False
Dictionary - Duration of single invocation 451.26 ns 398.80 ns 0.88 0.04 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorGivenSize<String>.HashSet(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 329.03160333797115 < 332.43746172814826.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 14.310216482193265 (T) = (0 -328.96496221041735) / Math.Sqrt((179.97077250228153 / (299)) + (90.46599075113991 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07848041895660136 = (356.98097900203476 - 328.96496221041735) / 356.98097900203476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<String>.Queue(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.06373357643109 < 116.68051716860782.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 5.733217806028427 (T) = (0 -115.62000028708721) / Math.Sqrt((10.219015939156423 / (299)) + (51.995718901952735 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06281669936676806 = (123.36967614442723 - 115.62000028708721) / 123.36967614442723 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<String>.List(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 112.70915879590446 < 117.1153629898588.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 9.819711602422664 (T) = (0 -115.43485506288754) / Math.Sqrt((29.155766334681125 / (299)) + (16.147123527194264 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06438020887722545 = (123.37795347869022 - 115.43485506288754) / 123.37795347869022 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.CtorGivenSize<String>.Dictionary(Size: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 398.79892961372383 < 418.5702209471429.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 19.46403303228649 (T) = (0 -405.8547723088656) / Math.Sqrt((148.98911890953946 / (299)) + (58.92340825405066 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.0708772635635791 = (436.8150260378844 - 405.8547723088656) / 436.8150260378844 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 145.82 μs 107.96 μs 0.74 0.01 True
ReadLine - Duration of single invocation 23.93 μs 21.45 μs 0.90 0.02 False
ReadLine - Duration of single invocation 78.36 μs 69.33 μs 0.88 0.01 True
ReadLine - Duration of single invocation 113.62 μs 101.14 μs 0.89 0.03 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 107.95675066312998 < 138.5197923309949.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 138.78418400880423 (T) = (0 -108496.89526241974) / Math.Sqrt((4639842.014497902 / (299)) + (1645367.8203059884 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.2558634783202876 = (145802.40601215707 - 108496.89526241974) / 145802.40601215707 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.450794127747255 < 22.628731778870474.
IsChangePoint: Marked as a change because one of 4/25/2023 12:29:35 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 66.65626346859676 (T) = (0 -21098.683504396693) / Math.Sqrt((367479.66562281817 / (299)) + (16140.109492701935 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.1177754919855381 = (23915.322361517112 - 21098.683504396693) / 23915.322361517112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8])


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 69.33491203703704 < 74.51960145682338.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 141.89965089286167 (T) = (0 -69010.27594408779) / Math.Sqrt((492764.0536256683 / (299)) + (90635.52036622605 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.12565599518921156 = (78928.05985330898 - 69010.27594408779) / 78928.05985330898 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 101.14195564516129 < 107.88267691660697.
IsChangePoint: Marked as a change because one of 3/6/2023 11:36:59 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 27.674779898138144 (T) = (0 -103396.85691147698) / Math.Sqrt((3462932.4756100858 / (299)) + (4229282.352419844 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09600053169296012 = (114377.12137719824 - 103396.85691147698) / 114377.12137719824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 10.10 μs 9.15 μs 0.91 0.03 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryAddGiventSize<String>.Dictionary(Count: 512)


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.147549650025297 < 9.618575179190486.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 7:27:15 AM, 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 30.085473550039353 (T) = (0 -9188.53904361235) / Math.Sqrt((72817.31994984383 / (299)) + (20731.035118788717 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.09203495374006408 = (10119.925961314835 - 9188.53904361235) / 10119.925961314835 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository


Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 7.49 ns 6.36 ns 0.85 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "0")


Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.36027501696286 < 7.140182690545437.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 53.52905279320354 (T) = (0 -6.1078690619904705) / Math.Sqrt((0.011185647415314057 / (299)) + (0.011403482112376528 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.15388109026245939 = (7.218688758398135 - 6.1078690619904705) / 7.218688758398135 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions