Skip to content

[Perf] Windows/x64: 17 Improvements on 2/20/2023 2:42:44 PM #13531

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 9.92 ns 8.84 ns 0.89 0.05 False
ToArray - Duration of single invocation 30.96 ns 26.04 ns 0.84 0.09 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.842172171900094 < 9.370597397257834.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 1/3/2023 6:29:42 AM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 35.81831524155733 (T) = (0 -8.842962010451465) / Math.Sqrt((0.26620578956999674 / (299)) + (4.656085287170797E-06 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10783678190026882 = (9.911820876550582 - 8.842962010451465) / 9.911820876550582 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.ToArray(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.039744771738253 < 29.722822760695095.
IsChangePoint: Marked as a change because one of 1/5/2023 3:38:13 PM, 2/4/2023 4:58:40 AM, 2/9/2023 12:40:40 PM, 2/17/2023 9:53:52 PM, 2/24/2023 9:01:57 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 12.087410821528222 (T) = (0 -26.118968077487516) / Math.Sqrt((2.5025484876456336 / (299)) + (0.3939264318271335 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.07594817124224046 = (28.26569599737745 - 26.118968077487516) / 28.26569599737745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceCompareTo - Duration of single invocation 22.21 ns 20.03 ns 0.90 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.SequenceCompareTo(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.03297293152366 < 21.384065858043797.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/3/2023 6:29:42 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 17.29138547874804 (T) = (0 -20.615197347951817) / Math.Sqrt((1.2508053821462566 / (299)) + (0.10495397047078142 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07007151708908473 = (22.168583634970453 - 20.615197347951817) / 22.168583634970453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Collections.Tests.Perf_Dictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ContainsValue - Duration of single invocation 3.79 ms 2.98 ms 0.79 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_Dictionary*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_Dictionary.ContainsValue(Items: 3000)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.9750119047619052 < 3.598079131313131.
IsChangePoint: Marked as a change because one of 12/12/2022 5:25:39 PM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 22.557440731525286 (T) = (0 -2989449.829859938) / Math.Sqrt((147576466372.1359 / (299)) + (743033086.4623688 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1469379848889616 = (3504375.7392841103 - 2989449.829859938) / 3504375.7392841103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.06 ms 1.76 ms 0.85 0.12 False
Count - Duration of single invocation 2.14 ms 1.66 ms 0.78 0.06 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: None)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7619405753968254 < 1.9600048203124998.
IsChangePoint: Marked as a change because one of 1/21/2023 6:49:37 AM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 8.581862592197279 (T) = (0 -1741276.832617314) / Math.Sqrt((29464880936.186382 / (299)) + (3289990969.623798 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.0681786666499469 = (1868681.0124395129 - 1741276.832617314) / 1868681.0124395129 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\\n]*", Options: None)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.663747767857143 < 2.0333398723958336.
IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 1/11/2023 2:37:52 AM, 1/21/2023 1:51:00 AM, 2/23/2023 8:54:44 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 25.762764355023524 (T) = (0 -1669013.9254153587) / Math.Sqrt((29490255247.526848 / (299)) + (44992812.710571304 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.13416463220088254 = (1927634.2679993026 - 1669013.9254153587) / 1927634.2679993026 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in Benchstone.BenchI.Puzzle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 462.95 ms 425.41 ms 0.92 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Puzzle*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Puzzle.Test


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 425.4144285714286 < 437.20506066666667.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 12.134563215625082 (T) = (0 -428734964.4057794) / Math.Sqrt((105312103179046.25 / (299)) + (110913610478378.4 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.056402160760541914 = (454361960.76004237 - 428734964.4057794) / 454361960.76004237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CheckObjIsInterfaceNo - Duration of single invocation 55.69 μs 27.85 μs 0.50 0.04 False
CheckIsInstAnyIsInterfaceYes - Duration of single invocation 55.70 μs 27.86 μs 0.50 0.04 False
CheckObjIsInterfaceYes - Duration of single invocation 55.69 μs 27.86 μs 0.50 0.04 False
CheckIsInstAnyIsInterfaceNo - Duration of single invocation 55.71 μs 27.86 μs 0.50 0.04 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.CheckObjIsInterfaceNo


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.852813799762323 < 52.915077300192756.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 22.14513514411128 (T) = (0 -27862.28315147408) / Math.Sqrt((182914257.79318744 / (299)) + (86.64049473685749 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.38334685725541434 = (45183.07168186198 - 27862.28315147408) / 45183.07168186198 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.86470662507427 < 52.912904309325015.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 22.306097784173648 (T) = (0 -27860.28789113844) / Math.Sqrt((182317293.7408383 / (299)) + (65.02471571473511 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.38469014960842013 = (45278.46884526276 - 27860.28789113844) / 45278.46884526276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.85948242933537 < 52.90498472085077.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 22.304421068416733 (T) = (0 -27862.292845589414) / Math.Sqrt((182283234.3668688 / (299)) + (70.77986980036994 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.38463325384851205 = (45277.54062084208 - 27862.292845589414) / 45277.54062084208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.861300505050504 < 52.932477943357064.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 22.144891936363933 (T) = (0 -27862.15984125109) / Math.Sqrt((182959166.9617868 / (299)) + (64.82963688891763 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.383374167425292 = (45184.87285054737 - 27862.15984125109) / 45184.87285054737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetValue - Duration of single invocation 69.71 ns 56.78 ns 0.81 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElement.GetValue


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.7848529716662 < 66.19610231931843.
IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 1/13/2023 4:37:08 PM, 2/9/2023 4:10:34 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 9.662344462387297 (T) = (0 -57.608617879040004) / Math.Sqrt((30.32728134077376 / (299)) + (2.1625114687322617 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.0666935770161214 = (61.72529885185961 - 57.608617879040004) / 61.72529885185961 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDecimal - Duration of single invocation 34.59 ns 30.97 ns 0.90 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.965127896436908 < 32.97213684411503.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 9.808315388350614 (T) = (0 -31.82831531649099) / Math.Sqrt((0.6742893247593533 / (299)) + (1.0803352071264154 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05958073979613359 = (33.84481439649712 - 31.82831531649099) / 33.84481439649712 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 58.45 ns 52.73 ns 0.90 0.06 False
EncodeUtf8 - Duration of single invocation 87.10 ns 79.55 ns 0.91 0.01 False
EncodeUtf8 - Duration of single invocation 74.88 ns 66.35 ns 0.89 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 52.729690276719985 < 55.27106517436818.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/13/2023 6:54:14 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 43.29100301355805 (T) = (0 -52.74656884299926) / Math.Sqrt((10.003633414311203 / (299)) + (0.023441489783390126 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13198828268336282 = (60.76711614684129 - 52.74656884299926) / 60.76711614684129 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 79.5491159954374 < 82.40717038264104.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 1/30/2023 11:52:26 AM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 154.83131103631587 (T) = (0 -79.55752942870504) / Math.Sqrt((0.5938236518414773 / (299)) + (0.013751675869334278 / (14))) is greater than 1.9676211333067681 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (14) - 2, .975) and 0.09586584356594696 = (87.99305817898045 - 79.55752942870504) / 87.99305817898045 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+&lt;World&gt;!,512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 66.35087629878811 < 72.25061349456317.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 1/18/2023 3:20:52 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 16.67514594961958 (T) = (0 -68.01771651094475) / Math.Sqrt((3.5987760318209383 / (299)) + (4.239805723891728 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09157265035746744 = (74.87413994934192 - 68.01771651094475) / 74.87413994934192 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 06dd1262479775558b7b8ce4766ddc0e8ae41214
Compare 9ed6d997beaaebd9850181e8df1cbf4e65d1e9ef
Diff Diff

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetByteCount - Duration of single invocation 5.86 μs 4.55 μs 0.78 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetByteCount(Input: EnglishAllAscii)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.547614324195024 < 5.568188613977487.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/3/2023 6:29:42 AM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsImprovementStdDev: Marked as improvement because 20.261756423192583 (T) = (0 -4550.943856269265) / Math.Sqrt((419863.54923523834 / (299)) + (10.884720193371665 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14300034445560553 = (5310.321686626998 - 4550.943856269265) / 5310.321686626998 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions