Skip to content

[Perf] Changes at 1/31/2022 6:18:59 PM #3289

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 215c328e039b3a6b6bfdc6514253307105f25816
Compare 09f681f7aa481d441c45ebeb9807694aa6df547e
Diff Diff

Improvements in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 1.64 ns 0.32 ns 0.19 0.28 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Boolean.ToString(value: True)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.3155690640236791 < 1.5546685986048363.
IsChangePoint: Marked as a change because one of 10/28/2021 4:25:04 PM, 11/2/2021 10:51:49 PM, 11/8/2021 11:30:55 AM, 11/22/2021 3:34:51 PM, 1/31/2022 1:37:35 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 84.9585642922736 (T) = (0 -0.31567126949577307) / Math.Sqrt((0.08959401680309258 / (299)) + (2.028954752073215E-06 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.8233394436373626 = (1.78688030874183 - 0.31567126949577307) / 1.78688030874183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.19041
Baseline 215c328e039b3a6b6bfdc6514253307105f25816
Compare 09f681f7aa481d441c45ebeb9807694aa6df547e
Diff Diff

Improvements in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 28.79 μs 23.21 μs 0.81 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.210580882352946 < 27.34278708179865.
IsChangePoint: Marked as a change because one of 12/25/2021 1:34:58 AM, 1/31/2022 1:37:35 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 87.39603729780188 (T) = (0 -23169.61061491225) / Math.Sqrt((254111.73057435657 / (299)) + (26714.07840657594 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.18901972048868992 = (28569.881660838982 - 23169.61061491225) / 28569.881660838982 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 215c328e039b3a6b6bfdc6514253307105f25816
Compare 09f681f7aa481d441c45ebeb9807694aa6df547e
Diff Diff

Improvements in System.Drawing.Tests.Perf_Color

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FromArgb_Channels - Duration of single invocation 2.96 ns 0.52 ns 0.18 0.18 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Drawing.Tests.Perf_Color*'

Payloads

Baseline
Compare

Histogram

System.Drawing.Tests.Perf_Color.FromArgb_Channels


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5206726808968822 < 2.8155664776462657.
IsChangePoint: Marked as a change because one of 1/31/2022 1:37:35 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 41.08534722521091 (T) = (0 -0.6575099431481537) / Math.Sqrt((0.029947837083067756 / (299)) + (0.026562526281503664 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.7753715133660191 = (2.9270995544724845 - 0.6575099431481537) / 2.9270995544724845 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 215c328e039b3a6b6bfdc6514253307105f25816
Compare 09f681f7aa481d441c45ebeb9807694aa6df547e
Diff Diff

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 30.46 μs 24.25 μs 0.80 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.ImmutableHashSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.254151066042645 < 29.247097450946175.
IsChangePoint: Marked as a change because one of 12/28/2021 8:42:38 AM, 1/31/2022 1:37:35 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 42.07244232869073 (T) = (0 -24631.542803776247) / Math.Sqrt((1729114.0266752157 / (299)) + (105603.81878057461 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1843823900841282 = (30199.86633971391 - 24631.542803776247) / 30199.86633971391 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.19041
Baseline 215c328e039b3a6b6bfdc6514253307105f25816
Compare 09f681f7aa481d441c45ebeb9807694aa6df547e
Diff Diff

Improvements in System.Text.Json.Tests.Perf_Ctor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor - Duration of single invocation 26.51 ns 20.37 ns 0.77 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Ctor*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Ctor.Ctor(Formatted: True, SkipValidation: False)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 20.367052147941074 < 25.28777589370507.
IsChangePoint: Marked as a change because one of 11/22/2021 6:45:42 PM, 1/20/2022 9:28:17 PM, 1/31/2022 1:37:35 PM, 2/2/2022 11:04:24 PM falls between 1/25/2022 5:49:56 AM and 2/2/2022 11:04:24 PM.
IsImprovementStdDev: Marked as improvement because 52.37093008835673 (T) = (0 -19.962086122037054) / Math.Sqrt((0.8356211589963407 / (299)) + (0.17278816585848109 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2800951988931682 = (27.72878593300941 - 19.962086122037054) / 27.72878593300941 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions