Skip to content

[Perf] Changes at 6/2/2022 5:58:32 PM #5882

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f873951b78d0a8123f4cd2e28cba08f08d67fbc4
Compare 6a8f16e54fe2f91ef6457c94cab830f3c31baf9d
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 9.56 ns 12.48 ns 1.31 0.14 False
Replace_String - Duration of single invocation 27.97 ns 32.97 ns 1.18 0.03 False
Join_List - Duration of single invocation 52.66 ns 57.81 ns 1.10 0.03 False
Replace_Char - Duration of single invocation 14.13 ns 16.25 ns 1.15 0.05 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.48171244032753 > 9.79527240235181.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/21/2022 5:54:30 PM, 4/25/2022 5:27:54 PM, 5/9/2022 11:56:12 AM, 6/2/2022 2:22:06 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -7.102614423485864 (T) = (0 -10.613679391073983) / Math.Sqrt((0.04245840898994725 / (41)) + (0.9519724387118657 / (31))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (31) - 2, .025) and -0.1353701921654341 = (9.348210358447979 - 10.613679391073983) / 9.348210358447979 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "b")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.967502422367765 > 29.744048869627747.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -24.67038892438677 (T) = (0 -31.324022711953933) / Math.Sqrt((0.22162202628883687 / (21)) + (0.21674932250100817 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.10598075676858423 = (28.32239396594509 - 31.324022711953933) / 28.32239396594509 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_List

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.80739646206523 > 55.42526223196435.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/9/2022 11:56:12 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -20.32129058923329 (T) = (0 -55.86920185112732) / Math.Sqrt((0.279862609125997 / (21)) + (0.3582736445657471 / (51))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (51) - 2, .025) and -0.05472942878286732 = (52.9701744603818 - 55.86920185112732) / 52.9701744603818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'l', newChar: '!')

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.254620790257817 > 14.972427969793442.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/11/2022 9:09:01 PM, 5/30/2022 12:57:05 PM, 6/6/2022 10:51:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -20.58416624944336 (T) = (0 -16.260182765561293) / Math.Sqrt((0.1614294533990822 / (67)) + (0.013203728047660549 / (5))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (67) + (5) - 2, .025) and -0.09885533364165196 = (14.797382574168681 - 16.260182765561293) / 14.797382574168681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline f873951b78d0a8123f4cd2e28cba08f08d67fbc4
Compare 6a8f16e54fe2f91ef6457c94cab830f3c31baf9d
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleParallel - Duration of single invocation 44.80 ns 51.69 ns 1.15 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleParallel(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 51.692970238095235 > 47.29653061398239.
IsChangePoint: Marked as a change because one of 5/8/2022 12:35:08 PM, 6/2/2022 2:22:06 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -9.413787726023381 (T) = (0 -48.734152386496504) / Math.Sqrt((1.8922632022413566 / (41)) + (1.8975475456607398 / (31))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (31) - 2, .025) and -0.06757045041086823 = (45.649589090575276 - 48.734152386496504) / 45.649589090575276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f873951b78d0a8123f4cd2e28cba08f08d67fbc4
Compare 6a8f16e54fe2f91ef6457c94cab830f3c31baf9d
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ObjrefValueTypeObj - Duration of single invocation 779.82 μs 890.99 μs 1.14 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 890.9868287037037 > 818.7193515624999.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/28/2022 4:25:37 PM, 5/11/2022 3:41:15 PM, 5/28/2022 3:04:49 PM, 6/2/2022 2:22:06 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -12.589728107864358 (T) = (0 -941559.8861007346) / Math.Sqrt((3086336224.1759415 / (41)) + (279880439.2422121 / (31))) is less than -1.9944371117706434 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (31) - 2, .025) and -0.1399528279202919 = (825963.9022243564 - 941559.8861007346) / 825963.9022243564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions