Skip to content

[Perf] Changes at 5/30/2022 8:02:30 PM #5741

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 18.04
Baseline 486b4d1a36aef5bbe6a77bb2c3412772e712961e
Compare afacf6a99e1b98331af958caec638bd93dc46ffe
Diff Diff

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt16 - Duration of single invocation 10.78 ns 5.39 ns 0.50 0.76 False
TryParseBool - Duration of single invocation 16.56 ns 1.15 ns 0.07 0.75 False
TryParseUInt16 - Duration of single invocation 16.56 ns 6.55 ns 0.40 0.77 False
TryParseByte - Duration of single invocation 11.17 ns 9.86 ns 0.88 0.57 False
TryParseUInt16 - Duration of single invocation 16.95 ns 7.87 ns 0.46 0.70 False
TryParseUInt16 - Duration of single invocation 17.33 ns 6.93 ns 0.40 0.76 False
TryParseUInt32Hex - Duration of single invocation 16.25 ns 8.09 ns 0.50 0.68 False
TryParseInt16 - Duration of single invocation 16.27 ns 8.09 ns 0.50 0.74 False
TryParseSByte - Duration of single invocation 12.32 ns 11.02 ns 0.89 0.74 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.390942466723847 < 10.242046945361183.
IsChangePoint: Marked as a change because one of 5/12/2022 12:13:09 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 4.442486490391021 (T) = (0 -7.451073949771431) / Math.Sqrt((8.506872078816755 / (299)) + (3.8911188824618006 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.28811798871794575 = (10.466725990663145 - 7.451073949771431) / 10.466725990663145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: true)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.15472238026061 < 8.778549514179586.
IsChangePoint: Marked as a change because one of 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 8.709935879029002 (T) = (0 -2.7170759584087643) / Math.Sqrt((7.55233344367303 / (299)) + (3.582687052990574 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.6759261477863492 = (8.384125839987512 - 2.7170759584087643) / 8.384125839987512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.545813817855214 < 10.975991555464086.
IsChangePoint: Marked as a change because one of 5/13/2022 2:27:51 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 10.693979116918369 (T) = (0 -6.484345111133795) / Math.Sqrt((9.626767771749465 / (299)) + (0.9332209018812935 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3780876871677709 = (10.426462022601974 - 6.484345111133795) / 10.426462022601974 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.85968658514099 < 10.24274620254418.
IsChangePoint: Marked as a change because one of 5/12/2022 12:48:56 AM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 4.000064021725237 (T) = (0 -7.214503313253841) / Math.Sqrt((8.086167217735246 / (299)) + (4.301945741143833 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2826521709461865 = (10.057189861116354 - 7.214503313253841) / 10.057189861116354 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 65535)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.868914993411778 < 16.098240301599045.
IsChangePoint: Marked as a change because one of 5/12/2022 12:48:56 AM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 6.414829884304966 (T) = (0 -8.20950616200433) / Math.Sqrt((9.271827487061138 / (299)) + (2.1824311607323366 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2900988896597187 = (11.564295424286934 - 8.20950616200433) / 11.564295424286934 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 12345)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.932224071872212 < 14.267176068885467.
IsChangePoint: Marked as a change because one of 5/12/2022 12:13:09 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 2.5663263342096037 (T) = (0 -9.52534278896083) / Math.Sqrt((8.161972047470423 / (299)) + (5.429227090415783 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17623516370914508 = (11.56318207493579 - 9.52534278896083) / 11.56318207493579 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 0)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.086948872743806 < 10.060131040623139.
IsChangePoint: Marked as a change because one of 5/16/2022 8:28:13 AM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 8.328526895412972 (T) = (0 -6.931582492959426) / Math.Sqrt((8.472663533235636 / (299)) + (1.669125850069101 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.357146647716058 = (10.782525234305403 - 6.931582492959426) / 10.782525234305403 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.087271461313206 < 13.53438811212587.
IsChangePoint: Marked as a change because one of 5/13/2022 2:27:51 PM, 5/30/2022 8:02:30 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 2.9731752458608876 (T) = (0 -10.243630063622911) / Math.Sqrt((8.81893008128026 / (299)) + (5.7010868090261955 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.20002460499501046 = (12.804931411120489 - 10.243630063622911) / 12.804931411120489 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.015243588700532 < 11.705952479697734.
IsChangePoint: Marked as a change because one of 5/14/2022 8:55:12 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 3.7420754850836926 (T) = (0 -7.813110333745113) / Math.Sqrt((9.160951031220463 / (299)) + (6.141073240386201 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.28795977262784134 = (10.972849613539422 - 7.813110333745113) / 10.972849613539422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions