-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Collections.Tests.Perf_BitArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BitArrayGet - Duration of single invocation | 3.13 μs | 4.46 μs | 1.42 | 0.02 | False | |||||
| BitArrayCopyToByteArray - Duration of single invocation | 96.60 ns | 127.13 ns | 1.32 | 0.14 | False | |||||
| BitArrayGet - Duration of single invocation | 23.90 ns | 34.88 ns | 1.46 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'Payloads
Histogram
System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.461169894969991 > 3.289212771233317.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -14.192708936531552 (T) = (0 -4716.691601345517) / Math.Sqrt((9.822328743776907 / (32)) + (174040.5699213365 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.504894189657222 = (3134.234708168993 - 4716.691601345517) / 3134.234708168993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.12562998486031 > 101.52213180950997.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 2/24/2023 12:21:41 AM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -8.126516233947049 (T) = (0 -123.38147901596396) / Math.Sqrt((121.98309277943181 / (26)) + (42.6614140605677 / (20))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (20) - 2, .025) and -0.20782692933852506 = (102.15162124554939 - 123.38147901596396) / 102.15162124554939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Perf_BitArray.BitArrayGet(Size: 4)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.87986771120701 > 24.999256065982223.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -15.458065785909893 (T) = (0 -36.27278652177423) / Math.Sqrt((0.12361635534773109 / (32)) + (8.78341090100157 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.5119226217582704 = (23.991165949743692 - 36.27278652177423) / 23.991165949743692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Indexer - Duration of single invocation | 6.59 ns | 7.75 ns | 1.18 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'Payloads
Histogram
Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.754013429754235 > 6.91930064855726.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -18.06856534409327 (T) = (0 -7.849148221895618) / Math.Sqrt((0.08106357917856769 / (32)) + (0.01803876575879867 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.16592026403053478 = (6.732148384454233 - 7.849148221895618) / 6.732148384454233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Tests.Perf_Random
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ctor_seeded - Duration of single invocation | 288.88 ns | 334.54 ns | 1.16 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'Payloads
Histogram
System.Tests.Perf_Random.ctor_seeded
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 334.54439670556474 > 303.22118724924195.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 2/9/2023 4:10:34 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -28.13552155284546 (T) = (0 -335.3118175594031) / Math.Sqrt((120.06256187109301 / (32)) + (0.3903661036166245 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.19493348939057553 = (280.6112813278122 - 335.3118175594031) / 280.6112813278122 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| EncodeUtf16 - Duration of single invocation | 94.93 ns | 105.09 ns | 1.11 | 0.01 | False | |||||
| EncodeUtf16 - Duration of single invocation | 39.12 ns | 42.63 ns | 1.09 | 0.07 | False | |||||
| EncodeUtf16 - Duration of single invocation | 44.68 ns | 52.52 ns | 1.18 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'Payloads
Histogram
System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.08947862018871 > 99.79948470820757.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 1/13/2023 4:18:46 AM, 1/19/2023 8:10:32 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -56.346263592222016 (T) = (0 -105.41317440800262) / Math.Sqrt((0.6596366346693843 / (32)) + (0.2051774884768605 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.11158636240394233 = (94.83129514114734 - 105.41317440800262) / 94.83129514114734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,16)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.634198435479036 > 41.106720723386296.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -9.668193430807563 (T) = (0 -43.61043461158647) / Math.Sqrt((1.2671574185008387 / (32)) + (2.4781612593950197 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.11505061278282146 = (39.11072207094556 - 43.61043461158647) / 39.11072207094556 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.5210271986192 > 45.67678367654555.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -29.87002404008044 (T) = (0 -54.94319482671539) / Math.Sqrt((0.5996188002245728 / (32)) + (1.6887168745314194 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.25462849484147976 = (43.792401537681776 - 54.94319482671539) / 43.792401537681776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in Benchstone.BenchI.EightQueens
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Test - Duration of single invocation | 1.52 μs | 1.67 μs | 1.10 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'Payloads
Histogram
Benchstone.BenchI.EightQueens.Test
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.670665690263447 > 1.5945792605011049.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -71.42958256579422 (T) = (0 -1661.3700495909973) / Math.Sqrt((80.9257841259283 / (32)) + (17.92842994860785 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.09160360383729522 = (1521.9536137026407 - 1661.3700495909973) / 1521.9536137026407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in SciMark2.kernel
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| benchFFT - Duration of single invocation | 582.08 ms | 618.81 ms | 1.06 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'Payloads
Histogram
SciMark2.kernel.benchFFT
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 618.8149386428572 > 613.46088937.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -23.223727106585116 (T) = (0 -619396407.0052328) / Math.Sqrt((3549708936860.585 / (32)) + (33141561698981.934 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.06272655567516185 = (582837046.5549564 - 619396407.0052328) / 582837046.5549564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 2.29 ms | 2.44 ms | 1.07 | 0.11 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: NonBacktracking)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4422080998613045 > 2.318360970081688.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -6.462520023890406 (T) = (0 -2415307.357646216) / Math.Sqrt((5513564343.765095 / (32)) + (6282623070.951262 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.07144385544859774 = (2254254.710000612 - 2415307.357646216) / 2254254.710000612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| HashSet - Duration of single invocation | 2.06 μs | 2.43 μs | 1.18 | 0.10 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'Payloads
Histogram
System.Collections.ContainsFalse<Int32>.HashSet(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4325265260126057 > 2.188240009532451.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/12/2022 5:25:39 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -19.194996594965662 (T) = (0 -2420.3929016343386) / Math.Sqrt((7597.081606857718 / (32)) + (433.1677897629344 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.14931029161277198 = (2105.9525171726405 - 2420.3929016343386) / 2105.9525171726405 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Collections.TryGetValueTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 2.45 μs | 3.02 μs | 1.23 | 0.23 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'Payloads
Histogram
System.Collections.TryGetValueTrue<Int32, Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0160619943713036 > 2.583393129227729.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/13/2023 6:54:14 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -13.203644400238474 (T) = (0 -3072.6138806619056) / Math.Sqrt((32902.739846669465 / (32)) + (23682.775345265425 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.2888391207980084 = (2384.0166170308676 - 3072.6138806619056) / 2384.0166170308676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Collections.CreateAddAndRemove<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 9.54 μs | 10.85 μs | 1.14 | 0.36 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<Int32>*'Payloads
Histogram
System.Collections.CreateAddAndRemove<Int32>.Dictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.84762242552295 > 10.440739742627738.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -8.796371030115006 (T) = (0 -11319.443344266809) / Math.Sqrt((185974.49802628352 / (32)) + (147649.02918055665 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.11035901660176996 = (10194.3994465049 - 11319.443344266809) / 10194.3994465049 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | 6387a9eb56098a889021190747d31f07246dd9f2 |
| Compare | cabaab5e90e19256ae404970091350bb6f0f7139 |
| Diff | Diff |
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 288.33 ms | 325.90 ms | 1.13 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 325.899507 > 302.9142510925.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/24/2023 8:06:06 PM, 2/28/2023 7:53:18 AM falls between 2/19/2023 7:20:18 PM and 2/28/2023 7:53:18 AM.
IsRegressionStdDev: Marked as regression because -55.410304359968514 (T) = (0 -325987617.64383835) / Math.Sqrt((13451161913235.465 / (32)) + (47558701461.590576 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.12441372838461481 = (289917856.22554374 - 325987617.64383835) / 289917856.22554374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository











_1.png)
_1.png)
_1.png)
