Skip to content

[Perf] Linux/x64: 3 Improvements on 11/10/2022 12:18:51 PM #9736

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db213657acc53fc48212867d5728e83d9e36a558
Compare 6a676280564ce8e04f10c29fe5ebc346aa462ca1
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 103.12 ms 85.66 ms 0.83 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 85.66000971428572 < 97.9321525325.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 10/6/2022 2:13:23 PM, 10/26/2022 2:14:52 PM, 11/10/2022 9:13:35 AM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsImprovementStdDev: Marked as improvement because 21.800791186053768 (T) = (0 -85860542.00581282) / Math.Sqrt((20116716716563.727 / (299)) + (180427211287.57544 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.06536717850882288 = (91865532.67926654 - 85860542.00581282) / 91865532.67926654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline db213657acc53fc48212867d5728e83d9e36a558
Compare 6a676280564ce8e04f10c29fe5ebc346aa462ca1
Diff Diff

Improvements in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LessThanOrEqualAnyBenchmark - Duration of single invocation 5.95 ns 0.56 ns 0.09 0.14 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAnyBenchmark


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 0.5578050606039437 < 0.6600382816602391.
IsChangePoint: Marked as a change because one of 10/2/2022 4:08:26 PM, 11/10/2022 9:13:35 AM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsImprovementStdDev: Marked as improvement because 4.587206894703643 (T) = (0 -0.49926029634669283) / Math.Sqrt((0.18740673122748874 / (299)) + (0.01000534734667903 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.23382090783980025 = (0.6516234930648602 - 0.49926029634669283) / 0.6516234930648602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline db213657acc53fc48212867d5728e83d9e36a558
Compare 6a676280564ce8e04f10c29fe5ebc346aa462ca1
Diff Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 3.17 ms 2.59 ms 0.82 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: NonBacktracking)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.588720085565477 < 3.0137144128999167.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/10/2022 9:13:35 AM, 11/14/2022 7:25:10 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 7:25:10 PM.
IsImprovementStdDev: Marked as improvement because 19.182612454394107 (T) = (0 -2589758.395110432) / Math.Sqrt((37718688102.800644 / (299)) + (1195594821.327844 / (21))) is greater than 1.967451947860886 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (21) - 2, .975) and 0.09109422338071536 = (2849314.4853178877 - 2589758.395110432) / 2849314.4853178877 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions