Regressions in System.Globalization.Tests.StringSearch

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	c3843c592eb3dfdcb0a7849c6f4ba4e9fc2288a0
Compare	6410312a63f9bd552c7b9d777192368efc95535d
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsPrefix_FirstHalf - Duration of single invocation	158.82 ns	205.13 ns	1.29	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 205.12955646039745 > 166.6854562362098.
IsChangePoint: Marked as a change because one of 4/1/2022 9:47:46 AM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -821.1045760583897 (T) = (0 -205.31193459511871) / Math.Sqrt((0.036342227688011 / (40)) + (0.03480960980686019 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.29410378247662333 = (158.65183100090928 - 205.31193459511871) / 158.65183100090928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

#67370 cc @tarekgh

[tarekgh]

@EgorBo

My change is just doing extra one array access HighCharTable[charB] at the end of the search. I am not sure if this is enough to show this delta regression. And why is this happening on Ubuntu only?

CC @adamsitnik

[EgorBo]

@tarekgh hard to say, it could be a data or loop alignment issue

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	c3843c592eb3dfdcb0a7849c6f4ba4e9fc2288a0
Compare	6410312a63f9bd552c7b9d777192368efc95535d
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsPrefix_FirstHalf - Duration of single invocation	158.82 ns	205.13 ns	1.29	0.00	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 205.12955646039745 > 166.6854562362098.
IsChangePoint: Marked as a change because one of 4/1/2022 9:47:46 AM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -821.1045760583897 (T) = (0 -205.31193459511871) / Math.Sqrt((0.036342227688011 / (40)) + (0.03480960980686019 / (15))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (15) - 2, .025) and -0.29410378247662333 = (158.65183100090928 - 205.31193459511871) / 158.65183100090928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	area-System.Globalization, untriaged, refs/heads/main, ubuntu 18.04, RunKind=micro, Regression, CoreClr, x64
Milestone:	-