-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Closed
Labels
arch-x64area-System.Buffersruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Milestone
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 261574bf4121b40c7023c73571fb7a690397bd0f |
| Compare | eeb0c1551b5fc9ccd6ceeaf5d2b1504afab249d7 |
| Diff | Diff |
Regressions in System.Memory.ReadOnlySequence
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Slice_StartPosition_And_EndPosition - Duration of single invocation | 13.78 ns | 15.44 ns | 1.12 | 0.08 | False | |||||
| Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation | 53.63 ns | 60.54 ns | 1.13 | 0.00 | False | |||||
| Slice_StartPosition_And_Length - Duration of single invocation | 13.76 ns | 14.79 ns | 1.08 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'Payloads
Histogram
System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.440949852696159 > 14.315615126726636.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -12.982560291189806 (T) = (0 -15.592018757223562) / Math.Sqrt((0.19567802268918302 / (33)) + (0.2793781352398622 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.10793883942820115 = (14.072995911281966 - 15.592018757223562) / 14.072995911281966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.53911815824025 > 56.51169690618845.
IsChangePoint: Marked as a change because one of 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -78.32202883540016 (T) = (0 -60.54882126165877) / Math.Sqrt((0.08802749003401368 / (33)) + (0.15377110808905917 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.1207642615963715 = (54.02458245359775 - 60.54882126165877) / 54.02458245359775 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment: Multiple)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.79136944665426 > 14.5302685753443.
IsChangePoint: Marked as a change because one of 4/25/2022 5:27:54 PM, 5/11/2022 9:09:01 PM, 5/25/2022 1:42:32 AM, 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -13.592933460548643 (T) = (0 -14.670164875060912) / Math.Sqrt((0.00887993704715489 / (33)) + (0.1112432532953855 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.05675016350483825 = (13.882339820422224 - 14.670164875060912) / 13.882339820422224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 261574bf4121b40c7023c73571fb7a690397bd0f |
| Compare | eeb0c1551b5fc9ccd6ceeaf5d2b1504afab249d7 |
| Diff | Diff |
Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IterateTryGetTenSegments - Duration of single invocation | 44.31 ns | 49.51 ns | 1.12 | 0.05 | False | |||||
| FirstSpanTenSegments - Duration of single invocation | 5.99 ns | 7.01 ns | 1.17 | 0.02 | False | |||||
| SliceTenSegments - Duration of single invocation | 22.47 ns | 24.25 ns | 1.08 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 49.51110749728125 > 46.54549188272966.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 6/6/2022 10:12:27 PM, 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -25.620612236664037 (T) = (0 -50.76069277618435) / Math.Sqrt((0.09826238373819779 / (33)) + (2.1824784404985293 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.14585725487163467 = (44.299316132418994 - 50.76069277618435) / 44.299316132418994 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanTenSegments
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.010698722159374 > 6.2647678022924715.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 6/17/2022 12:17:51 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -20.117197724859327 (T) = (0 -7.086505846700433) / Math.Sqrt((0.0855579979926397 / (47)) + (0.003745441458601351 / (22))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (22) - 2, .025) and -0.1450260446035094 = (6.1889472995824235 - 7.086505846700433) / 6.1889472995824235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceTenSegments
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.24975770649659 > 23.57585806292471.
IsChangePoint: Marked as a change because one of 5/31/2022 4:05:26 AM, 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -9.622886707602175 (T) = (0 -24.453196855023368) / Math.Sqrt((0.7184939338052545 / (33)) + (0.23922640331470277 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.07105175531117386 = (22.831013285552157 - 24.453196855023368) / 22.831013285552157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.19042 |
| Baseline | 261574bf4121b40c7023c73571fb7a690397bd0f |
| Compare | eeb0c1551b5fc9ccd6ceeaf5d2b1504afab249d7 |
| Diff | Diff |
Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IterateTryGetTenSegments - Duration of single invocation | 39.19 ns | 41.74 ns | 1.07 | 0.07 | False | |||||
| IterateGetPositionTenSegments - Duration of single invocation | 79.13 ns | 85.82 ns | 1.08 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Byte>*'Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetTenSegments
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.742896551494745 > 41.39769712247376.
IsChangePoint: Marked as a change because one of 5/25/2022 1:42:32 AM, 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -49.36296392191485 (T) = (0 -41.88757011156184) / Math.Sqrt((0.046993689410258145 / (33)) + (0.04637099274925687 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.06538501733546524 = (39.316837978745866 - 41.88757011156184) / 39.316837978745866 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionTenSegments
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.82089165007152 > 83.11101442489134.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/20/2022 8:23:28 AM, 6/14/2022 6:55:13 PM, 6/21/2022 6:07:15 AM falls between 6/12/2022 5:56:07 PM and 6/21/2022 6:07:15 AM.
IsRegressionStdDev: Marked as regression because -17.75233327666725 (T) = (0 -83.97774842540112) / Math.Sqrt((0.03171121923477616 / (33)) + (2.6998220065498866 / (36))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (36) - 2, .025) and -0.06186447864681352 = (79.08518470494288 - 83.97774842540112) / 79.08518470494288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-x64area-System.Buffersruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark



_1.png)
_2.png)
_3.png)
_1.png)
_2.png)