[Perf] Windows 10.0.19041/arm64 : Improvement on 8/24/2022 5:19:48 PM #8232
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 03c21467c97e75103d429a0e67504e402f6da17d |
| Compare | 75beeeccaf1583ae412d7ab0a3d6a5ae766db5df |
| Diff | Diff |
Improvements in System.Buffers.Tests.RentReturnArrayPoolTests<Object>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MultipleSerial - Duration of single invocation | 886.31 ns | 752.39 ns | 0.85 | 0.10 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests<Object>*'Payloads
Histogram
System.Buffers.Tests.RentReturnArrayPoolTests<Object>.MultipleSerial(RentalSize: 4096, ManipulateArray: False, Async: False, UseSharedPool: True)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 752.3880666666666 < 842.0063382440476.
IsChangePoint: Marked as a change because one of 5/26/2022 4:20:35 PM, 6/20/2022 7:45:04 PM, 8/21/2022 5:29:20 AM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 17.143538005301775 (T) = (0 -758.7412702380952) / Math.Sqrt((1348.1659102114672 / (299)) + (139.48942968434696 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.064002249498463 = (810.6229633901768 - 758.7412702380952) / 810.6229633901768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 03c21467c97e75103d429a0e67504e402f6da17d |
| Compare | 75beeeccaf1583ae412d7ab0a3d6a5ae766db5df |
| Diff | Diff |
Improvements in Exceptions.Handling
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MultipleNestedTryCatch_LastCatches - Duration of single invocation | 11.58 μs | 10.73 μs | 0.93 | 0.17 | False | |||||
| ThrowAndCatchWhen - Duration of single invocation | 11.71 μs | 10.49 μs | 0.90 | 0.20 | False | |||||
| MultipleNestedTryCatch_FirstCatches - Duration of single invocation | 11.49 μs | 10.47 μs | 0.91 | 0.17 | False | |||||
| ThrowAndCatch - Duration of single invocation | 11.42 μs | 10.52 μs | 0.92 | 0.16 | False | |||||
| ThrowAndCatch_ManyCatchBlocks - Duration of single invocation | 11.85 μs | 10.84 μs | 0.92 | 0.20 | False | |||||
| ThrowAndCatchWhenFinally - Duration of single invocation | 11.59 μs | 10.55 μs | 0.91 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Exceptions.Handling*'Payloads
Histogram
Exceptions.Handling.MultipleNestedTryCatch_LastCatches(kind: Hardware)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.733931472298222 < 11.037708936840573.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 29.739897302630872 (T) = (0 -10596.466537625867) / Math.Sqrt((356563.3918228389 / (299)) + (24027.48304237977 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.11071075641525133 = (11915.658053965915 - 10596.466537625867) / 11915.658053965915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.ThrowAndCatchWhen(kind: Hardware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.491537768276324 < 10.968568374546386.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 27.57577998916394 (T) = (0 -10453.44690772067) / Math.Sqrt((368175.33373679843 / (299)) + (30910.088708736334 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.11073874979832231 = (11755.203440327472 - 10453.44690772067) / 11755.203440327472 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.MultipleNestedTryCatch_FirstCatches(kind: Hardware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.472540969899665 < 10.864256205315822.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 29.135359867289246 (T) = (0 -10369.23943889938) / Math.Sqrt((388496.89637583337 / (299)) + (21455.037076381625 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.11198874179347794 = (11676.92339829304 - 10369.23943889938) / 11676.92339829304 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.ThrowAndCatch(kind: Hardware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.519194881278779 < 10.850589146603163.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 25.58195540340655 (T) = (0 -10362.18341084659) / Math.Sqrt((356875.0391749885 / (299)) + (41211.71992691363 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.11032413137157533 = (11647.144512103634 - 10362.18341084659) / 11647.144512103634 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.ThrowAndCatch_ManyCatchBlocks(kind: Hardware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.841428820655965 < 11.265612177051672.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 19.453370269263083 (T) = (0 -10681.38930886793) / Math.Sqrt((518662.4056597883 / (299)) + (56337.760709492875 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.09866481468694831 = (11850.629469388872 - 10681.38930886793) / 11850.629469388872 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Exceptions.Handling.ThrowAndCatchWhenFinally(kind: Hardware)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.551778824585014 < 10.97943970270374.
IsChangePoint: Marked as a change because one of 6/15/2022 1:30:25 PM, 8/24/2022 9:42:22 AM, 9/1/2022 1:51:24 AM falls between 8/23/2022 7:19:33 AM and 9/1/2022 1:51:24 AM.
IsImprovementStdDev: Marked as improvement because 26.251403696583708 (T) = (0 -10481.864352829314) / Math.Sqrt((313564.5248971796 / (299)) + (37279.56802002336 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.10705031222032328 = (11738.471378933471 - 10481.864352829314) / 11738.471378933471 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository