-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableHashSet - Duration of single invocation | 9.55 μs | 10.41 μs | 1.09 | 0.04 | False | |||||
| HashSet - Duration of single invocation | 2.52 μs | 2.67 μs | 1.06 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'Payloads
Histogram
System.Collections.ContainsFalse<Int32>.ImmutableHashSet(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.406454386839483 > 10.17372654075638.
IsChangePoint: Marked as a change because one of 2/15/2023 3:19:23 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.954844071377268 (T) = (0 -10513.704699392054) / Math.Sqrt((24111.282769521145 / (26)) + (36718.63049997734 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.07695804401064926 = (9762.408812360467 - 10513.704699392054) / 9762.408812360467 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsFalse<Int32>.HashSet(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.67230336631481 > 2.644351157467016.
IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -35.45124210444055 (T) = (0 -2680.462407508129) / Math.Sqrt((39.969712483490866 / (11)) + (615.910384341103 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.06333194483223217 = (2520.8143332240816 - 2680.462407508129) / 2520.8143332240816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_DateTime
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToString - Duration of single invocation | 29.35 ns | 31.97 ns | 1.09 | 0.03 | False | |||||
| ToString - Duration of single invocation | 41.42 ns | 44.65 ns | 1.08 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'Payloads
Histogram
System.Tests.Perf_DateTime.ToString(format: "r")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.9662792608466 > 30.6985330592904.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 4/9/2023 4:09:45 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.0585577856107 (T) = (0 -31.458272567814532) / Math.Sqrt((1.1382234849319144 / (26)) + (0.3285843092318028 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.1206818906260693 = (28.070653082687326 - 31.458272567814532) / 28.070653082687326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_DateTime.ToString(format: "o")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.65170413011696 > 43.58310395205574.
IsChangePoint: Marked as a change because one of 2/24/2023 12:59:32 PM, 3/14/2023 5:06:59 PM, 4/12/2023 8:39:35 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -15.86348378384795 (T) = (0 -44.893365140152405) / Math.Sqrt((1.8103046018576885 / (26)) + (0.21299533530297257 / (22))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (22) - 2, .025) and -0.11050936014393867 = (40.4259223302121 - 44.893365140152405) / 40.4259223302121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Int32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ParseSpan - Duration of single invocation | 10.04 ns | 12.87 ns | 1.28 | 0.04 | False | |||||
| ParseSpan - Duration of single invocation | 13.19 ns | 14.99 ns | 1.14 | 0.04 | False | |||||
| Parse - Duration of single invocation | 13.51 ns | 14.89 ns | 1.10 | 0.07 | False | |||||
| TryParseSpan - Duration of single invocation | 9.87 ns | 12.78 ns | 1.30 | 0.06 | False | |||||
| TryParseSpan - Duration of single invocation | 13.24 ns | 15.13 ns | 1.14 | 0.06 | False | |||||
| Parse - Duration of single invocation | 10.17 ns | 12.79 ns | 1.26 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'Payloads
Histogram
System.Tests.Perf_Int32.ParseSpan(value: "4")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.86711452888599 > 10.541584891604964.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -34.01468767228998 (T) = (0 -12.759194467892694) / Math.Sqrt((0.03170193106819649 / (25)) + (0.09994248817461299 / (23))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .025) and -0.24958762648219918 = (10.210724080080713 - 12.759194467892694) / 10.210724080080713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.ParseSpan(value: "12345")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.985658142954462 > 13.87914673703071.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 2/28/2023 10:21:25 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -14.895403839551003 (T) = (0 -15.418045556622896) / Math.Sqrt((0.026923035149472856 / (16)) + (0.5778228656040483 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.15452213343401783 = (13.35448243920917 - 15.418045556622896) / 13.35448243920917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.Parse(value: "12345")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.891160229348904 > 14.184186638998632.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 2/28/2023 10:21:25 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -19.782095336348252 (T) = (0 -15.006184038866632) / Math.Sqrt((0.09288388073624108 / (26)) + (0.012518667732933057 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.09239740240963 = (13.736927610561613 - 15.006184038866632) / 13.736927610561613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParseSpan(value: "4")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.78123161081272 > 10.344646253851487.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -35.064014330132146 (T) = (0 -12.748326626785412) / Math.Sqrt((0.09037098672998609 / (26)) + (0.047122615944843835 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.2569714963825048 = (10.142096828348453 - 12.748326626785412) / 10.142096828348453 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.TryParseSpan(value: "12345")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.132671937505771 > 13.898637255737551.
IsChangePoint: Marked as a change because one of 2/6/2023 9:02:07 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/24/2023 7:28:10 PM, 4/4/2023 4:23:54 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -17.174217082468825 (T) = (0 -15.347677552361151) / Math.Sqrt((0.03299352017515052 / (16)) + (0.36111047850884465 / (32))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (32) - 2, .025) and -0.14847147764309954 = (13.36356875301575 - 15.347677552361151) / 13.36356875301575 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Int32.Parse(value: "4")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.788073837112874 > 10.683626632389936.
IsChangePoint: Marked as a change because one of 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -11.20283417924027 (T) = (0 -11.436759404902167) / Math.Sqrt((0.02246685706852503 / (25)) + (0.2281684267753229 / (23))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (23) - 2, .025) and -0.1134457879976079 = (10.271500892261436 - 11.436759404902167) / 10.271500892261436 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_String
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Replace_Char - Duration of single invocation | 99.02 ns | 109.65 ns | 1.11 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'Payloads
Histogram
System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+')
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.64749021207822 > 104.04009500395063.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -9.896255491607077 (T) = (0 -110.48898387108157) / Math.Sqrt((22.188763896681692 / (26)) + (0.3632143876956923 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.09111360485960471 = (101.2625847381844 - 110.48898387108157) / 101.2625847381844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_UInt64
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 9.97 ns | 12.13 ns | 1.22 | 0.04 | False | |||||
| ParseSpan - Duration of single invocation | 9.69 ns | 11.45 ns | 1.18 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'Payloads
Histogram
System.Tests.Perf_UInt64.Parse(value: "0")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.1306987326699 > 10.472873386100177.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/31/2023 8:05:26 PM, 4/4/2023 8:16:00 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -19.080673866576742 (T) = (0 -11.852860195526727) / Math.Sqrt((0.15319191748165384 / (26)) + (0.016907789211674927 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.15081255304693253 = (10.299557616177083 - 11.852860195526727) / 10.299557616177083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_UInt64.ParseSpan(value: "0")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.450567195756538 > 10.180020040004713.
IsChangePoint: Marked as a change because one of 3/4/2023 1:07:20 AM, 3/20/2023 8:31:51 AM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -22.8633948039552 (T) = (0 -10.937610888869946) / Math.Sqrt((0.01807251278659947 / (16)) + (0.0351871883082982 / (33))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (33) - 2, .025) and -0.10858911226653534 = (9.866244190787473 - 10.937610888869946) / 9.866244190787473 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.Span<Char>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SequenceEqual - Duration of single invocation | 15.82 ns | 20.05 ns | 1.27 | 0.10 | False | |||||
| Reverse - Duration of single invocation | 15.73 ns | 22.40 ns | 1.42 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'Payloads
Histogram
System.Memory.Span<Char>.SequenceEqual(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.052494047876813 > 16.779730428942077.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 2/14/2023 11:21:41 AM, 3/3/2023 11:43:56 AM, 3/29/2023 12:02:44 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -16.857643896483232 (T) = (0 -19.89227967607469) / Math.Sqrt((0.4971515418208281 / (26)) + (0.08822333560633372 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.14731721263239664 = (17.338081793817057 - 19.89227967607469) / 17.338081793817057 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Span<Char>.Reverse(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.403705784755637 > 16.725906951583735.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -13.992442343706022 (T) = (0 -20.926013046109226) / Math.Sqrt((0.10233301764300992 / (11)) + (4.125328245231515 / (37))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (37) - 2, .025) and -0.3027631529081198 = (16.062791612885814 - 20.926013046109226) / 16.062791612885814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.19042 |
| Queue | OwlWindows |
| Baseline | 33b18692fdf2ef438de3da38bc419c6616a746c5 |
| Compare | 7222061f18b8342a0bd4edbd37df318fd43c47d3 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Collections.CreateAddAndRemove<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| LinkedList - Duration of single invocation | 8.77 μs | 9.22 μs | 1.05 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<Int32>*'Payloads
Histogram
System.Collections.CreateAddAndRemove<Int32>.LinkedList(Size: 512)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.220822258318567 > 9.095379548717531.
IsChangePoint: Marked as a change because one of 3/8/2023 9:36:36 PM, 3/14/2023 8:42:09 PM, 3/17/2023 9:25:32 PM, 3/20/2023 8:31:51 AM, 4/9/2023 11:04:55 PM, 4/19/2023 8:38:27 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -28.964446859567953 (T) = (0 -9359.732478660419) / Math.Sqrt((9528.745789613324 / (26)) + (1337.3926951728472 / (23))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (23) - 2, .025) and -0.06811271280020367 = (8762.869654572876 - 9359.732478660419) / 8762.869654572876 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository















