Skip to content

[Perf] Changes at 6/23/2022 3:07:42 AM #6502

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetEncoder - Duration of single invocation 29.27 ns 23.57 ns 0.81 0.71 False
GetEncoder - Duration of single invocation 26.46 ns 17.35 ns 0.66 0.73 False
GetChars - Duration of single invocation 343.71 ns 247.89 ns 0.72 0.63 False
GetBytes - Duration of single invocation 204.31 ns 97.95 ns 0.48 0.56 False
GetString - Duration of single invocation 313.47 ns 202.57 ns 0.65 0.59 False
GetChars - Duration of single invocation 62.39 ns 48.58 ns 0.78 0.64 False
GetEncoder - Duration of single invocation 27.86 ns 19.73 ns 0.71 0.75 False
GetChars - Duration of single invocation 281.89 ns 201.90 ns 0.72 0.56 False
GetBytes - Duration of single invocation 282.04 ns 208.91 ns 0.74 0.66 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetEncoder(size: 512, encName: "ascii")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.572824390523085 < 24.76912846060416.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 7.305580690187543 (T) = (0 -22.61273491427412) / Math.Sqrt((7.6122627061536425 / (182)) + (8.433073681314736 / (26))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (182) + (26) - 2, .975) and 0.16352958904201037 = (27.03351441729576 - 22.61273491427412) / 27.03351441729576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetEncoder(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.351560385685637 < 25.148217464515874.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 11.14010503492233 (T) = (0 -21.098384843662576) / Math.Sqrt((7.521989709592829 / (181)) + (7.022865803939288 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.22480776211710848 = (27.216971239655155 - 21.098384843662576) / 27.216971239655155 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 247.8867677050097 < 339.54121281086816.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 21.864572616975902 (T) = (0 -245.25443571849596) / Math.Sqrt((533.2394531012558 / (181)) + (445.54084325798334 / (26))) is greater than 1.9716034992643237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (26) - 2, .975) and 0.2854654975949735 = (343.2366595216924 - 245.25443571849596) / 343.2366595216924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 97.94672464613588 < 184.82416749942777.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 17.15523505754197 (T) = (0 -141.51952657800607) / Math.Sqrt((170.57904174924894 / (181)) + (234.33892870323487 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.27326364728666575 = (194.7329675330406 - 141.51952657800607) / 194.7329675330406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 202.56946212722613 < 298.2228471423532.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 22.555781766190083 (T) = (0 -200.5883039819902) / Math.Sqrt((338.8224220748506 / (181)) + (511.77561796016164 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.3391358259268364 = (303.52425180757837 - 200.5883039819902) / 303.52425180757837 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.57548809676517 < 58.331791401566115.
IsChangePoint: Marked as a change because one of 6/22/2022 1:08:18 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 5.244639009154831 (T) = (0 -52.85439271913433) / Math.Sqrt((27.491971151246617 / (178)) + (20.77737126685949 / (28))) is greater than 1.9716608894935834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (178) + (28) - 2, .975) and 0.08588368979049806 = (57.82020529424848 - 52.85439271913433) / 57.82020529424848 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetEncoder(size: 16, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.725396562869054 < 25.080131857900206.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 9.366914587257034 (T) = (0 -22.098088401373033) / Math.Sqrt((9.445305009353925 / (181)) + (6.690060632746275 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.1884132323362671 = (27.228251225664387 - 22.098088401373033) / 27.228251225664387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "ascii")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 201.90264561803787 < 273.44970913846936.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 27.103793807797704 (T) = (0 -175.14562396826094) / Math.Sqrt((371.41320602715376 / (181)) + (345.71479456556466 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.3736163794957194 = (279.61399090745226 - 175.14562396826094) / 279.61399090745226 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 208.90773337756974 < 260.0793486476799.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 12.45333070807537 (T) = (0 -233.04794559136397) / Math.Sqrt((638.9626398108109 / (182)) + (351.8830880277884 / (26))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (182) + (26) - 2, .975) and 0.18074102846144208 = (284.461877974559 - 233.04794559136397) / 284.461877974559 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare 3205533cbb03414db5c05e307044606452ac963b
Diff Diff

Improvements in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NewCachedCustomConverter - Duration of single invocation 3.25 μs 2.59 μs 0.80 0.44 False
NewCustomizedOptions - Duration of single invocation 3.05 μs 2.36 μs 0.77 0.42 False
NewDefaultOptions - Duration of single invocation 3.01 μs 1.78 μs 0.59 0.49 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.592534030211985 < 2.9985664220431265.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 20.250885307544877 (T) = (0 -2516.289513912622) / Math.Sqrt((25695.520056462115 / (181)) + (18706.86326596273 / (25))) is greater than 1.9716608894935834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (25) - 2, .975) and 0.19362941216876148 = (3120.512518543452 - 2516.289513912622) / 3120.512518543452 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;.NewCustomizedOptions

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3617766132925655 < 2.9288783291180853.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 21.642276619034014 (T) = (0 -2383.005202077302) / Math.Sqrt((23744.16157524502 / (181)) + (22927.164615624697 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.22140100605443014 = (3060.6322646287576 - 2383.005202077302) / 3060.6322646287576 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;.NewDefaultOptions

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7764058283248372 < 2.75186745303252.
IsChangePoint: Marked as a change because one of 6/22/2022 5:33:38 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 20.267022598625367 (T) = (0 -2260.0116585004007) / Math.Sqrt((31358.4007593197 / (181)) + (21083.085938130014 / (27))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (27) - 2, .975) and 0.21691377796526548 = (2886.031697286274 - 2260.0116585004007) / 2886.031697286274 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare 3205533cbb03414db5c05e307044606452ac963b
Diff Diff

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithCultureInfo - Duration of single invocation 188.06 ns 155.04 ns 0.82 0.66 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 155.03768574479687 < 183.37756597320114.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 4.118655169805784 (T) = (0 -177.77881843364176) / Math.Sqrt((204.61958287326584 / (181)) + (224.4667610306183 / (25))) is greater than 1.9716608894935834 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (181) + (25) - 2, .975) and 0.06860664807753386 = (190.8740470035489 - 177.77881843364176) / 190.8740470035489 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NoDI - Duration of single invocation 13.61 ns 9.73 ns 0.72 0.81 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.DependencyInjection.GetService.NoDI


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.73485203298211 < 13.83954467717845.
IsChangePoint: Marked as a change because one of 5/24/2022 11:43:29 AM, 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 3.541463219588293 (T) = (0 -11.90774918900719) / Math.Sqrt((6.829958455371431 / (182)) + (2.683590846900752 / (26))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (182) + (26) - 2, .975) and 0.10037527046110607 = (13.236351556399024 - 11.90774918900719) / 13.236351556399024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AddThenRemove_ExpirationTokens - Duration of single invocation 114.21 μs 73.87 μs 0.65 0.78 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Caching.Memory.Tests.MemoryCacheTests.AddThenRemove_ExpirationTokens


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.86548145604395 < 101.60304915771431.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 7.282265428476283 (T) = (0 -84036.40401529055) / Math.Sqrt((193257894.212465 / (182)) + (128021035.12635356 / (26))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (182) + (26) - 2, .975) and 0.1749248689040718 = (101853.0323458749 - 84036.40401529055) / 101853.0323458749 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline c9007c8cd39a5b086a74dd5580b34311041a68ff
Compare b1221494186e7b8e5199d892dae76481a52f5935
Diff Diff

Improvements in MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 48.78 μs 39.30 μs 0.81 0.60 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Binary_FromStream&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_FromStream<IndexViewModel>.BinaryFormatter_


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 39.296589958906566 < 43.21559723890821.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 6/30/2022 12:21:07 PM falls between 6/21/2022 9:29:01 PM and 6/30/2022 12:21:07 PM.
IsImprovementStdDev: Marked as improvement because 11.829169070640827 (T) = (0 -39458.229585450186) / Math.Sqrt((9015770.010212397 / (182)) + (7995211.201800192 / (26))) is greater than 1.9715466694451473 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (182) + (26) - 2, .975) and 0.15192005533045927 = (46526.54485400585 - 39458.229585450186) / 46526.54485400585 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions