-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1f00a73b1a7a513c009bcd6e7b60381798908fe |
| Compare | 620b0db3a987bab2694a0043cacb3e2a86673df5 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in System.Reflection.Invoke
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Property_Set_int - Duration of single invocation | 515.45 ns | 417.89 ns | 0.81 | 0.32 | False | |||||
| Ctor4_int_string_struct_class - Duration of single invocation | 403.49 ns | 281.57 ns | 0.70 | 0.22 | False | |||||
| Property_Set_class - Duration of single invocation | 484.99 ns | 428.46 ns | 0.88 | 0.26 | False | |||||
| Method0_NoParms - Duration of single invocation | 381.32 ns | 314.93 ns | 0.83 | 0.19 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'Payloads
Histogram
System.Reflection.Invoke.Property_Set_int
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 417.8870850441788 < 466.01043614115724.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 33.79088076878789 (T) = (0 -416.2749258501975) / Math.Sqrt((3047.381661907347 / (299)) + (185.11757829466075 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.2471893757255644 = (552.9610136033964 - 416.2749258501975) / 552.9610136033964 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.Ctor4_int_string_struct_class
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 281.5657163742691 < 351.6793457356355.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 52.67342333576401 (T) = (0 -276.3626435540252) / Math.Sqrt((962.1532958911622 / (299)) + (104.27189930475224 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.32839118333489453 = (411.49347164069786 - 276.3626435540252) / 411.49347164069786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.Property_Set_class
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 428.45597693452385 < 472.8725964781069.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 35.33772704952686 (T) = (0 -430.0763955191536) / Math.Sqrt((3271.806930827732 / (299)) + (146.59081694856883 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.24543703243251455 = (569.9675361827098 - 430.0763955191536) / 569.9675361827098 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Reflection.Invoke.Method0_NoParms
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 314.9261539423077 < 353.338060680544.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 43.50250565188961 (T) = (0 -306.55019151378883) / Math.Sqrt((1995.078019232853 / (299)) + (73.95489900625786 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.29927956545749224 = (437.47859546001666 - 306.55019151378883) / 437.47859546001666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1f00a73b1a7a513c009bcd6e7b60381798908fe |
| Compare | 620b0db3a987bab2694a0043cacb3e2a86673df5 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 29.43 μs | 24.31 μs | 0.83 | 0.18 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Transient
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.314215718524252 < 27.949008574867516.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 36.91927735615808 (T) = (0 -24816.60756702939) / Math.Sqrt((7207904.828035406 / (299)) + (240360.37610306914 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.21049613952395752 = (31433.16810644329 - 24816.60756702939) / 31433.16810644329 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1f00a73b1a7a513c009bcd6e7b60381798908fe |
| Compare | 620b0db3a987bab2694a0043cacb3e2a86673df5 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in Microsoft.Extensions.DependencyInjection.GetService
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 2.80 μs | 2.49 μs | 0.89 | 0.16 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.GetService.Transient
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.4910230545951886 < 2.680025275514724.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 38.34964067733264 (T) = (0 -2404.212353320707) / Math.Sqrt((50965.87624111175 / (299)) + (3797.5380370725957 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21446443229469717 = (3060.6027940197073 - 2404.212353320707) / 3060.6027940197073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | ubuntu 18.04 |
| Queue | TigerUbuntu |
| Baseline | c1f00a73b1a7a513c009bcd6e7b60381798908fe |
| Compare | 620b0db3a987bab2694a0043cacb3e2a86673df5 |
| Diff | Diff |
| Configs | AOT:true, CompilationMode:wasm, RunKind:micro |
Improvements in Microsoft.Extensions.DependencyInjection.ScopeValidation
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Transient - Duration of single invocation | 2.80 μs | 2.38 μs | 0.85 | 0.18 | False | |||||
| TransientWithScopeValidation - Duration of single invocation | 3.00 μs | 2.51 μs | 0.84 | 0.17 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ScopeValidation*'Payloads
Histogram
Microsoft.Extensions.DependencyInjection.ScopeValidation.Transient
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.3841687282185324 < 2.663913867835063.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 40.27809802852529 (T) = (0 -2408.1210997407825) / Math.Sqrt((54998.296009156955 / (299)) + (2713.899266621124 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.21695616788542416 = (3075.3337182131377 - 2408.1210997407825) / 3075.3337182131377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Microsoft.Extensions.DependencyInjection.ScopeValidation.TransientWithScopeValidation
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.5125309111699283 < 2.7688158007258266.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/24/2023 11:08:13 AM, 5/2/2023 9:04:55 PM falls between 4/24/2023 8:06:00 AM and 5/2/2023 9:04:55 PM.
IsImprovementStdDev: Marked as improvement because 40.63724387223018 (T) = (0 -2456.8407933924013) / Math.Sqrt((49806.79159594405 / (299)) + (3020.5869186898635 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.21182805527838558 = (3117.138093845966 - 2456.8407933924013) / 3117.138093845966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository





