Skip to content

Avoid exceptions in ProcessBasicChecksTests #7781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Nov 11, 2025
Merged

Avoid exceptions in ProcessBasicChecksTests #7781

merged 7 commits into from
Nov 11, 2025

Conversation

@NachoEchevarria
Copy link
Collaborator

@NachoEchevarria NachoEchevarria commented Nov 5, 2025 •
edited
Loading

Summary of changes

We are getting the following error in master:

04:54:49 [DBG] Datadog.Trace.Tools.dd_dotnet.ArtifactTests: STARTED: Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.ProcessBasicChecksTests.WorkingWithContinuousProfiler(1, True)�[0m
04:54:49 [DBG] Datadog.Trace.Tools.dd_dotnet.ArtifactTests: STARTED: Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.AgentConnectivityCheckTests.DetectVersionNamedPipes()
...
04:54:51 [DBG]   Failed Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.ProcessBasicChecksTests.WorkingWithContinuousProfiler(enabled: "1", ssiInjectionEnabled: True) [390 ms]
04:54:51 [DBG]   Error Message:
04:54:51 [DBG]    System.ComponentModel.Win32Exception : Access is denied
04:54:51 [DBG]   Stack Trace:
04:54:51 [DBG]      at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
04:54:51 [DBG]    at System.Diagnostics.Process.Start()
04:54:51 [DBG]    at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
04:54:51 [DBG]    at Datadog.Trace.Tools.dd_dotnet.ArtifactTests.ConsoleTestHelper.<StartConsole>d__5.MoveNext()
04:54:51 [DBG] --- End of stack trace from previous location where exception was thrown ---
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
04:54:51 [DBG]    at Datadog.Trace.Tools.dd_dotnet.ArtifactTests.Checks.ProcessBasicChecksTests.<WorkingWithContinuousProfiler>d__13.MoveNext() in D:\a\_work\1\s\tracer\test\Datadog.Trace.Tools.dd_dotnet.ArtifactTests\Checks\ProcessBasicChecksTests.cs:line 244
04:54:51 [DBG] --- End of stack trace from previous location where exception was thrown ---
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
04:54:51 [DBG] --- End of stack trace from previous location where exception was thrown ---
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
04:54:51 [DBG] --- End of stack trace from previous location where exception was thrown ---
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
04:54:51 [DBG]    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
04:54:51 [DBG]   Standard Output Messages:
04:54:51 [DBG]  Platform: X86
04:54:51 [DBG]  TargetPlatform: X86
04:54:51 [DBG]  Configuration: Release
04:54:51 [DBG]  TargetFramework: net48
04:54:51 [DBG]  .NET Core: False
04:54:51 [DBG]  Native Loader DLL: D:\a\_work\1\s\shared\bin\monitoring-home\win-x86\Datadog.Trace.ClrProfiler.Native.dll
04:54:51 [DBG]  Searching for CorFlags.exe in C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin
04:54:51 [DBG]  CorFlags.exe found at C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\CorFlags.exe
04:54:51 [DBG]  Updating Samples.Console.exe using /32BITREQ+

The "Access is denied" error happens when trying to start the process. This suggests that:

  1. The WorkingWithContinuousProfiler test is running at the same time as AgentConnectivityCheckTests which uses the same console app
  2. Each test case calls StartConsole, which calls PrepareSampleApp, which calls ProfilerHelper.SetCorFlags on line 46
    of ConsoleTestHelper.cs
  3. SetCorFlags modifies the same executable file in-place using CorFlags.exe
  4. One test thread is still running CorFlags.exe to modify the executable
  5. While CorFlags.exe has the file locked for writing
  6. Another test thread completes SetCorFlags() and tries to start that same executable
  7. Windows denies access because the file is still locked by CorFlags.exe from the other thread

So the lock needs to protect not just the CorFlags modification, but also ensure the CorFlags.exe process has fully
released the file lock before another thread can use that executable.

This error have been observed under Windows, net4, x86, which makes sense based on the ProfilerHelper.SetCorFlags conditions:

    protected (string Executable, string Args) PrepareSampleApp(EnvironmentHelper environmentHelper)
    {
        var sampleAppPath = environmentHelper.GetSampleApplicationPath();
        var executable = EnvironmentHelper.IsCoreClr() ? environmentHelper.GetSampleExecutionSource() : sampleAppPath;
        var args = EnvironmentHelper.IsCoreClr() ? sampleAppPath : string.Empty;

        if (EnvironmentTools.IsWindows()
         && !EnvironmentHelper.IsCoreClr()
         && !EnvironmentTools.IsTestTarget64BitProcess())
        {
            ProfilerHelper.SetCorFlags(executable, Output, !EnvironmentTools.IsTestTarget64BitProcess());
        }

        return (executable, args);
    }

Reason for change

Implementation details

Test coverage

Other details

@NachoEchevarria NachoEchevarria changed the title Add a lock to avoid exceptions Add a lock to avoid exceptions in ProcessBasicChecksTests Nov 5, 2025
@github-actions github-actions bot added the area:tests unit tests, integration tests label Nov 5, 2025
@datadog-official

This comment has been minimized.

Sign in to view
@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Nov 5, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7781) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration76.01 ± (75.91 - 76.54) ms74.20 ± (74.22 - 74.84) ms-2.4%✅
.NET Framework 4.8 - Bailout
duration81.16 ± (80.88 - 81.52) ms78.27 ± (78.49 - 79.29) ms-3.6%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1115.67 ± (1118.53 - 1129.48) ms1105.79 ± (1109.04 - 1121.12) ms-0.9%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms23.18 ± (23.07 - 23.29) ms22.87 ± (22.79 - 22.95) ms-1.3%✅
process.time_to_main_ms89.02 ± (88.62 - 89.42) ms86.69 ± (86.31 - 87.08) ms-2.6%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.91 ± (10.90 - 10.91) MB10.87 ± (10.87 - 10.88) MB-0.3%✅
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms22.98 ± (22.89 - 23.07) ms22.78 ± (22.71 - 22.85) ms-0.9%✅
process.time_to_main_ms89.07 ± (88.63 - 89.52) ms89.26 ± (88.89 - 89.63) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.95 ± (10.94 - 10.95) MB10.92 ± (10.91 - 10.92) MB-0.3%✅
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms221.69 ± (220.39 - 222.98) ms220.37 ± (218.65 - 222.09) ms-0.6%✅
process.time_to_main_ms554.33 ± (553.23 - 555.42) ms547.77 ± (546.52 - 549.02) ms-1.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed52.74 ± (52.72 - 52.77) MB52.45 ± (52.43 - 52.47) MB-0.6%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.0%✅
.NET 6 - Baseline
process.internal_duration_ms21.71 ± (21.65 - 21.77) ms21.56 ± (21.49 - 21.63) ms-0.7%✅
process.time_to_main_ms76.13 ± (75.83 - 76.44) ms76.02 ± (75.66 - 76.37) ms-0.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.59 ± (10.59 - 10.59) MB10.62 ± (10.62 - 10.63) MB+0.3%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.65 ± (21.60 - 21.71) ms21.36 ± (21.31 - 21.41) ms-1.3%✅
process.time_to_main_ms78.17 ± (77.85 - 78.48) ms76.07 ± (75.76 - 76.39) ms-2.7%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.64 ± (10.64 - 10.64) MB10.65 ± (10.65 - 10.65) MB+0.1%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms213.22 ± (210.53 - 215.91) ms207.26 ± (205.61 - 208.91) ms-2.8%✅
process.time_to_main_ms516.32 ± (515.25 - 517.38) ms509.07 ± (508.08 - 510.05) ms-1.4%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed51.81 ± (51.78 - 51.83) MB51.55 ± (51.51 - 51.58) MB-0.5%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.0%✅
.NET 8 - Baseline
process.internal_duration_ms19.80 ± (19.74 - 19.85) ms19.78 ± (19.73 - 19.84) ms-0.1%✅
process.time_to_main_ms75.20 ± (74.93 - 75.46) ms74.36 ± (74.09 - 74.63) ms-1.1%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.64 ± (7.64 - 7.65) MB7.63 ± (7.63 - 7.64) MB-0.1%✅
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms19.86 ± (19.81 - 19.92) ms19.85 ± (19.80 - 19.91) ms-0.1%✅
process.time_to_main_ms76.63 ± (76.34 - 76.92) ms75.52 ± (75.28 - 75.76) ms-1.4%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.69 ± (7.69 - 7.70) MB7.68 ± (7.67 - 7.69) MB-0.2%✅
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms190.55 ± (189.68 - 191.41) ms191.40 ± (190.59 - 192.21) ms+0.4%✅⬆️
process.time_to_main_ms490.93 ± (490.03 - 491.82) ms486.36 ± (485.41 - 487.32) ms-0.9%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed39.12 ± (39.09 - 39.16) MB38.75 ± (38.71 - 38.79) MB-1.0%✅
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)+0.0%✅⬆️

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration192.91 ± (192.80 - 193.64) ms192.07 ± (191.72 - 192.33) ms-0.4%✅
.NET Framework 4.8 - Bailout
duration196.21 ± (196.07 - 196.78) ms195.93 ± (195.81 - 196.54) ms-0.1%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1158.48 ± (1166.13 - 1176.14) ms1159.15 ± (1160.58 - 1167.77) ms+0.1%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms188.15 ± (187.77 - 188.53) ms187.20 ± (186.86 - 187.54) ms-0.5%✅
process.time_to_main_ms80.88 ± (80.64 - 81.12) ms80.60 ± (80.39 - 80.80) ms-0.3%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.03 ± (16.00 - 16.05) MB16.11 ± (16.08 - 16.15) MB+0.5%✅⬆️
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (19 - 20)-0.1%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms187.06 ± (186.75 - 187.37) ms187.31 ± (186.97 - 187.65) ms+0.1%✅⬆️
process.time_to_main_ms81.83 ± (81.69 - 81.96) ms81.95 ± (81.82 - 82.09) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.12 ± (16.10 - 16.15) MB16.17 ± (16.09 - 16.24) MB+0.3%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (20 - 21)+0.0%✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms391.50 ± (388.85 - 394.14) ms396.06 ± (393.66 - 398.47) ms+1.2%✅⬆️
process.time_to_main_ms515.07 ± (514.44 - 515.70) ms517.19 ± (516.17 - 518.22) ms+0.4%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed62.68 ± (62.53 - 62.83) MB62.87 ± (62.72 - 63.03) MB+0.3%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 30)+0.1%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms191.20 ± (190.88 - 191.52) ms191.93 ± (191.51 - 192.34) ms+0.4%✅⬆️
process.time_to_main_ms69.95 ± (69.77 - 70.14) ms70.05 ± (69.87 - 70.23) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed15.90 ± (15.75 - 16.06) MB15.89 ± (15.72 - 16.05) MB-0.1%✅
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)+0.2%✅⬆️
.NET 6 - Bailout
process.internal_duration_ms190.34 ± (190.04 - 190.64) ms190.87 ± (190.52 - 191.23) ms+0.3%✅⬆️
process.time_to_main_ms70.70 ± (70.60 - 70.80) ms70.91 ± (70.78 - 71.05) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed15.69 ± (15.52 - 15.86) MB16.13 ± (15.98 - 16.29) MB+2.8%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 19)19 ± (19 - 19)+3.0%✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms406.09 ± (403.14 - 409.03) ms414.24 ± (411.08 - 417.41) ms+2.0%✅⬆️
process.time_to_main_ms485.90 ± (485.25 - 486.54) ms485.37 ± (484.56 - 486.19) ms-0.1%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed62.07 ± (61.89 - 62.25) MB62.21 ± (62.08 - 62.34) MB+0.2%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)-0.1%✅
.NET 8 - Baseline
process.internal_duration_ms190.37 ± (189.92 - 190.81) ms189.99 ± (189.64 - 190.35) ms-0.2%✅
process.time_to_main_ms69.51 ± (69.31 - 69.70) ms69.53 ± (69.38 - 69.68) ms+0.0%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.71 ± (11.68 - 11.73) MB11.74 ± (11.71 - 11.77) MB+0.3%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)-0.2%✅
.NET 8 - Bailout
process.internal_duration_ms189.13 ± (188.92 - 189.35) ms190.23 ± (189.90 - 190.56) ms+0.6%✅⬆️
process.time_to_main_ms70.50 ± (70.40 - 70.60) ms70.68 ± (70.55 - 70.82) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.67 ± (11.58 - 11.76) MB11.79 ± (11.77 - 11.82) MB+1.1%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)+2.3%✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms359.15 ± (357.69 - 360.60) ms357.29 ± (355.58 - 358.99) ms-0.5%✅
process.time_to_main_ms465.60 ± (465.02 - 466.18) ms458.60 ± (458.02 - 459.18) ms-1.5%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed50.67 ± (50.63 - 50.71) MB50.29 ± (50.26 - 50.33) MB-0.7%✅
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)+0.1%✅⬆️
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (75ms)  : 70, 79
    master - mean (76ms)  : 72, 81

    section Bailout
    This PR (7781) - mean (79ms)  : 73, 85
    master - mean (81ms)  : 77, 86

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (1,115ms)  : 1022, 1208
    master - mean (1,124ms)  : 1038, 1210
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (117ms)  : 110, 124
    master - mean (120ms)  : 112, 128

    section Bailout
    This PR (7781) - mean (119ms)  : 113, 126
    master - mean (120ms)  : 112, 128

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (810ms)  : 760, 859
    master - mean (816ms)  : 782, 849
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (105ms)  : 97, 113
    master - mean (105ms)  : 100, 110

    section Bailout
    This PR (7781) - mean (104ms)  : 99, 110
    master - mean (107ms)  : 101, 112

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (755ms)  : 698, 813
    master - mean (764ms)  : 710, 819
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (102ms)  : 97, 107
    master - mean (103ms)  : 98, 108

    section Bailout
    This PR (7781) - mean (104ms)  : 99, 108
    master - mean (105ms)  : 100, 110

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (715ms)  : 678, 751
    master - mean (720ms)  : 693, 747
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (192ms)  : 189, 195
    master - mean (193ms)  : 189, 197

    section Bailout
    This PR (7781) - mean (196ms)  : 193, 200
    master - mean (196ms)  : 193, 200

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (1,164ms)  : 1113, 1215
    master - mean (1,171ms)  : 1093, 1249
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (276ms)  : 271, 281
    master - mean (277ms)  : 272, 283

    section Bailout
    This PR (7781) - mean (277ms)  : 273, 282
    master - mean (277ms)  : 274, 280

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (954ms)  : 900, 1008
    master - mean (948ms)  : 905, 991
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (270ms)  : 265, 276
    master - mean (269ms)  : 264, 275

    section Bailout
    This PR (7781) - mean (270ms)  : 266, 274
    master - mean (269ms)  : 266, 272

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (936ms)  : 881, 990
    master - mean (928ms)  : 884, 973
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7781) - mean (269ms)  : 265, 274
    master - mean (270ms)  : 263, 276

    section Bailout
    This PR (7781) - mean (271ms)  : 264, 277
    master - mean (269ms)  : 266, 273

    section CallTarget+Inlining+NGEN
    This PR (7781) - mean (848ms)  : 831, 866
    master - mean (856ms)  : 837, 874
Loading

@pr-commenter
Copy link

pr-commenter bot commented Nov 5, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7781 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.120
  • 2 benchmarks are slower, with geometric mean 1.671
  • 6 benchmarks have fewer allocations
  • 6 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.49 KB 5.53 KB 37 B 0.67%

Fewer allocations 🎉 in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.06 KB 5.99 KB -62 B -1.02%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.5μs 59.9ns 436ns 0 0 0 5.49 KB
master StartStopWithChild netcoreapp3.1 13.3μs 71.8ns 400ns 0 0 0 5.72 KB
master StartStopWithChild net472 21.9μs 123ns 881ns 0.896 0.224 0 6.06 KB
#7781 StartStopWithChild net6.0 10.5μs 58.1ns 363ns 0 0 0 5.53 KB
#7781 StartStopWithChild netcoreapp3.1 13.8μs 73ns 365ns 0 0 0 5.7 KB
#7781 StartStopWithChild net472 22μs 120ns 732ns 0.963 0.428 0.107 5.99 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 935μs 181ns 700ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 187ns 723ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.23ms 555ns 2.15μs 0 0 0 3.31 KB
#7781 WriteAndFlushEnrichedTraces net6.0 936μs 223ns 863ns 0 0 0 2.71 KB
#7781 WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 217ns 811ns 0 0 0 2.7 KB
#7781 WriteAndFlushEnrichedTraces net472 1.19ms 153ns 594ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 1.04μs 6.12ns 60ns 0 0 0 1.22 KB
master AllCycleSimpleBody netcoreapp3.1 1.44μs 5.74ns 21.5ns 0 0 0 1.2 KB
master AllCycleSimpleBody net472 1.03μs 0.321ns 1.2ns 0.192 0 0 1.23 KB
master AllCycleMoreComplexBody net6.0 7.14μs 34.9ns 156ns 0 0 0 4.72 KB
master AllCycleMoreComplexBody netcoreapp3.1 8.93μs 41.4ns 166ns 0 0 0 4.62 KB
master AllCycleMoreComplexBody net472 7.62μs 3.99ns 14.9ns 0.725 0 0 4.74 KB
master ObjectExtractorSimpleBody net6.0 324ns 1.62ns 7.24ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 395ns 2.17ns 13.4ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 299ns 0.0309ns 0.116ns 0.0437 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.31μs 33ns 165ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.86μs 37.3ns 149ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.75μs 4.57ns 17.7ns 0.576 0 0 3.8 KB
#7781 AllCycleSimpleBody net6.0 1.07μs 5.58ns 26.8ns 0 0 0 1.22 KB
#7781 AllCycleSimpleBody netcoreapp3.1 1.4μs 7.52ns 43.2ns 0 0 0 1.2 KB
#7781 AllCycleSimpleBody net472 1.06μs 0.749ns 2.9ns 0.193 0 0 1.23 KB
#7781 AllCycleMoreComplexBody net6.0 6.95μs 38.1ns 219ns 0 0 0 4.72 KB
#7781 AllCycleMoreComplexBody netcoreapp3.1 9.05μs 46.7ns 219ns 0 0 0 4.62 KB
#7781 AllCycleMoreComplexBody net472 7.64μs 3.71ns 13.9ns 0.727 0 0 4.74 KB
#7781 ObjectExtractorSimpleBody net6.0 315ns 0.189ns 0.734ns 0 0 0 280 B
#7781 ObjectExtractorSimpleBody netcoreapp3.1 397ns 2.19ns 13.1ns 0 0 0 272 B
#7781 ObjectExtractorSimpleBody net472 298ns 0.137ns 0.531ns 0.0434 0 0 281 B
#7781 ObjectExtractorMoreComplexBody net6.0 6.3μs 28.7ns 111ns 0 0 0 3.78 KB
#7781 ObjectExtractorMoreComplexBody netcoreapp3.1 7.89μs 37.3ns 149ns 0 0 0 3.69 KB
#7781 ObjectExtractorMoreComplexBody net472 6.75μs 1.49ns 5.56ns 0.575 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 76.1μs 299ns 1.12μs 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 98.3μs 218ns 843ns 0 0 0 32.4 KB
master EncodeArgs net472 110μs 166ns 643ns 4.98 0 0 32.51 KB
master EncodeLegacyArgs net6.0 148μs 41ns 153ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 196μs 70.2ns 272ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 264μs 86.7ns 336ns 0 0 0 2.16 KB
#7781 EncodeArgs net6.0 76.1μs 291ns 1.13μs 0 0 0 32.4 KB
#7781 EncodeArgs netcoreapp3.1 97.6μs 257ns 996ns 0 0 0 32.4 KB
#7781 EncodeArgs net472 110μs 110ns 425ns 4.95 0 0 32.51 KB
#7781 EncodeLegacyArgs net6.0 143μs 84.1ns 326ns 0 0 0 2.14 KB
#7781 EncodeLegacyArgs netcoreapp3.1 197μs 150ns 560ns 0 0 0 2.14 KB
#7781 EncodeLegacyArgs net472 267μs 64.6ns 250ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #7781

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1 2.399 307,801.09 738,455.38 several?

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 407μs 159ns 616ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 836μs 13μs 128μs 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 446μs 57.8ns 224ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 296μs 33.8ns 126ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 308μs 189ns 732ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 331μs 42.5ns 165ns 0 0 0 2.29 KB
#7781 RunWafRealisticBenchmark net6.0 391μs 105ns 408ns 0 0 0 4.55 KB
#7781 RunWafRealisticBenchmark netcoreapp3.1 815μs 11.4μs 113μs 0 0 0 4.48 KB
#7781 RunWafRealisticBenchmark net472 428μs 36.1ns 135ns 0 0 0 4.66 KB
#7781 RunWafRealisticBenchmarkWithAttack net6.0 284μs 88.5ns 331ns 0 0 0 2.24 KB
#7781 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 649μs 13.1μs 131μs 0 0 0 2.22 KB
#7781 RunWafRealisticBenchmarkWithAttack net472 309μs 36.1ns 140ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 60.8μs 38.7ns 145ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 72.6μs 239ns 861ns 0 0 0 17.42 KB
master SendRequest net472 0.00857ns 0.00228ns 0.00882ns 0 0 0 0 b
#7781 SendRequest net6.0 62.2μs 205ns 896ns 0 0 0 14.52 KB
#7781 SendRequest netcoreapp3.1 70.3μs 74.9ns 280ns 0 0 0 17.42 KB
#7781 SendRequest net472 0.000959ns 0.000721ns 0.00279ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7781

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 1.164 809,319.23 942,121.15

More allocations ⚠️ in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472 0 b 73 B 73 B ∞
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472 0 b 48 B 48 B ∞

Fewer allocations 🎉 in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 7 B 4 B -3 B -42.86%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.93ms 955ns 3.57μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.12ms 6.81μs 25.5μs 0 0 0 640 KB
master OriginalCharSlice net472 2.62ms 385ns 1.44μs 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.39ms 160ns 599ns 0 0 0 7 B
master OptimizedCharSlice netcoreapp3.1 1.67ms 396ns 1.53μs 0 0 0 1 B
master OptimizedCharSlice net472 1.96ms 219ns 848ns 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 809μs 32.2ns 125ns 0 0 0 3 B
master OptimizedCharSliceWithPool netcoreapp3.1 804μs 280ns 1.05μs 0 0 0 1 B
master OptimizedCharSliceWithPool net472 1.17ms 75.7ns 283ns 0 0 0 0 b
#7781 OriginalCharSlice net6.0 1.93ms 2.39μs 9.27μs 0 0 0 640.01 KB
#7781 OriginalCharSlice netcoreapp3.1 2.17ms 5.86μs 21.1μs 0 0 0 640 KB
#7781 OriginalCharSlice net472 2.57ms 168ns 607ns 100 0 0 641.95 KB
#7781 OptimizedCharSlice net6.0 1.54ms 2.79μs 10.8μs 0 0 0 4 B
#7781 OptimizedCharSlice netcoreapp3.1 1.68ms 288ns 1.12μs 0 0 0 1 B
#7781 OptimizedCharSlice net472 1.97ms 465ns 1.8μs 0 0 0 73 B
#7781 OptimizedCharSliceWithPool net6.0 942μs 49ns 190ns 0 0 0 3 B
#7781 OptimizedCharSliceWithPool netcoreapp3.1 809μs 63.4ns 245ns 0 0 0 1 B
#7781 OptimizedCharSliceWithPool net472 1.17ms 84.9ns 329ns 0 0 0 48 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.75 KB 41.96 KB 210 B 0.50%

Fewer allocations 🎉 in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 56.56 KB 55.88 KB -682 B -1.21%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 42.22 KB 41.7 KB -516 B -1.22%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 709μs 2.9μs 11.2μs 0 0 0 42.22 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 748μs 1.59μs 6.94μs 0 0 0 41.75 KB
master WriteAndFlushEnrichedTraces net472 875μs 4.4μs 18.7μs 8.33 0 0 56.56 KB
#7781 WriteAndFlushEnrichedTraces net6.0 699μs 3.43μs 14.5μs 0 0 0 41.7 KB
#7781 WriteAndFlushEnrichedTraces netcoreapp3.1 764μs 3.16μs 12.2μs 0 0 0 41.96 KB
#7781 WriteAndFlushEnrichedTraces net472 883μs 3.32μs 12.9μs 8.33 0 0 55.88 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.92μs 7.57ns 30.3ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.55μs 12.4ns 54ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.82μs 4.2ns 16.3ns 0.156 0.0142 0 987 B
#7781 ExecuteNonQuery net6.0 1.92μs 2.17ns 8.39ns 0 0 0 1.02 KB
#7781 ExecuteNonQuery netcoreapp3.1 2.66μs 4.18ns 16.2ns 0 0 0 1.02 KB
#7781 ExecuteNonQuery net472 2.91μs 4.57ns 17.7ns 0.143 0.0143 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.74μs 4.27ns 16.5ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.24μs 10.3ns 40ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.45μs 2.8ns 10.9ns 0.156 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.91μs 0.697ns 2.51ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.37μs 1.02ns 3.66ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.58μs 1.94ns 7.28ns 0.161 0 0 1.1 KB
#7781 CallElasticsearch net6.0 1.68μs 8.33ns 37.2ns 0 0 0 1.03 KB
#7781 CallElasticsearch netcoreapp3.1 2.23μs 10.2ns 38.3ns 0 0 0 1.03 KB
#7781 CallElasticsearch net472 3.56μs 0.941ns 3.39ns 0.161 0 0 1.04 KB
#7781 CallElasticsearchAsync net6.0 1.86μs 9.35ns 43.8ns 0 0 0 1.01 KB
#7781 CallElasticsearchAsync netcoreapp3.1 2.46μs 7.62ns 29.5ns 0 0 0 1.08 KB
#7781 CallElasticsearchAsync net472 3.7μs 1.7ns 6.38ns 0.167 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.84μs 3.2ns 12.4ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.41μs 7.43ns 28.8ns 0 0 0 952 B
master ExecuteAsync net472 2.63μs 2.03ns 7.85ns 0.145 0 0 915 B
#7781 ExecuteAsync net6.0 1.89μs 2.86ns 11.1ns 0 0 0 952 B
#7781 ExecuteAsync netcoreapp3.1 2.52μs 9.77ns 37.8ns 0 0 0 952 B
#7781 ExecuteAsync net472 2.57μs 1.24ns 4.79ns 0.141 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.83μs 15ns 58.3ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.83μs 22.7ns 87.8ns 0 0 0 2.9 KB
master SendAsync net472 11.9μs 10.3ns 40.1ns 0.474 0 0 3.18 KB
#7781 SendAsync net6.0 6.93μs 17.9ns 64.6ns 0 0 0 2.36 KB
#7781 SendAsync netcoreapp3.1 8.67μs 22.5ns 84.2ns 0 0 0 2.9 KB
#7781 SendAsync net472 12.4μs 9.24ns 35.8ns 0.493 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.82 KB 44.14 KB 1.32 KB 3.08%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 256.72 KB 260.1 KB 3.38 KB 1.32%

Fewer allocations 🎉 in #7781

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 44.14 KB 43.78 KB -368 B -0.83%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 273.7 KB 257.95 KB -15.75 KB -5.76%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 47μs 358ns 3.34μs 0 0 0 44.14 KB
master StringConcatBenchmark netcoreapp3.1 58.1μs 952ns 9.47μs 0 0 0 42.82 KB
master StringConcatBenchmark net472 56.3μs 280ns 1.25μs 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 447μs 1.25μs 4.32μs 0 0 0 256.72 KB
master StringConcatAspectBenchmark netcoreapp3.1 554μs 2.63μs 10.8μs 0 0 0 273.7 KB
master StringConcatAspectBenchmark net472 405μs 2.21μs 12.1μs 0 0 0 277.8 KB
#7781 StringConcatBenchmark net6.0 46.1μs 273ns 2.62μs 0 0 0 43.78 KB
#7781 StringConcatBenchmark netcoreapp3.1 50.4μs 573ns 5.5μs 0 0 0 44.14 KB
#7781 StringConcatBenchmark net472 56.9μs 239ns 924ns 0 0 0 57.34 KB
#7781 StringConcatAspectBenchmark net6.0 468μs 1.98μs 9.09μs 0 0 0 260.1 KB
#7781 StringConcatAspectBenchmark netcoreapp3.1 502μs 2.47μs 10.2μs 0 0 0 257.95 KB
#7781 StringConcatAspectBenchmark net472 398μs 2.13μs 16μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.58μs 13ns 61.1ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.64μs 17.3ns 69.3ns 0 0 0 1.7 KB
master EnrichedLog net472 3.85μs 3.18ns 12.3ns 0.249 0 0 1.64 KB
#7781 EnrichedLog net6.0 2.68μs 12.9ns 53.2ns 0 0 0 1.7 KB
#7781 EnrichedLog netcoreapp3.1 3.74μs 17.9ns 69.2ns 0 0 0 1.7 KB
#7781 EnrichedLog net472 4.02μs 7.69ns 29.8ns 0.242 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #7781

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 1.120 141,289.13 126,140.31

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 131μs 673ns 3.16μs 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 141μs 637ns 2.47μs 0 0 0 4.31 KB
master EnrichedLog net472 181μs 892ns 3.78μs 0 0 0 4.52 KB
#7781 EnrichedLog net6.0 123μs 39.4ns 142ns 0 0 0 4.31 KB
#7781 EnrichedLog netcoreapp3.1 126μs 62.9ns 227ns 0 0 0 4.31 KB
#7781 EnrichedLog net472 168μs 128ns 496ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.06μs 2.6ns 9.71ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.78μs 6.6ns 25.5ns 0 0 0 2.26 KB
master EnrichedLog net472 7.56μs 8.06ns 31.2ns 0.3 0 0 2.08 KB
#7781 EnrichedLog net6.0 5.17μs 5.41ns 19.5ns 0 0 0 2.26 KB
#7781 EnrichedLog netcoreapp3.1 6.9μs 26.4ns 102ns 0 0 0 2.26 KB
#7781 EnrichedLog net472 7.5μs 7.48ns 28ns 0.3 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.01μs 10.1ns 43.9ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.68μs 6.08ns 23.6ns 0 0 0 1.2 KB
master SendReceive net472 3.02μs 2.73ns 10.6ns 0.18 0 0 1.2 KB
#7781 SendReceive net6.0 2.01μs 9.24ns 35.8ns 0 0 0 1.2 KB
#7781 SendReceive netcoreapp3.1 2.65μs 13.1ns 57ns 0 0 0 1.2 KB
#7781 SendReceive net472 3.18μs 1.54ns 5.98ns 0.19 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.5μs 21.5ns 83.3ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.51μs 13.7ns 52.9ns 0 0 0 1.63 KB
master EnrichedLog net472 6.93μs 9.31ns 36.1ns 0.309 0 0 2.03 KB
#7781 EnrichedLog net6.0 4.36μs 1.13ns 4.39ns 0 0 0 1.58 KB
#7781 EnrichedLog netcoreapp3.1 5.63μs 7.46ns 27.9ns 0 0 0 1.63 KB
#7781 EnrichedLog net472 6.59μs 6.32ns 24.5ns 0.295 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 772ns 3.64ns 15ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 959ns 4.77ns 21.4ns 0 0 0 576 B
master StartFinishSpan net472 949ns 0.0993ns 0.371ns 0.0917 0 0 578 B
master StartFinishScope net6.0 916ns 1.69ns 6.31ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.2μs 6.11ns 29.3ns 0 0 0 696 B
master StartFinishScope net472 1.14μs 0.215ns 0.806ns 0.103 0 0 658 B
#7781 StartFinishSpan net6.0 791ns 0.18ns 0.673ns 0 0 0 576 B
#7781 StartFinishSpan netcoreapp3.1 973ns 5.25ns 27.8ns 0 0 0 577 B
#7781 StartFinishSpan net472 925ns 0.422ns 1.64ns 0.0898 0 0 578 B
#7781 StartFinishScope net6.0 943ns 4.43ns 18.3ns 0 0 0 696 B
#7781 StartFinishScope netcoreapp3.1 1.21μs 6.41ns 31.4ns 0 0 0 696 B
#7781 StartFinishScope net472 1.12μs 0.403ns 1.45ns 0.101 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.05μs 4.93ns 20.3ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.41μs 7.32ns 35.1ns 0 0 0 696 B
master RunOnMethodBegin net472 1.46μs 2.39ns 9.25ns 0.102 0 0 658 B
#7781 RunOnMethodBegin net6.0 1.08μs 5.57ns 25.5ns 0 0 0 696 B
#7781 RunOnMethodBegin netcoreapp3.1 1.41μs 7.28ns 34.9ns 0 0 0 696 B
#7781 RunOnMethodBegin net472 1.45μs 1.08ns 4.2ns 0.101 0 0 658 B

@NachoEchevarria NachoEchevarria marked this pull request as ready for review November 5, 2025 16:02
@NachoEchevarria NachoEchevarria requested a review from a team as a code owner November 5, 2025 16:02
@NachoEchevarria NachoEchevarria marked this pull request as draft November 5, 2025 16:13
@NachoEchevarria NachoEchevarria changed the title Add a lock to avoid exceptions in ProcessBasicChecksTests Avoid exceptions in ProcessBasicChecksTests Nov 6, 2025
andrewlock
andrewlock reviewed Nov 6, 2025
View reviewed changes
tracer/test/Datadog.Trace.TestHelpers/ProfilerHelper.cs Outdated
if (string.IsNullOrEmpty(corFlagsExe))
// Use the absolute path as the key to track which executables have been modified
var executablePath = Path.GetFullPath(executable);
var lockObj = _corFlagsLocks.GetOrAdd(executablePath, _ => new object());
Copy link
Member

@andrewlock andrewlock Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think it's fine to have a single lock object that's used when applying the flags tbh, and to have a fast-path outside the lock that checks if they're already applied.

e.g.

private readonly object _lockObj = new();

//...

var executablePath = Path.GetFullPath(executable);
if (_corFlagsApplied.TryGetValue(executablePath, out _))
{
    return;
}

// not applied
lock (_lockObj)
{
   // check again to be safe
  if (_corFlagsApplied.TryGetValue(executablePath, out _))
  {
      return;
  }

  // do the check
}

That way we don't need a lock per executable, and we avoid taking the lock (and having that contention) every time we start an app? wdyt?

Another tiny thing is I wonder if we need to check that we've set CorFlags correctly i.e. 32 bit or 64 bit. I don't think so, because I think we only set that based on whether we run the whole test suite as 32bit or 64bit, is that correct?

Copy link
Collaborator Author

@NachoEchevarria NachoEchevarria Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single lock is more simple and the performance impact would be negligible. I will apply that. Thanks!
Yes, we don't need to check the corflags because the whole test suite would run either in x86 or x64

@NachoEchevarria NachoEchevarria marked this pull request as ready for review November 6, 2025 11:40
andrewlock
andrewlock approved these changes Nov 6, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

tracer/test/Datadog.Trace.TestHelpers/ProfilerHelper.cs
.ToArray();

private static readonly object _lockObj = new();
private static readonly ConcurrentDictionary<string, bool> _corFlagsApplied = new ConcurrentDictionary<string, bool>(StringComparer.OrdinalIgnoreCase);
Copy link
Member

@andrewlock andrewlock Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should we ignore case 🤔 On Linux for example there could be two different samples with the same name, just different cases 😅 I don't think it's a problem in reality though

Copy link
Collaborator Author

@NachoEchevarria NachoEchevarria Nov 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we only change cor vars in Windows x86 net4, so I would keep it

bouwkast
bouwkast approved these changes Nov 11, 2025
View reviewed changes
tracer/test/Datadog.Trace.TestHelpers/ProfilerHelper.cs Outdated
CreateNoWindow = true,
UseShellExecute = false
};
var executedSuccessfully = Process.Start(opts).WaitForExit(20_000);
Copy link
Collaborator

@bouwkast bouwkast Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think we should dispose of the Process from Process.Start?

It was already like this though so 🤷

Copy link
Collaborator Author

@NachoEchevarria NachoEchevarria Nov 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! I have modified the code. Even thought it was already like that, it's better to dispose it. Thanks!

@NachoEchevarria NachoEchevarria merged commit 080f300 into master Nov 11, 2025
152 checks passed
@NachoEchevarria NachoEchevarria deleted the nacho/FixWin32ExceptionInTests branch November 11, 2025 14:53
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 11, 2025
@NachoEchevarria
Copy link
Collaborator Author

NachoEchevarria commented Nov 11, 2025

Thanks for your feedback and reviews!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:tests unit tests, integration tests

Projects

None yet

Milestone

vNext-v3

Development

Successfully merging this pull request may close these issues.

4 participants

@NachoEchevarria @bouwkast @andrewlock