`test-windows-full` CI job failed on disassemble tests for x86

[filzrev]

On GitHub Actions CI test-windows-full job failed by following tests.

• CanDisassembleAllMethodCalls(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [FAIL]
• CanDisassembleInlinableBenchmarks(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [FAIL]
• CanDisassembleAllMethodCallsUsingFilters(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [FAIL]
• CanDisassembleGenericTypes(jit: LegacyJit, platform: X86, runtime: .NET Framework 4.6.2) [FAIL]

I've tried 4-times But it seems failed always.
And it can be reproduced by running workflow on latest master branch. (It's succussed on 3 days ago)
So it's expected to be CI environment related problems.

But as far as I've confirmed, there is no noticeable changes exists.
Both image using 20250609.2.0 OS image.

Exception Log.

Following Exception is recorded on benchmark logs.

Failed to disassemble with following exception:
2025-06-21T00:29:13.9032696Z  Object reference not set to an instance of an object.
2025-06-21T00:29:13.9033779Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.DataStream.Read(PdbStreamHelper reader, Int32 position, Byte[] bytes, Int32 offset, Int32 data)
2025-06-21T00:29:13.9035235Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.DataStream.Read(PdbStreamHelper reader, BitAccess bits)
2025-06-21T00:29:13.9036676Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.MsfDirectory..ctor(PdbStreamHelper reader, PdbFileHeader head, BitAccess bits)
2025-06-21T00:29:13.9038400Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.PdbFile.LoadFunctions(Stream read, BitAccess bits, Boolean readAllStrings, Int32& ver, Int32& sig, Int32& age, Guid& guid, IEnumerable`1& sources)
2025-06-21T00:29:13.9039381Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.PdbReader.Init(Stream pdbStream)
2025-06-21T00:29:13.9039943Z     at Microsoft.Diagnostics.Runtime.Utilities.Pdb.PdbReader..ctor(String fileName)
2025-06-21T00:29:13.9040530Z     at BenchmarkDotNet.Disassemblers.SourceCodeProvider.GetReaderForMethod(ClrMethod method)
2025-06-21T00:29:13.9041404Z     at BenchmarkDotNet.Disassemblers.SourceCodeProvider.GetSourceLocation(ClrMethod method, Int32 ilOffset)
2025-06-21T00:29:13.9042024Z     at BenchmarkDotNet.Disassemblers.SourceCodeProvider.<GetSource>d__3.MoveNext()
2025-06-21T00:29:13.9042980Z     at BenchmarkDotNet.Disassemblers.ClrMdV1Disassembler.DisassembleMethod(MethodInfo methodInfo, State state, Settings settings, SourceCodeProvider sourceCodeProvider)
2025-06-21T00:29:13.9043872Z     at BenchmarkDotNet.Disassemblers.ClrMdV1Disassembler.Disassemble(Settings settings, State state)
2025-06-21T00:29:13.9044531Z     at BenchmarkDotNet.Disassemblers.ClrMdV1Disassembler.AttachAndDisassemble(Settings settings)
2025-06-21T00:29:13.9045064Z     at BenchmarkDotNet.Disassemblers.Program.Main(String[] args)

[timcassell]

Strange. Not sure what changed, but maybe it's time to update BenchmarkDotNet.Disassembler.x64/x86. ClrMD v1 is quite old.

[filzrev]

On failed tests.
I've confirmed some failed test outputting following logs additionally.

---------------------------
2025-06-21T04:17:12.1286918Z \ ---------------------------
2025-06-21T04:17:12.1287343Z Failed to read source code location!
2025-06-21T04:17:12.1287949Z Please make sure that the project, which defines benchmarks contains following settings:
2025-06-21T04:17:12.1288552Z 	 <DebugType>pdbonly</DebugType>
2025-06-21T04:17:12.1288899Z 	 <DebugSymbols>true</DebugSymbols>
2025-06-21T04:17:12.1289235Z \ ---------------------------
---------------------------

those logs are outputted by following code (same code shared with x86 version)

BenchmarkDotNet/src/BenchmarkDotNet.Disassembler.x64/Program.cs

Lines 31 to 39 in f8390f8

	catch (OutOfMemoryException) // thrown by clrmd when pdb is missing or in invalid format
	{
	Console.WriteLine("\\ ---------------------------");
	Console.WriteLine("Failed to read source code location!");
	Console.WriteLine("Please make sure that the project, which defines benchmarks contains following settings:");
	Console.WriteLine("\t <DebugType>pdbonly</DebugType>");
	Console.WriteLine("\t <DebugSymbols>true</DebugSymbols>");
	Console.WriteLine("\\ ---------------------------");
	}

[filzrev]

As a workaround.
It seems issue can be avoided by setting print:printSource:false to failed tests.

I don't know why this error are started to occurs.
But It can now reproduceable on my environment (Windows) also.

[timcassell]

Looks like another Framework update broke the x86 disassembler again for #2336. I think the ClrMD v1 disassembler is too old and needs to be updated.

I'm playing around with the idea of running the disassembler in-process so we won't need to keep separate versions for different architectures in the package (and it might enable it to work on MacOS, too!).