[NativeAOT] System.Runtime.Tests occasionally crashes on linux-arm-NativeAOT

[VSadov]

The crash does not happen often - just once in a few CI runs.
So far I was unable to get a dump or any other info for the failure. I'd need to set up a local repro.

For example seen in #99031 :

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-99031-merge-ad45d5f9cac945e0af/System.Runtime.Tests/1/console.a6a20f16.log?helixlogtype=result

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The crash does not happen often - just once in a few CI runs.
So far I was unable to get a dump or any other info for the failure. I'd need to set up a local repro.

For example seen in #99031 :

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-99031-merge-ad45d5f9cac945e0af/System.Runtime.Tests/1/console.a6a20f16.log?helixlogtype=result

Author:	VSadov
Assignees:	-
Labels:	area-NativeAOT-coreclr
Milestone:	-

[filipnavara]

Preliminary notes:

• We may be missing some of the Align8 fixes done in https://github.com/dotnet/coreclr/pull/25403/files
• ParentOfFuncletStackFrame functionality from CoreCLR is missing on NativeAOT (ref: Allow async interruptions on safepoints #95565 (comment))

[VSadov]

We may be missing some of the Align8 fixes done

I once saw a failure at this stack:

[SKIP] System.Collections.Concurrent.Tests.ConcurrentBagTests.DebuggerTypeProxy_Ctor_NullArgument_Throws
Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException.
   at System.RuntimeExceptionHelpers.FailFast(String, Exception, String, RhFailFastReason, IntPtr, IntPtr) + 0x161
   at System.RuntimeExceptionHelpers.GetRuntimeException(ExceptionIDs) + 0x2a1
   at System.Runtime.EH.GetClasslibException(ExceptionIDs, IntPtr) + 0x27
   at System.Runtime.EH.RhThrowHwEx(UInt32, EH.ExInfo&) + 0x8d
   at System.Threading.PortableThreadPool.HillClimbing.GetWaveComponent(Double[], Int32, Double) + 0x99
   at System.Threading.PortableThreadPool.HillClimbing.Update(Int32, Double, Int32) + 0x2fb
   at System.Threading.PortableThreadPool.AdjustMaxWorkersActive() + 0x133
   at System.Threading.ThreadPoolWorkQueue.Dispatch() + 0x335
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() + 0x12b
   at System.Threading.Thread.StartThread(IntPtr) + 0xd5
   at System.Threading.Thread.ThreadEntryPoint(IntPtr) + 0xf
./RunTests.sh: line 179:    86 Aborted                 (core dumped) ./System.Collections.Concurrent.Tests -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing -xml testResults.xml $RSP_FILE

Not necessarily because of double[] alignment, but there are some double[] involved.

I kind of think if this system was configured to fail on misaligned accesses, it would fail in more tests. Also this looks like an AV, not alignment problem, so maybe it was something else.

[EgorBo]

Speaking of align8 - does NAOT respect align8 for e.g.

static readonly double[] arr = new double[100];

during preinitialization?

[VSadov]

Speaking of align8 - does NAOT respect align8 for e.g.

static readonly double[] arr = new double[100];
during preinitialization?

Last time I checked RequiresAlign8Flag was not set for anything so types that want 8-align could not report it. So I doubt it.

It is not a lot of changes to enable this, but it would be "a bit here, a bit there."
Respecting 8-align is basically an ARM32 feature, so it was not implemented and would not be testable, even if implemented until there is ARM32 support.

[VSadov]

                // Currently we don't support frozen objects with special alignment requirements
                // TODO: We should also give up on arrays of doubles on 32-bit platforms.
                // (we currently never allocate them on frozen segments)
#if FEATURE_64BIT_ALIGNMENT
                if (type->RequiresAlign8)
                {
                    // Align8 objects are not supported yet
                    return null;
                }
#endif

So, nope.

[EgorBo]

So, nope.

The code you quoted is used to allocate objects on nongc in run-time, it's different from preinitialization that also allocates on nongc.

[EgorBo]

I kind of think if this system was configured to fail on misaligned accesses, it would fail in more tests. Also this looks like an AV, not alignment problem, so maybe it was something else.

Afair, some floating-point related loads/store (as well as atomicity-related) are not fixable by OS and may fail

[VSadov]

Right, that is not even in NAOT. Sorry.

Generally aligning frozen arrays should be doable. Just do what GC does - overallocate space and pad in front with empty objects (dummy byte[] would work too). I do not see any code attempting to do that.

[VSadov]

Afair, some floating-point related loads/store (as well as atomicity-related) are not fixable by OS and may fail

That what I suspected too. Perhaps OS was configured to handle misaligned accesses, but could not handle all the cases. I am not very familiar with how robust the emulation is. I'd assume it is robust, since it is in OS, but would not be too surprised if there are limitations.