Support dynamically creating tailored continuation layouts #120411

jakobbotsch · 2025-10-04T23:16:29Z

Add ability for the VM to dynamically create continuation layout types and for the JIT to request such types to be created.

dotnet-policy-service · 2025-10-04T23:17:32Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

jkotas · 2025-10-05T00:04:52Z

src/coreclr/vm/jitinterface.cpp

+
+    AllocMemTracker amTracker;
+    // TODO: table to share/deduplicate these
+    result = (CORINFO_CLASS_HANDLE)m_pMethodBeingCompiled->GetModule()->CreateContinuationMethodTable((unsigned)dataSize, objRefs, dataOffsets, &amTracker);


I think these should be created on the loader allocator.

Do you mean place the method on another class? CreateContinuationMethodTable uses the high frequency heap of the module's loader allocator for the allocation.

Definition module loader allocator != method loader allocator.

Consider MethodInNonCollectibleAssembly<CollectibleType>(): This method instantiation should be allocated on collectible loader allocator. As written, it will be allocated on non-collectible loader allocator of MethodInNonCollectibleAssembly and the collectability of the instantiation argument won't be taken into account. We will have a memory leak.

Loader allocator is the scope of unloading. If you are allocating something collectible, loader allocator is the right place to attach it to.

That makes a lot of sense. Will change this.

What do we expect pMT->GetLoaderModule()/pMT->GetModule() to return for these types?

If we want these to return something meaningful:

pMT->GetLoaderModule() should return m_pMethodBeingCompiled->GetLoaderModule()

pMT->GetModule() is used as a context for interpreting metadata tokens and for reflection. I assume that these types won't have metadata token and won't be visible to reflection, so pMT->GetModule() can be anything. Returning S.P.Corelib as pMT->GetModule() would help with sharing.

Alternatively, we may want to make sure that these methods are never called on these types. If these methods are called on the continuation types, it is a likely a bug. It may take a while to chase down all places.

Alternatively, we may want to make sure that these methods are never called on these types. If these methods are called on the continuation types, it is a likely a bug. It may take a while to chase down all places.

That does seem best. For now I have set the fields as you suggested above, but will look into asserting that we don't access them for the continuation types.

The parts of the system that will likely get in trouble are if we (via reflection) get a System.Type of one of these, and the various debugger and profiler apis which examine the heap and various values. Please get @tommcdon or @noahfalk to take a look at this before we signoff.

One thing that is already confused is SOS. !dumpobj, !dumpmt and !verifyheap report invalid objects/MT when the continuation method tables are involved. Hopefully that's a simple check somewhere.

jkotas · 2025-10-05T00:14:48Z

src/coreclr/vm/methodtable.h

+    inline BOOL IsContinuation()
+    {
+        LIMITED_METHOD_DAC_CONTRACT;
+        return GetFlag(enum_flag_IsContinuationType);


Can this check whether the parent type is a CLASS__CONTINUATION instead?

Yeah, that seems better, no need to add this flag

jkoritzinsky · 2025-10-05T18:05:21Z

src/coreclr/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncHelpers.CoreCLR.cs

-            Debug.Assert(continuationContext != null);
-            return continuationContext;
+            // Only the special continuation sub types are expected.
+            Debug.Assert(GetType() != typeof(Continuation));


Can we make this type abstract instead of having this assert?

We currently create an instance of Continuation inside FinalizeTaskReturningThunk, though I think with change and some more refactoring we can probably get rid of that.

Hmm, we also create one in AwaitAwaiter and UnsafeAwaitAwaiter. That one is harder to remove.

Now that we store directly into the ThunkTask's result we do not need to allocate a tail continuation at all.

jakobbotsch · 2025-10-07T08:17:22Z

src/coreclr/vm/jitinterface.cpp

+    LoaderAllocator* allocator = m_pMethodBeingCompiled->GetLoaderAllocator();
+    AsyncContinuationsManager* asyncConts = allocator->GetAsyncContinuationsManager();
+    AllocMemTracker amTracker;
+    result = (CORINFO_CLASS_HANDLE)asyncConts->LookupOrCreateContinuationMethodTable((unsigned)dataSize, objRefs, dataOffsets, m_pMethodBeingCompiled, &amTracker);


Strictly speaking we can delay this call until the continuation needs to be created. But in any case we need to persist some description of the continuation somewhere. However, we would be able to store that description compressed until it is needed.
I don't have good intuition about the expense of creating these or the number of them that we are going to be creating (likely a lot), so it is hard to know if this will be good enough or if it will show up on the radar once we start measuring.

I do not think this will be a problem (with de-duping of the layout in place). Also, the de-duping can be helped by sorting the fields, so that there is higher probability of finding dups.

jakobbotsch · 2025-10-07T08:26:55Z

src/coreclr/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncHelpers.CoreCLR.cs

-        public uint State;
+        public delegate*<Continuation, ref byte, Continuation?> Resume;
        public CorInfoContinuationFlags Flags;
+        public int State;


We don't actually need this field in Continuation, however this part is padding on 64-bit anyway. For 32-bit we could save space in methods with 1 suspension point by not having it at all, but then we make the 64-bit continuations with State fields in them larger. And doing one thing in 64-bit and another in 32-bit is also more complex than I figured was worth it.

jkotas · 2025-10-07T13:17:37Z

src/coreclr/vm/jitinterface.cpp

+
+    LoaderAllocator* allocator = m_pMethodBeingCompiled->GetLoaderAllocator();
+    AsyncContinuationsManager* asyncConts = allocator->GetAsyncContinuationsManager();
+    AllocMemTracker amTracker;


AllocMemTracker should be created around Create part of LookupOrCreate.

jkotas · 2025-10-07T13:20:35Z

src/coreclr/vm/methodtable.cpp

        return (pCanonMT->GetClass() == pClass);
    else
-        return (pCanonMT == this) || IsArray();
+        return (pCanonMT == this) || IsArray() || IsContinuation(); // TODO -- allocate an EEClass instead?


allocate an EEClass instead?

I do not think we want to do that to minimize overhead. These fake types will be special.

…uation-layouts

src/coreclr/vm/methodtable.inl

jakobbotsch added 4 commits October 4, 2025 15:20

Initial

7d27090

Fix build, make tests run

6743534

Tailored continuation layouts

0f8cd9c

Add ability for the VM to dynamically create continuation layout types and for the JIT to request such types to be created.

Update type

a642f42

github-actions bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Oct 4, 2025

dotnet-policy-service bot assigned jakobbotsch Oct 4, 2025

jakobbotsch added 3 commits October 5, 2025 01:24

Fix bug

03a640d

Fix some bugs

a99ce96

Run thunk generator

fa1820a

jkotas reviewed Oct 5, 2025

View reviewed changes

jkoritzinsky reviewed Oct 5, 2025

View reviewed changes

jakobbotsch added 4 commits October 6, 2025 15:59

Create a manager in the LoaderAllocator to hold map and create types

e4a75f1

Use SPC for continuation's module

82995d5

Remove unnecessary continuation allocation

38bdf11

Now that we store directly into the ThunkTask's result we do not need to allocate a tail continuation at all.

Fix some bugs

97de35e

jakobbotsch mentioned this pull request Oct 7, 2025

Add debug information for runtime async methods #120303

Open

jakobbotsch commented Oct 7, 2025

View reviewed changes

jkotas reviewed Oct 7, 2025

View reviewed changes

jakobbotsch added 5 commits October 7, 2025 16:40

Fix type GC info, add debug name, remove comments

fb296e3

Refactor building of obj refs bitmap into helper class

a3e223f

Properly allocate space for KeepAlive object

71b47b0

Merge branch 'main' of github.com:dotnet/runtime into tailored-contin…

7df6464

…uation-layouts

Fix bitmap building for implicit byrefs

badee7d

jakobbotsch commented Oct 7, 2025

View reviewed changes

src/coreclr/vm/methodtable.inl Outdated Show resolved Hide resolved

Run jit-format, remove copilot comment

56c403a

davidwrighton requested a review from noahfalk October 7, 2025 23:22

davidwrighton requested a review from tommcdon October 7, 2025 23:22

jakobbotsch added 8 commits October 8, 2025 14:37

Move Continuation MT into a static

ca7ef69

Remove added newline

9d6d275

Undo another change

04cd9f4

Undo name change

2c30029

Add a cache for created layout types

21ab2fc

Fix a bug

a45427b

Fix another bug

37fd434

Fix debug counting

15f205a

Support dynamically creating tailored continuation layouts #120411

Are you sure you want to change the base?

Support dynamically creating tailored continuation layouts #120411

Conversation

jakobbotsch commented Oct 4, 2025

Uh oh!

dotnet-policy-service bot commented Oct 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakobbotsch Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakobbotsch Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jakobbotsch Oct 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jakobbotsch Oct 6, 2025 •

edited

Loading

jakobbotsch Oct 7, 2025 •

edited

Loading

jakobbotsch Oct 7, 2025 •

edited

Loading