Skip to content

[ComInterfaceGenerator] [Proposal] Separate cleanup stages into CleanupIn and CleanupOut #89791

@jtschuster

Description

@jtschuster

To fix #89747, one solution would be to separate out the cleanup stages into two: one where the native values passed to the callee are cleaned up (CleanupIn), and one where the native values allocated by the callee are cleaned up (CleanupOut). This would work similarly to the idea of the CleanupFailure stage in #89329.

In ManagedToUnmanaged stubs, CleanupOut should cleanup the out parameters and [Out] elements of parameters (anything only allocated by the callee. This would be done in an if(_invokeSucceeded) statement in the finally block (same place as the GuaranteedUnmarshal stage). CleanupIn would be all other parameters and would be cleaned up in the finally block, without any guards.

In UnmanagedToManaged stubs, CleanupOut should cleanup out and ref parameters, as well as [In, Out], and [Out] elements of arrays (Anything allocated by the callee). These statements would be placed in a catch block if the method throws at any point. CleanupIn would be in, ref, [In], and [In,Out], and would be at the end of the try block.

Since the ManagedToUnmanaged direction is simpler and has more risks than leaking memory, we could only implement ManagedToUnmanaged in .Net 8.

This could be implemented in the GeneratedStatements without modification to any ICustomTypeMarshallingStrategy. The GeneratedStatements constructor would determine if the parameter should be cleaned up in CleanupIn or CleanupOut stage. In UnmanagedToManaged, we'd put all cleanup statements to go in the CleanupIn stage.

ManagedToUnmanaged stubs would now look like:

<< Variable Declarations >>
<< Setup >>
try
{
    << Marshal >>
    << Pin >> (fixed)
    {
        << Pinned Marshal >>
        << Invoke >>
    }
    << Notify For Successful Invoke >>
    << Unmarshal Capture >>
    << Unmarshal >>
}
finally
{
    if(_invokeSucceeded)
    {
        << GuaranteedUnmarshal >>
        << CleanupOut >>
    }
    << CleanupIn >>
}
<< AssignOut >>
return <returnValue>;

TODO:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions