Support backing the allocation of a string by user controlled memory

[davidfowl]

There's been lots of discussion about how to optimize strings for the future including things like supporting utf8 but it would be less of a breaking change let users control the backing memory of a particular string so that the memory can be reused. This would especially be useful in lots of places inside of ASP.NET (headers, connection ids etc), where today it's impossible to control that memory and still use the exchange type we want to use (string). Strings are the last kinds of allocations we can't get rid of today.

Strings are one of the only special major reference (and exchange) types that force you to allocate. They are immutable and allocation is controlled by the runtime. If you need large string, it's going into the LOH. There's no way to pool it since strings are immutable. Allowing this would open up a new set of performance optimizations not possible before, or possible with lots of runtime hackery.

cc @jkotas @davidwrighton @Maoni0

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[jkotas]

How would you propose to make this possible?

ReadOnlySpan<char> or ReadOnlyMemory<char> are existing types for strings backed by user-controlled memory. If we knew how to change System.String (and arrays) to be backed by user-controlled memory, we would not need to introduce Span<T> and Memory<T> in the first place. We have brainstormed what potential designs may look like, but there was nothing that looked even remotely feasible.

Also, see the discussion in #4368 and linked issues.

[davidfowl]

Right. ArrayPool is interesting here, because it allows you to use byte[] and char[] as exchange types but have them be "backed" by user controlled memory (sorta). Maybe what I'm looking for is something similar but for strings. Today we hand out a byte[] or char[] that came from the ArrayPool, there's no harm, the GC will collect it as it'll no longer be rooted. I wonder if we could do something similar with strings (or maybe there's something I'm missing).

[davidfowl]

OK doing some more thinking (stream of consciousness here so bare with me). It occurs to me that I'm asking for the free keyword for strings 😄. But I want to categorize it as solving the same category of problem as ArrayPool<byte> and ArrayPool<char>.

Or maybe an alternative to this whole thing would be to expose a Runtime API on MemoryMarshl to Cast/Convert/As a char[] to a string and back to a char[]. Then the pooling part would be managed by ArrayPool or any allocated char[] with a type safe way to represent those characters as a string.

[davidfowl]

How close are the array layout and string layout?

[jkotas]

How close are the array layout and string layout?

On CoreCLR 64-bit: String payload start at offset 12, array payload starts at offset 16. Length includes the zero terminator for char[], but not for String.

I am not sure how this would help.

[davidfowl]

I was thinking craziness (runtime can effectively cast between char[] and string). This looks interesting CommunityToolkit/WindowsCommunityToolkit#3380

I don't know if this is runtime safe though

[stephentoub]

This looks interesting CommunityToolkit/WindowsCommunityToolkit#3380

We've floated various ideas in the past for enabling lookups into a dictionary with a ReadOnlySpan<char>. We should consider just adding some extension methods (with access to dictionary internals) to dictionary, e.g.

public static bool TryGetValue<TValue>(this Dictionary<string, TValue> dictionary, ReadOnlySpan<char> span, out TValue value);

The challenge here has been that Dictionary<> can use an arbitrary comparer, and if it's not one we're aware of, we're stuck holding a span when the comparer expects a string. But we could special-case the comparers we know about, e.g. StringComparer.*, and make it work with those, which would cover the 99% case, and then we'd just need to decide on a behavior for the 1% case, which for now could just be throwing an exception. (We could even add a new virtual to StringComparer that supports comparing a ReadOnlySpan<char> to a string, override it on all StringComparers, and use that as the special-casing in dictionary, but the base implementation would need to either throw or allocate a new string, which is a pit of failure.)

[jkotas]

#27229