[Custom Descriptors] Edge case in Heap2Local #7755
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When Heap2Local optimized an allocation that flowed into a ref.cast_desc
that admitted nulls as the cast descriptor, it reasoned that the only
way the original cast could have succeeded was if the cast reference was
null, so Heap2Local optimized the cast to a drop of the descriptor
operand followed by a cast to null of the reference operand.
This missed the edge case where the same allocation flows in as both the
reference and descriptor operands. In that case, the optimized
allocation passed the cast to null even though the original cast would
have failed. Fix the bug by only using the code path that produces the
null cast when the optimized allocation does not also flow into the cast
as the reference operand. Casts where the same allocations flows in as
both operands are now optimized to a sequence ending in unreachable.
We had test coverage for the same allocation flowing in as both the
reference and descriptor operands, but only when the cast was to a
non-nullable type. Add coverage for nullable casts, and also add new
versions of the tests that exercise a slightly different combination of
conditions leading to the same code path.