Skip to content

[Custom Descriptors] Edge case in Heap2Local #7755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tlively
Copy link
Member

@tlively tlively commented Jul 25, 2025

When Heap2Local optimized an allocation that flowed into a ref.cast_desc
that admitted nulls as the cast descriptor, it reasoned that the only
way the original cast could have succeeded was if the cast reference was
null, so Heap2Local optimized the cast to a drop of the descriptor
operand followed by a cast to null of the reference operand.

This missed the edge case where the same allocation flows in as both the
reference and descriptor operands. In that case, the optimized
allocation passed the cast to null even though the original cast would
have failed. Fix the bug by only using the code path that produces the
null cast when the optimized allocation does not also flow into the cast
as the reference operand. Casts where the same allocations flows in as
both operands are now optimized to a sequence ending in unreachable.

We had test coverage for the same allocation flowing in as both the
reference and descriptor operands, but only when the cast was to a
non-nullable type. Add coverage for nullable casts, and also add new
versions of the tests that exercise a slightly different combination of
conditions leading to the same code path.

When Heap2Local optimized an allocation that flowed into a ref.cast_desc
that admitted nulls as the cast descriptor, it reasoned that the only
way the original cast could have succeeded was if the cast reference was
null, so Heap2Local optimized the cast to a drop of the descriptor
operand followed by a cast to null of the reference operand.

This missed the edge case where the same allocation flows in as both the
reference and descriptor operands. In that case, the optimized
allocation passed the cast to null even though the original cast would
have failed. Fix the bug by only using the code path that produces the
null cast when the optimized allocation does not also flow into the cast
as the reference operand. Casts where the same allocations flows in as
both operands are now optimized to a sequence ending in unreachable.

We had test coverage for the same allocation flowing in as both the
reference and descriptor operands, but only when the cast was to a
non-nullable type. Add coverage for nullable casts, and also add new
versions of the tests that exercise a slightly different combination of
conditions leading to the same code path.
@tlively tlively requested a review from kripken July 25, 2025 23:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant