Fix a 32-bit specific bug in fgMorphCast (#52828)

* Added a test

Verifying that checked '.un' casts from floats to small types are not treated as casts from unsigned types.

* Do not mark casts from FP types with GTF_UNSIGNED

It used to be that in the importer, whether the cast was to
be marked as GTF_UNSIGNED was decided exclusively based on
the incoming opcode. However, the flag only makes sense
for casts from integral sources, and it turns out morph
had a bug where it failed to clear this flag which resulted in
bad codegen.

The bug went as follows: "gtMorphCast" turns casts from an FP
type to a small integer into a chain of casts:
CAST(small integer <- FP) => CAST(small integer <- CAST(TYP_INT <- FP)).
On 32 bit platforms, the code failed to clear the GTF_UNSIGNED flag
from the original tree, which meant that the outer cast thought it
had TYP_UINT as the source. This matters for checked casts:
conv.ovf.i2.un(-2.0d), which is a legitimate conversion, was interpreted
wrongly as an overflowing one as the resulting codegen only checked the
upper bound via an unsigned compare.

The fix is two-fold: clear GTF_UNSIGNED for GT_CAST nodes with FP sources
on creation and unify the 64 bit and 32 bit paths in "gtMorphCast",
which, after the removal of GTF_UNSIGNED handling, are identical.

This is a zero-diff change across all SPMI collections for Windows x64, Linux x64,
Linux ARM64.

This **is not** a zero-diff change for Windows x86.
Instances of bad codegen have been corrected in some tests.

* Assert instead of normalizing

Instead of normalizing GTF_UNSIGNED for FP sources in "gtNewCastNode",
assert that it is not set in GenTreeCast's constructor and fix the
importer to respect that constraint.

Author: SingleAccretion

src/coreclr/jit/compiler.hpp

@@ -1365,8 +1365,9 @@ inline void GenTree::SetOper(genTreeOps oper, ValueNumberUpdate vnUpdate)
 
 inline GenTreeCast* Compiler::gtNewCastNode(var_types typ, GenTree* op1, bool fromUnsigned, var_types castType)
 {
-    GenTreeCast* res = new (this, GT_CAST) GenTreeCast(typ, op1, fromUnsigned, castType);
-    return res;
+    GenTreeCast* cast = new (this, GT_CAST) GenTreeCast(typ, op1, fromUnsigned, castType);
+
+    return cast;
 }
 
 inline GenTreeCast* Compiler::gtNewCastNodeL(var_types typ, GenTree* op1, bool fromUnsigned, var_types castType)
@@ -1378,9 +1379,10 @@ inline GenTreeCast* Compiler::gtNewCastNodeL(var_types typ, GenTree* op1, bool f
 
     /* Make a big node first and then change it to be GT_CAST */
 
-    GenTreeCast* res =
+    GenTreeCast* cast =
         new (this, LargeOpOpcode()) GenTreeCast(typ, op1, fromUnsigned, castType DEBUGARG(/*largeNode*/ true));
-    return res;
+
+    return cast;
 }
 
 inline GenTreeIndir* Compiler::gtNewMethodTableLookup(GenTree* object)

src/coreclr/jit/gentree.h

@@ -3531,6 +3531,11 @@ struct GenTreeCast : public GenTreeOp
     GenTreeCast(var_types type, GenTree* op, bool fromUnsigned, var_types castType DEBUGARG(bool largeNode = false))
         : GenTreeOp(GT_CAST, type, op, nullptr DEBUGARG(largeNode)), gtCastType(castType)
     {
+        // We do not allow casts from floating point types to be treated as from
+        // unsigned to avoid bugs related to wrong GTF_UNSIGNED in case the
+        // CastOp's type changes.
+        assert(!varTypeIsFloating(op) || !fromUnsigned);
+
         gtFlags |= fromUnsigned ? GTF_UNSIGNED : GTF_EMPTY;
     }
 #if DEBUGGABLE_GENTREE

src/coreclr/jit/importer.cpp

@@ -13456,11 +13456,17 @@ void Compiler::impImportBlockCode(BasicBlock* block)
                     callNode = varTypeIsFloating(impStackTop().val->TypeGet());
                 }
 
-                // At this point uns, ovf, callNode all set
-
                 op1 = impPopStack().val;
                 impBashVarAddrsToI(op1);
 
+                // Casts from floating point types must not have GTF_UNSIGNED set.
+                if (varTypeIsFloating(op1))
+                {
+                    uns = false;
+                }
+
+                // At this point uns, ovf, callNode are all set.
+
                 if (varTypeIsSmall(lclTyp) && !ovfl && op1->gtType == TYP_INT && op1->gtOper == GT_AND)
                 {
                     op2 = op1->AsOp()->gtOp2;

src/coreclr/jit/morph.cpp

@@ -183,20 +183,15 @@ GenTree* Compiler::fgMorphCast(GenTree* tree)
         // do we need to do it in two steps R -> I, '-> smallType
         CLANG_FORMAT_COMMENT_ANCHOR;
 
-#if defined(TARGET_ARM64) || defined(TARGET_AMD64)
         if (dstSize < genTypeSize(TYP_INT))
         {
-            oper = gtNewCastNodeL(TYP_INT, oper, tree->IsUnsigned(), TYP_INT);
-            oper->gtFlags |= (tree->gtFlags & (GTF_OVERFLOW | GTF_EXCEPT));
-            tree->gtFlags &= ~GTF_UNSIGNED;
-        }
-#else
-        if (dstSize < TARGET_POINTER_SIZE)
-        {
-            oper = gtNewCastNodeL(TYP_I_IMPL, oper, false, TYP_I_IMPL);
+            oper = gtNewCastNodeL(TYP_INT, oper, /* fromUnsigned */ false, TYP_INT);
             oper->gtFlags |= (tree->gtFlags & (GTF_OVERFLOW | GTF_EXCEPT));
+            // We must not mistreat the original cast, which was from a floating point type,
+            // as from an unsigned type, since we now have a TYP_INT node for the source and
+            // CAST_OVF(BYTE <- INT) != CAST_OVF(BYTE <- UINT).
+            assert(!tree->IsUnsigned());
         }
-#endif
         else
         {
             /* Note that if we need to use a helper call then we can not morph oper */