Detectron2 inference accuracy got failed

[mengfei25]

🐛 Describe the bug

Model	FP32	BF16	FP16	AMP_BF16	AMP_FP16
detectron2_fasterrcnn_r_101_c4	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_fasterrcnn_r_101_dc5	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_fasterrcnn_r_101_fpn	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_fasterrcnn_r_50_c4	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_fasterrcnn_r_50_dc5	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_fasterrcnn_r_50_fpn	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_maskrcnn_r_101_c4	fail_accuracy	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_maskrcnn_r_101_fpn	fail_accuracy	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_maskrcnn_r_50_c4	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy
detectron2_maskrcnn_r_50_fpn	pass	eager_fail_to_run	fail_accuracy	eager_fail_to_run	fail_accuracy

Versions

Envirnoments:
Device: PVC 1100
torch-xpu-ops: 18bcd9a
python: 3.10
TRITON_COMMIT_ID: e98b6fcb8df5b44eb0d0addb6767c573d37ba024
TORCH_COMMIT_ID: b9fbd65dfd5e703bacbc6c25258d1215108b4faf
TORCHBENCH_COMMIT_ID: 766a5e3a189384659fd35a68c3b17b88c761aaac
TORCHVISION_COMMIT_ID: d23a6e1664d20707c11781299611436e1f0c104f
TORCHAUDIO_COMMIT_ID: b6d4675c7aedc53ba04f3f55786aac1de32be6b4
DRIVER_VERSION: 1.23.10.49.231129.50 (803.61)
KERNEL_VERSION: 5.15.0-73-generic #80 SMP Mon May 15 15:18:26 UTC 2023
BUNDLE_VERSION: 2025.0.1.20241113 (DL-Essential 2025.0.1)
OS_PRETTY_NAME: Ubuntu 22.04.2 LTS
GCC_VERSION: 11

[daisyden]

@chuanqi129 @mengfei25 please help to update latest status.

[jianyizh]

roi_align_forward_kernel_xpu not inplemented for bf16. Is this the regression because previously we fall back to cpu?

[jianyizh]

nms on cpu fp64 will have error that dets should have the same type as scores, so it uses cosine to measure accuracy. If we put it fp64 reference on xpu, one of the result shape is different from fp32 reference and result...

For detectron2_fasterrcnn_r_50_c4 fp16 inference, the cosine similarity can increase from 0.955 to 0.986 (threshold is 0.99) if we keep the same calculation sequence in BatchNorm inference between eager and inductor

diff --git a/torch/_decomp/decompositions.py b/torch/_decomp/decompositions.py
index d2b92342b7..0c342afcc6 100644
--- a/torch/_decomp/decompositions.py
+++ b/torch/_decomp/decompositions.py
@@ -1806,12 +1806,12 @@ def native_batch_norm_helper(
             save_rstd = input.new_zeros((0,))
         mean = _unsqueeze_to_dim(mean, input.dim() - 1)
         invstd = _unsqueeze_to_dim(invstd, input.dim() - 1)
+    if weight is None:
         output = (input - mean) * invstd
-
-    if weight is not None:
+    else:
         weight = weight.flatten()
         weight = _unsqueeze_to_dim(weight, input.dim() - 1)
-        output = output * weight
+        output = weight * (input - mean) * invstd
 
     if bias is not None:
         bias = bias.flatten()