`BUILD_SEPARATE_OPS=ON` leads to failure when building PyTorch for XPU on Windows

[Oscilloscope98]

Flag BUILD_SEPARATE_OPS=ON leads to failure when building PyTorch for XPU on Windows.

Parts of error log:

[410/889] Linking CXX shared library bin\torch-xpu-ops-sycl-NestedTensorTransformerFunctionKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-NestedTensorTransformerFunctionKernels.dll lib/torch-xpu-ops-sycl-NestedTensorTransformerFunctionKernels.lib 
... failed (exit code 1181) with the following output:
LINK : fatal error LNK1181: 无法打开输入文件“caffe2\aten_xpu\src\CMakeFiles\torch-xpu-ops-sycl-NestedTensorTransformerFunctionKernels.dir\ATen\native\nested\xpu\sycl\torch-xpu-ops-sycl-NestedTensorTransformerFunctionKernels_gen_NestedTensorTransformerFunctionKernels.cpp.obj”

[411/889] Linking CXX shared library bin\torch-xpu-ops-sycl-QuantizedMaxPool2d.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-QuantizedMaxPool2d.dll lib/torch-xpu-ops-sycl-QuantizedMaxPool2d.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-QuantizedMaxPool2d_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-QuantizedMaxPool2d.lib 和对象 lib\torch-xpu-ops-sycl-QuantizedMaxPool2d.exp

torch-xpu-ops-sycl-QuantizedMaxPool2d_gen_QuantizedMaxPool2d.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclDeviceMaxWorkGroupSize(signed char)" (?syclDeviceMaxWorkGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-QuantizedMaxPool2d.dll : fatal error LNK1120: 1 个无法解析的外部命令

[412/889] Linking CXX shared library bin\torch-xpu-ops-sycl-FusedObsFakeQuantKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-FusedObsFakeQuantKernels.dll lib/torch-xpu-ops-sycl-FusedObsFakeQuantKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-FusedObsFakeQuantKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-FusedObsFakeQuantKernels.lib 和对象 lib\torch-xpu-ops-sycl-FusedObsFakeQuantKernels.exp

torch-xpu-ops-sycl-FusedObsFakeQuantKernels_gen_FusedObsFakeQuantKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-FusedObsFakeQuantKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[413/889] Linking CXX shared library bin\torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.dll lib/torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.lib 和对象 lib\torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.exp

torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels_gen_MakePerTensorQuantizedTensorKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-MakePerTensorQuantizedTensorKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[414/889] Linking CXX shared library bin\torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.dll lib/torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.lib 和对象 lib\torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.exp

torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels_gen_SparseBinaryOpIntersectionKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-SparseBinaryOpIntersectionKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[415/889] Linking CXX shared library bin\torch-xpu-ops-sycl-AffineQuantizerKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-AffineQuantizerKernels.dll lib/torch-xpu-ops-sycl-AffineQuantizerKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-AffineQuantizerKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-AffineQuantizerKernels.lib 和对象 lib\torch-xpu-ops-sycl-AffineQuantizerKernels.exp

torch-xpu-ops-sycl-AffineQuantizerKernels_gen_AffineQuantizerKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-AffineQuantizerKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[416/889] Linking CXX shared library bin\torch-xpu-ops-sycl-FakeQuantizeCoreKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-FakeQuantizeCoreKernels.dll lib/torch-xpu-ops-sycl-FakeQuantizeCoreKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-FakeQuantizeCoreKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-FakeQuantizeCoreKernels.lib 和对象 lib\torch-xpu-ops-sycl-FakeQuantizeCoreKernels.exp

torch-xpu-ops-sycl-FakeQuantizeCoreKernels_gen_FakeQuantizeCoreKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-FakeQuantizeCoreKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[417/889] Linking CXX shared library bin\torch-xpu-ops-sycl-SparseCsrTensorMathKernels.dll
FAILED: [code=4294967295] bin/torch-xpu-ops-sycl-SparseCsrTensorMathKernels.dll lib/torch-xpu-ops-sycl-SparseCsrTensorMathKernels.lib 
... failed (exit code 1120) with the following output:
torch-xpu-ops-sycl-SparseCsrTensorMathKernels_sycl_device_obj.obj : warning LNK4078: 找到多个“__CLANG_OFFLOAD_BUNDLE__sycl-spi”节，它们具有不同的特性(40500040)

  正在创建库 lib\torch-xpu-ops-sycl-SparseCsrTensorMathKernels.lib 和对象 lib\torch-xpu-ops-sycl-SparseCsrTensorMathKernels.exp

torch-xpu-ops-sycl-SparseCsrTensorMathKernels_gen_SparseCsrTensorMathKernels.cpp.obj : error LNK2019: 无法解析的外部符号 "__declspec(dllimport) struct c10::xpu::DeviceProp * __cdecl at::xpu::getDeviceProperties(signed char)" (__imp_?getDeviceProperties@xpu@at@@YAPEAUDeviceProp@1c10@@C@Z)，函数 "__int64 __cdecl xpu::sycl::syclMaxSubGroupSize(signed char)" (?syclMaxSubGroupSize@sycl@xpu@@YA_JC@Z) 中引用了该符号

bin\torch-xpu-ops-sycl-SparseCsrTensorMathKernels.dll : fatal error LNK1120: 1 个无法解析的外部命令

[418/889] Linking CXX shared library bin\torch_xpu.dll
[419/889] Building CXX object caffe2\aten_xpu\src\CMakeFiles\torch_xpu_ops_aten.dir\__\__\__\xpu\ATen\RegisterXPU_0.cpp.obj
ninja: build stopped: subcommand failed.
-- Checkout nccl release tag: v2.27.5-1

Build details:

• Base commit: pytorch/pytorch@d9832d8
• oneAPI: 2025.2
• OS: Windows 11
• Built-on device: U7 258V

[xuhancn]

It can reproduced in my local machine, need more debug work.

[xuhancn]

1. I found the Windows cmake(

   torch-xpu-ops/src/BuildOnWindows.cmake

   Lines 7 to 24 in 542d111

   	add_library(
   	torch_xpu_ops
   	STATIC
   	${ATen_XPU_CPP_SRCS})
   	set(PATH_TO_TORCH_XPU_OPS_ATEN_LIB \"torch_xpu_ops_aten.dll\")
   	target_compile_options(torch_xpu_ops PRIVATE -DPATH_TO_TORCH_XPU_OPS_ATEN_LIB=${PATH_TO_TORCH_XPU_OPS_ATEN_LIB})
   	add_library(
   	torch_xpu_ops_aten
   	SHARED
   	${ATen_XPU_MKL_SRCS}
   	${ATen_XPU_NATIVE_CPP_SRCS}
   	${ATen_XPU_GEN_SRCS})
   	install(TARGETS torch_xpu_ops_aten DESTINATION "${TORCH_INSTALL_LIB_DIR}")
   	target_compile_definitions(torch_xpu_ops_aten PRIVATE TORCH_XPU_BUILD_MAIN_LIB)
   	target_link_libraries(torch_xpu_ops_aten PUBLIC torch_xpu)
   	target_link_libraries(torch_xpu_ops_aten PUBLIC torch_cpu)
   	target_link_libraries(torch_xpu_ops_aten PUBLIC c10)

   ) file is different to Linux cmake(

   torch-xpu-ops/src/BuildOnLinux.cmake

   Lines 7 to 21 in 542d111

   	add_library(
   	torch_xpu_ops
   	STATIC
   	${ATen_XPU_CPP_SRCS}
   	${ATen_XPU_MKL_SRCS}
   	${ATen_XPU_NATIVE_CPP_SRCS}
   	${ATen_XPU_GEN_SRCS}
   	${ATen_XPU_XCCL_SRCS})
   	if(USE_C10D_XCCL)
   	target_compile_definitions(torch_xpu_ops PRIVATE USE_C10D_XCCL)
   	target_link_libraries(torch_xpu_ops PUBLIC torch::xccl)
   	target_link_libraries(torch_xpu_ops PUBLIC fmt::fmt-header-only)
   	endif()
   	list(APPEND TORCH_XPU_OPS_LIBRARIES torch_xpu_ops)

   ), I talked with @fengyuan14 and know it is that Windows can't cycle dependency.

2. I talked with @chunhuanMeng and confirmed it is a regression issue.

3. I tried to draft a PR to fix this issue: fix Windows cmake for BUILD_SEPARATE_OPS #2147

a. The original function at::xpu::getDeviceProperties is fixed.
b. But still have some fundanmental function linked failed, such as at::native::xpu::batch_norm_elemt_kernel, at::native::xpu::sqrt_kernel.
c. I think if we want to fix this issue, my need re-structure code, it is a big effort. We need to discuss the priority to fix it.