Skip to content

Commit 46e7f68

Browse files
committed
fix crash during torch model reload with inter op thread paramter set
torch allows setting the interop thread count only once and before any inter-op parallel work is started because it creates threadpool statically. To follow this constraint, check the interop thread count and set only if it's not yet set.
1 parent db70751 commit 46e7f68

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

src/libtorch.cc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -532,7 +532,9 @@ ModelState::ParseParameters()
532532
TRITONSERVER_ErrorDelete(err);
533533
}
534534
} else {
535-
if (inter_op_thread_count > 0) {
535+
if ((inter_op_thread_count > 0) &&
536+
(inter_op_thread_count != at::get_num_interop_threads()) &&
537+
(at::get_num_interop_threads() == std::thread::hardware_concurrency())) {
536538
at::set_num_interop_threads(inter_op_thread_count);
537539
LOG_MESSAGE(
538540
TRITONSERVER_LOG_INFO,

0 commit comments

Comments
 (0)