Is botorch using multiple jobs (available CPU cores) by default?

[Hrovatin]

I am planning to use BoTorch on a HPC and was wondering if botorch by default uses multiple jobs (e.g. workers in its scipy-based optimization) or do I need to set something after having requested a job with more CPU cores?

[saitcakmak]

Hi @Hrovatin. BoTorch does use multiple CPU cores (or threads) for its operations, but it is primarily due to PyTorch tensor operations rather than the scipy optimizer. We actually intentionally limit the scipy optimizer to a single thread, since having it multi-threaded while the underlying PyTorch operations are also multi-threaded leads to too many threads being spawned (aka oversubscription), which can slow things down significantly.

In short, BoTorch by default will try to utilize all CPU cores available to it where appropriate (though you won't see 100% utilization). If you'd like to limit it to use only a subset of the CPU, you can use torch.set_num_threads to specify this. This might be a useful reference: https://docs.pytorch.org/docs/stable/notes/multiprocessing.html#avoid-cpu-oversubscription

[Hrovatin]

Thank you very much for clarifying @saitcakmak