Pyinstaller's use of SetDllDirectory can interfere with DLL loading in applications launched from Python Manager #376
Description
The Windows API provides a function called SetDllDirectory that allows a process to override the typical search order for DLLs. Calls to this function will impact any future launched child processes. The influence over the DLL search path is not accomplished by altering the PATH environment variable, so it can be difficult to determine when this has happened.
pyinstaller uses this function in frozen applications in order to provide support for custom ctypes DLLs in "onefile" deployments. Specifically, it adds the app's _internal directory to the DLL search path. The current version of the Python Manager ships a number of DLLs in this directory. When a user uses the Python Manager to launch a command prompt into a venv, this change to the DLL search order is inherited by that command prompt, and then by any Python interpreter launched within that command prompt.
Practically speaking, this prevents applications which might depend on a different version of some of these DLLs from finding them, even if they would otherwise be able to. For example, when attempting to load a ModelCenter workflow containing an embedded DOE via pyModelCenter, if this attempt is made from a command prompt launched by the Python Manager, the DOE tool is unable to load the oSL DOE algorithms. It seems likely to me that this is also the root cause of bug #375. In the ModelCenter use case, the specific problem is the presence of a copy of libcrypto-3-x64. As an aside, this DLL doesn't seem to be actually used by the Python Manager -- removing it did not seem to have any impact on the usability of the program, and happened to allow loading of those workflows in this specific case.
This is a "known issue" with pyinstaller. The pyinstaller wiki contains a suggested workaround, which is essentially to call that same Windows API function in such a manner that the changes to the search path are undone. Because of the intended use of the Ansys Python Manager (i.e. to be a convenient way to manage Python installations and .venvs), I think we should consider either applying his workaround to the Python Manager, or modifying it such that it launches command prompts for .venvs in such a way that they do not inherit this change to the DLL search path.
Workarounds
- Because Ansys Python manager creates regular venvs in a location accessible to the user, launching a command prompt directly and activating the venv yourself manually sidesteps the issue; the problem is only introduced when Ansys Python Manager launches a command prompt for you, not when Python is installed / the venv is created / packages are installed in the venv.
- The aforementioned workaround suggested by the
pyinstallerwiki can also be applied by a Python script which is about to launch a native application which would be affected by this. I was able to test this for the specific ModelCenter use case mentioned above. It's worth noting that the general principle behind the workaround (callingSetDllDirectorywith aNULLargument) could be applied by any code with access to the Windows API which has a chance to execute before the affected attempt to load a DLL that exists in the Python Manager's installation directory.