Make it unnecessary to load Enzyme and FiniteDifferences to specify AutoEnzyme and AutoFiniteDifferences

[PTWaade]

Dear ADTypes maintainers,

Thank you for your work!

This issue is just to request that the AutoEnzyme and AutoFIniteDifferences constructors be updated so that it is not necessary to have Enzyme and FiniteDifferences loaded in order to construct them.
The reason this is currently necessary, is because AutoEnzyme requires the Forward and Reverse types to specify whether Enzyme is used in forward or reverse mode, and also requires set_runtime if it is to be used with Turing (which is my use-case); similarly, AutoFiniteDifferences requires importing central_fdm from FiniteDifferences to be constructed.

There are two reasons that it would be useful to not have to load the packages themselves to use the AutoAD constructors. Firstly, it strikes me as much easier to read and use, especially by people who just want to "turn on" their choice of AD in e.g. a Turing model, if things like forward and reverse mode can just be specified with a flag or an otherwise normal keyword argument. Secondly, and more importantly, in my case I would like to use AutoEnzyme as a default argument for a function, but I cannot do that without making Enzyme a hard dependency of my package.
Since the AutoEnzyme functionality is optional, I would like to avoid making it a hard dependency, and that users should just import Enzyme to use it.
I think in Turing's case there is the same problem, but with wanting to use FiniteDifferences as default, @penelopeysm tells me.

I'll note that this works fine for other constructors, such as AutoForwardDiff (whcih is currently used as default from Turing because of this), AutoReverseDiff and AutoMooncake.
I also discussed this with Enzyme, but they said that this isn't an issue they can fix, since it is in your package.

There may also be other AutoAD constructors in your (very useful!) list that have the same limitation; I haven't checked them all, Enzyme and FiniteDifferences are just the two I ran into. My suggestion would be to make it a general policy, however, that the ADTypes constructors do not require loading the corresponding AD package in order to be constructed.

There may of course be many good reasons that I am ignorant about for why using AD package specific types and functions in your constructors is useful, or that changing it now is complicated - so forgive me if this is a stupid question.

And thank you - for the consideration and time, and again, for your work :)

[gdalle]

Hi @PTWaade, thanks for opening an issue!

Firstly, it strikes me as much easier to read and use, especially by people who just want to "turn on" their choice of AD in e.g. a Turing model, if things like forward and reverse mode can just be specified with a flag or an otherwise normal keyword argument.

Note that AutoEnzyme() already works, and picks a reasonable mode depending on the context. For custom settings like runtime activity you do have to import Enzyme, but then again that makes sense: the natural way to specify a mode is the Enzyme mode object. Adding a second way to parametrize the backend object would only make things more confusing, because then you'd have ADTypes.EnzymeForward() mapping to Enzyme.Forward for no clear reason.

As for FiniteDifferences, we could indeed make it so that AutoFiniteDifferences() returns something, but that would require picking a default finite difference method, which the authors of the package haven't done. If it feels important,

Secondly, and more importantly, in my case I would like to use AutoEnzyme as a default argument for a function, but I cannot do that without making Enzyme a hard dependency of my package. Since the AutoEnzyme functionality is optional, I would like to avoid making it a hard dependency, and that users should just import Enzyme to use it.

Why not use a package extension?

Define this function in your core package:

function default_backend_choice

and then this method in your Enzyme-triggered extension

default_backend_choice() = AutoEnzyme(mode=Enzyme.Forward)

That way, Enzyme is not a hard dependency, but you can specify you default backend choice with as much granularity as you like, and it will only be callable when Enzyme as a package is present in the environment (which is necessary to compute gradients anyway).

[PTWaade]

Hey @gdalle

Thank you for your thoughtful response, it's appreciated!

A package extension is a good idea, I'll have a look at that. I don't know if it satisfies Turing's needs for the check_ad functionality (@penelopeysm); I'd have imagined they'd have considered the option already, being much more skilled programmers than I.

Re. AutoEnzyme, then you're right, the issue only comes up when setting runtime activity. The reason I forgot that is because I am using it with Turing, which I expect is a common use-case; and Turing needs runtime to be set. Until that might change (I am unsure how likely that is to happen), do you think there's any possibility of making setting runtime a keyword-like setting, just because I expect there to be many Turing users who would need it? The same with the Forward and Reverse options. If it is complicated to do (like I said, I and other Turing users just need to set runtime with one of the directions, so I haven't explored other settings), then I'll make do with it as it is.

And re FiniteDifferences: perhaps I should ask them if they have a reason that a default finite differences method isn't set? I suppose the reason it needs an import and not just a keyword is that there are many methods and possibly new ones that could be added, so it has to be dispatch based.

In any case, thank you for the time, and all the best!