Make Linux's webpki-roots optional and opt-in at runtime

[complexspaces]

Currently, the use of webpki-roots in the Linux verifier is not configurable and they are always loaded to supplement what might come from on-disk CA bundles. This, and the previous behavior of falling back to webpki-roots, was implemented at 1Password because we found openssl-probe to be way too unreliable and had multiple users write in that they were unable to use the app as a result.

However, this is likely not what everyone wants and makes it harder to control exactly what trust roots a consumer of rustls-platform-verifier might allow in addition to some other concerns:

I think webpki-roots should not be included on Linux, since they will inevitably bitrot - @Ralith

But there we're also some solutions presented as well:

I have mixed feelings about a feature, because those can end up silently enabled, which is a bit of a security hazard
maybe support manually adding certs and let people pipe in webpki-roots themselves if they really want to? - @Ralith

The use of webpki-roots can be an optional feature enabled with explicit API - @djc

Out of these, the last idea to enable it with an explicit API seems to be the best balance between convenience, a simple crate API, and control/security. So I believe that is the way we should go.

There are then two things needed to resolve this:

• Make all webpki-roots dependencies optional via an opt-in feature called webpki-roots or webpki-roots-augment.
• Add an API to the top-level crate like this which sets a global atomic and then update others.rs to use it on Linux:

pub fn enable_webpki_fall() {
   ...
}

[Ralith]

which sets a global atomic

Is it not practical to pass this configuration in when constructing a verifier, rather than using hidden global state?

[Ralith]

If posed in terms of "also trust these extra roots" in the verifier config, this could avoid a direct dep on webpki-roots and be useful for other cases as well.

[complexspaces]

Is it not practical to pass this configuration in when constructing a verifier, rather than using hidden global state?

Yes, that would be better 👍. The verifier isn't currently exposed in the public API (see #7) but we probably should do that to make the crate more usable.

If posed in terms of "also trust these extra roots" in the verifier config, this could avoid a direct dep on webpki-roots and be useful for other cases as well.

That could work too. We would already have an API asymmetry between UNIX and everything else with the webpki roots opt-in, so having a Linux-specific verifier method to do so isn't much further. My counterpoint is that users should really be putting any roots they want to trust in the platform's CA store instead but Linux has already strayed from that, so...

The last thing that comes to mind is that for a good percentage of users, make using this crate more annoying in a workspace since they'd need to write a shim crate that depends on webpki-roots instead for them vs just importing this crate and calling a method when constructing the verifier.

[Ralith]

On further consideration, use cases which need to trust arbitrary certs could straightforwardly wrap the rustls-platform-verifier Verifier (once exposed), so unless there's actually a very common demand for that, a less opinionated but less convenient API would have little value.

[djc]

I think the common pattern for this would be to have a Cargo feature that then enables an explicit opt-in API.

[djc]

Other idea I recently came up with is to have the verifier pull the Mozilla roots straight from the CCADB and refresh the in-memory set of roots on an interval (on a separate thread).

[jsha]

CCADB is super slow and unreliable (it's based on a Salesforce report), so it's not suitable for direct fetch by applications. Also any setup where we fetch fresh roots over a TLS connection puts a lot of trust in that connection and the server on the other side of it.

[cpu]

I put up a draft for this in #24. Input welcome.