Hermetic build mode

[matklad]

Meta: this should be an RFC, but I am unlikely to write one soon. Creating mostly to coordinate discussion.

Problem: today, it's possible to get byte-for-byte identical results compilation results (especially with wasm), but it is impossible to guarantee, because ambient environment (~/.cargo/config, CARGO_HOME, etc) can influence the build. That is, builds are reproducible, but are not hermetic. As a specific example, if I publish a wasm project with Cargo.lock and rust-toolchain.toml, folks running cargo build will generally get the same results, unless someone has RUSTFLAGS set.

Solution: add a mechanism to Cargo to out-out of ambient environment. Strawman proposal -- when --hermetic flag is passed, cargo guarantees that the builds are only affected by the contents of the current working directory.

I don't know an exhaustive list of things that hypothetical --hermetic should forbid, but here are some candidates:

• access to .cargo/config outside of the current directory
• access to env vars like RUSTFLAGS, unless explicitly opted into (a-la --hermetic --allow-env RUSTFLAGS)
• access to PATH? This intersects with rustup. compiler version obviously affects the results, and we need compiler, so PATH is importaet. Perhaps printing rustc -vv during hermetic build is the way to go?
• --hermetic should imply --locked (valid lockfile)
• --hermetic should not imply --frozen. That is, Cargo should be able to fetch sources from the internet (and uses CARGO_HOME for caches), as long as it guarantees that the result is reproducible.

A related feature is #7894, #7887. It allows to control .cargo/config specifically. I, however, feel that we want to make a more specific promise about properties (hermeticity), and tread config handling as an implementation detail.

cc @jsgf

[jsgf]

access to PATH? This intersects with rustup. compiler version obviously affects the results, and we need compiler, so PATH is importaet. Perhaps printing rustc -vv during hermetic build is the way to go?

It's definitely an antipattern to use env!() or std::env::var at build time to get PATH as a runtime default, but it's not unknown. Similarly, searching for libraries at build time with LD_LIBRARY_PATH or similar.

In general, should cargo control what env vars are available to builds? I have a proposal for rustc to make env control much tighter, by supplying a logical environment to rustc via --env parameters, so that the effective environment is completely controllable (including things like overriding PATH even if its needed to actually execute rustc).

[matklad]

--hermetic should also pass appropriate --remap-path-prefix: #5505

I think the current state of the art is to export RUST_FLAGS="--remap-path-prefix $PWD=/pwd --remap-path-prefix $CARGO_HOME=/cargo_home"

[LegNeato]

access to PATH? This intersects with rustup. compiler version obviously affects the results, and we need compiler, so PATH is importaet. Perhaps printing rustc -vv during hermetic build is the way to go?

Perhaps require a rust-toolchain.toml?

[bjorn3]

What about the linker or the C compiler? Different versions can result in different compilation outputs. Should rust-lld be forcefully used instead of the platform linker (probably uncontroversial) and usage of a C compiler prevented (probably controversial)?

Should this use seccomp-bpf or other os mechanisms to prevent not just filesystem access, but as much system calls as possible?

Should RUSTFLAGS=-Ctarget-cpu=native be ignored? Should other RUSTFLAGS or RUSTC be ignored?

[FloLo100]

People might not only want to protect the binary from side-effects of the build-system but also the build-system from side effects of building (and therefore subsequent builds on the same system) when using the catchphrase "hermetic builds".

To offer these features thoroughly, the use of container-solutions (Docker/linux-namespaces, I'm unsure what exists on Windows or MacOS) is necessary. The question then is, whether cargo itself should offer a mode where it spawns such containers, or if merely compatibility and ease of use of third-party tools that spawn build-containers should be the goal here. It's important to note, that container-in-container won't work. So if we spawn containers, people cannot put cargo into a container while using that mode.

At my company we have started to implement a container solution using bubblewrap-containers. I could share our development if that's the direction we want to go into.