Rollup merge of #147888 - ZuseZ4:autodiff-with-download-ci, r=jieyouxu

enzyme/autodiff is compatible with download-ci=true

To my surprise autodiff just works out of the box with download-ci=true. Thanks to all the bootstrap people who over the past helped me to link properly against the sysroot LLVM, which seems to pay off here.
That also helps with enabling Rust in Enzyme CI, since CCache for some reason doesn't seem to cache LLVM correctly on their runners.

I verified that this works with
` ./configure --release-channel=nightly --enable-llvm-enzyme --enable-option-checking --disable-docs --enable-llvm-assertions --set llvm.download-ci-llvm=true`. However, shouldn't download-ci-llvm already be the default? Why do I still have to manually set it with this pr?

I tested it afterwards with
`./x.py test --stage 1 tests/codegen-llvm/autodiff`

r? bootstrap

closes #147535

Author: Zalathar

src/bootstrap/src/core/config/toml/llvm.rs

@@ -117,7 +117,7 @@ pub fn check_incompatible_options_for_ci_llvm(
         enable_warnings,
         download_ci_llvm: _,
         build_config,
-        enzyme,
+        enzyme: _,
     } = ci_llvm_config;
 
     err!(current_llvm_config.optimize, optimize);
@@ -139,7 +139,6 @@ pub fn check_incompatible_options_for_ci_llvm(
     err!(current_llvm_config.clang, clang);
     err!(current_llvm_config.build_config, build_config);
     err!(current_llvm_config.plugins, plugins);
-    err!(current_llvm_config.enzyme, enzyme);
 
     warn!(current_llvm_config.enable_warnings, enable_warnings);
 

src/bootstrap/src/utils/change_tracker.rs

@@ -571,4 +571,9 @@ pub const CONFIG_CHANGE_HISTORY: &[ChangeInfo] = &[
         severity: ChangeSeverity::Warning,
         summary: "`rust.lld = true` no longer automatically causes the `x86_64-unknown-linux-gnu` target to default into using the self-contained LLD linker. This target now uses the LLD linker by default. To opt out, set `target.x86_64-unknown-linux-gnu.default-linker-linux-override = 'off'`.",
     },
+    ChangeInfo {
+        change_id: 147888,
+        severity: ChangeSeverity::Info,
+        summary: "`llvm.enzyme` now works with `download-ci-llvm=true`.",
+    },
 ];

src/doc/rustc-dev-guide/src/autodiff/debugging.md

@@ -25,7 +25,7 @@ The actual numbers will depend on your code.
 
 ## 2) Check your llvm-ir reproducer
 
-To confirm that your previous step worked, we will use llvm's `opt` tool. find your path to the opt binary, with a path similar to `<some_dir>/rust/build/<x86/arm/...-target-triple>/build/bin/opt`. also find `llvmenzyme-19.<so/dll/dylib>` path, similar to `/rust/build/target-triple/enzyme/build/enzyme/llvmenzyme-19`. Please keep in mind that llvm frequently updates it's llvm backend, so the version number might be higher (20, 21, ...). Once you have both, run the following command:
+To confirm that your previous step worked, we will use llvm's `opt` tool. Find your path to the opt binary, with a path similar to `<some_dir>/rust/build/<x86/arm/...-target-triple>/ci-llvm/bin/opt`. If you build LLVM from source, you'll likely need to replace `ci-llvm` with `build`. Also find `llvmenzyme-21.<so/dll/dylib>` path, similar to `/rust/build/target-triple/enzyme/build/enzyme/llvmenzyme-21`. Please keep in mind that llvm frequently updates it's llvm backend, so the version number might be higher (20, 21, ...). Once you have both, run the following command:
 
 ```sh
 <path/to/opt> out.ll -load-pass-plugin=/path/to/build/<target-triple>/stage1/lib/libEnzyme-21.so -passes="enzyme" -enzyme-strict-aliasing=0  -s

src/doc/rustc-dev-guide/src/autodiff/installation.md

@@ -8,7 +8,7 @@ First you need to clone and configure the Rust repository:
 ```bash
 git clone [email protected]:rust-lang/rust
 cd rust
-./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
+./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true
 ```
 
 Afterwards you can build rustc using:
@@ -47,7 +47,7 @@ Then build rustc in a slightly altered way:
 ```bash
 git clone https://github.com/rust-lang/rust
 cd rust
-./configure --enable-llvm-link-shared --enable-llvm-plugins --enable-llvm-enzyme --release-channel=nightly --enable-llvm-assertions --enable-clang --enable-lld --enable-option-checking --enable-ninja --disable-docs
+./configure --release-channel=nightly --enable-llvm-enzyme --enable-llvm-assertions --enable-option-checking --disable-docs --set llvm.download-ci-llvm=true
 ./x dist
 ```
 We then copy the tarball to our host. The dockerid is the newest entry under `docker ps -a`.
@@ -84,5 +84,5 @@ cd build
 cmake .. -G Ninja -DLLVM_DIR=<YourLocalPath>/llvm-project/build/lib/cmake/llvm/ -DLLVM_EXTERNAL_LIT=<YourLocalPath>/llvm-project/llvm/utils/lit/lit.py -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=YES -DBUILD_SHARED_LIBS=ON
 ninja
 ```
-This will build Enzyme, and you can find it in `Enzyme/enzyme/build/lib/<LLD/Clang/LLVM>Enzyme.so`. (Endings might differ based on your OS).
+This will build Enzyme, and you can find it in `Enzyme/enzyme/build/lib/<LLD/Clang/LLVM/lib>Enzyme.so`. (Endings might differ based on your OS).