Support optional builds

[henryiii]

I think this can be supported with a few more override settings. We could provide:

• if.failed: If the build fails, the config settings gets rerun, and if any match this, the new configuration is used. This only happens once.
• if.system-cmake and if.cmake-wheel.
• message.after-failure: This will display if the build fails.
• fail = true: This will fail the build. Might be only allowed in overrides.

Should we allow failed to take the condition of failure? I think the conditions would be "configure", "build", and "install". Though maybe it's better to just wait until someone asks for the ability to differentiate?

The cmake one would trigger if you were not on a platform with a known cmake wheel and there's no sufficient CMake present, or always if there's no system CMake. If this matches,
then it affects if cmake is requested, as well.

This is what pure-python on non-supported CMake system would look like:

[[tool.scikit-build.overrides]]
if.no-system-cmake = "known-wheel"
wheel.cmake = false

Or if the build fails:

[[tool.scikit-build.overrides]]
if.failed = true
wheel.cmake = false

Custom message on failure:

[tool.scikit-build]
error-message = "The build failed!"

I think this would add a lot of flexibility; for an unrelated example, you could fail on unsupported Pythons:

[[tool.scikit-build.overrides]]
if.python-version = ">=3.13"
error-message = "Python 3.13+ is not supported yet!"
fail = true

I'm looking at a way to allow the cmake wheel to trigger the bootstrapping feature, and I think this would be a step in that direction too.

This is a very rough draft of what I'm thinking.

[maxbachmann]

So this would mean, that on installation failure a hook is called, which would allow me to handle the error? This would be awesome. I currently handle this using try/except inside my setup.py: https://github.com/maxbachmann/RapidFuzz/blob/main/setup.py which I would love to get rid of.

[henryiii]

Yes, this is exactly what this refers to.

[maxbachmann]

One thing that could lead to problems is the cmake/ninja dependency, because as far as I know there is no notion of try to install for dependencies in Python. So when cmake/ninja are not installed and no wheel is available on the corresponding platform the build could fail when building them. In rapidfuzz I currently handle this by always installing the Python fallback in this case. This is suboptimal when the dependency build would actually succeed, but at least it is guaranteed to succeed this way.

[henryiii]

Good point. I have the information about platforms (in https://github.com/scikit-build/scikit-build-core/blob/main/src/scikit_build_core/resources/known_wheels.toml ) so could do that, at least as an option.

[maxbachmann]

An alternative I thought of when I was working on this for rapidfuzz was an optional_cmake package. This package would do the same as the cmake package, but on failure it would simply install an empty package.

[henryiii]

I think we do the following to make this mostly possible with the new overrides.

1. Support wheel.cmake = False. This will disable requiring CMake/Ninja during the wheel step too, just like the opposite of what sdist.cmake = True does.
2. Support wheel.compiled = False. The default will be to match wheel.cmake. This will target purelib instead of platlib.
3. At this point, we could make scikit-build-core bootstrap itself if we wanted to. :)
4. Add a way to opt-into an override. Maybe provide a user-extendable namespace like custom or opt? For example:

pip install -e -Copt.compile="true"

Would match

[[tool.scikit-build.overrides]]
opt.compile = ".+"

(API might need a little work there, maybe would be an inline table with options or something)

We could also add if.env.<VAR>=<regex>.

[maxbachmann]

Hm so if I understand you correctly: wheel.cmake = False would

1. try to install cmake/ninja. Probably based on whether it is already installed / a wheel is available.
2. try to build the c extension
3. if 1) + 2) succeeds install everything
4. if 1) or 2) failed only install the pure python parts

A user would be able to override the behavior while installing using something like -Copt.compile="true".

The only other thing I would require is the ability to enforce the compile based on environment variables. In my case I do enforce the build when I am in a known packaging environment:

packaging = "1" in {
    os.environ.get("CIBUILDWHEEL", "0"),
    os.environ.get("CONDA_BUILD", "0"),
    os.environ.get("PIWHEELS_BUILD", "0"),
    os.environ.get("RAPIDFUZZ_BUILD_EXTENSION", "0"),
}
if packaging:
    run_setup(True)
else:
    try:
        run_setup(True)
    except BaseException:
        show_message(
            "WARNING: The C extension could not be compiled, speedups are not enabled.",
            "Failure information, if any, is above.",
            "Retrying the build without the C extension now.",
        )
        run_setup(False)
        show_message(
            "WARNING: The C extension could not be compiled, speedups are not enabled.",
            "Plain-Python build succeeded.",
        )

[henryiii]

No, I think it's the exact opposite part of the solution. This would give you the ability to enforce the compile based on environment variables. It doesn't handle the try/fail case. wheel.cmake = False would not add cmake to the build requirements, and would only do the pure Python part of the build. By putting it in an overrides section, that would make it conditional on whatever is in the if part of the overrides section (+ proposed opt section I guess).

Basically this would solve the if/else part above, but not the try/except. But I think the try/except part solution might be easy to get to from this. For example, maybe you could set one to be a fallback on failure or similar...

So if you had this:

packaging = "1" in {
    os.environ.get("CIBUILDWHEEL", "0"),
    os.environ.get("CONDA_BUILD", "0"),
    os.environ.get("PIWHEELS_BUILD", "0"),
    os.environ.get("RAPIDFUZZ_BUILD_EXTENSION", "0"),
}
if packaging:
    run_setup(True)
else:
    run_setup(False)

You could write this:

[tool.scikit-build]
wheel.cmake = false

[[tool.scikit-build.overrides]]
if.env.CIBUILDWHEEL = "1"
wheel.cmake = true

[[tool.scikit-build.overrides]]
if.env.CONDA_BUILD = "1"
wheel.cmake = true

[[tool.scikit-build.overrides]]
if.env.PIWHEELS_BUILD = "1"
wheel.cmake = true

[[tool.scikit-build.overrides]]
if.env.RAPIDFUZZ_BUILD_EXTENSION = "1"
wheel.cmake = true

(Probably should come up with a way to do OR...)

[maxbachmann]

Thanks for the clarification. Yes I completely misunderstood which part this was going to solve.

[henryiii]

See discussion at rapidfuzz/Levenshtein#65 for use case. I think this is what it would look like:

[tool.scikit-build]
wheel.cmake = true

[[tool.scikit-build.overrides]]
if.any.no-system-cmake = "known-wheel"
if.any.failed = true
wheel.cmake = false

[[tool.scikit-build.overrides]]
if.any.env.CIBUILDWHEEL = "1"
if.any.env.CONDA_BUILD = "1"
if.any.env.PIWHEELS_BUILD = "1"
if.any.env.RAPIDFUZZ_BUILD_EXTENSION = "1"
wheel.cmake = true
error-message = "failed to build C++ Extension in a packaged build"