CI docbuild: Run containers explicitly with docker run, docker exec

[mkoeppe]

Running the container manually instead of using the (convenient, but restrictive) container feature of GH Actions gives us more control:

• we can create more space on the host if necessary; we just scraped by an out of space condition in CI docbuild: Do not rebuild sagelib from scratch #36473 / Revert to non-incremental docbuild #36469 (comment) (however, there also exist a solution that calls out from the container to the host for the same effect)
• we can run some operations outside of the container but in the same job; this will make the separate "Get CI fixes" jobs unnecessary, addressing the cosmetic concerns from CI: Merge open blocker PRs in all CI workflows + other improvements #36338 (comment), ci: improve things around auto-apply of fixes #36349
• creating the container explicitly is also preparation for build sharing / caching between the various workflows, as discussed in CI: Merge workflows into build.yml #36446 and implemented in the follow-up PR CI build, doc-build: Run containers explicitly, separate jobs for pyright, build, modularized tests, long tests #36498

Here we do the work for doc-build.yml and doc-build-pdf.yml only; the same is done in #36498 for build.yml.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

• Depends on CI: Make jobs more responsive to canceling #36617 (merged here to remove merge conflict)
• Depends on Deploy live doc preview with pdf docs #36714 (merged here to remove merge conflict)

[kwankyu]

LGTM.

Voilà. The cosmetic concern is now solved.

[mkoeppe]

Thanks!

[tobiasdiez]

As explained in the similar PR #36498, I'm strongly against reimplementing the container feature.

[kwankyu]

It seems to mostly complicate the setup and essentially reimplements the container feature of github.

It seems not very complicated. It's a bit ugly though. But beauty is the least concern in the business of pipelining.

For the out-of-space issue it would be nice to first know what's the root cause to be able to evaluate if steps to fixing it go into the correct direction.

Yes. I am curious about that. Do you have any clue?

For potential speedups by caching the docker image, I'm not sure if this is really needed. Yes, from a conceptual viewpoint you only want to build sagelib once, and then reuse it - but well, it mostly takes 2-10 mins of incremental compilation anyway.

Yes, there are potential speedups, so why not try?

Personally, I think we have more pressing issues with the ci setup

I believe Matthias is working on this PR for pressing issues. One of them is the out-of-space issue. This PR gives us some time to investigate the root cause leisurely.

What are your pressing issues? Does this PR hinders solving them?

and better not introduce more moving parts (atm at least).

It's working well. Why is it a "moving part"?

Personally most important is that it fixes the cosmetic issue!

[kwankyu]

If you disagree with this PR, then removing "positive review" is enough. "Requesting changes" seems inappropriate.

[mkoeppe]

Tobias wrote:

As explained in the similar PR #36498, I'm strongly against reimplementing the container feature.

As the record there shows, neither did he say that he is "strongly against" it, nor did he explain a basis for his opinion.

[tobiasdiez]

It seems to mostly complicate the setup and essentially reimplements the container feature of github.

It seems not very complicated. It's a bit ugly though. But beauty is the least concern in the business of pipelining.

It limits what we can do in the workflow. For example, I don't think it would be easily possible to run github actions in the container context (e.g. something like #36510).

For the out-of-space issue it would be nice to first know what's the root cause to be able to evaluate if steps to fixing it go into the correct direction.

Yes. I am curious about that. Do you have any clue?

My guess would be that the git acrobatics maybe replaced a symlink by a copy of the files. In my opinion step number one would be to clarify why our image got 10% bigger and step 2 would be to print the used space before the build (github gives 14gb, minus 7gb of the container image should be still 7gb available - enough for a full build).

For potential speedups by caching the docker image, I'm not sure if this is really needed. Yes, from a conceptual viewpoint you only want to build sagelib once, and then reuse it - but well, it mostly takes 2-10 mins of incremental compilation anyway.

Yes, there are potential speedups, so why not try?

Caching the docker image doesn't decrease the time until you see the test results and built docs. You cannot start testing before the build is done, so you have to wait for the build and then do the next steps. In fact, it would be slower as caching and restoring a 7gb image takes about 3-5 mins. It would only reduce github's cpu time, but this is free anyway.

and better not introduce more moving parts (atm at least).

It's working well. Why is it a "moving part"?

It's one more thing that we have to maintain and that might break (similar to the "apply ci fixes" that also seemed to work and then broke the ci in many different ways). The github container feature is maintained by github, and not by us.

Personally most important is that it fixes the cosmetic issue!

Yes, for me as well. But we all know that this can be also fixed within the boundaries of the docker container.

[tobiasdiez]

If you disagree with this PR, then removing "positive review" is enough. "Requesting changes" seems inappropriate.

It would be nice if the developer docs would contain a guide on how to handle such situations.

[mkoeppe]

there are potential speedups, so why not try?

Caching the docker image doesn't decrease the time until you see the test results and built docs. You cannot start testing before the build is done, so you have to wait for the build [...] It would only reduce github's cpu time, but this is free anyway.

That's an oversimplification. We do experience situations in which our CI queue is congested, namely in the 36 hours after a release has been made. Caching the built image can help reduce the congestion.

[mkoeppe]

It limits what we can do in the workflow. For example, I don't think it would be easily possible to run github actions in the container context

That's correct, but I am not aware of a need to run any Actions on the container in which the docbuild takes place.
Obviously it's easy to switch back to using the container feature, should a need for this arise that is more important than caching.

[mkoeppe]

Let's get this in please.

[kwankyu]

Hopefully Tobias doesn't object anymore.

[tobiasdiez]

Sorry, but in its current form the disadvantages of this PR outweigh the positives. I might change my opinion once its clear how the caching should work, but with the information provided so far I don't see how caching will give us any improvement (given the limited cache quota of 10GB and the relatively slow cache upload/retrieval speeds).

[jhpalmieri]

How do we resolve the conflict here? A poll on sage-devel?

[mkoeppe]

I am preparing the caching technique, which Tobias says he needs to see.

Should be ready in a bit.

[github-actions]

Documentation preview for this PR (built with commit 082585b; changes) is ready! 🎉

[mkoeppe]

Closing this PR because it is merged into #36498, where the work will continue.