Precompilation

[tfiers]

Would it make sense to add SnoopPrecompile and a short @precompile_all_calls block to Documenter?
It might speed up the first doc-build of a session.
I'd be up for making a PR.

[mortenpi]

Quite likely. The first run is always pretty slow. Would be happy to look at a PR.

[tfiers]

Just for reference for myself:
current doc build times on the linked GitHub Actions worfklow are very slow

Example Run.
Total workflow time (only docs/make.jl, no test/): about 24 minutes.

Breakdown, from timestamped log (lines):

Thu, 29 Dec 2022 11:21:11 GMT <makedocs> (Including Documenter.HTML(…) construction call)
Thu, 29 Dec 2022 11:21:15 GMT SetupBuildDirectory: setting up build directory.
Thu, 29 Dec 2022 11:29:17 GMT Doctest: running doctests.
Thu, 29 Dec 2022 11:33:41 GMT ExpandTemplates: expanding markdown templates.
Thu, 29 Dec 2022 11:43:10 GMT CrossReferences: building cross-references.
Thu, 29 Dec 2022 11:43:34 GMT CheckDocument: running document checks.
Thu, 29 Dec 2022 11:43:35 GMT Populate: populating indices.
Thu, 29 Dec 2022 11:43:37 GMT RenderDocument: rendering document.
Thu, 29 Dec 2022 11:43:39 GMT HTMLWriter: rendering HTML pages.
Thu, 29 Dec 2022 11:45:19 GMT </makedocs>

i.e.

SetupBuildDirectory: 482 seconds  (8 minutes)
            Doctest: 264 seconds  (4 minutes)
    ExpandTemplates: 569 seconds  (9 minutes)
    CrossReferences:  24 seconds 
      CheckDocument:   1 second  
           Populate:   2 seconds 
     RenderDocument:   2 seconds 
         HTMLWriter: 100 seconds  (2 minutes)

(parsing script)

[KristofferC]

I somewhat doubt that a lot of that time is compilation time of Documenter itself.

[tfiers]

The same make.jl on my laptop is quite fast

EDIT: timings on my laptop:

julia> @time include("docs/make.jl")  # first run
…
39.462901 seconds (56.96 M allocations: 2.966 GiB, 3.82% gc time, 87.34% compilation time)

julia> @time include("docs/make.jl")  # second run
…
2.798503 seconds (2.19 M allocations: 138.908 MiB, 2.27% gc time, 63.69% compilation time: 66% of which was recompilation)

[KristofferC]

Yes, but isn't that mostly code of your package that has to get compiled? To check Documenter compilation time you can try generate docs for a very small package.

[mortenpi]

For a tiny test case, I get 13s first run with precompilation, 12s first run without precompilation, and 0.2s second run. So I still think it would be worth looking into if we can shift more of that into precompilation (particularly useful for workflows where you call julia make.jl repeatedly).

[mortenpi]

Although, regarding SnoopCompile as a dependency: is this something that gets often broken by the Julia nightlies?

One of the main considerations for Documenter dependencies is that they wouldn't break as Julia master marches on, since you don't want to force people to be fixing patching external dependencies as they work on their PRs (to get doc builds passing). Are the APIs SnoopCompile depends on all public and documented?

[KristofferC]

This is the code for SnoopPrecompile https://github.com/timholy/SnoopCompile.jl/blob/master/SnoopPrecompile/src/SnoopPrecompile.jl.

So I wouldn't say that the API are public and documented. But on the other hand, if SnoopPrecompile breaks, then the whole ecosystem breaks so it is likely to be fixed quite fast.

[mortenpi]

if SnoopPrecompile breaks, then the whole ecosystem breaks

Maybe then having it in Documenter would actually be a useful canary.

[KristofferC]

Yeah, I don't think there is a big harm in depending on SnoopPrecompile.

I did a minimal test at #2004.

[tfiers]

I did a minimal test at #2004.

That's an amazing time improvement.

Good to merge for me
(No more need for

I'd be up for making a PR.

)