PGO + Bolt

[zamazan4ik]

parseable right now does not support building with more advanced optimization techniques like PGO and BOLT. This tooling has an increasing adoption in the community as a tool to additionally optimize programs. With this tooling, there is a huge chance to gain even more performance "for free".

Here I suggest considering an option at least to play with LTO ("fat" version) + PGO + Bolt pipeline (or any combination of them) and test, does it give a performance to the project or not. If yes, would be awesome to have prebuilt binaries with more advanced optimization from the scratch. Also, for the users will be helpful to have the ability to tweak manually their own binaries to their own workloads with the integrated into the build scripts functionality.

Since the project is quite small I do not expect (at least yet) significant improvement from the PGO right now. But at least a good option to try :)

Also, there are some caveats to consider like:

• Significantly increased build times
• BOLT could be still unstable (or even broken) on some architectures

Links:

• Rust experience with LTO + PGO + BOLT: https://kobzol.github.io/rust/rustc/2022/10/27/speeding-rustc-without-changing-its-code.html

[nitisht]

Thank you for the issue @zamazan4ik we’ll review the blog post, looks interesting for sure - but as you said, at our current size it may not be the most useful.

[nitisht]

Closing this for now because we don't have cycles to explore this. We may get to this at a later stage

[zamazan4ik]

@nitisht I suggest you re-open the issue, so someone in the future can start working on it since the issue will be open.

[nitisht]

Thanks @zamazan4ik . Reopening now..

[de-sh]

I did primary analysis on results with LTO, just tracking times and binary size, here are the results:

main
________________________________________________________
Executed in  291.94 secs    fish           external
   usr time   60.53 mins  474.00 micros   60.53 mins
   sys time    1.63 mins  724.00 micros    1.63 mins

size: 186822568

With Thin LTO
+ [profile.release]
+ lto = "thin"
+ codegen-units = 1
________________________________________________________
Executed in  522.91 secs    fish           external
   usr time   47.94 mins  704.00 micros   47.94 mins
   sys time    1.43 mins  219.00 micros    1.43 mins

size: 159024032
   
With Fat LTO
+ [profile.release]
+ lto = "fat"
+ codegen-units = 1
________________________________________________________
Executed in   18.26 mins    fish           external
   usr time   44.87 mins    1.13 millis   44.87 mins
   sys time    1.53 mins    4.44 millis    1.53 mins
   
size: 150544072

I believe thin LTO should be good enough for perf, let's see if there's demand to improve things in the future and move accordingly?

As for PGO and LLVM-bolt optimizations, it might be too much and anyone interested would still be able to do this on their own when compiling themselves. Not sure if we should even guide them to do the same!