[question] Multiple calls to `@snoopi_deep`

[tlienart]

Hello,

I was trying out a few things in a project to reduce the number of inference triggers when I realised that if I ran @snoopi_deep after restarting Julia, I got a huge number of triggers whereas after that I only got 4, all from Base. Which result should I consider?

Also is it possible to filter the inference triggers by module?

Thanks!

[timholy]

Sorry to be slow to respond, #225 has been taking a lot of time.

You generally want to run @snoopi_deep in a fresh session. Once you infer a method, you generally don't need to reinfer it. (Redefining the method of course forces it to be reinferred.)

As far as filtering goes, for @snoopr-measured invalidations, you can use filtermod. That only filters on the root; we might want something that "hits" anything in a given module?

For @snoopi_deep inference triggers, should be able to do something like

filter(itrig -> itrig.callerframes[end].linfo.def.module === Franklin, itrigs)

That should probably become filtermod on a list of itrigs.

[tlienart]

Ah crap then I have 160 of those :/ (this is for another, much smaller module, which will eventually replace Franklin's parser: FranklinParser).

I think once I get the hang of reducing those, I might try to make a PR to extend the docs :)

Thanks again Tim!

[timholy]

It's pretty in flux right now. suggest can triage them to help eliminate ones you don't need to address, or that might need to be addressed elsewhere (e.g., Julia itself). For example, my investigations have led me to tackle the concatenation pipeline, and led to these PRs:

• Force-specialize on T in cat_similar JuliaLang/julia#39292
• Improve inferability of shape::Dims for cat JuliaLang/julia#39294
• Use lispy tuples in cat (fixes #21673) JuliaLang/julia#39314

and the net effect is an improvement in both runtime performance and precompilability.

I will play with FranklinParser at some point and see if I can help provide some context and/or additional tools. I recognize the complexity of these tools and am working to make them accessible to a wide audience, and you're very much in that target audience. So complain if it's unclear or too much work and I'll see what I can do to decrease the effort.

For now, the invalidations should be a much narrower set than all inference failures. In the long run, it may be easier to identify problems from the inference failures, but as you say right now the list of inference failures can be disturbingly long.

[timholy]

The main thing I'd recommend is to check whether you've tried the profile-guided despecialization GUI yet. https://timholy.github.io/SnoopCompile.jl/dev/pgdsgui/
I find it's a nice way to get a sense for the latency cost of design decisions.

[tlienart]

Thanks a lot Tim! There's one very clear benefit of all this is that it made me think "wait, does the compiler know what that thing is or is it only known at runtime" and while this is painful in some cases, I think overall it might push me and others to just do things better (eg I now stumbled upon the section in the Julia sections in the docs warning against abusing the despatch system, something I was certainly doing).

I'll keep trying and will provide specific feedback once I'm a bit more comfortable with these tools!

[timholy]

Very good. You don't have to optimize everything; if "abusing dispatch" makes your code cleaner, and you don't really care about the runtime or latency hit, there's no problem with it. But if you are trying to make lean, low-latency, performant code, then yes, these things matter.

[tlienart]

But if you are trying to make lean, low-latency, performant code, then yes, these things matter.

Yes well ultimately I'd like time to first serve in Franklin to be as low as possible, so if there are low hanging fruits to do so (and if it makes me a better Julia programmer along the way) I'll do it!

[timholy]

👍 I see you still have a lot of invalidations. Fixing those is probably the highest payoff. To make your life easier I just pushed #226. I'll post a demo in JuliaDocs/Franklin.jl#752.

[tlienart]

Thanks so much!

[tlienart]

I went from 160 to 9 inference triggers in FranklinParser 😅 , the remaining ones are a bit hard to track down but the situation is likely significantly better than it was before! thanks again, this is an amazing package.

[timholy]

Woot! That's awesome news. Great work!

SnoopCompile 2.5.0 just came out a couple of moments ago. It has quite a few changes to the inference triggers analysis code. In particular, for #222 (comment) the new aggregation of triggers by method, plus the ability to filter, may help reduce the daunting number.

[tlienart]

It's now down to zero 😅 and I think the code is all the better for it.

One small note: to "remove" the final one, I had to inline a function that I used to create a const dictionary used elsewhere something like

const RULES = LittleDict{...}(
  key1 => method(params1),
  key2 => method(params2),
  ...
)

The suggestion from SnoopCompile was to try precompiling the function. It's not immediately clear to me why the inlining worked but maybe that could be an extra example in the docs?

[timholy]

That sounds good. Can you point me somewhere so I can understand the changes in greater detail?

[tlienart]

https://github.com/tlienart/FranklinParser.jl/tree/validator

Some of the changes are pretty major (the parsing before was running with "oracle functions" that were encapsulating a condition for a section of string to meet, now this is done with concrete objects).

this is the inline: https://github.com/tlienart/FranklinParser.jl/blob/07d2ecd8de9ebc0b135d586fbf8d2da03c5695a7/src/tokens/utils.jl#L102-L117

the function forward_match is only used as a way to define rules like here: https://github.com/tlienart/FranklinParser.jl/blob/07d2ecd8de9ebc0b135d586fbf8d2da03c5695a7/src/tokens/markdown_tokens.jl#L8-L18

this: https://github.com/tlienart/FranklinParser.jl/blob/validator/src/tokens/find_tokens.jl is the function I ended up significantly refactoring in view of the output from SnoopCompile.

if you compare it with https://github.com/tlienart/FranklinParser.jl/blob/main/src/tokens/find_tokens.jl you'll see some, probably obvious, elements:

• making TokenFinder a proper object
• refactoring of the validation of candidates (instead of using functions as fields of the "TokenFinder", use objects that do the validation).

I also got rid of a few NTuple{N, Char} where N to replace by Vector{Char}, looks like inference wise this is better?