Could we fix "Stack overflow in type inference" by moving compilation to another Task?

[NHDaly]

There have been several issues reporting Stack overflow during type inference, which can crash a julia process entirely.

Here are a few of them:

• Better error reporting for stack overflow in type inference: Line number of the function being inferred / call site #49237
• Stack overflow in type inference… julia 1.9.0-DEV #45286
• Internal error: stack overflow in type inference when displaying an array with 511 dimensions #48813
• Plus we've seen this a bunch in production, but haven't filed an issue for all of them.

One of the most common places this occurs is if we're in a try/catch, and we're trying to handle the exception we've caught, but it turns out that we're right up near the edge of the stack, and the exception we caught is a StackOverflowError, and so then in the catch block when trying to handle it (log it, switch on its type, report it to our users, abort the current operation, etc), we end up hitting a stack overflow during type inference since we're calling code for the first time that hasn't been compiled yet for this specific exception.

But it could also happen during normal julia operation: If you're about to dispatch to a function that would push you over the stack limit, but before you call, you need to compile it, you can hit this error.

---

It seems like this situation could be solved by having separate Task(s) where we do compilation, and switching over to them when we need to compile something. The benefit would be that we would avoid stack overflows during inference, which really screw up the runtime. The downside is that this task switch would likely make compilation more expensive.

Maybe first we need to work on performance of task switching?

[NHDaly]

really screw up the runtime

Including that we regularly see segfaults and/or signal 6 immediately following this error.

[nsajko]

As far as I understand/hope stack overflow in type inference is a thing of the past now after #55575.