Unsticky Tasks stick to threadpool since 1.10, causing issues since 1.12

[bbrehm]

Cf discourse https://discourse.julialang.org/t/is-the-operation-scheduling-a-task-blocking/133372

Consider

$ julia +1.11 -t 4 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
3
$ julia +1.12 -t 4 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
1

I think the responsible PR is Fix enq_work behavior when single-threaded by kpamnany · Pull Request #48702 · JuliaLang/julia · GitHub

This was a probably unintended breaking change because up to julia 1.11, the official and documented API for spawning a task in the :default threadpool was t=Task(fun); t.sticky=false; #= do some other setup =# schedule(t);.

Now, 1.12 silently changed the behavior (without deprecation warnings, not even with documentation update) such that this task now sticks to the threadpool. Hence, lots of old code will now silently degrade in performance / latency, in a way that unit tests are unlikely to notice. And some 1.12-era code will rely on the new (imo bad) behavior for correctness 😱

Sticky-ness now has 3 values: Stick-to-thread, Stick-to-pool, and unsticky, while we previously only had 2 values: Stick-to-thread and unsticky, and the update silently mapped unsticky->stick-to-pool.

[vchuravy]

FYI the official API to launch an unsticky task is https://docs.julialang.org/en/v1/base/multi-threading/#Base.Threads.@spawn

[sgaure]

The real problem occurs when you have more than 1 thread in :interactive, and you attempt to schedule a non-sticky compute-bound task to the :interactive pool. If the initial thread of that task happens to be the REPL-thread, the REPL will be blocked, even if there are more idle threads in the pool.

$ julia -t 4,2
julia> function busy(v)
           for i = 1:999999
               circshift!(v, rand(Int))
               reverse!(v)
           end
       end;
julia> const v = collect(1:99999);
julia> const u = collect(1:99999);
julia> Threads.@spawn busy(v);   # fine
julia> Threads.@spawn :interactive busy(u); #  hangs in 1.12, not in 1.11

EDIT: Er..., no, it's not deterministic in either of them. Sometimes hang, sometimes not.

[bbrehm]

FYI the official API to launch an unsticky task is https://docs.julialang.org/en/v1/base/multi-threading/#Base.Threads.@spawn

Cf https://docs.julialang.org/en/v1/base/parallel/#Core.Task

It would be horrible if @spawn was the only supported way:

1. Macro-only APIs suck. Learn from the mistakes of the C world and how much you hate macros in header files that obscure (or refuse to specify) the actual symbols you need to link against.
2. It is sometimes very useful to have separate steps for creating a Task and starting it; especially if you have multiple tasks that need to refer to each other. (in that case, you first create all the Tasks, then use them to fill in references, and only then start them)

The real problem occurs when you have more than 1 thread in :interactive, and you attempt to schedule a non-sticky compute-bound task to the :interactive pool

I think these are two different issues / bugs? This issue takes umbrage at Tasks sticking to the parent threadpool even if the sticky-bit has been cleared, which differs from 1.11 behavior and breaks coding patterns that were valid in 1.11.

I think what you are taking umbrage at is the REPL being a sticky task? Which can clearly lead to the REPL becoming unresponsive if a non-yielding task gets scheduled on the same thread as the REPL, which can only happen if the non-yielding task got scheduled on :interactive, either due to this issue or because of user error (like Threads.@spawn :interactive busy(u);).

I have no idea whether the REPL could be make non-sticky, but that could help. But it wouldn't solve this issue.

[vchuravy]

Cf https://docs.julialang.org/en/v1/base/parallel/#Core.Task
It would be horrible if @Spawn was the only supported way:

Yes and that documentation explicitly out, that you need to follow the behavior of the spawn macro.

I agree that the docs about threadpools could be better, but if you are using the runtime internal API, you also need to

julia/base/threadingconstructs.jl

Line 524 in ba1e628

_spawn_set_thrpool(task, tp)

The crux of your issue seems to me that Julia 1.12 starts with an interactive threadpool by default. IMO it's correct that the Task/@async API are equivalent to being sticky to the spacing threadpool.

[bbrehm]

Before 1.9, setting sticky was all that @spawn ever did, and before 1.12 setting the threadpool explicitly on a new task was not required to make it fully unsticky; and the current docs very clearly state

To obtain the behavior of Threads.@spawn set the sticky bit manually to false.

Hence, I think the old behavior should be preserved -- and now is still time to declare this a bug in 1.12.0 and 1.12.1 and revert to the old behavior for 1.12.2.

The alternative would be to declaring that this is a breaking / behavior change in 1.11 -> 1.12, with incorrect documentation in 1.12.0 and 1.12.1 (and possibly 1.12.2 if we can't get a docfix in before).

[vchuravy]

~> julia +1.11 -t 4 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
2
~> julia +1.11 -t 4,1 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
1

~> julia +1.12 -t 4,0 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
3
~> julia +1.12 -t 4,1 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
1

 ~> julia +1.10 -t 4 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
2
~> julia +1.10 -t 4,1 -e "t=Task(()->Threads.threadid()); t.sticky=false; schedule(t); println(fetch(t))"
1

Note how Julia 1.10, 1.11, and 1.12 all give the same consistent behavior for your code, and the only difference is that the default on 1.12 has changed to --threads=N,1 from --threads=N,0

[bbrehm]

Ok, so

To obtain the behavior of Threads.@spawn set the sticky bit manually to false.

became wrong in 1.9 or 1.10; but this breakage stayed unnoticed until 1.12 changed the default threadpool size (at the moment of breakage, one should arguably have done something ugly like having setproperty(t::Task, :sticky, false) also set the threadpool to :default, but that ship has sailed anyways).

This sucks, because it means that there is no sane way to fix it (we can't break 1.10 and 1.11 code to fix older code).

I guess we need to fix the documentation, and get a usable API in place for splitting task creation and task scheduling: Make threadpool and sticky optional keyword arguments for Task(...), document that tasks inherit the parent task's threadpool, and expose threadpool as a property on Task (for discoveribility, such that people see this field when they look at repl completions).