Fix Cthulhu.jl#541 by catching IOError when calling REPL.Terminals.raw! #53392
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix JuliaDebug/Cthulhu.jl#541 This allows Cthulhu.jl to precompile with Julia release-1.11 branch.
|
Do we know what upstream change broke this? |
|
I do not know which upstream change caused this. The triggering code only seems to cause a problem during precompilation. using PrecompileTools
@setup_workload begin
input = Base.link_pipe!(Pipe(), reader_supports_async=true, writer_supports_async=true)
term = REPL.Terminals.TTYTerminal("dumb", input.out, devnull, devnull)
write(input.in, 'q')
descend(gcd, (Int, Int); terminal=term)
# declare we are done with streams
close(input.in)
endExecuting this on release-1.11 does not result in an error. using Cthulhu, REPL
function cthulhu_541()
input = Base.link_pipe!(Pipe(), reader_supports_async=true, writer_supports_async=true)
term = REPL.Terminals.TTYTerminal("dumb", input.out, devnull, devnull)
write(input.in, 'q')
descend(gcd, (Int, Int); terminal=term)
# declare we are done with streams
close(input.in)
end
cthulhu_541()
I traced the closure of the stream using this function. function Base.setproperty!(io::Base.PipeEndpoint, x::Symbol, v)
@async begin
if x === :status && v === Base.StatusClosing
@info "setproperty!(::PipeEndpoint, ...)" io x v
Base.show_backtrace(stdout, backtrace())
end
end
invoke(Base.setproperty!, Tuple{Base.LibuvStream, Symbol, typeof(v)}, io, x, v)
endStacktrace:
[1] setproperty!
@ ~/.julia/dev/Cthulhu/src/Cthulhu.jl:883 [inlined]
[2] (::Base.var"#readcb_specialized#818")(stream::Base.PipeEndpoint, nread::Int64, nrequested::UInt64)
@ Base ./stream.jl:679
[3] uv_readcb(handle::Ptr{Nothing}, nread::Int64, buf::Ptr{Nothing})
@ Base ./stream.jl:709
[4] process_events
@ ./libuv.jl:125 [inlined]
[5] wait()
@ Base ./task.jl:1009
[6] wait(c::Base.GenericCondition{Base.Threads.SpinLock}; first::Bool)
@ Base ./condition.jl:130
[7] wait
@ ./condition.jl:125 [inlined]
[8] wait_readnb(x::Base.PipeEndpoint, nb::Int64)
@ Base ./stream.jl:416
[9] eof(s::Base.PipeEndpoint)
@ Base ./stream.jl:106
[10] read(this::Base.PipeEndpoint, ::Type{UInt8})
@ Base ./stream.jl:1001
[11] read(io::Base.PipeEndpoint, ::Type{Char})
@ Base ./io.jl:961
[12] readbyte
@ ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/util.jl:15 [inlined]
[13] _readkey(stream::Base.PipeEndpoint)
@ REPL.TerminalMenus ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/util.jl:22
[14] readkey(stream::Base.PipeEndpoint)
@ REPL.TerminalMenus ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/util.jl:20
[15] request(term::REPL.Terminals.TTYTerminal, m::Cthulhu.CthulhuMenu; cursor::Int64, suppress_output::Bool)
@ REPL.TerminalMenus ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/AbstractMenu.jl:204
[16] request
@ ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/AbstractMenu.jl:181 [inlined]
[17] request(term::REPL.Terminals.TTYTerminal, msg::String, m::Cthulhu.CthulhuMenu; kwargs::@Kwargs{})
@ REPL.TerminalMenus ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/AbstractMenu.jl:262
[18] request(term::REPL.Terminals.TTYTerminal, msg::String, m::Cthulhu.CthulhuMenu)
@ REPL.TerminalMenus ~/src/julia/usr/share/julia/stdlib/v1.11/REPL/src/TerminalMenus/AbstractMenu.jl:260
[19] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, curs::Cthulhu.CthulhuCursor; override::Nothing, debuginfo::Symbol, optimize::Bool, interruptexc::Bool, iswarn::Bool, hide_type_stable::Bool, verbose::Nothing, remarks::Bool, with_effects::Bool, exception_type::Bool, inline_cost::Bool, type_annotations::Bool, annotate_source::Bool, inlay_types_vscode::Bool, diagnostics_vscode::Bool, jump_always::Bool)
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:561
[20] _descend(term::REPL.Terminals.TTYTerminal, interp::Cthulhu.CthulhuInterpreter, mi::Core.MethodInstance; kwargs::@Kwargs{iswarn::Bool})
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:791
[21] _descend(term::REPL.Terminals.TTYTerminal, args::Any; interp::Core.Compiler.NativeInterpreter, kwargs::@Kwargs{iswarn::Bool})
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:807
[22] __descend_with_error_handling(args::Any; terminal::Any, kwargs...)
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:222
[23] _descend_with_error_handling(f::Any, argtypes::Any; kwargs::@Kwargs{iswarn::Bool, terminal::REPL.Terminals.TTYTerminal})
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:211
[24] descend_code_typed(::Any, ::Vararg{Any}; kwargs::@Kwargs{terminal::REPL.Terminals.TTYTerminal})
@ Cthulhu ~/.julia/dev/Cthulhu/src/Cthulhu.jl:169 |
|
Note that this change aims to make the Julia release-1.11 branch compatible with Cthulhu.jl v2.11.0. It does not resolve all the issues on the master branch of Julia due to issues outlined in JuliaDebug/Cthulhu.jl#547 . |
|
@KristofferC please backport to release-1.11 |
|
Imo, we should try figure out a bit more about what happened here. Catching an error and just ignoring it feels a bit odd since presumably something is going wrong then? |
|
I will bisect. Could someone remind me how the list the available nightlies so I can establish some coarse bounds? That said I still think we should do some error handling as I have done here. We are merely trying to reset raw mode on a TTY terminal. What is the point of failing when the underlying streams are already closed? |
|
The following works just fine. julia> using Cthulhu
julia> versioninfo()
Julia Version 1.11.0-DEV.1550
Commit 604609a3fc* (2024-02-10 19:26 UTC)
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 8 × AMD FX(tm)-8350 Eight-Core Processor
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, bdver1)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)
julia> using Pkg
julia> pkg"status"
Status `~/.julia/environments/v1.11/Project.toml`
[f68482b8] Cthulhu v2.11.0
julia> pkg"status -m"
Status `~/.julia/environments/v1.11/Manifest.toml`
[1520ce14] AbstractTrees v0.4.4
[da1fd8a2] CodeTracking v1.3.5
[f68482b8] Cthulhu v2.11.0
[1eca21be] FoldingTrees v1.2.1
[70703baa] JuliaSyntax v0.4.8
[aea7be01] PrecompileTools v1.2.0
[21216c6a] Preferences v1.4.1
[d265eb64] TypedSyntax v1.2.4
[b8c1c048] WidthLimitedIO v1.0.1
[2a0f44e3] Base64 v1.11.0
[ade2ca70] Dates v1.11.0
[b77e0a4c] InteractiveUtils v1.11.0
[d6f4376e] Markdown v1.11.0
[de0858da] Printf v1.11.0
[3fa0cd96] REPL v1.11.0
[9a3f8284] Random v1.11.0
[ea8e919c] SHA v0.7.0
[6462fe0b] Sockets v1.11.0
[f489334b] StyledStrings v1.11.0
[fa267f1f] TOML v1.0.3
[cf7118a7] UUIDs v1.11.0
[4ec0a83e] Unicode v1.11.0
julia> ccall(:uv_version, Cuint, ())
0x00020000
julia> ccall(:uv_version_string, Cstring, ()) |> unsafe_string
"2.0.0-dev" |
|
One observation of why this issue did not occur before is that under 604609a the call to The problem occurs now on master because the status is changed to |
|
I can confirm that the issue was introduced with e5496e0 (#49937) |
|
An alternative fix involves checking for in_stream = pipe_reader(term)
if isopen(in_stream) && in_stream.status != Base.StatusClosing
REPL.Terminals.raw!(term, false)
endI do not like that low-level libuv implementation details are leaking across abstraction barriers though. Another option is to define Lines 379 to 390 in 02699bb function isopen(x::Union{LibuvStream, LibuvServer})
if x.status == StatusUninit || x.status == StatusInit || x.handle === C_NULL
throw(ArgumentError("$x is not initialized"))
end
return x.status != StatusClosed && x.status != StatusClosing
end
function check_open(x::Union{LibuvStream, LibuvServer})
if !isopen(x)
throw(IOError("stream is closed or unusable", 0))
end
end Then could use this shorter snippet.. if isopen(term.in_stream)
REPL.Terminals.raw!(term, false)
endI went with the catching the IOError approach for the following reasons:
|
|
Thanks for the detailed investigation. So maybe just ignoring |
|
Should there be a method That probably should just be |
|
This is probably a libuv bug, though there is a small change it might be a kernel bug (I am still investigating with strace). It therefore can be worked around with |
The problem is that the stream is technically still open. It's closing,
Should we reconcile I think this is an orthogonal issue to the low-level libuv problems. |
That is a bit murky unfortunately. The kernel reported that the pipe was closed, but actually it wasn't closed. So libuv reported that the pipe was closed, even though the kernel lied about that, libuv didn't know that. So Julia reported that the pipe was closed, even though that was a lie from the kernel, it would be fairly hard for Julia to realize that was a lie, and fairly costly to have checked |
|
Fixed by libuv bump |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
catch IOError when using REPL.Terminals.raw!,Fix JuliaDebug/Cthulhu.jl#541
This allows Cthulhu.jl to precompile with Julia release-1.11 branch.