Fix path logic on 1.9.0-DEV.725 and newer #694

pfitzseb · 2022-07-20T09:27:43Z

Fixes #691. Fixes #687.

jishnub · 2022-07-20T12:05:04Z

This resolves #691 on binaries, but makes things worse on an installation from source. Using the tagged version of Revise, the following works (I'm tracking the source while using Cthulhu)

(@v1.9) pkg> st
Status `~/.julia/environments/v1.9/Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [f68482b8] Cthulhu v2.6.3
  [c3a54625] JET v0.6.0
  [295af30f] Revise v3.3.3

julia> using Cthulhu

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> versioninfo()
Julia Version 1.9.0-DEV.998
Commit e1739aa42a (2022-07-18 10:27 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.5 (ORCJIT, tigerlake)
  Threads: 1 on 8 virtual cores
Environment:
  LD_LIBRARY_PATH = :/usr/lib/x86_64-linux-gnu/gtk-3.0/modules
  JULIA_EDITOR = vim

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia/src"

but using this PR

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia"

julia> using Cthulhu

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

┌ Error: unable to find path containing source for Base, tracking is not possible
└ @ Revise ~/Dropbox/JuliaPackages/Revise.jl/src/recipes.jl:62
[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end┌ Warning: /home/jishnu/julia/usr/share/julia/base/special is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/strings is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/compiler/ssair is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/ryu is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/ is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570
┌ Warning: /home/jishnu/julia/usr/share/julia/base/docs is not an existing directory, Revise is not watching
└ @ Revise /home/jishnu/Dropbox/JuliaPackages/Revise.jl/src/packagedef.jl:570


Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> @which sin(1)
sin(x::Real)
     @ Base.Math ~/julia/usr/share/julia/src/base/math.jl:1460

This is a strange state, where it actually manages to track Base, but also prints an error indicating otherwise

pfitzseb · 2022-07-20T12:28:49Z

Does /home/jishnu/julia/usr/share/julia/base/ exist for you? It should be a symlink to /home/jishnu/julia/base/ (unless my build dir is in a weird state).

jishnub · 2022-07-20T12:35:46Z

/home/jishnu/julia/usr/share/julia/base/ doesn't exist for me.

julia> Revise.juliadir
"/home/jishnu/julia/usr/share/julia"

julia> isdir(Revise.juliadir)
true

julia> isdir(joinpath(Revise.juliadir, "base"))
false

$ ls /home/jishnu/julia/usr/share/julia
base.cache  cert.pem  julia-config.jl  src  stdlib  test

$ ls -dl /home/jishnu/julia/usr/share/julia/src/base
drwxrwxr-x 7 jishnu jishnu 4096 Jul 20 17:27 /home/jishnu/julia/usr/share/julia/src/base

/home/jishnu/julia/usr/share/julia/src/base exists, but it's not a symlink. I wonder if my installation is in an unusual state (although I've not done anything non-standard). Something similar was reported on discourse as well

pfitzseb · 2022-07-20T13:31:11Z

Ok, that should be fixed now. Turns out the symlink from julia/usr/share/julia/base/ to julia/src/base doesn't exist anymore (I still had it because of an old build).

jishnub · 2022-07-20T13:58:34Z

Can confirm that this is fixed now. Thank you so much for this!

I have one unrelated question, although I'm unsure if this is a Revise issue or a Cthulhu one:

julia> using Cthulhu

julia> @which sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460

julia> @descend sin(1)
sin(x::Real)
     @ Base.Math math.jl:1460
│ ─ %-1  = invoke sin(::Int64)::Float64
1461 1 ─ %1 = Base.sitofp(Float64, x)::Float64                                                                                                                            │╻╷╷ float
1463 2 ─ %2 = invoke Base.Math.sin(%1::Float64)::Float64                                                                                                                  │   
     └──      return %2                                                                                                                                                   │   
Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

[ Info: tracking Base
    @eval function ($f)(x::Real)
        xf = float(x)
        x === xf && throw(MethodError($f, (x,)))
        return ($f)(xf)
    end

Select a call to descend into or ↩ to ascend. [q]uit. [b]ookmark.
Toggles: [o]ptimize, [w]arn, [h]ide type-stable statements, [d]ebuginfo, [r]emarks, [e]ffects, [i]nlining costs, [t]ype annotations, [s]yntax highlight for Source/LLVM/Native.
Show: [S]ource code, [A]ST, [T]yped code, [L]LVM IR, [N]ative code
Actions: [E]dit source code, [R]evise and redisplay
Advanced: dump [P]arams cache.
 • %2  = invoke sin(::Float64)::Float64
   ↩

julia> @which sin(1)
sin(x::Real)
     @ Base.Math ~/julia/usr/share/julia/src/base/math.jl:1460

The file name display format in the output of @which seems to change after I track the source with Cthulhu. Why is this? Is there type-piracy involved in either of the packages?

pfitzseb · 2022-07-20T14:18:29Z

Mh, I think that happens because Revise writes the full path into the method table:

julia> @which(gcd(2,3)).file
Symbol("intfuncs.jl")

julia> using Revise

julia> @which(gcd(2,3)).file
Symbol("intfuncs.jl")

julia> Revise.track(Base)

# make some changes to gcd

julia> @which(gcd(2,3)).file
Symbol("/home/pfitzseb/Documents/Git/julia/usr/share/julia/src/base/intfuncs.jl")

No type piracy happening at all.

jishnub · 2022-07-20T14:25:17Z

I see. Perhaps type-piracy isn't the right word, but I wonder if writing the full path isn't something that should be done by Base in this case? If revise gets the path to the file wrong, it corrupts subsequent introspection, as seen in JuliaDebug/Cthulhu.jl#299. Getting the path wrong will likely be a bug anyway, but the introspection failure may be hard to trace back to Revise.

pfitzseb · 2022-07-20T14:33:51Z

So you'd want Base to write the full path into the method table? Not sure that's possible without a performance hit on startup. Anyways, this is unrelated to this PR (or did this not happen in the past?).

jishnub · 2022-07-20T15:24:21Z

It's unrelated to this PR. Maybe I'll discuss this elsewhere. Thanks again for this!

Keno · 2022-07-22T02:56:35Z

src/recipes.jl

+                # joinpath(juliadir, "..", "..", "..", "src", "base"), but that makes
+                # things even more confusing since e.g. `@edit` points towards the
+                # copy in usr/share
+                srcdir = fixpath(joinpath(juliadir, "src", "base"))


The same issue is present for Compiler later in this file.

timholy · 2022-07-23T08:14:17Z

I guess the key question is whether JuliaLang/julia#45441 is up for reconsideration.

timholy · 2022-08-25T07:16:01Z

With JuliaLang/julia#46420 merged, should we close this?

Fix path logic on 1.9.0-DEV.725 and newer

81b5b45

This was referenced Jul 20, 2022

juliadir is incorrectly inferred on nightly #691

Closed

Tracking Base is broken on master #687

Closed

Fix source-build Base path

1857853

jishnub mentioned this pull request Jul 20, 2022

unable to find path containing source for Base, tracking is not possible on nightly JuliaDebug/Cthulhu.jl#299

Closed

Keno reviewed Jul 22, 2022

View reviewed changes

also handle Core.Compiler

6897a20

This was referenced Jul 23, 2022

build: cleanup base files inclusion JuliaLang/julia#45441

Merged

bump coverage #700

Merged

t-bltg mentioned this pull request Aug 7, 2022

Remove dependency on Libc.TimeVal by directly calling time() instead #679

Merged

pfitzseb closed this Aug 25, 2022

timholy deleted the sp/fix-path-logic branch April 1, 2023 11:54

Uh oh!

Fix path logic on 1.9.0-DEV.725 and newer #694

Fix path logic on 1.9.0-DEV.725 and newer #694

Uh oh!

Conversation

pfitzseb commented Jul 20, 2022

Uh oh!

jishnub commented Jul 20, 2022

Uh oh!

pfitzseb commented Jul 20, 2022

Uh oh!

jishnub commented Jul 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pfitzseb commented Jul 20, 2022

Uh oh!

jishnub commented Jul 20, 2022

Uh oh!

pfitzseb commented Jul 20, 2022

Uh oh!

jishnub commented Jul 20, 2022

Uh oh!

pfitzseb commented Jul 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jishnub commented Jul 20, 2022

Uh oh!

Keno Jul 22, 2022

Choose a reason for hiding this comment

Uh oh!

timholy commented Jul 23, 2022

Uh oh!

timholy commented Aug 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jishnub commented Jul 20, 2022 •

edited

Loading

pfitzseb commented Jul 20, 2022 •

edited

Loading