feat(REPL): Make shell command handling extensible via dispatch; add Nushell

[MilesCranmer]

See #58525 for an alternative approach.

---

This PR cleans up the branching over shells for better extensibility and maintainability. It also adds a more compatible interface with nushell. This is an alternative to #58413 by @sandyspiers and supersedes it (if desired). Also supersedes #58518.

Fixes #54291.

It introduces a function prepare_shell_command which dispatches on a struct ShellSpecification{is_windows, shell_sym}. This is a bit cleaner than the manual handling of if !Sys.iswindows() and if shell_name == "fish". It also allows users to customize REPL logic for new shells, should they wish to. This addresses the issue I had with #54291 where there was simply no way to customize the REPL shell command handling. Now, the next time a user wants to interface with their shell, they can just overload

Base.prepare_shell_command(::ShellSpecification{false,:my_new_shell}, cmd, raw_string) = #= ... =#

(Another idea I had was passing the full Sys.KERNEL as the third type parameter, for customizations on other OSes. I thought it probably wasn't needed though.)

Note that repl_cmd now receives the raw command string as the second argument. This also means Nushell, which takes the raw string, can correctly handle pipes (e.g., ls | sort-by name).

Here's the main logic of this change:

struct ShellSpecification{is_windows,shell} end

function prepare_shell_command(::ShellSpecification{true,shell}, cmd, _) where {shell}
    return cmd # This was handled with the `if !Sys.iswindows()` branch
end
function prepare_shell_command(::ShellSpecification{false,shell}, cmd, _) where {shell}
    # This was handled with the `else` branch
    shell_escape_cmd = "$(shell_escape_posixly(cmd)) && true"
    return `$shell -c $shell_escape_cmd`
end
function prepare_shell_command(::ShellSpecification{false,:fish}, cmd, _)
    # This was handled with the `shell_name == "fish"` branch
    shell_escape_cmd = "begin; $(shell_escape_posixly(cmd)); and true; end"
    return `fish -c $shell_escape_cmd`
end
function prepare_shell_command(::ShellSpecification{false,:nu}, _, raw_string)
    return `nu -c $raw_string` # This is new logic to handle nushell
end

function repl_cmd(cmd, raw_string, out)  # repl_cmd now takes 3-args:
    #= ... =#
        shell_spec = ShellSpecification{Sys.iswindows(),Symbol(shell_name)}()
        prepared_cmd = prepare_shell_command(shell_spec, cmd, raw_string)
    #= ... =#
end

Another design I considered was having a dictionary SHELL_COMMANDS but I thought the method dispatch version was ultimately cleaner and more extensible.

---

If you want to try this without re-compiling Julia, here's some code you can put in your startup.jl. (This code also works on 1.11 so this lets you use nushell on stable Julia!)

Code: (click me)

@static if isinteractive()
    @eval begin
        using REPL: REPL, Prompt, SHELL_PROMPT, ShellCompletionProvider, respond, outstream

        atreplinit(repl -> REPL.numbered_prompt!(repl))

        struct ShellSpecification{is_windows,shell} end

        function prepare_shell_command(::ShellSpecification{true,shell}, cmd, _) where {shell}
            return cmd
        end
        function prepare_shell_command(::ShellSpecification{false,shell}, cmd, _) where {shell}
            shell_escape_cmd = "$(shell_escape_posixly(cmd)) && true"
            return `$shell -c $shell_escape_cmd`
        end
        function prepare_shell_command(::ShellSpecification{false,:fish}, cmd, _)
            shell_escape_cmd = "begin; $(shell_escape_posixly(cmd)); and true; end"
            return `fish -c $shell_escape_cmd`
        end
        function prepare_shell_command(::ShellSpecification{false,:nu}, _, raw_string)
            return `nu -c $raw_string`
        end

        function custom_repl_cmd(cmd, raw_string, out)
            shell = Base.shell_split(get(ENV, "JULIA_SHELL", get(ENV, "SHELL", "/bin/sh")))
            shell = isempty(shell) ? ["/bin/sh"] : shell
            shell_name = Base.basename(shell[1])

            # Immediately expand all arguments, so that typing e.g. ~/bin/foo works.
            cmd.exec .= expanduser.(cmd.exec)

            if isempty(cmd.exec)
                throw(ArgumentError("no cmd to execute"))
            elseif cmd.exec[1] == "cd"
                if length(cmd.exec) > 2
                    throw(ArgumentError("cd method only takes one argument"))
                elseif length(cmd.exec) == 2
                    dir = cmd.exec[2]
                    if dir == "-"
                        if !haskey(ENV, "OLDPWD")
                            error("cd: OLDPWD not set")
                        end
                        dir = ENV["OLDPWD"]
                    end
                else
                    dir = homedir()
                end
                try
                    ENV["OLDPWD"] = pwd()
                catch ex
                    ex isa IOError || rethrow()
                    # if current dir has been deleted, then pwd() will throw an IOError: pwd(): no such file or directory (ENOENT)
                    delete!(ENV, "OLDPWD")
                end
                cd(dir)
                println(out, pwd())
            else
                shell_spec = ShellSpecification{Sys.iswindows(),Symbol(shell_name)}()
                prepared_cmd = prepare_shell_command(shell_spec, cmd, raw_string)
                try
                    run(ignorestatus(prepared_cmd))
                catch
                    # Windows doesn't shell out right now (complex issue), so Julia tries to run the program itself
                    # Julia throws an exception if it can't find the program, but the stack trace isn't useful
                    lasterr = current_exceptions()
                    lasterr = ExceptionStack([(exception = e[1], backtrace = [] ) for e in lasterr])
                    invokelatest(display_error, lasterr)
                end
            end
            nothing
        end

        @async begin
            # This needs to be `@async`, because it has to run
            # AFTER the REPL is initialized, and can't do it
            # during `atreplinit` (since the interface gets overwritten!)
            while true
                if (
                    isdefined(Base, :active_repl) &&
                    isdefined(Base.active_repl, :interface) &&
                    isdefined(Base.active_repl.interface, :modes) &&
                    !isempty(Base.active_repl.interface.modes)
                )
                    break
                end
                sleep(0.001)
            end
            repl = Base.active_repl
            julia_prompt = repl.interface.modes[1]
            hascolor = true # HACK
            new_shell_prompt = Prompt(
                SHELL_PROMPT;
                prompt_prefix = hascolor ? repl.shell_color : "",
                prompt_suffix = hascolor ?
                    (repl.envcolors ? Base.input_color : repl.input_color) : "",
                repl = repl,
                complete = ShellCompletionProvider(),
                on_done = respond(repl, julia_prompt) do line
                    parsed = Base.shell_parse(line::String)[1]
                    return Expr(
                        :call,
                        custom_repl_cmd,
                        Expr(:call, Base.cmd_gen, parsed),
                        line::String,
                        outstream(repl)
                    )
                end,
                sticky = true
            )
            repl.interface.modes[2] = new_shell_prompt
            julia_mode = repl.interface.modes[1]
            julia_on_enter = julia_mode.on_done
            km = julia_mode.keymap_dict

            km[';'] = (s, rest...) -> begin
                buf = REPL.LineEdit.buffer(s)
                if buf.size == 0 && REPL.LineEdit.position(buf) == 0
                    REPL.LineEdit.transition(s, new_shell_prompt) do
                        # nothing to copy into the new prompt
                    end
                else
                    julia_on_enter(s, rest...)
                end
            end
        end
    end
end ### END REPL CUSTOMIZATION

[MilesCranmer]

@sandyspiers Check it out... Seems like nushell piping now works, since we are passing the raw string:

Although the one downside is $ doesn't seem to work yet, which limits some things: (Edit: I got this working)

[MilesCranmer]

Though it seems like defining anything for the variable is enough to get nushell closures to work. I guess it's because it is still creating the Cmd but not using it? Edit: I got it working. See #58525.

[MilesCranmer]

Moving to #58525 because that version also lets us enable Windows shells.