New clients started per `tsconfig.json` in a multi-project TypeScript codebase

[faergeek]

Description

With the migration to a new way of configuring LSPs I noticed a change in behavior when working with multiple TypeScript projects tied together using project references.

When using require('lspconfig').<lsp-name>.setup(...) and opening files from different projects there is only one client instance.

But with vim.lsp.config + vim.lsp.enable a new separate client instance is started for each project.

The issue is that with multiple instances finding references doesn't work reliably across all projects even when all of them are loaded.

I've tried ts_ls and vtsls, they both behave the same way before and after.

Here is a repository to demonstrate the issue: https://github.com/faergeek/nvim-ts-workspace-folders-repro

Steps to reproduce:

1. Clone the repo
2. npm install -g typescript typescript-language-server as per configs.md
3. Run nvim --clean -u init.new.lua -O a/index.ts b/index.ts, which uses the new way of configuration. Run :LspInfo. There will be two ts_ls instances under vim.lsp: Active Clients.
4. Run nvim --clean -u init.old.lua -O a/index.ts b/index.ts, which uses the old way. Run :LspInfo. There will be just one ts_ls instance under vim.lsp: Active Clients.

With the old way any unrelated project will also be added to the same instance as well (from a completely different repository in a parent directory, for example), which doesn't seem right, but I don't know if it actually causes any issues.

Based on :help vim.lsp.Config possible workaround outside of nvim-lspconfig would be either defining a custom reuse_client implementation or just setting root_dir to a cwd, for example:

vim.lsp.config('vtsls', {
  root_dir = vim.uv.cwd(),
})

I don't know what would be a generic solution to this other than looking for root markers top-down instead of bottom-up when detecting root_dir.

[justinmk]

This is decided by reuse_client. Its default condition is here: https://github.com/neovim/neovim/blob/0d658660c29e920e74c4dade3819d80dccad0dde/runtime/lua/vim/lsp.lua#L146

I don't know what would be a generic solution to this other than looking for root markers top-down instead of bottom-up when detecting root_dir.

Can you try defining a custom reuse_client predicate?

btw, root_markers is prioritized in-order so another option would be to list .git (and/or package.json) first, instead of near the end:

nvim-lspconfig/lsp/ts_ls.lua

Line 84 in 463b16b

root_markers = { 'tsconfig.json', 'jsconfig.json', 'package.json', '.git' },

If you see a way to improve the default reuse_client, or anything else, let us know :)

[faergeek]

I couldn't think of a way to fix this using only reuse_client besides just ignoring root_dir completely and only considering names. Was this the default behavior before the introduction of lsp directory and the reason there was only a single instance previously?

I couldn't think of any other solution than adjusting root_dir detection. Moving .git to the start of root_markers does help in this particular case, but from what I see removal of .git from root_markers was considered in #3728. And on top of that I think it's not hard to imagine a project setup where these TypeScript projects are plugged as git submodules into another repo. In which case this solution won't fix out-of-the-box behavior.

From what I understand currently given the buffer path like
/home/user/projects/nvim-ts-workspace-folders-repro/a/index.ts the search for a single marker will go like this:

1. Check if /home/user/projects/nvim-ts-workspace-folders-repro/a/tsconfig.json exists
2. Check if /home/user/projects/nvim-ts-workspace-folders-repro/tsconfig.json exists
3. Check if /home/user/projects/tsconfig.json exists
4. ...
5. Check if /tsconfig.json exists

What if there was a way to say that searching for a particular marker should go backwards?

1. Check if /tsconfig.json exists
2. Check if /home/tsconfig.json exists
3. Check if /home/user/tsconfig.json exists
4. ...
5. Check if /home/user/projects/nvim-ts-workspace-folders-repro/a/tsconfig.json exists

Having tsconfig.json in a home directory would result in false positives though.

To be fair, setting root_dir via .nvim.lua with exrc enabled already works perfectly fine for me. I'm using exrc for other reasons anyway and it depends on cwd where nvim is started, so I could as well set root_dir to cwd globally in my config and be done with it. Maybe it's not worth trying to solve it any other way.

[justinmk]

I couldn't think of a way to fix this using only reuse_client besides just ignoring root_dir completely and only considering names. Was this the default behavior before the introduction of lsp directory and the reason there was only a single instance previously?

Here's the reuse_client logic in the legacy nvim-lpsconfig framework, maybe there's a logic difference that could be upstreamed:

nvim-lspconfig/lua/lspconfig/manager.lua

Lines 133 to 144 in 463b16b

	reuse_client = function(existing_client)
	if (self._clients[root_dir] or {})[existing_client.name] then
	self:_notify_workspace_folder_added(root_dir, existing_client)
	return true
	end
	for _, dir_clients in pairs(self._clients) do
	if dir_clients[existing_client.name] then
	self:_notify_workspace_folder_added(root_dir, existing_client)
	return true
	end
	end

Moving .git to the start of root_markers does help in this particular case, but from what I see removal of .git from root_markers was considered in #3728.

Unlikely that we will move forward with that change.

imagine a project setup where these TypeScript projects are plugged as git submodules into another repo. In which case this solution won't fix out-of-the-box behavior.

Yeah, the common case is what we must target. Impossible to support every case.

What if there was a way to say that searching for a particular marker should go backwards?

Can easily fail because people (commonly) have stuff like .git and package.json in their home dir.

To be fair, setting root_dir via .nvim.lua with exrc enabled already works perfectly fine for me.

Can you show the code example?

We definitely want to start leveraging project-local config more heavily.

[faergeek]

Can you show the code example?

We definitely want to start leveraging project-local config more heavily.

I have this in .nvim.lua at the root of the project:

vim.lsp.config('vtsls', { root_dir = vim.uv.cwd() })

I also configure nvim-dap and add some custom user commands per project this way, which is quite convenient.

A possible inconvenience is that it only works when starting nvim at that same directory as well. But that's how I do it most of the time anyway, so that's not a big issue for me personally.

I also have to remember to call :trust and most likely :source % on every edit of .nvim.lua, but that could be easily automated with a BufWritePost autocmd I think.

For local configuration of just LSP https://github.com/folke/neoconf.nvim looks cool. It doesn't need :trust since it can't execute arbitrary code and doesn't rely on cwd as far as I know, but I couldn't figure out how to make it stop reading .vscode/settings.json and I often had to override some settings. It can also be extended with settings for other purposes. I used it for configuring nvim-dap, but that started taking too much effort, so I dropped it and moved to just using exrc instead.

[justinmk]

I have this in .nvim.lua at the root of the project:

vim.lsp.config('vtsls', { root_dir = vim.uv.cwd() })

Better to get the script path instead (debug.getinfo(1, 'S').source:sub(2)):

• Lua: get path of current script/module file (like <sfile>), show Lua plugins in :scriptnames neovim#32116

I also have to remember to call :trust and most likely :source % on every edit of .nvim.lua

Hmm that's weird, you should be prompted automatically.

[faergeek]

Better to get the script path instead (debug.getinfo(1, 'S').source:sub(2)):

But exrc file is only read if it's exactly in nvim cwd at the start, right? So making it relative to exrc file doesn't make much of a difference.

Hmm that's weird, you should be prompted automatically.

Prompt appears just fine, no problem with that, I just prefer calling :trust to not be presented with a prompt next time when I'm trying to edit something quickly.

[justinmk]

But exrc file is only read if it's exactly in nvim cwd at the start, right?

No that changed in Nvim 0.12, though we didn't communicate it very well. neovim/neovim#33889

[faergeek]

Here's the reuse_client logic in the legacy nvim-lpsconfig framework, maybe there's a logic difference that could be upstreamed:

If I understand it correctly, it looks like it tries to add a workspace folder to a client with matching root_dir and name first (assuming root_dir has all the same logic as it does now, this part roughly stays the same) and if it couldn't do that, then it tries to add root_dir to an existing client with just the same name (in current nvim's default it would return false and start a new instance instead). So root_dir is preferred to match, but not forced to. With this behavior it couldn't ever start another instance at all if there is an existing client with the same name already, which explains why it did work in such cases previously.

The old behavior works fine assuming you only open different workspaces which are related to each other (multiple related TypeScript projects, as discussed here). What issues it might cause in practice if you mix it with unrelated projects I'm not really sure. Maybe different package versions might result in conflicts or something like that.

The new behavior seems to be more correct in general, but might require setting root_dir explicitly.

On a side note, I've tried to compare it with what vscode does, based on whether it finds references properly or not. From what I've inferred, for vscode it's easy since it can just assume the folder you open in it is the root_dir. If I open a real root (git repo root form above) in it then it finds all the references (same as nvim with root_dir set), but if I open a inside of it, it doesn't find references in other projects (same as nvim out of the box, without configuration).

For neovim the only way to find a root_dir is to try to infer it based on presence of markers. But it's easy to set it explicitly.

In summary, both need a correct root_dir. In neovim you might need to set it explicitly if it can't be properly detected, in vscode you have to correctly set it implicitly by opening the right folder.

At this point I don't really think anything about neovim or nvim-lspconfig behavior should be changed. But maybe it's worth mentioning in the troubleshooting section in a point about root detection, that root_dir can be set explicitly using exrc + .nvim.lua.

[faergeek]

Seems like debug.getinfo(1, 'S').source:sub(2) could not be used since .source value it returns from inside of .nvim.lua is just "<nvim>", there's no file path.

And since in stable (0.11.*) exrc only looks in cwd, should I mention it in troubleshooting as just vim.lsp.config('<client name>', { root_dir = vim.uv.cwd() }) for now?

[justinmk]

debug.getinfo(1, 'S').source:sub(2) could not be used since .source value it returns from inside of .nvim.lua is just "<nvim>", there's no file path.

Oh that is probably due to how 'exrc' is currently implemented. Need to fix that, it's kind of important... neovim/neovim#34577

since in stable (0.11.*) exrc only looks in cwd, should I mention it in troubleshooting as just vim.lsp.config('<client name>', { root_dir = vim.uv.cwd() }) for now?

I would mention the 0.12 approach and say it requires 0.12. That is easier to remember to update later, than the other way.