If the workspace folder is a symlink, convert paths relative to it #314

MaskRay · 2019-03-10T08:30:53Z

madscientist had a good summary in #242 (comment)

I believe this was caused by issue #304 . Normally I create a /work directory for all my work. However on this system due to the vagaries of partition mounting I made /work a symlink to /home/work, so my Git worktrees are at the physical location /home/work/foo/bar but when I visit them I use the path /work/foo/bar. This appears to have caused the problem.

In order to work around it I had to (a) delete the existing .ccls-cache, (b) visit the files using the real path not the symlink, and (most importantly) (c) delete and recreate my compile_commands.json file having cd'd to the worktree using the real path. Apparently CMake uses the value of $PWD rather than the real path when it locates the current working directory so if you get to the root via different means you get different paths in the JSON file.

Suppose /tmp/real is a project. Consider different symlink types.

/tmp/symlink -> /tmp/real The project root directory may be a symlink. The decision whether the symlink should be followed is on the client side.
Some editors (vim) follow symlinks => The root is /tmp/real and the paths ccls receives are always resolved. There is no tricky thing in the interaction between the client and ccls. However, but we still have to ensure that "directory" "filename" paths are followed when parsing compile_commands.json.

If the editor doesn't (Emacs, VSCode), ccls should use /tmp/symlink/a.cc as the canonical path, instead of /tmp/real/a.cc. In this case, the user had better not open /tmp/real/a.cc directly (I'll call it "unsolicited open"), as it raises the question whether the symlink and the target should be considered the same file. The most sensible approach (which this PR does) is probably to convert every /tmp/real/ path to /tmp/symlink/ (there is some weird thing in VSCode: it opens two buffers and the second one may does not have semantic highlight)
/tmp/real/a.h -> /tmp/real/b.h Some files/directories under the root directory are symlinks. Ideally a.h and b.h should be considered different but it is impossible in clang internals. It always follows links and thinks the file is b.h.
/tmp/real/sub -> /tmp/sub /tmp/sub is another project but embedded in /tmp/real. After you jump from a /tmp/real/ buffer to a /tmp/sub buffer, you may find the two buffers are disconnected. You may also unsolicitedly open /tmp/sub buffers and find them disconneted from other /tmp/real/ buffers. IIRC no known client has good support for this (but I can hack emacs lsp-mode to do the sane thing!) Configure your client to make /tmp/real/sub a workspace folder (it is nested under the root /tmp/real but it is allowed!), then ccls will convert /tmp/sub paths to /tmp/real/sub. In addition, you have to hack your language client so that when it sees /tmp/sub, it sends requests to the ccls process with the root /tmp/real. This hack may not be easy in your client, though.

) If the workspace folder is a symlink and the client doesn't follow it. Treat /tmp/symlink/ as canonical and convert every /tmp/real/ path to /tmp/symlink/.

ilysym · 2019-03-15T22:08:24Z

I've noticed similar issue before, however this change doesn't fix it.

I believe many editors (emacs,vim) follow symlinks

No, Emacs with lsp-mode uses whatever path the file was originally opened at. It seems that symlink gets resolved somewhere in ccls (maybe clang?).

Here is ccls log (built from rev. c2a232647aea1c4eb0f77d58a9956f8ef87e2e74):

01:03:52 ccls           initialize.cc:271 I initialize in directory /home/ilya/Документы/projects/c-proj-test with uri file:///home/ilya/%D0%94%D0%BE%D0%BA%D1%83%D0%BC%D0%B5%D0%BD%D1%82%D1%8B/projects/c-proj-test/
01:03:52 ccls           initialize.cc:294 I initializationOptions: {"compilationDatabaseCommand":"","compilationDatabaseDirectory":"","cache":{"directory":".ccls-cache","format":"binary","hierarchicalPath":false,"retainInMemory":2},"capabilities":{"documentOnTypeFormattingProvider":{"firstTriggerCharacter":"}","moreTriggerCharacter":[]},"foldingRangeProvider":true,"workspace":{"workspaceFolders":{"supported":true,"changeNotifications":true}}},"clang":{"excludeArgs":[],"extraArgs":[],"pathMappings":[],"resourceDir":""},"client":{"hierarchicalDocumentSymbolSupport":true,"linkSupport":true,"snippetSupport":true},"codeLens":{"localVariables":true},"completion":{"caseSensitivity":2,"detailedLabel":true,"dropOldRequests":true,"duplicateOptional":true,"filterAndSort":true,"include":{"blacklist":[],"maxPathSize":30,"suffixWhitelist":[".h",".hpp",".hh",".inc"],"whitelist":[]},"maxNum":100},"diagnostics":{"blacklist":[],"onChange":1000,"onOpen":0,"onSave":0,"spellChecking":true,"whitelist":[]},"highlight":{"largeFileSize":2097152,"lsRanges":false,"blacklist":[],"whitelist":[]},"index":{"blacklist":[],"comments":2,"initialBlacklist":[],"initialWhitelist":[],"maxInitializerLines":5,"multiVersion":0,"multiVersionBlacklist":[],"multiVersionWhitelist":[],"name":{"suppressUnwrittenScope":false},"onChange":false,"parametersInDeclarations":true,"threads":0,"trackDependency":2,"whitelist":[]},"request":{"timeout":5000},"session":{"maxNum":10},"workspaceSymbol":{"caseSensitivity":1,"maxNum":1000,"sort":true},"xref":{"maxNum":2000}}
01:03:52 ccls           initialize.cc:323 I use -resource-dir=/home/ilya/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib/clang/7.0.1
01:03:52 ccls           initialize.cc:356 I workspace folder: /home/ilya/Документы/projects/c-proj-test/
01:03:52 ccls              project.cc:397 I loaded /home/ilya/Документы/projects/c-proj-test/compile_commands.json
01:03:52 ccls              project.cc:461 I search directory: /usr/include/ <
01:03:52 ccls              project.cc:461 I search directory: /usr/include/x86_64-linux-gnu/ <
01:03:52 ccls              project.cc:461 I search directory: /home/ilya/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib/clang/7.0.1/include/ <
01:03:52 ccls              project.cc:461 I search directory: /include/ <
01:03:52 ccls              project.cc:461 I search directory: /  
01:03:52 ccls              project.cc:461 I search directory: /usr/local/include/ <
01:03:52 ccls           initialize.cc:381 I start 4 indexers
01:03:52 ccls           initialize.cc:389 I dispatch initial index requests
01:03:52 indexer1         pipeline.cc:337 I parse /home/ilya/Документы/projects/c-proj-test/main.c
01:03:52 indexer0         pipeline.cc:337 I parse /home/ilya/Документы/projects/c-proj-test/funcs.c
01:03:52 ccls             pipeline.cc:471 I loaded project. Refresh semantic highlight for all working file.
01:03:52 indexer0         pipeline.cc:375 I store index for /home/ilya/Документы/projects/c-proj-test/funcs.h (delta: 0)
01:03:52 indexer0         pipeline.cc:375 I store index for /home/ilya/Документы/projects/c-proj-test/funcs.c (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/stdio_lim.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/sys_errlist.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/FILE.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/__FILE.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/sys/cdefs.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/__mbstate_t.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/libc-header-start.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /home/ilya/Документы/projects/c-proj-test/main.c (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/stdio.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/stdc-predef.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/__fpos_t.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/gnu/stubs.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /home/ilya/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib/clang/7.0.1/include/stddef.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/struct_FILE.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/features.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/gnu/stubs-64.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /home/ilya/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/lib/clang/7.0.1/include/stdarg.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/typesizes.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/wordsize.h (delta: 0)
01:03:52 indexer1         pipeline.cc:375 I store index for /usr/include/x86_64-linux-gnu/bits/types/__fpos64_t.h (delta: 0)
01:03:52 indexer2         pipeline.cc:337 I parse /home/ilya/p/c-proj-test/main.c
01:03:52 preamble     sema_manager.cc:730 I create session for /home/ilya/p/c-proj-test/main.c

/home/ilya/p is a link to /home/ilya/Документы/projects. Notice that both paths appear in the log.

ilysym · 2019-03-15T23:37:16Z

Sorry, my last comment is probably misleading. I looked into lsp-mode and it seems to have issues with symlinks. I'll look into it.

MaskRay · 2019-03-16T02:05:47Z

No, Emacs with lsp-mode uses whatever path the file was originally opened at. It seems that symlink gets resolved somewhere in ccls (maybe clang?).

Thanks for the clarification. I've checked vanilla emacs -Q doesn't use the truename as the buffer filename. I've updated the comment.

ilysym · 2019-03-16T11:07:59Z

Root cause is in projectile package which is used by lsp-mode. Relevant issue: bbatsov/projectile#1387

madscientist · 2019-03-16T14:17:36Z

I tried the latest ccls with my original setup and still see the same issues. I do use projectile as well and that does seem to be a likely culprit since the symlink modifies the project root location (the links are not internal to the project). I'll follow that issue, thanks @ilysym

) If the workspace folder is a symlink and the client doesn't follow it. Treat /tmp/symlink/ as canonical and convert every /tmp/real/ path to /tmp/symlink/.

MaskRay force-pushed the realpath branch from 563069f to 22a2134 Compare March 10, 2019 08:56

If the workspace folder is a symlink, convert paths relative to it (#314

1e3c683

) If the workspace folder is a symlink and the client doesn't follow it. Treat /tmp/symlink/ as canonical and convert every /tmp/real/ path to /tmp/symlink/.

MaskRay force-pushed the realpath branch from 22a2134 to 1e3c683 Compare March 10, 2019 08:57

MaskRay merged commit 1e3c683 into master Mar 10, 2019

MaskRay deleted the realpath branch March 10, 2019 08:59

This was referenced Mar 10, 2019

(Optionally) don't follow symlinks #102

Closed

always use complete path as index lookup #304

Closed

Receiving "code -32600" / message "not indexed" #242

Closed

MaskRay mentioned this pull request Mar 23, 2019

ccls crashes if compile_commands.json refers to non-existent file #338

Closed

MaskRay mentioned this pull request Mar 25, 2019

ccls seems to get confused when trying to find libc++ when libc++ itself is being built #344

Closed

MaskRay mentioned this pull request Jul 4, 2020

fix index error when there is a soft link file #643

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

If the workspace folder is a symlink, convert paths relative to it #314

If the workspace folder is a symlink, convert paths relative to it #314

Uh oh!

MaskRay commented Mar 10, 2019 •

edited

Loading

Uh oh!

ilysym commented Mar 15, 2019

Uh oh!

ilysym commented Mar 15, 2019

Uh oh!

MaskRay commented Mar 16, 2019

Uh oh!

ilysym commented Mar 16, 2019

Uh oh!

madscientist commented Mar 16, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

If the workspace folder is a symlink, convert paths relative to it #314

If the workspace folder is a symlink, convert paths relative to it #314

Uh oh!

Conversation

MaskRay commented Mar 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilysym commented Mar 15, 2019

Uh oh!

ilysym commented Mar 15, 2019

Uh oh!

MaskRay commented Mar 16, 2019

Uh oh!

ilysym commented Mar 16, 2019

Uh oh!

madscientist commented Mar 16, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MaskRay commented Mar 10, 2019 •

edited

Loading