Debugging VS Code with VS Code is flaky and frustrating

[jrieken]

Sorry, that's it. Cannot say more, the experience is a lottery - it sometimes works flawless, it sometimes doesn't work. That it works only randomly is frustrating but what's worst is that I don't feel empowered to provide any kind of qualified feedback. I cannot file good issues and have to resort to ranting. I believe most have confirmed the broken window theory and just moved back to devtools or never made the switch.

[jrieken]

For some background: I am trying to use the "VS Code" launch config. I am trying to debug tests with the newish test integration. Symptoms are hangs and crashes, sometimes right after starting, sometimes after reloading, sometimes after the Nth debug attempt.

[jrieken]

/cc @eamodio @rebornix @joaomoreno who have expressed positive or negative emotions towards debugging in the recent past

[jrieken]

related to @alexdima's issue: #126826

[rebornix]

I asked in general probably more than twice about how to reliably debug VS Code or tests and also asked for @connor4312 and @roblourens personally for help. Maybe with one more bug to fix the experience can be smooth but I want to share my overall feeling with debugging in VS Code for the last couple of years and workarounds I ended up with

• Debugging single unit/integration tests is flaky
  • At the beginning the breakpoint seldomly hit when debugging single file, the test always finished before the debugger is attached (Debug unit test in VS Code fail to attach randomly #125567). I then started to use await new Promise(c => setTimeout(c, 3000));
  • Recently there are good fixes in js debug so breakpoints will mostly hit but occasionally the process failed probably because of crash crash on debugging VS Code renderer crashes when setting certain breakpoints #122225
• Debug VS Code all processes fail pretty often, I can't call it random as when it fails it's 100%. No clue why is that
  • might be timeout Alex is seeing Connection refused debugging extension host #126826
  • might be crash as above VS Code renderer crashes when setting certain breakpoints #122225
  • sometimes it succeed connecting, but breakpoints never hit (breakpoints should just work for all processes, right?). I saw this happening in a few ppl's machines
  • disconnect doesn't always work, have to right click kills the process
  • At the end, whenever I want to troubleshoot end to end, say notebook, what I do now is (almost work every time)
    • Run OSS, open extension host code, set breakpoints
    • Debug a notebook extension, in the new window use chrome dev tools in the renderer process

In addition to above, with the extensions starting to use bundler, I sometime find breakpoints doesn't hit in ts file but only in js file, it seems random but when that happens my confidence with source map becomes low and by the end of the day, I stopped using source map in both VS Code and browser dev tools.

With that being said, I'm not happy with all the workarounds so often gave the builtin debugger another try, it's getting better though the randomness is still there. I would love to see a clear detailed guidance for troubleshooting (it would be great if it's a command like Toubleshoot Keyboard Shortcuts) and we can file good bug reports.

[bpasero]

Related:

• Breakpoint does not correctly apply on destructured variable expression #127397
• Debug tests: often first click to "Debug Test" does nothing #127396
• Debugger does not step into correct function #127345

[joaomoreno]

Recently talking with @isidorn and @bpasero, we had the idea to have a PM-driven selfhosting effort They would collect information from team members of what the biggest selfhosting pains are and whether and why they are or aren't using VS Code core features. They would then help prioritize work and get it on the plan accordingly. The goal is to maximize selfhosting by putting highlight on areas with most pain.

—-

Specifically on selfhost debugging: I share your sentiments today. While it was great back in January/February, I can no longer say the same. I subscribe to the lottery analogy, which is the worst situation since it makes us silently ditch the feature, which I’ve certainly done recently.

[jrieken]

They would collect information from team members of what the biggest selfhosting pains are

I like that and that's also the idea of the papercut label

[connor4312]

Here's the state of where the mentioned bugs stand for me:

• VS Code renderer crashes when setting certain breakpoints #122225 remains the big one, at least for me. @deepak1556 please let me know if there's anything I can do to help with that. If it's insurmountable I could try to just brute force and try to disable features or behavior in the debugger until the crashes stop, but this is not sustainable. I would assume this is being caused by setting breakpoints via regexes; while these patterns are long, they're also pretty simple.
• Connection refused debugging extension host #126826 is on my to do. I believe this is something going wrong in VS Code itself where the extension host doesn't get launched in debug mode, but it's racey. I want to look at this this week.

  • There is a similar error for the main process and renderer. Electron behaves like Chrome, where only the first "main process" will look at the debugging instructions, so if an OSS window or main process is already open not in debug mode, then the debug ports will not be opened and a connection refused will happen.

    This can be fixed by using a separate user data dir for VS code when launched in debug mode, like we do for Chrome, but this could make life difficult. In Chrome there is a lockfile which we warn about if it's present when launching, but we don't have a similar file in VS Code right now.

  • Update: the linked bug was actually 'expected' since Alex was paused at a point before the extension host was launched, we just needed a larger timeout. I've also added fixes for the error with the renderer, Debugging VS Code with VS Code is flaky and frustrating #127861 (comment)

• Breakpoint does not correctly apply on destructured variable expression #127397 / Debugger does not step into correct function #127345 we'll see about. Ben mentioned devtools does this as well, it may or may not be something we can control/work around.
  • Update: both of these appear to be Chrome issues, the former will be fixed when we update, the latter is also a current issue the happens in Chrome.
• Debug unit test in VS Code fail to attach randomly #125567 there's now code to wait for at least one breakpoint to be verified by js-debug before the test session starts. However, while breakpoints are set in parallel it's possible that this could be insufficient--and there's also a 7-second timeout which may not be sufficient on certain machines. @rebornix are you still seeing this today, and when you are is it the case that either you have multiple breakpoints set, or you see that log in the test output?
  • Update: I've increased this, @rebornix let me know if you see this again.

• In addition to above, with the extensions starting to use bundler, I sometime find breakpoints doesn't hit in ts file but only in js file, it seems random but when that happens my confidence with source map becomes low and by the end of the day, I stopped using source map in both VS Code and browser dev tools

  • This is, as far as I've seen, always a configuration issue in the compiler/bundler or launch config. If you have examples of cases where this happens, please share them and I can take a look.

• I would love to see a clear detailed guidance for troubleshooting (it would be great if it's a command like Toubleshoot Keyboard Shortcuts) and we can file good bug reports.

  • If you have breakpoints that are set but unverified, use the "Debug: Diagnose Breakpoint Problems" command while the debug session is running.
  • Otherwise, add "trace: true" to the launch config and share the resulting log.

I will follow up with VS Code team members to gather feedback.

[connor4312]

With regard to debugging tests specifically, I will use the new test run configurations API to allow running in browsers which should some most of these issues (by using a newer Chromium/V8 version where bugs aren't present.)