SSH disconnects when the laptop goes to sleep even for a short time while the native terminal is running

[thechargedneutron]

• VSCode Version: 1.62.0-insider
• Local OS Version: macOS Big Sur 11.5.2
• Remote OS Version: Ubuntu 18.04 LTS
• Remote Extension/Connection Type: SSH
• Logs:

Steps to Reproduce:

1. Open native terminal in macOS and connect to a remote server. Connect to the same server with vscode.
2. Close the laptop lid and instantly open it up. Alternatively, let the laptop sleep due to inactivity and then instantly wake it up using the trackpad.

The vscode is now disconnected and asks me to reload window. The macOS terminal is still connected to the remote and works perfectly. Is there a way to delay the session time-out time in vscode? It is a bit annoying to reload every time the screen goes off for a minute. I have to increase the sleep duration to tackle this issue.

Does this issue occur when you try this locally?: Yes
Does this issue occur when you try this locally and all extensions are disabled?: Yes

[tanhakabir]

@roblourens I'm guessing this has to do with the ssh process closing when the computer goes to sleep and when we try to reconnect, we aren't able to within the 5 times so we close the connection?

[roblourens]

It should be trying to reconnect at this point. Please share the log from the Remote-SSH output channel, it should tell us why it failed to reconnect

[thechargedneutron]

@roblourens After I put the laptop lid down and instantly put it up, the password prompt comes, if I do not enter the password and click exit I get the following log. Though, my concern is that vscode should not ask for password again if I instantly open up my lid again (similar to macOS native terminal).

[19:57:04.914] [Forwarding server 52217] Got connection 3
[19:57:04.925] ------




[19:57:04.925] SSH Resolver called for "ssh-remote+`hostname`", attempt 2, (Reconnection)
[19:57:04.925] SSH Resolver called for host: `hostname`
[19:57:04.925] Setting up SSH remote `"hostname"`
[19:57:04.927] Acquiring local install lock: /var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-remote-ssh-76b40f39-install.lock
[19:57:04.934] Looking for existing server data file at /Users/thechargedneutron/Library/Application Support/Code - Insiders/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-76b40f39-34eeebdd66033633ac22efd5fab928ae37916168-0.65.8/data.json
[19:57:04.938] Found local server running: {"remoteListeningOn":{"port":37801},"osReleaseId":"ubuntu","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","display":"","tmpDir":"/run/user/41042","platform":"linux","connectionToken":"a11a111a-1a11-11aa-1a1a-11a11aa111aa","pid":83938,"ipcHandlePath":"/var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-ssh-askpass-2ef0d9ce17bbe074fbc7bf86195d17bc1a93c36e.sock","socksPort":52193,"startupTime":1635555338990}
[19:57:04.938] Running server is stale. Ignoring
[19:57:04.939] Using commit id "34eeebdd66033633ac22efd5fab928ae37916168" and quality "insider" for server
[19:57:04.942] Install and start server if needed
[19:57:04.954] PATH: /Users/thechargedneutron/.rbenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Users/thechargedneutron/Library/Android/sdk/emulator:/Users/thechargedneutron/Library/Android/sdk/tools:/Users/thechargedneutron/Library/Android/sdk/tools/bin:/Users/thechargedneutron/Library/Android/sdk/platform-tools
[19:57:04.954] Checking ssh with "ssh -V"
[19:57:04.963] > OpenSSH_8.1p1, LibreSSL 2.7.3

[19:57:04.964] askpass server listening on /var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-ssh-askpass-243fffb2716a3c88297ba544bff67b3ee7beb0c2.sock
[19:57:04.964] Spawning local server with {"serverId":1,"ipcHandlePath":"/var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-ssh-askpass-0dabf9d3e7ca1a73ca37809a7142fdfd3c525eb0.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","52243","-o","ConnectTimeout=15","`hostname`"],"dataFilePath":"/Users/thechargedneutron/Library/Application Support/Code - Insiders/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-76b40f39-34eeebdd66033633ac22efd5fab928ae37916168-0.65.8/data.json"}
[19:57:04.964] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Applications/Visual Studio Code - Insiders.app/Contents/Frameworks/Code - Insiders Helper (Renderer).app/Contents/MacOS/Code - Insiders Helper (Renderer)","VSCODE_SSH_ASKPASS_MAIN":"/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-ssh-askpass-243fffb2716a3c88297ba544bff67b3ee7beb0c2.sock"}
[19:57:04.967] Spawned 83978
[19:57:05.101] > local-server-1> Spawned ssh, pid=83983
[19:57:05.102] stderr> OpenSSH_8.1p1, LibreSSL 2.7.3
[19:57:05.324] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:n09TbVQEP2lfMtVM1hdZAsZs5lfm4IakAEmuWgAcfOg
[19:57:05.620] Got askpass request: {"request":"Enter passphrase for key '/Users/thechargedneutron/.ssh/id_rsa':"}
[19:57:05.621] Detected passphrase message
[19:57:05.621] Listening for interwindow password on /var/folders/bn/wchw2lf52zdc_y9p6bm7217m0000gn/T/vscode-ssh-askpass-edfb48cf23ea357f834d86795ca6b6fb713bbc2c.sock
[19:57:05.621] Writing password prompt to globalState
[19:57:10.883] Passphrase dialog canceled
[19:57:10.883] Interactor signaled cancel
[19:57:10.884] Cleaning up other-window auth server
[19:57:10.903] stderr> `username@hostname`: Permission denied (publickey).
[19:57:10.904] > local-server-1> ssh child died, shutting down
[19:57:10.906] Local server exit: 0
[19:57:10.906] Received install output: local-server-1> Spawned ssh, pid=83983
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:n09TbVQEP2lfMtVM1hdZAsZs5lfm4IakAEmuWgAcfOg
`username@hostname`: Permission denied (publickey).
local-server-1> ssh child died, shutting down

[19:57:10.908] Resolver error: Error: Permission denied (publickey).
	at Function.Create (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:397041)
	at /Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:395235
	at Object.t.handleInstallOutput (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:395620)
	at Object.e [as tryInstallWithLocalServer] (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:435292)
	at processTicksAndRejections (internal/process/task_queues.js:93:5)
	at async /Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:452665
	at async Object.t.withShowDetailsEvent (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:456460)
	at async /Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:433816
	at async C (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:430392)
	at async Object.t.resolveWithLocalServer (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:433431)
	at async Object.t.resolve (/Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:454065)
	at async /Users/thechargedneutron/.vscode-insiders/extensions/ms-vscode-remote.remote-ssh-0.65.8/out/extension.js:1:526942
[19:57:10.910] ------

[roblourens]

Well, the socket might close for some reason in the time that the laptop was asleep. Seems that sometimes ssh in the terminal will survive a brief sleep, sometimes not, same for vscode. Also, we implement our own timeout where once the remote is not reachable for some amount of time, we assume it's disconnected and try to reconnect. So I have to just call this "by design". @alexdima I wonder whether it could be improved for scenarios like this, if we can check the previous socket again after waking from sleep.

It's much more smooth if you set up SSH with your passphrase in the ssh agent so you don't need to type anything to connect

[alexdima]

@roblourens The core will invoke the resolver again. You can definitely return the same port again from the resolver if the previous SSH tunnel is still healthy. For example, once it started the server, Remote WSL always returns the same port from the resolver.

[thechargedneutron]

Honestly, I think there should be at least this level of robustness to keep the session alive for cases when the laptop has just gone to sleep. If I set my screen sleep time to 10 minutes the ssh remains active for this long. But if I set my screen sleep to 1 minute the ssh disconnects as soon as the screen goes off even for a few seconds. I appreciate the your efforts in discussing this issue. Thanks!

[roblourens]

We have the ability to test the connection again, but wouldn't it make more sense for the core to do that?

[alexdima]

We have the ability to test the connection again, but wouldn't it make more sense for the core to do that?

Doing that in all cases means that in many cases we would waste around 30s (the timeout we have) trying to connect to a possibly broken resolver port. So we always just invoke the resolver which can trivially return the same port if it so wishes.

e.g. of what you could do in the SSH resolver (if you want to):

class Resolver {
	private lastPort: number = 0;
	private lastPortReusedTime: number = 0;

	public resolve(ctx) {
		const shouldReuseLastPort = (
			this.lastPort
			&& (Date.now() - this.lastPortReusedTime > 5 * 60 * 1000) // the port was not reused within the past 5 minutes
		);
		if (shouldReuseLastPort /* && tunnel process still alive */) {
			// try to reuse the same tunnel, maybe it still works fine
			this.lastPortReusedTime = Date.now();
			return this.lastPort;
		}
		this.lastPortReusedTime = 0;
		// ...
		// this.lastPort = ...
		return this.lastPort;
	}
}

The core calling the resolver gives the most freedom. Resolvers that know that they are really disconnected can recreate the connection, while resolvers which don't know can try to be lucky and return first the old port and only on a subsequent resolve call create a new tunnel.

What do you think?

[roblourens]

If we reuse the port, and it turns out to be stuck, will vscode spend 30s trying to connect to it? Because that will happen sometimes. The resolver can also ping the server through the port with a shorter timeout.

[alexdima]

If we reuse the port, and it turns out to be stuck, will vscode spend 30s trying to connect to it? Because that will happen sometimes. The resolver can also ping the server through the port with a shorter timeout.

👍 Yes, that is exactly the point I was trying to make. When the core tries to reconnect to a host:port provided by the resolver, it sets a 30s timeout. If the timeout fires without the connection succeeding, it goes again to the resolver to resolve again and so on...