Remote SSH connection hangs at Checking "/home/user/.vscode-server/cli/server/Stable-{commit_id}/log.txt and /home/user/.vscode-server/cli/servers/Stable-{commit_id}/pid.txt"

[chl19940220]

Is there an existing issue for this bug?

• I have searched the existing open issues and found none that apply.
• If I find any issue of interest that is related or closed, I will included a link to it in this issue.

Required Troubleshooting Steps

• I have read and performed the troubleshooting steps
• I have tried both values of the remote.SSH.useLocalServer setting
• My issue was not covered in the Tips and Tricks linked from the Troubleshooting Wiki.
• I will include a complete copy of my Remote - SSH logs by running Remote-SSH: Show Log in the command palette or from View > Output in the menu bar

[Optional] Diagnose with Copilot

I asked @remote-ssh but it did not help

In step 2 of the troubleshooting wiki, what was the result of running the generated SSH command verbatim outside of VS Code?

I did not try step 2 of the troubleshooting steps

Remote-SSH Log

Remote-SSH Log

[14:53:30.582] Log Level: 2
[14:53:30.597] SSH Resolver called for "ssh-remote+192.168.3.55", attempt 1
[14:53:30.599] remote.SSH.useLocalServer = false
[14:53:30.599] remote.SSH.useExecServer = true
[14:53:30.599] remote.SSH.showLoginTerminal = true
[14:53:30.599] remote.SSH.remotePlatform = {}
[14:53:30.599] remote.SSH.path =  C:\Windows\System32\OpenSSH\ssh.exe
[14:53:30.599] remote.SSH.configFile = C:\Users\Win10\.ssh\config
[14:53:30.599] remote.SSH.useFlock = true
[14:53:30.599] remote.SSH.lockfilesInTmp = false
[14:53:30.600] remote.SSH.localServerDownload = auto
[14:53:30.600] remote.SSH.remoteServerListenOnSocket = false
[14:53:30.600] remote.SSH.defaultExtensions = []
[14:53:30.600] remote.SSH.loglevel = 2
[14:53:30.600] remote.SSH.enableDynamicForwarding = true
[14:53:30.600] remote.SSH.enableRemoteCommand = false
[14:53:30.600] remote.SSH.serverPickPortsFromRange = {}
[14:53:30.600] remote.SSH.serverInstallPath = {}
[14:53:30.600] remote.SSH.permitPtyAllocation = false
[14:53:30.600] remote.SSH.preferredLocalPortRange = undefined
[14:53:30.600] remote.SSH.useCurlAndWgetConfigurationFiles = false
[14:53:30.600] remote.SSH.experimental.chat = true
[14:53:30.600] remote.SSH.experimental.enhancedSessionLogs = true
[14:53:30.606] VS Code version: 1.96.3
[14:53:30.606] Remote-SSH version: [email protected]
[14:53:30.606] win32 x64
[14:53:30.608] SSH Resolver called for host: 192.168.3.55
[14:53:30.608] Setting up SSH remote "192.168.3.55"
[14:53:30.615] Using commit id "91fbdddc47bc9c09064bf7acf133d22631cbf083" and quality "stable" for server
[14:53:30.618] Script variables:
 {
  "ExitCode.0": "OK",
  "ExitCode.64": "LinuxPrereqs",
  "ExitCode.65": "ProxyAuthenticationRequired",
  "ExitCode.128": "UnparsableOutput",
  "ExitCode.129": "EmptyOutput",
  "ExitCode.131": "UnhandledResolverExitCode",
  "ExitCode.132": "OfflineError",
  "ExitCode.133": "SshInstall",
  "ExitCode.134": "FindLocalPort",
  "ExitCode.135": "Timeout",
  "ExitCode.136": "ExhaustedRetries",
  "ExitCode.137": "PermissionDenied",
  "ExitCode.138": "MitmPortForwardingDisabled",
  "ExitCode.139": "Canceled",
  "ExitCode.140": "BadLocalDownloadRequest",
  "ExitCode.141": "LocalDownloadFailed",
  "ExitCode.142": "LocalLockTimeout",
  "ExitCode.143": "NoBash",
  "ExitCode.144": "InvalidHostName",
  "ExitCode.145": "BadInstallScriptResult",
  "ExitCode.146": "ShellRequestFailedOnChannel",
  "ExitCode.147": "InvalidPortRange",
  "ExitCode.148": "SendDisconnectBrokenPipe",
  "ExitCode.149": "LocalServerCrashed",
  "ExitCode.150": "NoFreePortInRange",
  "ExitCode.151": "InvalidCustomServerPath",
  "ExitCode.192": "AlreadyInProgress",
  "ExitCode.193": "ServerDownloadFailed",
  "ExitCode.194": "NoDownloaderAvailable",
  "ExitCode.195": "NoDownloaderAvailableForStatusCheck",
  "ExitCode.196": "UnsupportedArch",
  "ExitCode.197": "StatusCheckFailed",
  "ExitCode.198": "NeedInsidersArch",
  "ExitCode.199": "ServerTransferFailed",
  "ExitCode.200": "ServerFailedToStart",
  "ExitCode.201": "NeedInsidersWindows",
  "ExitCode.202": "CreateInstallDirFailed",
  "ExitCode.203": "UnsupportedPlatform",
  "ExitCode.204": "ServerTerminatedCVE20201416",
  "ExitCode.205": "UnpackFailed",
  "ExitCode.206": "ChangeDirFailed",
  "ExitCode.1023": "UnclassifiedError",
  "ExitCode.OK": "0",
  "ExitCode.LinuxPrereqs": "64",
  "ExitCode.ProxyAuthenticationRequired": "65",
  "ExitCode.UnparsableOutput": "128",
  "ExitCode.EmptyOutput": "129",
  "ExitCode.UnhandledResolverExitCode": "131",
  "ExitCode.OfflineError": "132",
  "ExitCode.SshInstall": "133",
  "ExitCode.FindLocalPort": "134",
  "ExitCode.Timeout": "135",
  "ExitCode.ExhaustedRetries": "136",
  "ExitCode.PermissionDenied": "137",
  "ExitCode.MitmPortForwardingDisabled": "138",
  "ExitCode.Canceled": "139",
  "ExitCode.BadLocalDownloadRequest": "140",
  "ExitCode.LocalDownloadFailed": "141",
  "ExitCode.LocalLockTimeout": "142",
  "ExitCode.NoBash": "143",
  "ExitCode.InvalidHostName": "144",
  "ExitCode.BadInstallScriptResult": "145",
  "ExitCode.ShellRequestFailedOnChannel": "146",
  "ExitCode.InvalidPortRange": "147",
  "ExitCode.SendDisconnectBrokenPipe": "148",
  "ExitCode.LocalServerCrashed": "149",
  "ExitCode.NoFreePortInRange": "150",
  "ExitCode.InvalidCustomServerPath": "151",
  "ExitCode.AlreadyInProgress": "192",
  "ExitCode.ServerDownloadFailed": "193",
  "ExitCode.NoDownloaderAvailable": "194",
  "ExitCode.NoDownloaderAvailableForStatusCheck": "195",
  "ExitCode.UnsupportedArch": "196",
  "ExitCode.StatusCheckFailed": "197",
  "ExitCode.NeedInsidersArch": "198",
  "ExitCode.ServerTransferFailed": "199",
  "ExitCode.ServerFailedToStart": "200",
  "ExitCode.NeedInsidersWindows": "201",
  "ExitCode.CreateInstallDirFailed": "202",
  "ExitCode.UnsupportedPlatform": "203",
  "ExitCode.ServerTerminatedCVE20201416": "204",
  "ExitCode.UnpackFailed": "205",
  "ExitCode.ChangeDirFailed": "206",
  "ExitCode.UnclassifiedError": "1023",
  "InstallUnpackCode.Success": "success",
  "InstallUnpackCode.Error": "error",
  "InstallUnpackCode.MissingFiles": "missingFiles",
  "uuid": "c620204616ec",
  "startMarker": "c620204616ec: running",
  "commitId": "91fbdddc47bc9c09064bf7acf133d22631cbf083",
  "quality": "stable",
  "token": "aaa11a11-aaa1-11aa-aa11-1a111aa1aaa1",
  "vscodeAgentFolder": "$HOME/.vscode-server",
  "allowClientDownload": "1",
  "forceClientDownload": "0",
  "cliNameInArchive": "code",
  "ignoreWgetConfigFlag": " --no-config ",
  "ignoreCurlConfigFlag": " --disable ",
  "wgetTriesSegment": "--tries=1",
  "listenArgs": "--on-host=127.0.0.1 --on-port",
  "getDownloadServerStartTrigger": "c620204616ec:trigger_server_download",
  "getDownloadServerEndTrigger": "c620204616ec:trigger_server_download_end",
  "getProgressDownloading": "c620204616ec%%1%%",
  "getProgressInstalling": "c620204616ec%%2%%",
  "printVars": "echo \"SSH_AUTH_SOCK==$SSH_AUTH_SOCK==\"\necho \"DISPLAY==$DISPLAY==\""
}
[14:53:30.620] Install and start server if needed
[14:53:31.328] Checking ssh with " C:\Windows\System32\OpenSSH\ssh.exe -V"
[14:53:31.330] Got error from ssh: spawn  C:\Windows\System32\OpenSSH\ssh.exe ENOENT
[14:53:31.330] The specified path  C:\Windows\System32\OpenSSH\ssh.exe is not a valid SSH binary
[14:53:31.330] Checking ssh with "C:\Program Files\Common Files\Oracle\Java\javapath\ssh.exe -V"
[14:53:31.331] Got error from ssh: spawn C:\Program Files\Common Files\Oracle\Java\javapath\ssh.exe ENOENT
[14:53:31.331] Checking ssh with "C:\Windows\system32\ssh.exe -V"
[14:53:31.332] Got error from ssh: spawn C:\Windows\system32\ssh.exe ENOENT
[14:53:31.332] Checking ssh with "C:\Windows\ssh.exe -V"
[14:53:31.333] Got error from ssh: spawn C:\Windows\ssh.exe ENOENT
[14:53:31.333] Checking ssh with "C:\Windows\System32\Wbem\ssh.exe -V"
[14:53:31.333] Got error from ssh: spawn C:\Windows\System32\Wbem\ssh.exe ENOENT
[14:53:31.333] Checking ssh with "C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe -V"
[14:53:31.334] Got error from ssh: spawn C:\Windows\System32\WindowsPowerShell\v1.0\ssh.exe ENOENT
[14:53:31.334] Checking ssh with "C:\Windows\System32\OpenSSH\ssh.exe -V"
[14:53:31.358] > OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2

[14:53:31.361] Using SSH config file "C:\Users\Win10\.ssh\config"
[14:53:31.361] Running script with connection command: "C:\Windows\System32\OpenSSH\ssh.exe" -T -D 63509 -F "C:\Users\Win10\.ssh\config" "192.168.3.55" sh
[14:53:31.362] Generated SSH command: 'type "C:\Users\Win10\AppData\Local\Temp\vscode-linux-multi-line-command-192.168.3.55-363982854.sh" | "C:\Windows\System32\OpenSSH\ssh.exe" -T -D 63509 -F "C:\Users\Win10\.ssh\config" "192.168.3.55" sh'
[14:53:31.362] Using connect timeout of 7 seconds
[14:53:31.363] Terminal shell path: C:\Windows\System32\cmd.exe
[14:53:31.589] > 
[14:53:31.589] Got some output, clearing connection timeout
[14:53:31.603] > 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
[14:53:31.666] > [email protected]'s password: 
[14:53:36.167] > 
[14:53:36.285] > c620204616ec: running
> Script executing under PID: 349007
> Found existing installation at /home/chenghl/.vscode-server...
> Starting VS Code CLI...
> printenv:
>     USER=chenghl
>     SSH_CLIENT=192.168.2.125 63512 22
>     LC_TIME=zh_CN.UTF-8
>     XDG_SESSION_TYPE=tty
>     SHLVL=0
>     MOTD_SHOWN=pam
>     HOME=/home/chenghl
>     LC_MONETARY=zh_CN.UTF-8
>     DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1005/bus
>     LOGNAME=chenghl
>     _=/usr/bin/sh
>     XDG_SESSION_CLASS=user
>     XDG_SESSION_ID=6046
>     PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
>     LC_ADDRESS=zh_CN.UTF-8
>     XDG_RUNTIME_DIR=/run/user/1005
[14:53:36.291] > 
>     LANG=en_US.UTF-8
>     LC_TELEPHONE=zh_CN.UTF-8
>     SHELL=/bin/bash
>     LC_NAME=zh_CN.UTF-8
>     LC_MEASUREMENT=zh_CN.UTF-8
>     LC_IDENTIFICATION=zh_CN.UTF-8
>     PWD=/home/chenghl
>     SSH_CONNECTION=192.168.2.125 63512 192.168.3.55 22
>     LC_NUMERIC=zh_CN.UTF-8
>     LC_PAPER=zh_CN.UTF-8
> Removing old logfile at /home/chenghl/.vscode-server/.cli.91fbdddc47bc9c09064bf7acf133d22631cbf083.log
> Spawned remote CLI: 349027
> Waiting for server log...
[14:53:36.315] > 
[14:53:36.338] > c620204616ec: start
> listeningOn==127.0.0.1:34975==
> osReleaseId==ubuntu==
> arch==x86_64==
> vscodeArch==x64==
> bitness==64==
> tmpDir==/run/user/1005==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> serverStartTime==35==
> execServerToken==aaa11a11-aaa1-11aa-aa11-1a111aa1aaa1==
> platformDownloadPath==cli-alpine-x64==
> SSH_AUTH_SOCK====
> DISPLAY====
> c620204616ec: end
[14:53:36.338] Received install output: 
listeningOn==127.0.0.1:34975==
osReleaseId==ubuntu==
arch==x86_64==
vscodeArch==x64==
bitness==64==
tmpDir==/run/user/1005==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
serverStartTime==35==
execServerToken==aaa11a11-aaa1-11aa-aa11-1a111aa1aaa1==
platformDownloadPath==cli-alpine-x64==
SSH_AUTH_SOCK====
DISPLAY====

[14:53:36.339] Remote server is listening on port 34975
[14:53:36.339] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":34975},"osReleaseId":"ubuntu","arch":"x86_64","sshAuthSock":"","display":"","tmpDir":"/run/user/1005","platform":"linux","execServerToken":"aaa11a11-aaa1-11aa-aa11-1a111aa1aaa1"},"serverStartTime":35,"installUnpackCode":""}
[14:53:36.341] Starting forwarding server. local port 63523 -> socksPort 63509 -> remotePort 34975
[14:53:36.342] Forwarding server listening on port 63523
[14:53:36.342] Waiting for ssh tunnel to be ready
[14:53:36.343] [Forwarding server port 63523] Got connection 0
[14:53:36.344] Tunneled port 34975 to local port 63523
[14:53:36.344] Resolved "ssh-remote+192.168.3.55" to "port 63523"
[14:53:36.351] Initizing new exec server for ssh-remote+192.168.3.55
[14:53:36.351] Resolving exec server at port 63523
[14:53:36.352] [Forwarding server port 63523] Got connection 1
[14:53:36.356] > 
[14:53:36.378] Exec server for ssh-remote+192.168.3.55 created and cached
[14:53:36.382] ------




[14:53:36.383] No hints found in the recent session.
[14:53:36.390] [server] Checking /home/chenghl/.vscode-server/cli/servers/Stable-91fbdddc47bc9c09064bf7acf133d22631cbf083/log.txt and /home/chenghl/.vscode-server/cli/servers/Stable-91fbdddc47bc9c09064bf7acf133d22631cbf083/pid.txt for a running server...

Expected Behavior

I can't find "log.txt" and "pid.txt" in the specified path. Actually, the ".vscode-server/cli" finder is empty. I got the same issue as discussed here: Remote-SSH freeze at ''Checking #####.log #####.pid for a running server #4458.
I have followed the suggested solution: I ran the command "Kill VS Code Server on Host..." from the command palette and set remote.SSH.useExecServer to false. However, it does not work. I have tried nearly everything that I could find online, but none of them have resolved this issue.

Actual Behavior

Any help would be much appreciated.

Steps To Reproduce

log Level: 2
VS Code version: 1.96.3
Remote-SSH version: [email protected]
win32 x64

Anything else?

No response

[joshspicer]

Do you see a file named /home/chenghl/.vscode-server/.cli.<commit>.log on your remote? Note that it's a hidden file (with a leading .). In a linux terminal you would need to use ls -la to see all files. It would be interesting to see the contents of this log file

Could you share a bit about the remote machine? Is the filesystem where /home/ exists on a remote file share (cifs) or similar? We've seen issues in the past of funky filesystems and reading/writing files.

Additionally, could there be any proxy or filewall in place that could be blocking a connection to a Microsoft domain? I would expect the next lines to be:

[10:51:00.521] [server] Installing and setting up Visual Studio Code Server...
[10:51:00.658] [server] Downloading Visual Studio Code server -> /tmp/.tmpUf2HJz/vscode-server-linux-x64.tar.gz
[10:51:00.658] [server] server download progress: 0/61847744 (0%)
[10:51:00.712] [server] server download progress: 2105344/61847744 (3%)
[10:51:00.730] [server] server download progress: 4210688/61847744 (7%)

where the exact server for your installed version of VS Code is fetched. We have various fallbacks in place but perhaps in this case we're hanging on a particular firewall rule instead of failing gracefully