VSCode 1.88.0 changed paths => nixos-vscode-server no longer functional

[samuela]

I just upgraded to VSCode 1.88.0 and it has broken nixos-vscode-server by moving the installation path of binaries. Logs reveal that there is a failure to execute /home/skainswo/.vscode-server/cli/servers/Stable-5c3e652f63e798a5ac2f31ffd0d863669328dc4c.staging/server/bin/code-server, which contains:

#!/usr/bin/env sh
#
# Copyright (c) Microsoft Corporation. All rights reserved.
#

case "$1" in
	--inspect*) INSPECT="$1"; shift;;
esac

ROOT="$(dirname "$(dirname "$(readlink -f "$0")")")"

"$ROOT/node" ${INSPECT:-} "$ROOT/out/server-main.js" "$@"

"$ROOT/node" resolves to /home/skainswo/.vscode-server/cli/servers/Stable-5c3e652f63e798a5ac2f31ffd0d863669328dc4c.staging/server/node which differs from previous node installation locations AFAICT. This file is not patched:

❯ ldd /home/skainswo/.vscode-server/cli/servers/Stable-5c3e652f63e798a5ac2f31ffd0d863669328dc4c.staging/server/node
	linux-vdso.so.1 (0x0000ffffba48b000)
	libdl.so.2 => /nix/store/sk71q8mwavsax8piwjx4jjzy45vkrixr-glibc-2.38-44/lib/libdl.so.2 (0x0000ffffba420000)
	libstdc++.so.6 => not found
	libm.so.6 => /nix/store/sk71q8mwavsax8piwjx4jjzy45vkrixr-glibc-2.38-44/lib/libm.so.6 (0x0000ffffba370000)
	libgcc_s.so.1 => /nix/store/9cha78hyh74ys0q8cszrzggm49c69ixx-xgcc-12.3.0-libgcc/lib/libgcc_s.so.1 (0x0000ffffba330000)
	libpthread.so.0 => /nix/store/sk71q8mwavsax8piwjx4jjzy45vkrixr-glibc-2.38-44/lib/libpthread.so.0 (0x0000ffffba300000)
	libc.so.6 => /nix/store/sk71q8mwavsax8piwjx4jjzy45vkrixr-glibc-2.38-44/lib/libc.so.6 (0x0000ffffba150000)
	/lib/ld-linux-aarch64.so.1 => /nix/store/sk71q8mwavsax8piwjx4jjzy45vkrixr-glibc-2.38-44/lib/ld-linux-aarch64.so.1 (0x0000ffffba44e000)

IIUC adjustments to the installPath and bin_dir logic in https://github.com/nix-community/nixos-vscode-server/blob/master/pkgs/auto-fix-vscode-server.nix will be required.

[samuela]

Solution proposal: Scan for all binaries in the ~/.vscode-server directory tree, and patch each of them if they have not been patched already.

[samuela]

The script

#! /usr/bin/env nix-shell
#! nix-shell -i sh -p file patchelf

libpath=$(nix eval --raw --impure --expr 'with import <nixpkgs> {}; lib.makeLibraryPath [ stdenv.cc.libc stdenv.cc.cc curl icu libunwind libuuid lttng-ust openssl zlib krb5 ]')
files_to_patch=$(find ~/.vscode-server -type f -name '*node' -exec file {} + | grep 'ELF' | grep 'executable' | grep 'interpreter /lib/' | awk -F: '{print $1}')
interpreter="$(cat $(nix eval --raw --impure --expr '(import <nixpkgs> {}).stdenv.cc.outPath')/nix-support/dynamic-linker)"

for elf in $files_to_patch; do
  echo "patching $elf"
  patchelf --set-interpreter "$interpreter" --add-rpath "$libpath" "$elf"
done

succeeds in making the executables runnable, but it's not enough to satisfy VSCode 1.88.0 which is apparently not fooled by my patchelf shenanigans.

[samuela]

Update: running

cp -r /home/skainswo/.vscode-server/cli/servers/Stable-5c3e652f63e798a5ac2f31ffd0d863669328dc4c.staging/ /home/skainswo/.vscode-server/cli/servers/Stable-5c3e652f63e798a5ac2f31ffd0d863669328dc4c

along with the above script works.

Altogether now, my script is

#! /usr/bin/env nix-shell
#! nix-shell -i sh -p file patchelf

# Move the <foo>.staging directories to <foo>. Necessary since VSCode 1.88.0.
find ~/.vscode-server/cli/servers -type d -name '*.staging' -exec bash -c 'for dir; do mv "$dir" "${dir%.staging}"; done' bash {} +

libpath=$(nix eval --raw --impure --expr 'with import <nixpkgs> {}; lib.makeLibraryPath [ stdenv.cc.libc stdenv.cc.cc curl icu libunwind libuuid lttng-ust openssl zlib krb5 ]')
files_to_patch=$(find ~/.vscode-server -type f -name '*node' -exec file {} + | grep 'ELF' | grep 'executable' | grep 'interpreter /lib/' | awk -F: '{print $1}')
interpreter="$(cat $(nix eval --raw --impure --expr '(import <nixpkgs> {}).stdenv.cc.outPath')/nix-support/dynamic-linker)"

for elf in "$files_to_patch"; do
  echo "patching $elf"
  patchelf --set-interpreter "$interpreter" --add-rpath "$libpath" "$elf"
done

[lucianthorr]

@samuela, thanks for looking into this. I think I'm running into the same problem. I just booted my little nixos machine last night for the first time in a few months and cannot get Remote Explorer to connect. It seems to be stuck looping the "Downloading VS Code Server."

Unfortunately the above script doesn't seem to work. I get a patchelf: getting info about '': No such file or directory error running from the home directory. Can I run it as a simple shell script or does it need to be wired into something? I'm a nixos novice so I might just be missing some context. Thanks.

[samuela]

It seems to be stuck looping the "Downloading VS Code Server."

Yes, this is exactly the behavior I was experiencing as well.

Can I run it as a simple shell script or does it need to be wired into something?

It should run fine just as a shell script... What version of VSCode are you using?

[n-hass]

thanks for the script @samuela. I changed the files_to_patch= line to files_to_patch=$(find ~/.vscode-server -type f -name '*node' -exec file {} + | grep 'ELF' | grep 'executable' | awk -F: '/interpreter \/lib64?\// {print $1}') becuase /lib/ wasn’t giving me any results. @lucianthorr that might be your problem&fix too.

[lucianthorr]

@n-hass That totally did it. Thanks!

One slight typo in the above. The last single quote got "auto-incorrected" to a ’. It needs to be a '

[lblasc]

With #78 new binary paths are handled correctly. I've switched few nixos 23.11 machines to it, so far no problems.

[essential-randomness]

I've upgraded my machine to use a version that I believe includes #78, but I'm still unable to connect on newer versions of VSCode. Downgrading VSCode to 1.86.2 works and I can connect as before.

Is there more that needs to be done?

The error is:

[14:47:35.362] [server] Error installing server: error checking server integrity: failed to run command "/home/msboba/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685.staging/server/bin/code-server --version" (code 127): /home/msboba/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685.staging/server/bin/code-server: line 12: /home/msboba/.vscode-server/cli/servers/Stable-dc96b837cf6bb4af9cd736aa3af08cf8279f7685.staging/server/node: cannot execute: required file not found

(note that these files actually exist on my machine when I SSH into it)

My configuration is:

  services.vscode-server = {
    enable = true;
    installPath = "$HOME/.vscode-server";
   };

and in my lockfile:

 "vscode-server": {
      "inputs": {
        "flake-utils": "flake-utils_4",
        "nixpkgs": "nixpkgs_3"
      },
      "locked": {
        "lastModified": 1713958148,
        "narHash": "sha256-8PDNi/dgoI2kyM7uSiU4eoLBqUKoA+3TXuz+VWmuCOc=",
        "owner": "nix-community",
        "repo": "nixos-vscode-server",
        "rev": "fc900c16efc6a5ed972fb6be87df018bcf3035bc",
        "type": "github"
      },
      "original": {
        "owner": "nix-community",
        "repo": "nixos-vscode-server",
        "type": "github"
      }
    }
    ```

[Ten0]

I've upgraded my machine to use a version that I believe includes #78, but I'm still unable to connect on newer versions of VSCode.

Can you try removing your .vscode-server/cli/servers/ folder and attempting to connect again? I fear that if the folder is left in a dirty state by incompatible versions it may not recover.

[ashkan-leo]

I am also facing similar issues. When connecting with VS Code Version: 1.93.1 I get the error

Failed to connect to the remote extension host server (Error: WrappedError(WrappedError { message: "error checking server integrity", original: "failed to run command \"/home/ashkan/.vscode-server/cli/servers/Stable-38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40.staging/server/bin/code-server --version\" (code 127): Could not start dynamically linked executable: /home/ashkan/.vscode-server/cli/servers/Stable-38c31bc77e0dd6ae88a4e9cc93428cc27a56ba40.staging/server/node\nNixOS cannot run dynamically linked executables intended for generic\nlinux environments out of the box. For more information, see:\nhttps://nix.dev/permalink/stub-ld\n" }))

Any advise on how to fix?

[Ten0]

VS Code Version: 1.93.1

This seems like a relatively old version. Are you sure that's the correct version number, and if so, have you tried removing the .vscode-server folder, then restarting the user service ?

#86 (comment)

[essential-randomness]

I've upgraded my machine to use a version that I believe includes #78, but I'm still unable to connect on newer versions of VSCode.

Can you try removing your .vscode-server/cli/servers/ folder and attempting to connect again? I fear that if the folder is left in a dirty state by incompatible versions it may not recover.

Sorry, i hadn't seen this! Deleting that folder didn't work, but deleting .vscode-server/ itself worked!