Nvidia Vaapi video decode not working on nvidia beta drivers 525.53 works on 520 stable

[jrgiacone]

Describe the bug

nvidia video decode is not working on the beta drivers. It works on stable

Steps To Reproduce

Steps to reproduce the behavior:

1. install latest nvidia and run vainfo note codecs show up
2. install beta nvidia and run vainfo note no codecs show up
3. ...

Expected behavior

Expected behavior is for the video codecs to show and vaapi to work with the nvidia vaapi driver

Screenshots

[jrgiacone@nixos:~]$ nix-shell -p libva-utils --run vainfo
libva info: VA-API version 1.15.0
libva info: User environment variable requested driver 'nvidia'
libva info: Trying to open /run/opengl-driver/lib/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0

When on the 520 drivers all codecs are shown and video decode is working

Additional context

Add any other context about the problem here.

Notify maintainers

@Kiskae

Metadata

Please run `` and paste the result.

jrgiacone@nixos:~]$ nix-shell -p nix-info --run "nix-info -m"
this path will be fetched (0.00 MiB download, 0.00 MiB unpacked):
  /nix/store/m142rmlj114xwgwgf6k2gvvm33zxkbia-nix-info
copying path '/nix/store/m142rmlj114xwgwgf6k2gvvm33zxkbia-nix-info' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 6.0.9, NixOS, 23.05 (Stoat), 23.05.20221127.a115bb9`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.0`
 - channels(jrgiacone): `""`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Here is my nvidia module:

{ config, pkgs, ... }:
  {
      services.xserver.videoDrivers = [ "nvidia" ];
      hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.beta;
      hardware.opengl.enable = true;
      hardware.nvidia.modesetting.enable = true;
      #hardware.nvidia.open = true;
      #hardware.nvidia.powerManagement.enable = true;
      boot.kernelModules = ["nvidia" "nvidia_modeset" "nvidia_uvm" "nvidia_drm"];
      environment.systemPackages = [
        pkgs.nvidia-vaapi-driver
      ];
  }

[Kiskae]

Just to confirm did you restart your system after changing drivers?
Because nvidia really doesn't like mismatched kernel and userspace components communicating and I am getting the same behavior when trying to use the 520 branch with my 515 branch kernel modules.

For context performing ltrace while overriding the driver with LD_LIBRARY_PATH shows that it just quits during initialization, resulting in the lack of output you're observing:

[pid 4045714] vaInitialize(0x432fc0, 0x7ffd7398a508, 0x7ffd7398a510, 0x433320libva info: VA-API version 1.16.0
libva info: Trying to open /run/opengl-driver/lib/dri/nvidia_drv_video.so
<no return ...>

[jrgiacone]

Just to confirm did you restart your system after changing drivers? Because nvidia really doesn't like mismatched kernel and userspace components communicating and I am getting the same behavior when trying to use the 520 branch with my 515 branch kernel modules.

For context performing ltrace while overriding the driver with LD_LIBRARY_PATH shows that it just quits during initialization, resulting in the lack of output you're observing:

[pid 4045714] vaInitialize(0x432fc0, 0x7ffd7398a508, 0x7ffd7398a510, 0x433320libva info: VA-API version 1.16.0
libva info: Trying to open /run/opengl-driver/lib/dri/nvidia_drv_video.so
<no return ...>

Yea I have restarted, currently running kernel 6.0.9 with ZFSunstable. I just tried on my arch installation to see if it is the nvidia driver playing funky or other packages that interact with it.

When installing 525.6 on arch and running vainfo it just states checking for wayland, checking for x11 and then stops

[Kiskae]

This appears to be an open issue upstream:

elFarto/nvidia-vaapi-driver#126
NVIDIA/open-gpu-kernel-modules#414
https://forums.developer.nvidia.com/t/cueglstreamproducerconnect-returns-error-801-on-525-53-driver/233610

[Kiskae]

#204104 makes it so the vaapi driver will at least not crash, but it will simply report that there are no decoders available.

[Kiskae]

It looks like this was fixed in the 525.85.05 driver release. Since 23.05 is still on the 515 branch it should be fixed in all channels.