Severe degradation using obs-vkcapture together with gamescope on Steam games

[Plarpoon]

Describe the bug
Hello, my name is Alex,
I have been having this bug for almost a week now, I have opened issues at Valve's gamescope, RedHat BugZilla and even talked with the OBS support on their Discord. This is the last place I have left then I don't know how to solve this issue anymore, so please if you have any idea on how I could better identify the problem help me out.

Here is the issue!
Since the day after I updated to Fedora 40 KDE (from Fedora 39 KDE) whenever I record a video that is taken from gamescope with the Game Capture function on OBS, using obs-vkcapture the video in the preview and the recorded (or streamed) format will look like the video embedded below.

Here is the list of the odd things:

• The recording worked well until well over 24h after the upgrade to Fedora 40, so it's not something that became off after the upgrade.
• When recording a game using gamescope and obs-vkcapture outside of Steam the video works perfectly fine (example WoW running from Bottles)
• When recording a game on Steam without using gamescope the error doesn't show itself (for example capturing it with pipewire's window capture utility

The solutions I have tried as of now are as follows:

• I have tried downgrading the gamescope package to the version that existed until immediately before the Fedora 40 update, to no anvail.
• I have tried running a Steam game without gamescope enabled and it's being recorded fine.
• I have tried recording a game with gamescope enabled outside Steam and it's working fine
• I have tried completely removing OBS (which I am using the flatpak version btw) and reinstalling it removing every single setting in the .var directory and restarting clean, still nothing.
• I have also tried with or withouth HDR enabled on my system and going back to the standard profile all together, but that wasn't the problem either.

Screenshots or video
https://youtu.be/7svRzATLp_0

System (please complete the following information):

• OS: Fedora Linux 40 (KDE Plasma) x86_64
• GPU: AMD Radeon RX 7900 XTX
• Mesa version: Mesa 24.0.6
• obs-vkcapture version:

OBS log:

23:23:04.188: Platform: Wayland
23:23:04.189: CPU Name: AMD Ryzen 9 7900X3D 12-Core Processor
23:23:04.189: CPU Speed: 4785.582MHz
23:23:04.189: Physical Cores: 12, Logical Cores: 24
23:23:04.189: Physical Memory: 64002MB Total, 47288MB Free
23:23:04.189: Kernel Version: Linux 6.8.7-300.fc40.x86_64
23:23:04.189: Flatpak Branch: stable
23:23:04.189: Flatpak Arch: x86_64
23:23:04.189: Flatpak Runtime: runtime/org.kde.Platform/x86_64/6.6
23:23:04.189: App Extensions:
23:23:04.189:  - com.obsproject.Studio.Plugin.OBSVkCapture
23:23:04.189:  - org.freedesktop.LinuxAudio.Plugins.TAP
23:23:04.189:  - org.freedesktop.LinuxAudio.Plugins.swh
23:23:04.189:  - com.obsproject.Studio.Locale
23:23:04.189: Runtime Extensions:
23:23:04.189:  - org.freedesktop.Platform.GL.default
23:23:04.189:  - org.freedesktop.Platform.openh264
23:23:04.189:  - org.freedesktop.Platform.VulkanLayer.OBSVkCapture
23:23:04.189:  - org.freedesktop.Platform.VulkanLayer.gamescope
23:23:04.189:  - org.gtk.Gtk3theme.Breeze
23:23:04.189:  - org.kde.KStyle.Adwaita
23:23:04.189:  - org.kde.Platform.Locale
23:23:04.189:  - org.freedesktop.Platform.GL.default
23:23:04.189: Flatpak Framework Version: 1.15.8
23:23:04.189: Desktop Environment: KDE (KDE)
23:23:04.189: Session Type: wayland
23:23:04.191: Qt Version: 6.6.3 (runtime), 6.6.2 (compiled)
23:23:04.191: Portable mode: false
23:23:04.212: OBS 30.1.2 (linux)
23:23:04.212: ---------------------------------
23:23:04.212: ---------------------------------
23:23:04.212: audio settings reset:
23:23:04.212: 	samples per sec: 48000
23:23:04.212: 	speakers:        2
23:23:04.212: 	max buffering:   960 milliseconds
23:23:04.212: 	buffering type:  dynamically increasing
23:23:04.215: ---------------------------------
23:23:04.215: Initializing OpenGL...
23:23:04.215: Using EGL/Wayland
23:23:04.223: Initialized EGL 1.5
23:23:04.232: Loading up OpenGL on adapter AMD AMD Radeon RX 7900 XTX (radeonsi, navi31, LLVM 17.0.6, DRM 3.57, 6.8.7-300.fc40.x86_64)
23:23:04.232: OpenGL loaded successfully, version 4.6 (Core Profile) Mesa 24.0.6 (git-c659c7e660), shading language 4.60
23:23:04.252: ---------------------------------
23:23:04.252: video settings reset:
23:23:04.252: 	base resolution:   1920x1080
23:23:04.252: 	output resolution: 1920x1080
23:23:04.252: 	downscale filter:  Bicubic
23:23:04.252: 	fps:               60/1
23:23:04.252: 	format:            NV12
23:23:04.252: 	YUV mode:          Rec. 709/Partial
23:23:04.252: NV12 texture support enabled
23:23:04.252: P010 texture support not available
23:23:04.254: Audio monitoring device:
23:23:04.254: 	name: Default
23:23:04.254: 	id: default
23:23:04.254: ---------------------------------
23:23:04.268: No AJA devices found, skipping loading AJA UI plugin
23:23:04.268: Failed to initialize module 'aja-output-ui.so'
23:23:04.279: No AJA devices found, skipping loading AJA plugin
23:23:04.279: Failed to initialize module 'aja.so'
23:23:04.284: Failed to load 'en-US' text for module: 'decklink-captions.so'
23:23:04.291: Failed to load 'en-US' text for module: 'decklink-output-ui.so'
23:23:04.297: A DeckLink iterator could not be created.  The DeckLink drivers may not be installed
23:23:04.298: Failed to initialize module 'decklink.so'
23:23:04.454: [pipewire] Available captures:
23:23:04.454: [pipewire]     - Desktop capture
23:23:04.454: [pipewire]     - Window capture
23:23:04.476: v4l2loopback not installed, virtual camera disabled
23:23:04.486: [obs-browser]: Version 2.23.4
23:23:04.486: [obs-browser]: CEF Version 103.0.5060.134 (runtime), 103.0.0-5060-shared-textures.2594+g17f8588+chromium-103.0.5060.134 (compiled)
23:23:04.501: VAAPI: API version 1.19
23:23:04.502: FFmpeg VAAPI H264 encoding supported
23:23:04.507: FFmpeg VAAPI AV1 encoding supported
23:23:04.512: FFmpeg VAAPI HEVC encoding supported
23:23:04.596: [obs-websocket] [obs_module_load] you can haz websockets (Version: 5.4.2 | RPC Version: 1)
23:23:04.596: [obs-websocket] [obs_module_load] Qt version (compile-time): 6.6.2 | Qt version (run-time): 6.6.3
23:23:04.596: [obs-websocket] [obs_module_load] Linked ASIO Version: 102900
23:23:04.600: [obs-websocket] [obs_module_load] Module loaded.
23:23:04.632: [linux-vkcapture] plugin loaded successfully (version 1.5.0)
23:23:04.632: ---------------------------------
23:23:04.632:   Loaded Modules:
23:23:04.632:     linux-vkcapture.so
23:23:04.632:     text-freetype2.so
23:23:04.632:     rtmp-services.so
23:23:04.632:     obs-x264.so
23:23:04.632:     obs-websocket.so
23:23:04.632:     obs-webrtc.so
23:23:04.632:     obs-vst.so
23:23:04.632:     obs-transitions.so
23:23:04.632:     obs-qsv11.so
23:23:04.632:     obs-outputs.so
23:23:04.632:     obs-libfdk.so
23:23:04.632:     obs-filters.so
23:23:04.632:     obs-ffmpeg.so
23:23:04.632:     obs-browser.so
23:23:04.632:     linux-v4l2.so
23:23:04.632:     linux-pulseaudio.so
23:23:04.632:     linux-pipewire.so
23:23:04.632:     linux-jack.so
23:23:04.632:     linux-capture.so
23:23:04.632:     image-source.so
23:23:04.632:     frontend-tools.so
23:23:04.632:     decklink-output-ui.so
23:23:04.632:     decklink-captions.so
23:23:04.632: ---------------------------------
23:23:04.632: ==== Startup complete ===============================================
23:23:04.662: All scene data cleared
23:23:04.662: ------------------------------------------------
23:23:04.666: pulse-input: Server name: 'PulseAudio (on PipeWire 1.0.5) 15.0.0'
23:23:04.666: pulse-input: Audio format: s32le, 48000 Hz, 8 channels
23:23:04.666: pulse-input: Started recording from 'alsa_output.pci-0000_08_04.0.analog-surround-71.monitor' (default)
23:23:04.666: [Loaded global audio device]: 'Desktop Audio'
23:23:04.666: pulse-input: Server name: 'PulseAudio (on PipeWire 1.0.5) 15.0.0'
23:23:04.666: pulse-input: Audio format: s16le, 48000 Hz, 2 channels
23:23:04.666: pulse-input: Started recording from 'alsa_input.usb-Blue_Microphones_Yeti_Stereo_Microphone_REV8-00.pro-input-0' (default)
23:23:04.666: [Loaded global audio device]: 'Mic/Aux'
23:23:04.666: ext_screencopy_manager_v1 not available
23:23:04.668: Switched to scene 'Scene'
23:23:04.668: ------------------------------------------------
23:23:04.668: Loaded scenes:
23:23:04.668: - scene 'Scene':
23:23:04.668:     - source: 'Game Capture' (vkcapture-source)
23:23:04.668: ------------------------------------------------
23:23:05.215: adding 21 milliseconds of audio buffering, total audio buffering is now 21 milliseconds (source: Desktop Audio)
23:23:05.215: 
23:23:05.237: adding 21 milliseconds of audio buffering, total audio buffering is now 42 milliseconds (source: Mic/Aux)
23:23:05.237: 
23:23:39.373: [linux-vkcapture] Client 1 connected (pid=0)
23:23:41.804: [linux-vkcapture] Creating texture from dmabuf 1272x680 modifier:144115188757872388
23:23:41.804: [linux-vkcapture]  [0] fd:45 stride:5120 offset:0
23:23:41.804: [linux-vkcapture]  [1] fd:46 stride:2048 offset:3932160
23:23:41.825: [linux-vkcapture] Client 2 connected (pid=0)
23:23:56.232: [linux-vkcapture] Client 2 disconnected
23:23:56.330: [linux-vkcapture] Client 1 disconnected
23:24:21.162: ==== Shutting down ==================================================
23:24:21.164: pulse-input: Stopped recording from 'alsa_output.pci-0000_08_04.0.analog-surround-71.monitor'
23:24:21.164: pulse-input: Got 3058 packets with 3669600 frames
23:24:21.164: pulse-input: Stopped recording from 'alsa_input.usb-Blue_Microphones_Yeti_Stereo_Microphone_REV8-00.pro-input-0'
23:24:21.164: pulse-input: Got 3059 packets with 3670800 frames
23:24:21.164: [linux-vkcapture] destroy
23:24:21.185: All scene data cleared
23:24:21.185: ------------------------------------------------
23:24:21.226: [linux-vkcapture] plugin unloaded
23:24:21.227: [obs-websocket] [obs_module_unload] Shutting down...
23:24:21.227: Tried to call obs_frontend_remove_event_callback with no callbacks!
23:24:21.227: [obs-websocket] [obs_module_unload] Finished shutting down.
23:24:21.232: [Scripting] Total detached callbacks: 0
23:24:21.232: Freeing OBS context data
23:24:21.244: == Profiler Results =============================
23:24:21.244: run_program_init: 565.333 ms
23:24:21.244:  ┣OBSApp::AppInit: 3.156 ms
23:24:21.244:  ┃ ┗OBSApp::InitLocale: 0.516 ms
23:24:21.244:  ┗OBSApp::OBSInit: 490.412 ms
23:24:21.244:    ┣obs_startup: 2.325 ms
23:24:21.244:    ┗OBSBasic::OBSInit: 466.688 ms
23:24:21.244:      ┣OBSBasic::InitBasicConfig: 0.091 ms
23:24:21.244:      ┣OBSBasic::ResetAudio: 0.187 ms
23:24:21.244:      ┣OBSBasic::ResetVideo: 41.718 ms
23:24:21.244:      ┃ ┗obs_init_graphics: 39.719 ms
23:24:21.244:      ┃   ┗shader compilation: 19.793 ms
23:24:21.244:      ┣OBSBasic::InitOBSCallbacks: 0.003 ms
23:24:21.244:      ┣OBSBasic::InitHotkeys: 0.018 ms
23:24:21.244:      ┣obs_load_all_modules2: 377.492 ms
23:24:21.244:      ┃ ┣obs_init_module(aja-output-ui.so): 0.064 ms
23:24:21.244:      ┃ ┣obs_init_module(aja.so): 0.072 ms
23:24:21.244:      ┃ ┣obs_init_module(decklink-captions.so): 0 ms
23:24:21.244:      ┃ ┣obs_init_module(decklink-output-ui.so): 0 ms
23:24:21.244:      ┃ ┣obs_init_module(decklink.so): 0.106 ms
23:24:21.244:      ┃ ┣obs_init_module(frontend-tools.so): 53.143 ms
23:24:21.244:      ┃ ┣obs_init_module(image-source.so): 0.01 ms
23:24:21.244:      ┃ ┣obs_init_module(linux-capture.so): 0 ms
23:24:21.244:      ┃ ┣obs_init_module(linux-jack.so): 0.002 ms
23:24:21.244:      ┃ ┣obs_init_module(linux-pipewire.so): 9.43 ms
23:24:21.244:      ┃ ┣obs_init_module(linux-pulseaudio.so): 0.002 ms
23:24:21.244:      ┃ ┣obs_init_module(linux-v4l2.so): 7.214 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-browser.so): 0.048 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-ffmpeg.so): 18.362 ms
23:24:21.244:      ┃ ┃ ┗nvenc_check: 1.363 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-filters.so): 0.021 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-libfdk.so): 0.002 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-outputs.so): 0.008 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-qsv11.so): 12.176 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-transitions.so): 0.006 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-vst.so): 0.003 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-webrtc.so): 0.008 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-websocket.so): 3.552 ms
23:24:21.244:      ┃ ┣obs_init_module(obs-x264.so): 0.003 ms
23:24:21.244:      ┃ ┣obs_init_module(rtmp-services.so): 0.735 ms
23:24:21.244:      ┃ ┣obs_init_module(text-freetype2.so): 0.019 ms
23:24:21.244:      ┃ ┗obs_init_module(linux-vkcapture.so): 0.115 ms
23:24:21.244:      ┣OBSBasic::InitService: 0.662 ms
23:24:21.244:      ┣OBSBasic::ResetOutputs: 0.14 ms
23:24:21.244:      ┣OBSBasic::CreateHotkeys: 0.017 ms
23:24:21.244:      ┣OBSBasic::InitPrimitives: 0.073 ms
23:24:21.244:      ┗OBSBasic::Load: 35.147 ms
23:24:21.244: obs_hotkey_thread(25 ms): min=0 ms, median=0 ms, max=0.001 ms, 99th percentile=0 ms, 100% below 25 ms
23:24:21.244: audio_thread(Audio): min=0.005 ms, median=0.017 ms, max=0.57 ms, 99th percentile=0.037 ms
23:24:21.244: obs_graphics_thread(16.6667 ms): min=0.065 ms, median=0.18 ms, max=111.566 ms, 99th percentile=0.369 ms, 99.8262% below 16.667 ms
23:24:21.244:  ┣tick_sources: min=0 ms, median=0.003 ms, max=0.197 ms, 99th percentile=0.01 ms
23:24:21.244:  ┣output_frame: min=0.041 ms, median=0.067 ms, max=3.365 ms, 99th percentile=0.19 ms
23:24:21.244:  ┃ ┗gs_context(video->graphics): min=0.041 ms, median=0.067 ms, max=3.364 ms, 99th percentile=0.19 ms
23:24:21.244:  ┃   ┣render_video: min=0.006 ms, median=0.013 ms, max=1.625 ms, 99th percentile=0.11 ms
23:24:21.244:  ┃   ┃ ┗render_main_texture: min=0.004 ms, median=0.011 ms, max=1.618 ms, 99th percentile=0.107 ms
23:24:21.244:  ┃   ┗gs_flush: min=0.001 ms, median=0.04 ms, max=0.367 ms, 99th percentile=0.079 ms
23:24:21.244:  ┗render_displays: min=0.001 ms, median=0.091 ms, max=111.488 ms, 99th percentile=0.175 ms
23:24:21.244: =================================================
23:24:21.244: == Profiler Time Between Calls ==================
23:24:21.244: obs_hotkey_thread(25 ms): min=25.009 ms, median=25.056 ms, max=26.326 ms, 99.9675% within ±2% of 25 ms (0% lower, 0.0325415% higher)
23:24:21.244: obs_graphics_thread(16.6667 ms): min=0.263 ms, median=16.667 ms, max=111.572 ms, 99.4134% within ±2% of 16.667 ms (0.304149% lower, 0.282425% higher)
23:24:21.244: =================================================
23:24:21.245: Number of memory leaks: 0

Game log:
steam-1113000.log

[nowrep]

Try gamescope -- obs-vkcapture %command%

[Plarpoon]

Here is the odd thing, the artifacts are completely gone on Persona 4 Golden using that launch parameter. But I cannot make OBS see Portal 2 as an available source if I put obs-vkcapture after the two lines. Keeping same settings and everything obviously.

If you have a spare second would you mind explaining what those two lines do? Is it some sort of piping? Btw, thank you for the quick reply!

(here is the launch parameters I have used to test both Person 4 Golden and Portal 2 just now)

PROTON_LOG=1 OBS_VKCAPTURE_QUIET=0 gamemoderun gamescope -- obs-vkcapture %command%

Logs

• Portal 2 logs:
  steam-620.log
• OBS logs with Portal 2:
  2024-04-29 13-36-05.txt

---

• Persona 4 Golden logs:
  steam-1113000.log
• OBS logs with Persona 4 Golden:
  2024-04-29 13-37-19.txt

[nowrep]

You're using old obs-vkcapture version (1.4.3), current is 1.5.0.

But I cannot make OBS see Portal 2 as an available source

Do you have 32bit obs-vkcapture installed?

If you have a spare second would you mind explaining what those two lines do? Is it some sort of piping?

https://unix.stackexchange.com/questions/11376/what-does-double-dash-double-hyphen-mean

[Plarpoon]

I actually have the latest flatpak versions installed of both these packages:

com.obsproject.Studio.Plugin.OBSVkCapture

org.freedesktop.Platform.VulkanLayer.OBSVkCapture

As far as I know a --version switch doesn't exist so I unfortunately cannot tell you for certain which version it is and I do not even know if a 32 bit flatpak version exists

---

I am going to be honest, I pretty much just followed the instructions on this README file. If I did something wrong please let me know and I will try to fix it.

Thanks again!

[Plarpoon]

@SuNNjek perhaps you can help me solving this problem, I see you are the main maintainer of the flatpaks. I would be really glad if you could give me your feedback too.

Thank you in advance!

[SuNNjek]

@Plarpoon To answer the question about a 32-bit flatpak: There's no separate flatpak for the 32-bit libraries, both the 64-bit and 32-bit libraries are included in the flatpak. If you are getting an output (albeit corrupted) I doubt that flatpak is the problem here honestly.

Unfortunately, I currently don't have a gaming capable Linux-machine ready to reproduce this on. Maybe Portal 2 will run on my underpowered ThinkPad. If it does, I'll try and look into it 😅

[Plarpoon]

I mean.. it's a 2011 game so perhaps it could (wish you luck 😃).

Anyway, I think nowrep might have nailed the other issue with the question raised before. I just tested Half-Life (the 1998 one) which should also be at 32 bit and it's not detected at all by Game Capture.

If this issue gets also solved together with the corruption issue already being mitigated thanks to the usage of the end of options delimiter we are nearing the closure of this GitHub issue.

Btw, let me know if you would prefer for me to close this issue right now and open a new one in @SuNNjek's repos instead.

[SuNNjek]

@Plarpoon It seems to work fine on my system (ThinkPad T480s, Arch Linux, using the same command you used). For some reason my mouse input is broken, that's why I'm spinning uncontrollably, but that has nothing to do with the recording, so ignore that 😅

test2.mp4

Does the output look corrupted in the stream preview in OBS as well or is it only an output issue? Have you tried different encoder settings?

[Plarpoon]

I think there was a miscommunication sorry for my poor English, the issue of video corruption was resolved when I modified the launch parameters. Initially, I was using:

PROTON_LOG=1 obs-vkcapture OBS_VKCAPTURE_QUIET=0 gamemoderun gamescope %command%

However, I switched to the following format based on a suggestion from nowrep here, which resolved the video degradation issue:

PROTON_LOG=1 OBS_VKCAPTURE_QUIET=0 gamemoderun gamescope -- obs-vkcapture %command%

Despite this, a new problem arose. Even though I have installed both of these flatpak packages:

obsproject.Studio.Plugin.

org.freedesktop.Platform.VulkanLayer.OBSVkCapture

I’m unable to record any 32-bit video game if I place the obs-vkcapture string after the --. It works when it’s placed before the --, but not after.

For example:

This command makes video capturing be recognized by OBS, but it results in video corruption:

PROTON_LOG=1 obs-vkcapture OBS_VKCAPTURE_QUIET=0 gamemoderun gamescope %command%

This command doesn’t cause video corruption, but OBS doesn’t make the video source appear if the game is in 32-bit:

PROTON_LOG=1 OBS_VKCAPTURE_QUIET=0 gamemoderun gamescope -- obs-vkcapture %command%

[Plarpoon]

I have updated my previous comment after realizing how much badly it was written in English, please let me know if you have doubts on what I am meaning, sorry again

[nowrep]

I think LD_PRELOAD (which is used for OpenGL capture) may not be correct in flatpak, there were other issues about this too.

[Plarpoon]

If I can help testing something or you need any more feedback I am available for anything, from providing logs, testing things to straight up join one of you in a VC with screen-sharing. I am at your complete availability as this is a problem I would like to see solved as-well.

Thank you all!

[Plarpoon]

@SuNNjek if you think you can confirm nowrep's suspicion about LD_PRELOAD being incorrect in the FlatPaks I will close the issues on Valve's GitHub and RedHat's BugZilla.

Take all the time you need, relax and enjoy your first of May both of you! 😃

[nowrep]

The reason why capturing gamescope results in partially corrupted image is because gamescope presents without wait semaphore which works for compositors but not for capture. Probably it would be best to just blacklist gamescope, same as what mangohud does.

[Plarpoon]

Is there anything that I can do to make it work with the 32 bit game scope apps? Because unfortunately this is a hit or miss issue, I recorded the entire first half of Portal 2 and it worked perfectly fine, the day after booting the PC as usual and couldn't make it work without corruption.

My understanding as of now is that the best option is to just wait for an update where gamescope is blacklisted as you suggested but of-course that means waiting for it to be approved and imported into the flatpaks distribution system too.

[nowrep]

You can check LD_PRELOAD value in the flatpak runtime (eg. run obs-gamecapture sh -c 'echo $LD_PRELOAD') and confirm that the path is correct. If not, set LD_PRELOAD to correct value and see if it fixes capture.

EDIT: I just checked and this seems to be correct. No idea why it doesn't work for you, maybe something with Steam sandboxing again.

[Plarpoon]

If there is anything I can do to test it with sandboxing and relying you the results let me know, I am more than available to test it