Wayland support for vsg

[VasanthBalguri]

Hi All,

I am working on wayland support for vsg, I would like to share my progress:

I added my commits over 1.1.0 release as it seems stable...
⚠️ Limitations: (Help appreciated 🙏)

1. It uses server side decorations for now, so it will work on those DEs which support it (KDE,sway,... but not Gnome)
2. The muti-window example failed to work
3. Qt examples are not working as expected

👇 How to get it:

• Clone my fork
  • git clone https://github.com/VasanthBalguri/VulkanSceneGraph
• Checkout wayland-testing branch
  • git checkout wayland-testing
• Run following commands to build
  • mkdir build
  • cd build
  • cmake .. -DBUILD_WAYLAND=1
  • make
• Let me know if there is a way to distribute it as compiled library 🙏

I am open for any feedback or queries 😄

[VasanthBalguri]

Prequesites

• xkbcommon and wayland development libraries must be installed
  • sudo apt install libxkbcommon-dev libwayland-dev

[robertosfield]

Thanks for posting your work on Wayland support. I just got back from a day our so only done a quick code review. If I have an opportunity tomorrow I will try it out on Kubuntu 22.04 desktop.

After my first once through questions/suggestion that spring to mind are:

1. Longer term perhaps we need an option for users to compile against Wayland and/or Xcb and have the VSG have a mechanism for choosing which implementation they want at runtime. Right now though just having a compile option will be fine.
2. The button mask being cached in Wayland_Window will probably need to be done via a thread safe singleton so that when users have multiple windows the button state isn't lost, or better still if there is a way of asking the window system the pointer state.
3. For Qt use we might not need the input handling, perhaps just Wayland integration would be required, if so then perhaps whether input events need handling could be toggled at compile/runtime.

What were the Qt problems? The way vsgQt works is to use Qt to create the window but defer the vkInstance and vkSurface creation to the VSG's vsg::Window/Surface classes, with the vsg::Window subclass taking the Qt native window id from the WindowTraits. The Win32_Window.cpp and Xcb_Window.cpp both do this so you can look them to see what they do.

[VasanthBalguri]

1. Longer term perhaps we need an option for users to compile against Wayland and/or Xcb and have the VSG have a mechanism for choosing which implementation they want at runtime. Right now though just having a compile option will be fine.

I will look into it, also if the application can intelligently choose its implementation based on the target machine will be good(like if it has wayland and supports our required extensions), although it will require some more effort from our end.

2. The button mask being cached in Wayland_Window will probably need to be done via a thread safe singleton so that when users have multiple windows the button state isn't lost, or better still if there is a way of asking the window system the pointer state.

Unlike X11, in wayland window doesn't have access to global state of input devices.
When window gets into focus the emitted events go directly to that window,so things like global key state, drag and drop, placing window at given location on screen require dedicated extensions which are universally not supported.This makes multi window application tricky.

Hence, having a separate key state will not be of an issue for now, as every window receives it own set of inputs.
Unfortunately multi-window example doesn't work. I need to look into it...

I am thinking of using KDE/QT as a base for this implementation, it has better extensions.

As of now I am facing following issue with QT:

I will need to dig into QT source to find the root cause...

[VasanthBalguri]

3. For Qt use we might not need the input handling, perhaps just Wayland integration would be required, if so then perhaps whether input events need handling could be toggled at compile/runtime

I didn't have time to dig into Qt source code, seems like I need to refactor my source to handle this scenario...
Also the 2. point makes sense now for QT scenario, I will look into it. 😅

[VasanthBalguri]

This is off-topic..

Is there any reason we are packaging the vsg in a single library?
splitting the library will be helpful I suppose, like having a dedicated library built for each platform, instead of configuring at compile time:
libvsg -> libvsgwindow -> binary
Here if we can swap the platform library(if shared library then it is possible at runtime I suppose)
Correct me if I am wrong 🙏

[robertosfield]

glfw has a Wayland implementation. glfw is what I used as my once of reference for learning how to do Xcb/Vulkan integration.

W.r.t multiple libraries vs a single library, I have tried to keep things simple for developers with the ability to just link to single library if all they need is basic 3d rendering and loading of native file formats. There are pros and cons for doing single and multiple libraries, supporting multiple windowing backends at the same time makes multiple dynamic libraries preferable but so far this possibility hasn't existed for the VSG - we've only had one implementation per platform.

The input device issue does sound problematic for Wayland, I can't believe the situation is so bad. I was expecting modern well handled solutions rather than a regression on decades old X11's capabilities.

[VasanthBalguri]

Wayland is a mess unfortunately, there is de-facto implementation like wl-roots, but the default DE is gnome which has very restrictive set of extensions. It seems like developing for a DE is equivalent to developing for an OS 😞 which is very demotivating. The situation might get better over time though.

To give an example,even glfw had to use another library called libdecor to handle window decorations, which seems ridiculous..
glfw/glfw#2212

Right now, focusing our efforts on KDE/QT+Wayland is ideal I think. I will take a look at glfw and implement 1 to 1 for vsg.

[robertosfield]

Thanks for the link. Seems a sorry state of affairs that Wayland has been pushed as being ready for prime time long before it is.

If you come up with helpful links to KDE/Qt+Wayland discussions/code please post them here. I expect others will be interested in the topic of Wayland support.

I don't know if this OpenSceneGraph PR will help, it's a bit old now. Unfortunately I've been so swamped with VSG related work that I haven't had a chance to review this PR, let along merge it, so I can't say whether it will help with aspects but it might provide another perspective:

openscenegraph/OpenSceneGraph#1170

[VasanthBalguri]

Thank you for providing this, I will use this as a reference.

[robertosfield]

I don't now if this might be workable, but perhaps we could dynamically load native windowing libraries at runtime and get the function names to use rather than compiling directly against the library. This approach would provide the flexibility of not needing to directly link the VSG to a specific native windowing library.

This approach is essentially what we do with the vsg::InstanceExtensions and vsg::DeviceExtensions classes for handling Vulkan extensions. However, it's more complicated and more work to implement and use than just using a libraries functions directly.

[robertosfield]

To help wider testing I have merged @VasanthBalguri's wayland-testing branch into the main VulkanSceneGraph repo as the branch:

https://github.com/vsg-dev/VulkanSceneGraph/tree/VasanthBalguri-wayland-testing

You'll need to set the CMake var BUILD_WAYLAND to 1 as per the instructions in the first post in this thread.

I have now a build of the VSG, vsgXchange etc. & vsgExamples and everything builds, though there are many warnings on the VSG build that we'll need to address in time, no worries there. When I run on vsgviewer on my system I do see an exception through about lack of wayland support:

$ vsgviewer models/lz.vsgt
vsgviewer models/lz.vsgt
[Exception] - failed to create Wayland connection result = 0

I am using Kubuntu 22.04 right now so I'll need to look into whether it's possible to enable a Wayland back-end. I'm a newbie on Wayland so happy to stumble through and learn as I go.

--

On the topic of references for Wayland support, perhaps SDL might also be another reference we can look at for inspiration.

Another thought, I believe Wayland is common on automotive infotainment systems, the OSG found a home in this market, the VSG is now a much more natural fit than OSG as it's smaller, faster and more modern, so Wayland support might help those considering the VSG in this embedded market. The question I would raise would be about what happens with event handling on embedded Wayland platforms, I wouldn't be surprised if libxkbcommon isn't a thing, though.... I haven't kept up with recent automotive platforms so this hunch might well be off. If it's right though we might want to make it an optional code path.

[robertosfield]

I have now installed the wayland-plasma and selected this on log-in but when I run vsgviewer I see a new error:

 vsgviewer models/lz.vsgt
Authorization required, but no authorization protocol specified

[robertosfield]

I am now testing the wayland-testing branch on my Ubuntu 22.04 laptop and found I had to install wayland-protocols in addition to libxkbcommon-dev and libwayland-dev, so the install dependencies line is:

sudo apt install libxkbcommon-dev libwayland-dev wayland-protocols

Everything compiles OK, but still haven't got things working cleanly when I run vsgviewer I get a seg fault:

#0 0x00007ffff7f23e34 in wl_proxy_get_version () from /lib/x86_64-linux-gnu/libwayland-client.so.0
#1 0x00005555557d6cbc in vsgWayland::Wayland_Window::Wayland_Window(vsg::ref_ptrvsg::WindowTraits) ()
#2 0x00005555557d6e55 in vsg::Window::create(vsg::ref_ptrvsg::WindowTraits) ()
#3 0x0000555555636b32 in main ()

Will need to check the Wayland status of Ubuntu 22.04

[VasanthBalguri]

Any specific line number where it is segfaulting?

[robertosfield]

Just rebuilt with debug:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f23e34 in wl_proxy_get_version () from /lib/x86_64-linux-gnu/libwayland-client.so.0
(gdb) where
#0  0x00007ffff7f23e34 in wl_proxy_get_version () from /lib/x86_64-linux-gnu/libwayland-client.so.0
#1  0x0000555555945c94 in zxdg_decoration_manager_v1_get_toplevel_decoration (zxdg_decoration_manager_v1=0x0, 
    toplevel=0x55555622aca0) at /home/robert/Dev/VulkanSceneGraph/include/vsg/platform/wayland/xdg-decoration-client.h:197
#2  0x0000555555947196 in vsgWayland::Wayland_Window::Wayland_Window (this=0x7fffee6a5ac0, traits=...)
    at /home/robert/Dev/VulkanSceneGraph/src/vsg/platform/wayland/Wayland_Window.cpp:334
#3  0x0000555555947e55 in vsg::Inherit<vsg::Window, vsgWayland::Wayland_Window>::create<vsg::ref_ptr<vsg::WindowTraits>&> ()
    at /home/robert/Dev/VulkanSceneGraph/include/vsg/core/Inherit.h:37
#4  0x0000555555945d96 in vsg::Window::create (traits=...)
    at /home/robert/Dev/VulkanSceneGraph/src/vsg/platform/wayland/Wayland_Window.cpp:22
#5  0x00005555556aa727 in main (argc=2, argv=0x7fffffffdb98)
    at /home/robert/Dev/vsgExamples/examples/app/vsgviewer/vsgviewer.cpp:190

[VasanthBalguri]

Pls add following line here and share the results

void Wayland_Window::registry_add_object(void *data, struct wl_registry *registry, uint32_t id, const 
   char *interface, uint32_t version) {
Wayland_Window *window = static_cast<Wayland_Window*>(data);
 + std::cout<<" interface:"<<interface<<" id:"<<id<<" version:"<<version<<std::endl
    if (strcmp(interface,"wl_compositor") == 0) {

I want ti see what all interfaces are available in your system

[robertosfield]

I'm working under Windows today so can't look at testing this right away,

[VasanthBalguri]

Hi Robert, Seems like there are many issues. I was able to get mine up and running, but seems like I didn`t provide enough info. I run following system: Kubuntu 22.04 Kde plasma 5 I am away and I dont have access to my computer will be back on March 11 to verify. I built mine on version 1.1.0 as I got compilation errors on latest version. I will try to setup a VM and verify different distros once I am back home Regards, Vasanth

…

On Sun, 25 Feb, 2024, 10:44 pm Robert Osfield, ***@***.***> wrote: I am now testing the wayland-testing branch on my Ubuntu 22.04 laptop and found I had to install wayland-protocols in addition to libxkbcommon-dev and libwayland-dev, so the install dependencies line is: sudo apt install libxkbcommon-dev libwayland-dev wayland-protocols Everything compiles OK, but still haven't got things working cleanly when I run vsgviewer I get a seg fault: #0 0x00007ffff7f23e34 in wl_proxy_get_version () from /lib/x86_64-linux-gnu/libwayland-client.so.0 #1 <#1> 0x00005555557d6cbc in vsgWayland::Wayland_Window::Wayland_Window(vsg::ref_ptrvsg::WindowTraits) () #2 <#2> 0x00005555557d6e55 in vsg::Window::create(vsg::ref_ptrvsg::WindowTraits) () #3 <#3> 0x0000555555636b32 in main () Will need to check the Wayland status of Ubuntu 22.04 — Reply to this email directly, view it on GitHub <#1099 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6DBOIQFLWHK25QZDS7PC3YVNWORAVCNFSM6AAAAABDX4JUAOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKOBUGMYTE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[VasanthBalguri]

Hi Robert, Yes the xkb is exclusive to linux desktop, although automotive uses linux kernel, it is not garuntee to be present there. I was initially in the same dillema, but got over it as I had no other option.As wayland has no standard for keboard mapping and it is upto the compositor to decide it. My compositor sends the raw values stored in linux/input.h header file, so it is common on all linuxs I suppose. Regards, Vasanth

…

On Sun, 25 Feb, 2024, 10:00 pm Robert Osfield, ***@***.***> wrote: To help wider testing I have merged @VasanthBalguri <https://github.com/VasanthBalguri>'s wayland-testing branch into the main VulkanSceneGraph repo as the branch: https://github.com/vsg-dev/VulkanSceneGraph/tree/VasanthBalguri-wayland-testing You'll need to set the CMake var BUILD_WAYLAND to 1 as per the instructions in the first post in this thread. I have now a build of the VSG, vsgXchange etc. & vsgExamples and everything builds, though there are many warnings on the VSG build that we'll need to address in time, no worries there. When I run on vsgviewer on my system I do see an exception through about lack of wayland support: $ vsgviewer models/lz.vsgt vsgviewer models/lz.vsgt [Exception] - failed to create Wayland connection result = 0 I am using Kubuntu 22.04 right now so I'll need to look into whether it's possible to enable a Wayland back-end. I'm a newbie on Wayland so happy to stumble through and learn as I go. -- On the topic of references for Wayland support, perhaps SDL might also be another reference we can look at for inspiration. Another thought, I believe Wayland is common on automotive infotainment systems, the OSG found a home in this market, the VSG is now a much more natural fit than OSG as it's smaller, faster and more modern, so Wayland support might help those considering the VSG in this embedded market. The question I would raise would be about what happens with event handling on embedded Wayland platforms, I wouldn't be surprised if libxkbcommon isn't a thing, though.... I haven't kept up with recent automotive platforms so this hunch might well be off. If it's right though we might want to make it an optional code path. — Reply to this email directly, view it on GitHub <#1099 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6DBOJLEOCYQ76Z6LEGRITYVNRKNAVCNFSM6AAAAABDX4JUAOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKOBUGA2TE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[VasanthBalguri]

Hi Robert,

Sorry for delay, I was busy with personal and professional work.

I will try merge 1.1.4 release changes and add logging facility to this branch soon enough.
Currently I am working on other projects, so I cannot commit on when I can deliver it.

feel free to message me regarding anything.