Idea: Protocol for server-side window decorations in a separate process

[probonopd]

I find it really annoying that in the Wayland world, the code that generates server-side window decorations (a.k.a. window title bars) is baked into the "display server" itself (Wayland compositor). This makes every compositor depend on a certain GUI toolkit or at least font rendering engine, and in turn makes Wayland compositors unsuitable to be used across desktops with various different GUI toolkits.

To be specific, I'd like to use, e.g., wayfire or wlmaker, but I don't like it's dependencies on parts of the Gnome stack, such as Cairo, Pango and PangoCairo, which are essentially the GTK rendering stack (Pango depends on GLib which is GTK).

If I understand it correctly, those dependencies are there just for rendering the server-side window decorations (a.k.a. window title bars). So, if there was a protocol that would let the Wayland compositor reach out to a small toolkit-specific helper binary to get the window decorations, then the compositor itself would not need to depend on any particular GUI toolkit or font rendering engine.

So, the compositor would need to be able to request window decorations (title bars) from a separate binary (let's call it the " "decorations renderer") by sending

• The title of the window
• Whether it is active or inactive
• Which buttons it has, and in which position

Something along these lines.

Maybe something similar for scroll bars.

Advantages:

• The same Wayland compositor could be used in Qt, GNUstep, Fyre,.. based desktop environments and would always have native look and feel (and no dependencies on an "alien" GUI toolkit)
• It would be easy to write custom styles by just writing a small "decorations renderer"

What do you think?

Edit: Shared libraries (.so) might actually work better for this, so possibly no real need for a portal (but clearly defined APIs).

[myownfriend]

What is the compositor requesting the binary from and what happens if no binary is provided?

Wayland is for communication between the compositor and clients and the only reason I can imagine the compositor requesting this binary from the client is if the compositor wants to draw SSDs that match the toolkit of the client. Also the only reason I can imagine the client provides this binary is because it also wants to be drawn with decorations that match it's toolkit. If that's the case then the client might use CSDs. If it doesn't want to write the decoration itself then it can use libdecor and toolkit specific plugin for it.

You don't need a Wayland protocol to do what you want to do. If the compositor is determined to use SSDs but doesn't want to be dependent on a specific toolkit to draw them, it could make a hard requirement that the shell provides this binary and the compositor will draw it. The shell is made for the compositor so the requirements of this binary and how it and the compositor communicate with each other can be completely up to the compositor.

[i509VCB]

While I agree with this idea in principle, I feel that the implementation is going to be a bit more complicated. My experience with this idea is from my attempt to write a Wayland compositor that used a WM process and it got quite complicated so I just moved to making the WM programmable inside my compositor.

So in order to make this work under Wayland's "every frame is perfect" mantra, you will need some synchronization between the client and decoration client. Transactions are a great way to implement this on the server, but it does require some extra internal state.

Let's consider the normal way you'd create a window:

1. Create an xdg_toplevel
2. Send an initial commit
3. Wait for a configure
4. Ack the configure
5. Commit

Within that process, the configure will dictate a few things: Is the proposed size of the toplevel flexible or must the size be followed? Some of the xdg_toplevel.state enum add additional (or remove) constraints for the size. For example a tiler could just set the maximized state and bob's your uncle.

However not every app or user wants that. If you have a terminal which uses cell resizing and you didn't set maximized, your terminal might only actually change it's size every time the width and height of the buffer changes by once character (which could be something like 20x20 pixels on a high DPI display). In this scenario without the maximized state, you might not know the true size of the window until the commit (after the configure). In the most ideal case clients send the ack and then start rendering immediately and at worse you'd have to wait for the frame callback that comes after the ack for the client to start rendering.

So then for the case where state does not contain maximized you would have psuedocode like this:

toplevel.configure(state).await
// Since you don't know the size until commit, this is a hard barrier for configuring the decoration client.
toplevel.wait_for_ack_and_commit().await 
decoration_client.configure(state).await
// Another hard barrier, the decoration client might have rendering work
decoration_client.wait_for_ack_and_commit().await
// We can only update the toplevel's texture after the toplevel and decoration client finish
toplevel.update_texture()

In the case that state does contain maximized, we can send both configures at the same time and apply the texture update after both acks and commits occur.

Furthermore there are things we need to consider: what if the decoration client hangs? Do we just deadlock or do we try to respawn the client after a failure to respond after some time.

[ssokolow]

What is the compositor requesting the binary from and what happens if no binary is provided?

Wayland is for communication between the compositor and clients and the only reason I can imagine the compositor requesting this binary from the client is if the compositor wants to draw SSDs that match the toolkit of the client. Also the only reason I can imagine the client provides this binary is because it also wants to be drawn with decorations that match it's toolkit. If that's the case then the client might use CSDs. If it doesn't want to write the decoration itself then it can use libdecor and toolkit specific plugin for it.

You don't need a Wayland protocol to do what you want to do. If the compositor is determined to use SSDs but doesn't want to be dependent on a specific toolkit to draw them, it could make a hard requirement that the shell provides this binary and the compositor will draw it. The shell is made for the compositor so the requirements of this binary and how it and the compositor communicate with each other can be completely up to the compositor.

They're basically looking to re-introduce the XServer/WM split. However, given my understanding of their motivations (having a single standard implementation which isn't tied to a specific toolkit), I imagine the WM "binary" could also be a .so file, like how Weston shells are implemented.

Think "the separation between the xdg-desktop-portal backend and frontends like xdg-desktop-portal-kde but for the compositor's SSD support".

In fact, if I understand correctly, they're more or less literally asking for what Weston does using .so files, but without developer neglect making it not suitable for purpose.

(However, I could be wrong. This is probonopd and so "WMs distributable as Appimages" might also be a strong requirement in which case it would have to be an out-of-process thing.)

[probonopd]

They're basically looking to re-introduce the XServer/WM split.

Correct. I am coming from the idea that it is a gigantic waste to have different compositors (with different features and bugs each) for every UI toolkit. So the idea is to make the compositor UI toolkit (and look and feel) agnostic, just like the X server.

Shared libraries (.so) might actually work for this, so possibly no real need for a portal (but clearly defined APIs). (If someone really wanted to ship a compositor as an AppImage, they might just bundle that library.)

[i509VCB]

They're basically looking to re-introduce the XServer/WM split.

Correct. I am coming from the idea that it is a gigantic waste to have different compositors (with different features and bugs each) for every UI toolkit. So the idea is to make the compositor UI toolkit (and look and feel) agnostic, just like the X server.

Shared libraries (.so) might actually work for this, so possibly no real need for a portal (but clearly defined APIs). (If someone really wanted to ship a compositor as an AppImage, they might just bundle that library.)

I get a feeling you'll struggle to get compositors to agree on a specific kind of API for that job.

I kind of have something similar (very WIP and unusable right now) that kind of follows the idea of using a library to implement a wm. It's not .so libraries, but rather the wm can inject it's runtime for window management: https://codeberg.org/i509vcb/aerugo/src/branch/master/examples/mini-wm/src/lib.rs

Mir has a similar idea with window management policies.

There is also the incredibly cursed idea I've come to calling "waylandx" which uses xwayland to provide both x11 client support and window management for Wayland clients.

[markusbkk]

There is also the incredibly cursed idea I've come to calling "waylandx" which uses xwayland to provide both x11 client support and window management for Wayland clients.

Out of morbid curiosity, is there any code implementing this or is just a "brainfart" at this stage?

[i509VCB]

There is also the incredibly cursed idea I've come to calling "waylandx" which uses xwayland to provide both x11 client support and window management for Wayland clients.

Out of morbid curiosity, is there any code implementing this or is just a "brainfart" at this stage?

I have not written anything related to the idea.

[markusbkk]

There is also the incredibly cursed idea I've come to calling "waylandx" which uses xwayland to provide both x11 client support and window management for Wayland clients.

Out of morbid curiosity, is there any code implementing this or is just a "brainfart" at this stage?

I have not written anything related to the idea.

Too bad. Would've been an interesting thought experiment (that might have some use-cases).

I kind of have something similar (very WIP and unusable right now) that kind of follows the idea of using a library to implement a wm. It's not .so libraries, but rather the wm can inject it's runtime for window management: https://codeberg.org/i509vcb/aerugo/src/branch/master/examples/mini-wm/src/lib.rs

This is nice though. Kinda looks like the "meta compositor" I discussed in #6.
I think this is the right approach, since there's very little chance any of the compat protocols discussed in this repository will ever be accepted upstream.