GTK apps on Linux can use either X11 or Wayland, depending on what type of desktop environment they’re running on. At first this may seem simple, but a number of factors complicate matters. This post will explain how GTK backend selection works, and how it can be controlled by desktop developers, application developers and users.
Why it’s complicated
Many (but not all) Wayland desktops support X11 apps via XWayland. As a result, apps which support both backends can choose which they connect to. By default GTK apps run natively on Wayland, or X11 if Wayland’s not available. However, some apps only support one platform or the other. Some apps support both platforms, but work better on one.
App developers can set which backends they support and in what order GTK will try them using gdk_set_allowed_backends(). The first backend in the comma-separated list is tried first. If, for example, allowed backends is set to
“x11,wayland” then either can be used but it will run on X11 instead of Wayland if both are available (opposite the default).
GDK_BACKEND environment variable
I mentioned the environment variable
GDK_BACKEND before in a previous post. Note it’s GDK_BACKEND, not GTK_BACKEND (GDK is the subsystem of GTK that handles windowing).
It’s a way to filter the set of supported backends and override the order. That is, you can not add support for a backend that was omitted from a call to
gdk_set_allowed_backends(), but you can stop GTK using one that is supported by the app. You can also change the priority order in which supported backends are tried.
Example: a bug in Mir shells
Most GTK apps work great in Mir-based shells, but we recently noticed Chrome and Electron apps (which use GTK under-the-hood) along with all X11-only GTK3 apps were failing with a similar error:
(chrome:2683326): Gtk-WARNING **: 13:54:39.151: cannot open display: :0
The message isn’t particularly helpful (it implies GTK is trying and failing to open the X11 display :0, which is not actually the case). The cause turned out to be that we were setting
GDK_BACKEND=wayland in Mir startup. This was originally introduced before GTK used Wayland by default and before Mir supported XWayland.
This forced all GTK3 apps to use Wayland, which is a problem for apps that explicitly don’t support it. These apps were left with zero allowed backends to try and thus failed to start.
How we fixed it
We considered not setting
GDK_BACKEND (which is what GNOME does). This would let X11-only GTK3 apps work and allow most GTK3 apps to run on Wayland as usual, however there was a small problem with that. Some apps (such as Firefox) support Wayland, but choose X11 by default. These apps may indeed work better on XWayland than native Wayland on some desktops, but Mir’s XWayland support is still comparatively underdeveloped. We prefer to run all apps natively on Wayland if possible.
What we settled on was setting
wayland,x11. This results in all apps that support both platforms using Wayland, but apps that support only X11 still working.