A better way to debug Wayland

Mir
1

If you’ve never tried to debug or figure out a Wayland program, you might not get much out this. If you do hack on Wayland, (especially if you work with custom protocols or compositors) keep reading.

Terminology

Feel free to skip

  • Object: More of a concept than a thing-with-memory-address. Wayland objects are often represented by the programming language’s version of an object/struct in both the client and the server. If you’re using libwayland in C or C++, that means a wl_proxy in a client and a wl_resource in a server. Wayland objects are for messaging only, they don’t hold properties of any sort. An object is identified by a numeric ID. At any given time, an ID is unique to a connection. Object IDs are not unique in a program (multiple clients connected to the same server can have objects with the same IDs) or unique over the lifespan of a client (an object can be destroyed, and a new object with the same ID can then be created).
  • Message: Every message is called on exactly one object. Messages sent by a client to a server are called requests, and messages sent from a server to a client are called events. Messages can have any number of arguments with various types (ints, strings, Wayland objects, etc). An argument of type new_id allows a request to create a new object.
  • Protocol/Protocol Extension: Wayland protocols are defined in XML files. The core Wayland protocol is in wayland.xml, and additional protocol extensions can be used. Protocols define object types, and what messages can be called on them. Protocol files are often used to generate typesafe language bindings.

Debugging basics

To solve Wayland problems, you first have to figure out what messages are being sent. You might already know about the WAYLAND_DEBUG environment variable. If you run a program with it set to client, server or just 1 for both, it will dump every message it sends or receives to stderr. It looks something like this:

Output from libwayland
Output from libwayland1908×1936 573 KB

That output can be enormously useful, but there are still some issues:

  • There are two many messages, and grepping for the ones you want with long regexes can be cumbersome
  • To keep track of specific objects you have to redirect to a file, check which object IDs your interested in and search for them
  • It is very hard to keep track of which objects are which when IDs are reused
  • It is impossible to know why a message was sent
  • When debugging a server, multiple clients become an indistinguishable jumble
  • It is ugly and difficult to pick out what you’re looking for

All of these problems are fixed in a new Wayland debug tool I’ve built

Wayland Debug Features

First of all, it looks a heck of a lot nicer:

Output from Wayland Debug
Output from Wayland Debug1770×1937 640 KB

But aesthetics is just the tip of the iceberg. Wayland Debug has three modes. In the first two mode, it simply reads the WAYLAND_DEBUG=1 output above (either piped in, or from a file), and does additional processing. The advantages you get over the raw stream are:

  • Syntax highlighting
  • DSL for filtering Wayland messages: Want to know when any XDG object has a configure message, and when any object has a commit message? -f 'xdg_*[configure],[commit]' will do that.
  • Generation awareness: In addition to the type and object ID, each object also has a “generation”. This distinguishes objects on the same connection with the same ID (the first to have the ID is .0, the 2nd is .1, etc.)
  • Smart delete_id: Instead of just showing an int and making you dig through the log to find what ID was deleted, Wayland Debug shows the full object (type, ID and generation)

The 2nd mode is a lot more exciting. In that, it runs your program in GDB, with itself loaded as a plugin. To use Wayland Debug this way, you do need to have libwayland debug symbols installed. This mode has all the features of the former, plus additional ones:

  • Multiple connections: handled properly
  • Commands while execution is paused (after Ctrl+C, hitting a breakpoint, etc)
    • Adjust object filter (this allows for only showing a specific object instance, because you can only know its ID once it’s been created)
    • Put all messages so far through a different filter
    • Switch between connections

And my favorite one of all:

  • Breakpoints on Wayland messages: You can specify a matcher to break on, and when a matching message is processed GDB will stop just like a normal breakpoint. The best part is you get a full stack trace showing why that message was sent, and can do all the normal GDB things.

Here, I instruct Wayland debug to break when a specific wl_pointer sends a motion event, and I inspect the stack trace that lead to that event being sent:

Stack trace from Wayland breakpoint
Stack trace from Wayland breakpoint3753×1986 660 KB

Getting and Using

If you’re interested, just clone the repo from git@github.com:wmww/wayland-debug.git. It’s pure Python, so no building necessary. Just run ./main.py -h. I symlink it into /usr/bin/wldbg with sudo ln -s $PWD/main.py /usr/bin/wldbg to make it easy to access, but you can put it wherever you like.

The readme should contain all the info you need to use it. If something is unclear to you, please open an issue or comment on this post. Same goes if you come across a bug or crash.

7 Likes