Ubuntu Frame OSK not showing up with Flutter / GTK apps

Hi,
Following these instruction in a running ubuntu-core 22 with a GTK snap installed, the keyboard fails to appear.

snap logs -f ubuntu-frame-osk displays:

2022-09-15T15:04:19Z ubuntu-frame-osk.daemon[1689]: Setting up watches.
2022-09-15T15:04:19Z ubuntu-frame-osk.daemon[1689]: Watches established.
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1689]: /run/user/0/ CREATE wayland-0.lock
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1463]: Setting input sources to [(‘xkb’, ‘us’)]
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1463]: Debug: Tried file “/root/snap/ubuntu-frame-osk/143/.local/share/squeekboard/keyboards/us.yaml”, but it’s missing: No such file or directory (os error 2)
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1463]: Info: Loaded layout Resource: us
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1463]: Debug: Tried file “/root/snap/ubuntu-frame-osk/143/.local/share/squeekboard/keyboards/us.yaml”, but it’s missing: No such file or directory (os error 2)
2022-09-15T15:04:21Z ubuntu-frame-osk.daemon[1463]: Info: Loaded layout Resource: us
2022-09-15T15:04:21Z -[1463]: Could not connect to session manager: Cannot autolaunch D-Bus without X11 $DISPLAY

2022-09-15T15:04:21Z -[1463]: Failed to init libfeedback: Cannot autolaunch D-Bus without X11 $DISPLAY

`App build using the following snapcraft.yaml

name: feenics-kiosk
version: 22.8.1
summary: Feenics visitor Management Kiosk
description: Feenics visitor Management Kiosk
confinement: strict
compression: lzo
grade: stable
base: core20

apps:
  feenics-kiosk:
    command-chain:
      - bin/wayland-launch
    command: bobcat
    daemon: simple
    restart-condition: always
    plugs:
      - network
      - cups-control
      - opengl
      - wayland
    environment:
      XDG_DATA_HOME: $SNAP_USER_DATA
      XDG_DATA_DIRS: $SNAP/usr/share

plugs:
  graphics-core20:
    interface: content
    target: $SNAP/graphics
    default-provider: mesa-core20

environment:
  LD_LIBRARY_PATH:    $SNAP/graphics/lib
  LIBGL_DRIVERS_PATH: $SNAP/graphics/dri
  LIBVA_DRIVERS_PATH: $SNAP/graphics/dri
  __EGL_VENDOR_LIBRARY_DIRS: $SNAP/graphics/glvnd/egl_vendor.d
  XDG_CACHE_HOME:  $SNAP_USER_COMMON/.cache
  XDG_CONFIG_HOME: $SNAP_USER_DATA/.config
  XDG_CONFIG_DIRS: $SNAP/etc/xdg
  XKB_CONFIG_ROOT: $SNAP/usr/share/X11/xkb

layout:
  /usr/share/libdrm:
    bind: $SNAP/graphics/libdrm
  /usr/share/drirc.d:
    bind: $SNAP/graphics/drirc.d
  /usr/share/fonts:
    bind: $SNAP/usr/share/fonts
  /usr/share/icons:
    bind: $SNAP/usr/share/icons
  /usr/share/sounds:
    bind: $SNAP/usr/share/sounds
  /etc/fonts:
    bind: $SNAP/etc/fonts
  /usr/lib/$SNAPCRAFT_ARCH_TRIPLET/gdk-pixbuf-2.0:
    bind: $SNAP/usr/lib/$SNAPCRAFT_ARCH_TRIPLET/gdk-pixbuf-2.0
  /usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/gtk-3.0:
    bind: $SNAP/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/gtk-3.0
  /usr/share/mime:
    bind: $SNAP/usr/share/mime
  /etc/gtk-3.0:
    bind: $SNAP/etc/gtk-3.0

parts:
  feenics-kiosk:
    source: ./build/linux/x64/release/bundle
    plugin: dump
    build-packages:
      - libgdk-pixbuf2.0-0
      - librsvg2-common
      - shared-mime-info
    override-build: |
      update-mime-database ${SNAPCRAFT_PART_INSTALL}/usr/share/mime
      snapcraftctl build
    filesets:
      include-dir: [ bobcat, data, lib, usr ]
    stage:
      - $include-dir
    stage-packages:
      - libgtk-3-0
      - libgl1
      - librsvg2-common
      - gsettings-desktop-schemas
      - libglib2.0-bin
      - libsecret-1-0
      - libsecret-1-dev
      - libjsoncpp-dev
      - libjsoncpp1
    override-prime: |
      snapcraftctl prime
      echo "$SNAPCRAFT_PRIME"
      /usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/glib-2.0/glib-compile-schemas "$SNAPCRAFT_PRIME/usr/share/glib-2.0/schemas"

      LOADERS_PATH=$(echo ${SNAPCRAFT_PRIME}/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/gdk-pixbuf-2.0/*/loaders)
      QUERY_LOADERS=/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/gdk-pixbuf-2.0/gdk-pixbuf-query-loaders
      GDK_PIXBUF_MODULEDIR=${LOADERS_PATH} ${QUERY_LOADERS} > ${LOADERS_PATH}/../loaders.cache
      sed s!$SNAPCRAFT_PRIME!!g --in-place ${LOADERS_PATH}/../loaders.cache

  wayland-launch:
    plugin: dump
    source: ./wayland-launch
    override-build: |
      PLUGS="opengl wayland graphics-core20"
      sed --in-place "s/%PLUGS%/$PLUGS/g" $SNAPCRAFT_PART_BUILD/bin/wayland-launch
      sed --in-place "s/%PLUGS%/$PLUGS/g" $SNAPCRAFT_PART_BUILD/bin/setup.sh
      snapcraftctl build
    stage-packages:
      - inotify-tools

  cleanup:
    after:
      - feenics-kiosk
      - wayland-launch
    plugin: nil
    build-snaps: [ mesa-core20 ]
    override-prime: |
      set -eux
      cd /snap/mesa-core20/current/egl/lib
      find . -type f,l -exec rm -f $SNAPCRAFT_PRIME/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/{} \;
      rm -fr "$SNAPCRAFT_PRIME/usr/lib/${SNAPCRAFT_ARCH_TRIPLET}/dri"
      for CRUFT in bug drirc.d glvnd libdrm lintian man; do
        rm -rf "$SNAPCRAFT_PRIME/usr/share/$CRUFT"
      done
      for CRUFT in alsa bash-completion dbus-1 help kconf_update kservices5 libinput locale openal pkgconfig \
                   applications doc kf5 kservicetypes5 libthai metainfo perl qlogging-categories5 thumbnailers xml \
                   apport config.kcfg doc-base GConf gnupg info knotifications5 kxmlgui5 libwacom themes sounds ; do
        rm -rf "$SNAPCRAFT_PRIME/usr/share/$CRUFT"
      done

architectures:
  - build-on: amd64
  - build-on: arm64

Any help appreciated, thanks.

Hi @feenicsdev,

The logs show nothing out of the ordinary, can you confirm that the OSK comes up when using e.g. wpe-webkit-mir-kiosk with an input field (e.g. snap set wpe-webkit-mir-kiosk url=https://duckduckgo.com)?

Note it’s the client application that needs to use the appropriate Wayland protocol extension (text-input). Mir supports v1…v3, covering all the toolkits we knew to try.

Can you give more info on your application? What toolkit it’s using?

You can see the wayland traffic between the app and the compositor by running it under WAYLAND_DEBUG=1.

1 Like

Hi Saviq,

Thanks for your help.
I can confirm the keyboard does work with wpe-webkit-mir-kiosk.
The app is a gtk application. It is a flutter app built using the flutter toolkit via flutter build linux. This is turned into a snap via the snapcraft dump plugin. Would the snapcraft.yaml file be usefull here?

I set the WAYLAND_DEBUG=1 environment variable, but how do I see the output? Ubuntu core doesn’t seem to have a syslog and redirect standard error gets me permissions denied, even when run with sudo.

it uses journald, just use the journalctl command to read the system log

2 Likes

@feenicsdev I’ll get back to you on this, I can confirm from e.g. this:

The OSK isn’t showing up for the Flutter app (or for other GTK apps, for that matter) - but is for e.g. WPE. It does mean that something isn’t right between the app and the compositor.

1 Like

@feenicsdev actually having tried again it does work fine. Can you try this snap, built from the above documentation?

https://people.canonical.com/~msawicz/iot-example-graphical-snap_0+git.2ee561d_amd64.snap

1 Like

The example app works as expected, thank you.

I have reconfigufed my snapcraft.yaml to use:

build-snaps:
  - flutter/latest/stable

and provided required stanzas for building the flutter app with snapcraft. The keyboard now works!

In both cases however, on first boot, ubuntu-frame crashes. when the app is manually restarted everything is fine. Looking at this now.

Thanks again.

1 Like