Easy multi-user LXD setup

One nifty new feature of the newly released LXD 4.22 is the ability for regular users to safely interact with LXD.

Until now, LXD has suffered from much the same issue as the likes of Docker and Libvirt in indirectly granting full root access to anyone who’s allowed to interact with it. This was possible through a number of different options (device passthrough, privileged container, custom ID maps, …). While not a problem on a developer’s laptop, it’s a no-go for most shared environments, especially in a corporate setting.

Over the years, LXD has grown support for proper remote authentication, fancy access control (RBAC) and projects as ways to restrict specific users to a subset of LXD’s features so multiple people can safely share a LXD server or cluster.

With LXD 4.22, this is now all doable locally with normal local users making it a great fit for desktop systems, especially multi-users desktop in a corporate environment.

Here is an overview of the feature from our release live stream:
https://www.youtube.com/watch?v=Blx7cdygiS8&t=848s

For those who aren’t into video watching, the instructions basically are:

snap install lxd
snap set lxd daemon.user.group=users

With that done, any user in the users group will be allowed to interact with LXD despite not being in the all-powerful lxd group.

The first time one such user interacts with LXD, they will automatically get their own restricted project which will look like this:

foo@v1:~$ lxc project list
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |               DESCRIPTION                | USED BY |
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+
| user-1001 (current) | YES    | YES      | YES             | NO       | User restricted project for "foo" (1001) | 3       |
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+

foo@v1:~$ lxc project show user-1001
config:
  features.images: "true"
  features.networks: "false"
  features.profiles: "true"
  features.storage.volumes: "true"
  restricted: "true"
  restricted.containers.nesting: allow
  restricted.devices.disk: allow
  restricted.devices.disk.paths: /home/foo
  restricted.devices.gpu: allow
  restricted.idmap.gid: "1003"
  restricted.idmap.uid: "1001"
description: User restricted project for "foo" (1001)
name: user-1001
used_by:
- /1.0/instances/welcome-earwig?project=user-1001
- /1.0/images/ced57a80f2b761c3cdab867c2296b801c6adfe521f811bacdd61410da4bc2734?project=user-1001
- /1.0/profiles/default?project=user-1001

Which in practice allows the user to just do:

foo@v1:~$ lxc launch ubuntu:20.04
Creating the instance
Instance name is: welcome-earwig        
Starting welcome-earwig

foo@v1:~$ lxc list
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+
|      NAME      |  STATE  |        IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+
| welcome-earwig | RUNNING | 10.31.36.109 (eth0) | fd42:fa4a:d38d:1c7b:216:3eff:fed1:63aa (eth0) | CONTAINER | 0         |
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+

And get a container, or virtual machine, running immediately, with no configuration and without ever having needed any kind of elevated privileges.

Should that user try to create a privileged container, pass in paths outside of their home directory or do any device passthrough other than GPUs, it will be rejected by LXD.

Login as another user on the system and you’ll get the exact same behavior, every user gets their own personal project and can’t see the others. Well, unless they are part of the lxd group, then they can see everything going on on the system.

We’re hoping that this feature can be used to provide experiences similar to that of WSL on Windows or Crostini on ChromeOS where getting containers going is just a few clicks away and the main system remains nice and safe.

Enjoy!

3 Likes

This is awesome :slight_smile: Is there an “easy” way to migrate an existing LXD install to this new less-privileged scenario?

Currently my user is in the lxd group and I have a bunch of (unprivileged) containers and VMs running under the default project - can I easily export my containers / VMs, remove myself from the lxd group and then set those back up under a restricted personal project for my user?

This is possible if slightly more involved :slight_smile:

I’ve tried it here, starting with:

ubuntu@u2004-desktop:~$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),44(video),100(users),998(lxd)
ubuntu@u2004-desktop:~$ lxc project list
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
|       NAME        | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |     DESCRIPTION     | USED BY |
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
| default (current) | YES    | YES      | YES             | YES      | Default LXD project | 5       |
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
ubuntu@u2004-desktop:~$ lxc list
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
ubuntu@u2004-desktop:~$ 

I’ve then removed my user from the lxd group and logged in again:

ubuntu@u2004-desktop:~$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),44(video),100(users)
ubuntu@u2004-desktop:~$ lxc list
Error: not authorized
ubuntu@u2004-desktop:~$ lxc project list
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
|   NAME    | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |                 DESCRIPTION                 | USED BY |
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
| user-1000 | YES    | YES      | YES             | NO       | User restricted project for "ubuntu" (1000) | 1       |
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
ubuntu@u2004-desktop:~$ lxc project switch user-1000
ubuntu@u2004-desktop:~$ lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
ubuntu@u2004-desktop:~$ 

So the user is now restricted to an empty project. Next we need to move the instances from a privileged user:

root@u2004-desktop:~# lxc list
To start your first container, try: lxc launch ubuntu:20.04
Or for a virtual machine: lxc launch ubuntu:20.04 --vm

+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
root@u2004-desktop:~# lxc stop --all
root@u2004-desktop:~# lxc move a1 --target-project user-1000
root@u2004-desktop:~# lxc move a2 --target-project user-1000
root@u2004-desktop:~# lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

And after that, starting everything back as the user:

ubuntu@u2004-desktop:~$ lxc list
+------+---------+------+------+-----------+-----------+
| NAME |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+------+------+-----------+-----------+
| a1   | STOPPED |      |      | CONTAINER | 0         |
+------+---------+------+------+-----------+-----------+
| a2   | STOPPED |      |      | CONTAINER | 0         |
+------+---------+------+------+-----------+-----------+
ubuntu@u2004-desktop:~$ lxc start --all
ubuntu@u2004-desktop:~$ lxc list
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
ubuntu@u2004-desktop:~$ 

Note that the lxc move step requires that any project or custom volume you need exist in the target project, so if you had anything fancy on that front, you’ll want to re-create your profiles and move any custom storage volumes ahead of time.

If your instances use features which aren’t allowed under a restricted project, the lxc move will fail with Forbidden, normally telling you what’s being rejected. You can either modify the instance to comply with the restrictions or use lxc project edit user-1000 to alter the restrictions to allow what you need.

Thanks @stgraber - that worked like a charm :slight_smile:

For posterity, it was as simple as:

# remove my user from the lxd group
sudo gpasswd -d $USER lxd

# reboot to cleanly logout and back in again without being in
# the lxd group anymore

# set the lxd user group to just my user's primary group since 
# there is no other  users on my local machine
sudo snap set lxd daemon.user.group=$USER

# lxc will have created a new project called user-$UID for your user
lxc project list
# set this as the default project
lxc project switch user-$UID

# then migrate instances to this user's project by stopping them all
# and then moving them across
sudo lxc stop --all
# moving may take a while depending on how many containers/vms you have
for instance in $(sudo lxc list --format json | jq ".[].name" -r); do
  sudo lxc move $instance --target-project user-$UID
done

# then I can use those instances from my local user as before
2 Likes