Easy multi-user LXD setup

One nifty new feature of the newly released LXD 4.22 is the ability for regular users to safely interact with LXD.

Until now, LXD has suffered from much the same issue as the likes of Docker and Libvirt in indirectly granting full root access to anyone who’s allowed to interact with it. This was possible through a number of different options (device passthrough, privileged container, custom ID maps, …). While not a problem on a developer’s laptop, it’s a no-go for most shared environments, especially in a corporate setting.

Over the years, LXD has grown support for proper remote authentication, fancy access control (RBAC) and projects as ways to restrict specific users to a subset of LXD’s features so multiple people can safely share a LXD server or cluster.

With LXD 4.22, this is now all doable locally with normal local users making it a great fit for desktop systems, especially multi-users desktop in a corporate environment.

Here is an overview of the feature from our release live stream:
https://www.youtube.com/watch?v=Blx7cdygiS8&t=848s

For those who aren’t into video watching, the instructions basically are:

snap install lxd
snap set lxd daemon.user.group=users

With that done, any user in the users group will be allowed to interact with LXD despite not being in the all-powerful lxd group.

The first time one such user interacts with LXD, they will automatically get their own restricted project which will look like this:

foo@v1:~$ lxc project list
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+
|        NAME         | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |               DESCRIPTION                | USED BY |
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+
| user-1001 (current) | YES    | YES      | YES             | NO       | User restricted project for "foo" (1001) | 3       |
+---------------------+--------+----------+-----------------+----------+------------------------------------------+---------+

foo@v1:~$ lxc project show user-1001
config:
  features.images: "true"
  features.networks: "false"
  features.profiles: "true"
  features.storage.volumes: "true"
  restricted: "true"
  restricted.containers.nesting: allow
  restricted.devices.disk: allow
  restricted.devices.disk.paths: /home/foo
  restricted.devices.gpu: allow
  restricted.idmap.gid: "1003"
  restricted.idmap.uid: "1001"
description: User restricted project for "foo" (1001)
name: user-1001
used_by:
- /1.0/instances/welcome-earwig?project=user-1001
- /1.0/images/ced57a80f2b761c3cdab867c2296b801c6adfe521f811bacdd61410da4bc2734?project=user-1001
- /1.0/profiles/default?project=user-1001

Which in practice allows the user to just do:

foo@v1:~$ lxc launch ubuntu:20.04
Creating the instance
Instance name is: welcome-earwig        
Starting welcome-earwig

foo@v1:~$ lxc list
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+
|      NAME      |  STATE  |        IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+
| welcome-earwig | RUNNING | 10.31.36.109 (eth0) | fd42:fa4a:d38d:1c7b:216:3eff:fed1:63aa (eth0) | CONTAINER | 0         |
+----------------+---------+---------------------+-----------------------------------------------+-----------+-----------+

And get a container, or virtual machine, running immediately, with no configuration and without ever having needed any kind of elevated privileges.

Should that user try to create a privileged container, pass in paths outside of their home directory or do any device passthrough other than GPUs, it will be rejected by LXD.

Login as another user on the system and you’ll get the exact same behavior, every user gets their own personal project and can’t see the others. Well, unless they are part of the lxd group, then they can see everything going on on the system.

We’re hoping that this feature can be used to provide experiences similar to that of WSL on Windows or Crostini on ChromeOS where getting containers going is just a few clicks away and the main system remains nice and safe.

Enjoy!

4 Likes

This is awesome :slight_smile: Is there an “easy” way to migrate an existing LXD install to this new less-privileged scenario?

Currently my user is in the lxd group and I have a bunch of (unprivileged) containers and VMs running under the default project - can I easily export my containers / VMs, remove myself from the lxd group and then set those back up under a restricted personal project for my user?

This is possible if slightly more involved :slight_smile:

I’ve tried it here, starting with:

ubuntu@u2004-desktop:~$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),44(video),100(users),998(lxd)
ubuntu@u2004-desktop:~$ lxc project list
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
|       NAME        | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |     DESCRIPTION     | USED BY |
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
| default (current) | YES    | YES      | YES             | YES      | Default LXD project | 5       |
+-------------------+--------+----------+-----------------+----------+---------------------+---------+
ubuntu@u2004-desktop:~$ lxc list
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
ubuntu@u2004-desktop:~$ 

I’ve then removed my user from the lxd group and logged in again:

ubuntu@u2004-desktop:~$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),44(video),100(users)
ubuntu@u2004-desktop:~$ lxc list
Error: not authorized
ubuntu@u2004-desktop:~$ lxc project list
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
|   NAME    | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS |                 DESCRIPTION                 | USED BY |
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
| user-1000 | YES    | YES      | YES             | NO       | User restricted project for "ubuntu" (1000) | 1       |
+-----------+--------+----------+-----------------+----------+---------------------------------------------+---------+
ubuntu@u2004-desktop:~$ lxc project switch user-1000
ubuntu@u2004-desktop:~$ lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
ubuntu@u2004-desktop:~$ 

So the user is now restricted to an empty project. Next we need to move the instances from a privileged user:

root@u2004-desktop:~# lxc list
To start your first container, try: lxc launch ubuntu:20.04
Or for a virtual machine: lxc launch ubuntu:20.04 --vm

+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
root@u2004-desktop:~# lxc stop --all
root@u2004-desktop:~# lxc move a1 --target-project user-1000
root@u2004-desktop:~# lxc move a2 --target-project user-1000
root@u2004-desktop:~# lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+

And after that, starting everything back as the user:

ubuntu@u2004-desktop:~$ lxc list
+------+---------+------+------+-----------+-----------+
| NAME |  STATE  | IPV4 | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+------+------+-----------+-----------+
| a1   | STOPPED |      |      | CONTAINER | 0         |
+------+---------+------+------+-----------+-----------+
| a2   | STOPPED |      |      | CONTAINER | 0         |
+------+---------+------+------+-----------+-----------+
ubuntu@u2004-desktop:~$ lxc start --all
ubuntu@u2004-desktop:~$ lxc list
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a1   | RUNNING | 10.225.65.247 (eth0) | fd42:2c06:5b38:f668:216:3eff:fe38:f20 (eth0)  | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| a2   | RUNNING | 10.225.65.206 (eth0) | fd42:2c06:5b38:f668:216:3eff:fefa:2294 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
ubuntu@u2004-desktop:~$ 

Note that the lxc move step requires that any project or custom volume you need exist in the target project, so if you had anything fancy on that front, you’ll want to re-create your profiles and move any custom storage volumes ahead of time.

If your instances use features which aren’t allowed under a restricted project, the lxc move will fail with Forbidden, normally telling you what’s being rejected. You can either modify the instance to comply with the restrictions or use lxc project edit user-1000 to alter the restrictions to allow what you need.

2 Likes

Thanks @stgraber - that worked like a charm :slight_smile:

For posterity, it was as simple as:

# remove my user from the lxd group
sudo gpasswd -d $USER lxd

# reboot to cleanly logout and back in again without being in
# the lxd group anymore

# set the lxd user group to just my user's primary group since 
# there is no other  users on my local machine
sudo snap set lxd daemon.user.group=$USER

# lxc will have created a new project called user-$UID for your user
lxc project list
# set this as the default project
lxc project switch user-$UID

# then migrate instances to this user's project by stopping them all
# and then moving them across
sudo lxc stop --all
# moving may take a while depending on how many containers/vms you have
for instance in $(sudo lxc list --format json | jq ".[].name" -r); do
  sudo lxc move $instance --target-project user-$UID
done

# then I can use those instances from my local user as before
3 Likes

I have tried it with local users and it works fine

But if I try with AD users (joined via SSSD), I get the following error

Error: Get "http://unix.socket/1.0": read unix @->/var/snap/lxd/common/lxd-user/unix.socket: read: connection reset by peer

The domain user is member of the local group “users”

We merged a fix for this. It will be in LXD 5.3 for sure (and in 5.0.1 LTS) but may also make it as a hotfix in a future update of LXD 5.2 (we’re collecting some other fixes for it now).

1 Like

Thanks for this great feature!

I’ve been playing around with it a bit and it seems like daemon.user.group has to be a local group. I’ve tried to use a LDAP group but then get this error:

Error: Get "http://unix.socket/1.0": dial unix /var/snap/lxd/common/lxd/unix.socket: connect: permission denied

Is this some sort of limitation related to name spaces?

After changing this parameter to a local group, it works as intended with a new user. But trying to use the user account I used first (when daemon.user.group was set to a LDAP group) I then get this error instead:

Error: not authorized

Any idea how to recover from that for this user account? I tried deleting the project, hoping that it would get recreated when running lxc list, but that didn’t happen.

So indeed the snap environment doesn’t have access to the host NSS stack, meaning it can’t resolve group names that come from a remote system like LDAP.

I would have expected group numbers to work though as all we really do is feed that to a chgrp call.
But there’s been some other users suggesting this may not work either? Would be great to confirm though.

Now for the second issue. What happened is that the LXD client wasn’t able to talk to lxd-user and so initialized a local configuration of its own. This most likely means it’s trying to access the default project rather than the user-specific project.

To fix that, you can do lxc project list to see the list of allowed projects and then lxc project switch to have the client use the correct one.