Accessing host's ZFS inside an LXD container

hadmut · June 23, 2025, 11:11pm

Hi,

for many years I had this setup running:

Host with LXD and several disks, mounted as ZPOOL / ZFS volumes
ZFS filesystems mounted into an LXD guest machine
In the guest machine, running a daemon, which not just accesses the files in the volume, but also reads ZFS properties like
zfs get -H guid ZFSPATH

Since upgrading LXD to version 6, this does not work anymore.

The process can still access the files in the filesystem, mounted in to the LXD container, but it does not see the zpools and zfs file systems of the host anymore, probably because

/sys/module/zfs/*

have been isolated, like a namespace. They still do exist, and are readable, and look like fully functional, but do not show the host’s zfs file systems.

The question: Is there a way to configure LXD 6 to have this behave like LXD 5, i.e. allow the guest to query the ZFS properties of the host?

hadmut · June 24, 2025, 12:02am

Maybe I found the answer here:

https://klarasystems.com/articles/isolating-containers-with-zfs-and-linux-namespaces/

Will have to try this.

tomp · June 24, 2025, 8:34am

You’re correct that LXD 5.21 onwards has support for ZFS delegation:

github.com/canonical/lxd

Add support for ZFS delegation

canonical:main ← stgraber:zfs-delegate

opened 10:05PM - 20 Jul 23 UTC

stgraber

+405 -81

This adds support for the ZFS dataset delegation feature found in OpenZFS 2.2. … On the LXD front, this is done through a new `zfs.delegate` configuration option on storage volumes. One can therefore do: ``` lxc init images:ubuntu/22.04 u1 lxc storage volume set default container/u1 zfs.delegate=true lxc start u1 ``` To get a container that has delegated ZFS access to its dataset and anything underneath it. The same works as expected with custom storage volumes which can therefore allow multiple containers to interact with the dataset. As part of implementing this, the ZFS storage driver had to be updated to handle recursive operations throughout, so that those nested datasets don't get lost during a backup or migration. Lastly, the way this is implemented is through new `DelegateInstance` and `DelegateCustomVolume` functions on the storage backend which are called after the instance is running (as the PID is required). This allowed for the work to be mostly contained to the ZFS storage driver. That's with the exception of the logic enabling /dev/zfs in the container as that did need to be directly to the LXC driver and has to be conditional on the correct ZFS version being on the system as one doesn't want to expose /dev/zfs to containers on prior versions (potential security risk there). This work was sponsored by Buddy (https://buddy.works) who also sponsored the matching OpenZFS work. Closes https://github.com/canonical/lxd/issues/11796

However I am interested to know if you are seeing this behaviour change on the same host when only switching between LXD 5.0 and LXD 6 or whether there has also been an OS kernel update at the same time?

hadmut · June 24, 2025, 4:36pm

Actually, I don’t recall when exactly I found this problem first.

I had a process running in an LXD machine dealing with some distributed directory updates, and since I do need a unique identifier for each client (file system), I just used the zfs guid as an identifier, pulled with

zfs get -H guid ZFSFILESYSTEM

and I think it happened when I upgraded my server from 22.04 to 24.04, but correlation is not causation. During the upgrade process, I probably have checked the LXD version, since I am not using the latest snap channel, but a particular version, and thus upgraded LXD from 5 to 6, and noticed that it didn’t work anymore. I had considered this as a bug, and ignored it for a while, had the process running on the host instead of the guest.

But since this is not really clean, I just wanted to get things straight, and if it had been a bug, it would have been fixed, so my guess was it is not necessarily a bug, it might be a feature.

And yes, that zone thing solves the problem. Once you know what it is …