Failed instance creation with no space left on device

Hi All,

As shown below, I am using a storage pool of 10TB in my LXD default profile.

$ lxc storage list
+------------------+--------+------------+-------------+---------+---------+
|       NAME       | DRIVER |   SOURCE   | DESCRIPTION | USED BY |  STATE  |
+------------------+--------+------------+-------------+---------+---------+
| lxc-storage-pool | lvm    | op1_md1_vg |             | 6       | CREATED |
+------------------+--------+------------+-------------+---------+---------+

$ lxc profile show default 
name: default
description: Default LXD profile
config: {}
devices:
  eth0:
    network: lxdbr0
    type: nic
  root:
    path: /
    pool: lxc-storage-pool
    type: disk
used_by: []

When i try to create the new instance I am seeing “no space left on device”

$ lxc launch ubuntu:22.04 u2204 --vm -c limits.cpu=8 -c limits.memory=32GiB -p default
Creating u2204
Error: Failed instance creation: write /var/snap/lxd/common/lxd/images/e78b98611cf12d2dc6d6aac3ac084aa570ba2bb8f967a9233654a1aed8018e0b.rootfs: no space left on device

As shown below my / path is 100% used and is this causing my above issue ? Sadly my os disk is a 64GB ssd but I do have other volume groups which are larger in size. Instead of resizing the / is it possible to make LXD use other volumegroups ?

$ df -h
Filesystem                         Size  Used Avail Use% Mounted on
tmpfs                               51G  2.6M   51G   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv   28G   27G     0 100% /
tmpfs                              252G     0  252G   0% /dev/shm
tmpfs                              5.0M     0  5.0M   0% /run/lock
/dev/sde2                          2.0G  253M  1.6G  14% /boot
/dev/sde1                          1.1G  6.1M  1.1G   1% /boot/efi
tmpfs                               51G   16K   51G   1% /run/user/1000
tmpfs                              1.0M     0  1.0M   0% /var/snap/lxd/common/ns
/dev/mapper/op1_md0_lvm-lxc--vol    10T  3.6M   10T   1% /mnt/lxc-vol

If you have other disks/partitions available, you could use them to grow/extend the VG.

You could create a new LXD storage pool using the larger volume group and then move the instances to it:

lxc storage create foo lvm source=<large vg>
lxc move <instance> --storage foo

My existing lxc-storage-pool in lxd is 10TB and with ample amount of storage space.

As per my understanding the issue is with the / path being full. Is LXD dependent on / path ?

The host’s filesystem is used to store images and backups unless you tweaked the storage.backups_volume and storage.images_volume configuration keys, see
https://documentation.ubuntu.com/lxd/en/latest/server/#miscellaneous-options

1 Like

To expand on @sdeziel1 suggestion, if you create a storage pool (e.g. like Failed instance creation with no space left on device - #3 by tomp) you can then create a custom volume on there using:

lxc storage volume create <pool> <vol name> size=...

And then instruct LXD to use that for image storage rather than the root filesystem by doing:

lxc config set storage.images_volume=<pool>/<vol name>
1 Like

Thanks @tomp

Is this option available under LXD-UI? Well I can only see configuration settings under lxd instance but nothing at the global level.

Also when I try below command I dont see any output. So the default is host’s filesystem ?

$ lxc config get storage.images_volume

@edlerd can global server settings be controlled in the UI yet?

@codingfreak thats right if its not set it will default to the host’s filesystem.