Remove destroyed Ceph cluster storage pool from LXD cluster

[vosdev]

I need to remove a Ceph 20.2.0 cluster from a LXD 6.6 cluster. The Ceph cluster has already been destroyed, but we forgot to remove the Ceph storage pool from LXD beforehand (And also a single VM stored on this ceph storage pool).

I am unable to delete the VM because the rbd command hangs and I am unable to remove the pool because it is still in use. What’s next?

[jpelizaeus]

Do you still have any custom volumes/instances on the pool present in LXD’s database? You can check with lxc storage volume list <name of pool>.

[vosdev]

Yeah the VM and ~10 snapshots.

[jpelizaeus]

If there is no way to get the Ceph cluster back up and running as it’s destroyed already, you can delete the entries from the database. Keep in mind that this modifies the actual database used by LXD.

Can you please provide the output of the following commands so I can post you the necessary queries:

lxd sql global 'SELECT * FROM instances WHERE name = "<name of your inst>"' 
lxd sql global 'SELECT * FROM storage_pools WHERE name = "<name of your ceph pool>"

[vosdev]

+----+---------+---------+--------------+------+-----------+--------------------------------+----------+--------------------------------+----------------+------------+----------------------+
| id | node_id |  name   | architecture | type | ephemeral |         creation_date          | stateful |         last_use_date          |  description   | project_id |     expiry_date      |
+----+---------+---------+--------------+------+-----------+--------------------------------+----------+--------------------------------+----------------+------------+----------------------+
| 51 | 4       | openvox | 2            | 1    | 0         | 2025-10-23T09:18:57.483769313Z | 0        | 2026-02-18T08:28:50.345215049Z | OpenVox master | 2          | 0001-01-01T00:00:00Z |
+----+---------+---------+--------------+------+-----------+--------------------------------+----------+--------------------------------+----------------+------------+----------------------+

+----+------+--------+-------------+-------+
| id | name | driver | description | state |
+----+------+--------+-------------+-------+
| 4  | ceph | ceph   |             | 1     |
+----+------+--------+-------------+-------+

[jpelizaeus]

Double check if the IDs are correct and then run:

# Delete the instance. This will also delete (cascade) the snapshots.
lxd sql global 'DELETE FROM instances WHERE id = 51'

# Remove the pool
lxd sql global 'DELETE FROM storage_pools WHERE id = 4'

[vosdev]

Thank you, that seemed to have done the trick!

I looked at the storage pool after deleting the instances via the first sql command. The storage pool still showed all the old data.

The pool is now also deleted

Is my database now in a similar state as if I deleted the instances properly via lxc delete <name>? Or is there still old lingering data?

And is it safe to recreate a storage pool named ceph and create similarly named instances again?

I’ve also created a feature request: --force flag for instance deletion for offline storage pools · Issue #17682 · canonical/lxd · GitHub

[jpelizaeus]

There isn’t anything lingering left and you should be able to recreate the pool, instance and snapshots.

I looked at the storage pool after deleting the instances via the first sql command. The storage pool still showed all the old data.

Removing the instance entry from the instances table doesn’t yet touch the corresponding instance volume record. When deleting the pool, it deletes (cascade) also all of the volumes (including the instance volume) attached to it.

[jpelizaeus]

You can confirm by running lxd sql global 'SELECT * FROM storage_volumes'. In the output there shouldn’t be any volume left referencing the pool with id 4.