Ceph RBD features

Hey,

By default the ceph.rbd.features are set to layering. This is a pretty thinned out feature set, likely for old kernel compatibility.

Running Ubuntu 24.04 and Ceph 19, would it be okay to simply enable the majority of features?

ceph.rbd.features=layering,exclusive-lock,object-map,fast-diff,deep-flatten

https://docs.ceph.com/en/squid/rbd/rbd-config-ref/#image-features

The only feature I didn’t add is striping. Is it beneficial for a virtualization environment?

Striping spreads data across multiple objects. Striping helps with parallelism for sequential read/write workloads.

And does it make sense for LXD to update the defaults of this setting for future LXD versions? Some of these features can only be enabled at image creation.

These features rely on kernel 4.9, 5.1 and 5.3 or above, which is everything Ubuntu 20.04 and newer (5.4).

The docs do not mention anything about the Ceph RBD features or recommended features to enable on a modern environment.

1 Like

I created an instance with the above features enabled and WOW! LXD UI is so much more responsive. Previous instances had to load for ~3-5 seconds but this new instance is almost instant. The fast-diff makes rbd du so much quicker.

In my lxd cluster (version 5.21.4 LTS) deployed on Ubuntu 24.04.3 hosts using microcloud (version 5.21.4 LTS) with microceph (19.2.1-0ubuntu0.24.04.2) I have these settings set by default:

$ lxc storage get remote ceph.rbd.features
layering,striping,exclusive-lock,object-map,fast-diff,deep-flatten

Hi, thanks for following up on this. When you manually create a Ceph RBD pool in LXD directly, the features are set to layering only.

However when MicroCloud sets up the remote pool, we configure another set of features, namely layering,striping,exclusive-lock,object-map,fast-diff,deep-flatten.

Then I feel safe also enabling the striping feature :slight_smile:

If it already is the default for microcloud, why not make this the default for LXD as well?

The overall experience is so much smoother that I cannot imagine running a big cluster with only layering enabled.

I would be open to making these the default once we confirm what each item does and whether it whether there are any drawbacks/compromises/compatibility issues we need to be aware of so we can document this - as LXD outside of MicroCloud is not tied to a specific version of ceph.

1 Like

@vosdev would you mind creating a GH issue for this so it can be tracked? Thanks :slight_smile:

1 Like