Upstream Ceph uses cephadm based containerised ceph deployment method to orchestrate a ceph cluster. Although we package and distribute cephadm on ubuntu but the default image it uses for deploying ceph services is not based out of ubuntu packages.
This document outlines the process to migrate an existing cephadm cluster to ubuntu rocks based ceph cluster.
1. Check ceph version compatibilty
As a rule if thumb you should not switch versions without planning proper maintainence.
Additionally, the new ceph image should not be of an older version (or point release).
- Shell into a ceph container
$ sudo cephadm shell
Inferring fsid 5db0335a-058b-11f0-a493-09ff4446d1ad
Inferring config /var/lib/ceph/5db0335a-058b-11f0-a493-09ff4446d1ad/mon.node2/config
root@node2:/# ceph -s
cluster:
id: 5db0335a-058b-11f0-a493-09ff4446d1ad
health: HEALTH_WARN
services:
mon: 3 daemons, quorum node2,node3,node1 (age 63m)
mgr: node1.pwidfn(active, since 6h), standbys: node3.cfcdqd
osd: 3 osds: 3 up (since 88m), 3 in (since 89m)
data:
pools: 1 pools, 1 pgs
objects: 2 objects, 577 KiB
usage: 875 MiB used, 279 GiB / 279 GiB avail
pgs: 1 active+clean
3. Check ceph cluster status
root@node1:/# ceph orch ls
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
crash 3/3 8m ago 3h *
mgr 2/2 8m ago 3h count:2
mon 3/5 8m ago 3h count:5
osd 3 8m ago - *
4. Configure source as ubuntu rock and redeploy service
ceph config set mon.node1 container_image "ghcr.io/canonical/ceph:quincy"
ceph orch daemon redeploy mon.node1
Note
Repeat the same for other mon daemons and subsequently for other daemons as well.
Check and wait for cluster status to settle post each daemon redeploy.
5. Configure the default container source to ubuntu rocks
ceph config set global container_image "ghcr.io/canonical/ceph:quincy"
Info
The global config ensures that future daemons also use ubuntu rocks as the container image source.