Upgrading node operating system

The operating system running on a Juju machine is known as a Juju “series” and corresponds to an Ubuntu release (e.g. the ‘Ubuntu 18.04 LTS’ release corresponds to the ‘bionic’ series). Depending on the current state of the cluster and the goal state of the upgrade, a series upgrade may occur before or after a Ceph upgrade.

The command used to manage series upgrades on a per-machine basis is the upgrade-series command.

The below example will cover the case of a bionic-to-focal series upgrade.

  1. Set the default series for the current model:

    juju model-config default-series=focal
    
  2. Ensure that the cluster is in a healthy state. On any ceph-mon unit run a status check:

    juju ssh ceph-mon/1 sudo ceph status
    
  3. Prevent data rebalancing from occurring when OSDs start disappearing from the network. On any ceph-mon unit set the ‘noout’ option for the cluster:

    juju run-action --wait ceph-mon/1 set-noout
    

Group upgraded machines by application. In terms of the upgrade order, begin with the machines hosting the ceph-mon application, after which the remaining applications/machines can be upgraded in any order. The below example commands will feature the ceph-mon application.

  1. Set the default series for the application so any future application units (scaleout) will use the new series:

    juju set-series ceph-mon focal
    
  2. Assuming that the ceph-mon units are on machines 0, 1, and 2 begin with machine 0:

    juju upgrade-series 0 prepare focal
    juju run --machine=0 -- sudo apt update
    juju ssh 0 sudo apt full-upgrade
    juju ssh 0 sudo do-release-upgrade # reboot when prompted
    juju upgrade-series 0 complete
    
  3. Repeat step #4 for the remaining machines associated with the ceph-mon application (machines 1 and 2 in this example).

  4. Repeat steps #3, #4, and #5 for each remaining application (e.g. ceph-osd, ceph-fs, etc).

Verify the upgrade by checking for errors in juju status output.