[Spec] Curtin v2 storage format

Status Drafting
Authors @mwhudson @dbungert
Release Ubuntu 22.04
Internal ID FO028

Abstract

We want to define a “version 2” of curtin’s storage config format to enable some features for the new desktop installer. We should see if there are other changes it would benefit users (mostly MAAS users) to make.

Rationale

Curtin’s current (and original) storage config has a few design issues that make it harder to add new features or fix some reported bugs. Because users of subiquity’s autoinstall facility and MAAS can provide raw curtin storage config, we have to be very careful about changing the meaning of existing configs.

Specification

Curtin has had, since day 1, had a version field on the storage config. So far the only supported version has been 1. Before the 22.04 release we will add support for a version 2. This will make the changes described in this section.

(From an implementation point of view, we will go for an “if version==1: old-code; else new-code;” sort of approach but that isn’t really necessary for this spec).

Partitioning behaviour

Rationale

It is not reasonably possible to edit a partition table for at least two reasons:

  1. existing partitions are never deleted unless you repartition a device entirely – any partition not listed in the configuration is simply ignored.
  2. partitions are assumed to be contiguous (up to alignment), so you cannot control where a new partition goes accurately enough to ensure it fits in a gap between existing partitions.

Specification

In a v2 storage configuration, an offset in a partition action will not only be respected but it will become quite fundamental as it will be used to identify existing partitions, and any partition not referenced by the config will be deleted.

There is a lot more detail about this in the dedicated specification.

Bootloader specification

Rationale

Curtin assumes that the target system will boot the same way the installer environment does and almost all of its behaviour around bootloader installation depends on this. We want installations of Ubuntu to not depend on this – the user should be able to install in BIOS mode and then switch to UEFI boot and continue using their system.

It also uses a few different and somewhat “do what I mean” ish ways to identify where bootloaders should be installed.

Specification

A v2 storage config will be much more explicit and use an ‘install_bootloader’ flag to indicate that grub should be installed to a particular device. Other flags on the device will indicate the kind of bootloader that is required – for a partition the ‘flag’ attribute could be ‘esp’ or ‘prep’ to indicate the nature of the bootloader supported, and setting it on a disk will trigger the installation of grub-pc for bios boot.

This change has its own specification too.

Location of block devices

Rationale

Ambiguities in how disk actions are connected to actual block devices mean that some buggy situations fail to install in situations where a successful install could be performed regardless. In particular, sometimes distinct drives from a vendor report identical wwns.

In addition, the way a provided wwn or uuid ‘val’ is matched to a device is a little complicated.

Specification

Currently curtin considers in turn the ‘wwn:’, ‘serial:’, and ‘path:’ fields on a block device and takes the first device that is located by one of these fields. In v2 configs, all fields that are present must match for a device.

Currently the way a provided wwn or uuid ‘val’ is matched to a device is (the same code is used for both fields):

  1. First ‘val’ is ‘escaped’ by replacing spaces with underscores.
  2. Then curtin finds the longest filename in /dev/disk/by-id that has ‘val’ as a substring
  3. Then curtin replaces a reference to a multipath device or multipath member with a /dev/mapper/$name-type path.

In a v2 storage config, this will change: ‘val’ must exactly match the value found into the udev database (and so any escaping must match the escaping udev and its rules do). Looking up a device by ‘wwn:’ will match on the DM_WWN field. Looking up a device by ‘serial:’ will match on DM_SERIAL first (to prefer the multipath disk over any path to it), and then ID_SERIAL.