Modifying kernel boot parameters

When using the default GNU GRUB bootloader, kernel boot parameters can be customised without breaking the integrity of the TPM backed full disk encryption (requires snapd 2.50+).

Parameters can be modified either statically, by editing cmdline files, or dynamically, by specific system settings (by dynamically we mean that snapd API can be used to set them, but the parameters do survive across reboots). Both of these methods are covered below.

Ubuntu Core boot-time kernel parameters can be viewed from /proc/cmdline:

$ cat /proc/cmdline
snapd_recovery_mode=run console=ttyS0,115200n8 console=tty1 panic=-1

In addition to parameters supported by the kernel, there are also Ubuntu Core-specific kernel boot parameters.


Dynamic kernel parameter modifications

There are two system options that can be used to add new kernel boot parameters to a system that has been deployed and is running:

  1. system.kernel.cmdline-append
  2. system.kernel.dangerous-cmdline-append

The first setting will permit only boot parameters verified against an allow list in the gadget snap. See gadget.yaml for further details.

The second option is valid only for dangerous grade models and permits any parameter to be added.

System options can be set either through the snapd API, or with the snap set command:

snap set system system.kernel.dangerous-cmdline-append="core.bootchart"

A reboot is required for the changes to be applied. The system will not restart automatically.

To remove a dynamically set kernel command, use the snap unset command:

snap unset system system.kernel.dangerous-cmdline-append

Static boot option modifications

Static kernel boot-time options are formed from mode arguments set by snapd, a static element declared in the bootloader configuration script, and optional extra arguments.

For example, the run mode kernel command line is:

snapd_recovery_mode=run console=ttyS0 console=tty1 panic=-1

The recovery mode of a system labeled 20210512 would look like this:

snapd_recovery_mode=recover snapd_recovery_system=20210512 console=ttyS0 console=tty1 panic=-1

In the above examples, the mode arguments are snapd_recovery_mode and snapd_recovery_system. The static command line content for the grub bootloader is console=ttyS0 console=tty1 panic=-1 which is defined in the snapd source code.

Below we describe how to add additional static parameters (static with the meaning that they cannot be manipulated dynamcally from the snapd API).

Adding static parameters from the gadget

The recommended way to add static parameters is by using the kernel-cmdline stanza as described in gadget.yaml. This allows to add new parameters by using the append list or removing some of the default Ubuntu Core parameters by putting them in the remove list. As already explained, a list of the allowed dynamic parameters can be set from there too.

Files for modification

The legacy way of customizing statically the kernel command line is by adding one of two possible files to the top level of the filesystem in the Gadget snap:

  1. Add a cmdline.extra file containing the extra kernel command line arguments, such as custom.option=1.

    These arguments are automatically appended to the command line:

    run mode:

    snapd_recovery_mode=run console=ttyS0 console=tty1 panic=-1 custom.option=1

    recovery mode of system 20210512:

    snapd_recovery_mode=recover snapd_recovery_system=20210512 console=ttyS0 console=tty1 panic=-1 custom.option=1

  2. Add a cmdline.full file containing the full kernel command line to replace the built-in static command line entirely.

    For example, if cmdline.full file has the following contents:

    # my custom option
    custom.option=1
    # use only ttyS0
    console=ttyS0
    

    The kernel command line will contain the following:

    run mode:
    snapd_recovery_mode=run custom.option=1 console=ttyS0

    recovery mode of system 20210512:

    snapd_recovery_mode=recover snapd_recovery_system=20210512 custom.option=1 console=ttyS0

Both kernel command line extension methods also apply to install mode.

The gadget snap can contain only one of cmdline.full or cmdline.extra; the presence of both files at the same time is treated as an error.

Extending the kernel command line using drop-in files is also supported on systems using the full disk encryption. See Full disk encryption for more details.

Customising the kernel command line

If setting the parameters in gadget.yaml, you will just need to edit that file in your sources and build the gadget as usual with snapcraft.

If using the legacy files, there are two general approaches to incorporate them into an Ubuntu Core image.

The first requires that the snapcraft.yaml is edited to add the kernel command line file to the root of the gadget filesystem, using the dump plugin, for example.

The second is to modify an existing gadget snap directly, and this procedure is outlined below.

Splash screen configuration:

Enabling the Ubuntu Core splash screen requires a modified kernel command line. See Splash screen configuration for details.

Modify an existing gadget snap

When building your own custom Ubuntu Core image, the gadget snap that you include can be modified manually to include the kernel command line file.

To do this, first retrieve the gadget snap you wish to use in the image. The following command, for example, will download the PC gadget snap with a base of core24:

$ snap download pc --channel=24/stable
Fetching snap "pc"
Fetching assertions for "pc"
Install the snap with:
   snap ack pc_178.assert
   snap install pc_178.snap

In the above example, the downloaded gadget snap is called pc_178.snap, but this will change according to revisions and architectures. You may even source your own gadget snap locally.

All snaps are compressed with the Squashfs filesystem and the gadget snap next needs to be decompressed locally with the unsquashfs command:

$ unsquashfs pc_178.snap
Parallel unsquashfs: Using 8 processors
16 inodes (41 blocks) to write

[============|] 41/41 100%

created 15 files
created 10 directories
created 0 symlinks
created 0 devices
created 0 fifos
created 0 sockets

The above command will create a new directory called squashfs-root containing the files and folders of the gadget snap. We can now create the required cmdline.extra or cmdline.full file (see above) containing our kernel command line arguments:

echo "option1=foo option2=bar" > squashfs-root/cmdline.extra

With the cmdline.extra or cmdline.full file created, the gadget snap can be recompressed back into a snap using the snap pack <squashfs-root> command:

snap pack squashfs-root

The final step is to build a new Ubuntu Core image with the modified gadget snap. This requires a model assertion with grade: dangerous set (as we are using local snaps for the build - this would not be necessary if we have uploaded the gadget to the store) and the ubuntu-image command to compile the image. See Custom images for more details.

You can then build the image with the new gadget snap using ubuntu-image:

$ ubuntu-image snap my-model.model --snap pc_24-0.4_amd64.snap
Fetching snapd
Fetching pc-kernel
Fetching core24
Fetching htop
WARNING: "pc" installed from local snaps disconnected from a store cannot be refreshed subsequently!
Copying "pc_24-0.4_amd64.snap" (pc)

The resultant image can now be installed on your device and will include your custom kernel command line options.

1 Like

“All use snaps …” I have no idea what that should say.

It should just read “all snaps are …” I edited it just now

1 Like

This page claims that the gadget snap can have only one of cmdline.full or cmdline.extra at the top level or it’s an error. Where is this enforced? I wanted to check the code that actually tests this.

Haven’t been able to update the kernel boot option of a machine after installation with a customized gadget snap:

  • adding quiet splash as mentioned there https://ubuntu.com/core/docs/splash-screen
  • in the cmdline.extra file at the root level of the gadget snap as mentioned in this very page / doc
  • updating (adding 1) the edition of each and every structure of the volume (by lack of knowing which one does contain the actual kernel boot options) as mentioned there https://snapcraft.io/docs/gadget-boot-assets (kudos to anyone who can point me to some doc about what structure is actually containing the kernel options :star_struck:)
  • rebuilding the gadget snap
  • scp’ing it and snap installing it with the --dangerous flag will reboot the machine:
me@mymachine:~$ snap install --dangerous pc_customized.snap
Stop snap "pc" services                                                                                  -
Broadcast message from root@mymachine (Sun 2022-12-04 09:07:44 UTC):

reboot scheduled to update the system
The system is going down for reboot at Sun 2022-12-04 09:17:44 UTC!

snapd is about to reboot the system

But then the quiet splash options aren’t added at all:

me@mymachine:~$ sudo cat /proc/cmdline
snapd_recovery_mode=run console=ttyS0,115200n8 console=tty1 panic=-1

Rebuilding the whole image with the newest gadget snap does work: it adds the splash screen options. But that needs physical presence to reinstall the machines, and a keyboard to go through the installer, so it’s no more an option for deployed machines…

Did I miss something? Can’t we change the kernel boot parameters after the installation?


Update: even worse, after installing the new image (which brings the splash screen) updating some stuff from the gadget snap did remove the quiet splash kernel options after that gadget snap being scp’ed and updated on a machine :exploding_head:

https://github.com/snapcore/snapd/blob/78a3721ab7200d0d40cfb2be52b2cf8220d77e4d/gadget/gadget.go#L1311 and its call in https://github.com/snapcore/snapd/blob/78a3721ab7200d0d40cfb2be52b2cf8220d77e4d/boot/cmdline.go#L110 should be the relevant bits.

Hi @degville,

Could you please expend this documentation by including the brand new feature snapd_system_disk kernel parameter?

https://github.com/snapcore/snapd/commit/b3cb341d9f96848a234896012ac458196419e2b5#diff-e8bdaf38399d14829a21d8bec556c27a5b067be9c02dc89f597495bf9048ba50

The new feature will allow developers to hard-code bootable disk.

Thanks,
Bugra

Yes, of course. I’ll create a task for it. Thanks for letting me know, and for the links.

Some suggestions (sorry, it looks like I cannot edit directly):

breaking the integrity of the UEFI Secure Boot mechanism → breaking TPM backed FDE

The second (dangerous) option permits any parameter to be added: → The second option is valid only for dangerous grade models and permits any parameter to be added

Static boot option modifications

The preferred way to add static parameters is by using the kernel-cmdline stanza as described in gadget.yaml. This allows to add new parameters by using the append list or removing some of the default Ubuntu Core parameters by putting them in the remove list. As already explained, a list of the allowed dynamic parameters can be set from there too.

There is a legacy way of adding kernel command parameters by one of two possible files to the top level of the filesystem in the Gadget snap:…

EDIT: I could edit directly in the end

1 Like