Kernel Regression - Mitigation

Lounge Tips & Tricks
1

Introduction

This tip is relevant to Ubuntu systems, where two kernels are installed and a new kernel introduces a regression.

An interim release initially will only have one kernel.
If a regression appears in the new (second) kernel, simply boot into the first working kernel and wait for another kernel update.

A regression may be evident in a newly installed kernel, for example:

  • Failure to boot
  • Bluetooth or Wi-Fi malfunction
  • Graphics instability

In such cases, temporarily boot into an earlier working kernel until the issue is resolved with a subsequent update.

Recommendation

When a regression occurs:

  1. Ensure that you have personal data backups
  2. Boot into the last known working kernel from the GRUB menu
  3. Leave the regressed kernel installed temporarily
  4. Wait for the next kernel update
  5. Test the new kernel when it becomes available

It is generally simpler and safer to keep the full kernel stack installed until the user has tested that the regression is fixed.

Regression Fixed - Remove Unused Kernel

List installed kernels

ls /boot | grep vmlinuz-

Example

vmlinuz-6.14.0-35-generic
vmlinuz-6.14.0-36-generic
vmlinuz-6.14.0-37-generic
  • 6.14.0-35 — working kernel
  • 6.14.0-36 — regression present
  • 6.14.0-37 — regression fixed

Kernel 6.14.0-36 was retained to maintain normal upgrade tracking.
Once 6.14.0-37 is verified working, 6.14.0-36 may be removed.

Remove unused kernel

sudo apt remove *6.14.0-36*

Scroll down to the end of the output and there will be a list of packages to be removed plus a double-check continuation prompt.
Review the list carefully before confirming.

Example

The following packages will be REMOVED
  linux-headers-6.14.0-36-generic linux-hwe-6.14-headers-6.14.0-36
  linux-hwe-6.14-tools-6.14.0-36 linux-image-6.14.0-36-generic
  linux-modules-6.14.0-36-generic linux-modules-extra-6.14.0-36-generic
  linux-tools-6.14.0-36-generic
0 to upgrade, 0 to newly install, 7 to remove and 0 not to upgrade.
After this operation, 297 MB disk space will be freed.
Do you want to continue? [Y/n]

Residual Configuration Files

After removing a kernel, configuration remnants may remain

List kernel image packages and their status

dpkg --list | grep linux-image

Example

ii  linux-image-6.14.0-35-generic                 6.14.0-35.35~24.04.1                       amd64        Signed kernel image generic
rc  linux-image-6.14.0-36-generic                 6.14.0-36.36~24.04.1                       amd64        Signed kernel image generic
ii  linux-image-6.14.0-37-generic                 6.14.0-37.37~24.04.1                       amd64        Signed kernel image generic

Status codes

  • ii - installed correctly
  • rc - removed, configuration files remain
  • iF - installed, failed configuration

To purge residual configuration files

sudo apt purge $(dpkg -l | egrep '^rc' | awk '{print $2}')

Review the list carefully before confirmation

Summary

  • Verify that your data backups are intact and restorable
  • Never remove the latest kernel
  • Keep the regressed kernel installed until a fix is confirmed
  • Only remove older specific kernels once verified unnecessary
  • Always review removal lists carefully

Further Information

Terminal vs GUI

Ubuntu’s Software Updater typically

  • Retains the currently booted kernel
  • Retains the newest installed kernel
  • Removes unused older kernels

The GUI presents a checkbox Unused kernel updates to be removed
Descriptions are less detailed than terminal output.

sudo apt autoremove provides explicit package lists and confirmation prompts, allowing finer control.

If you have suffered a kernel regression, terminal-based management is generally clearer and safer.

Meta Packages

Kernel upgrades are controlled by meta packages and should not be removed.
The better option is the complete retention of the latest kernel inc. meta packages.

Examples (HWE 24.04)

  • linux-generic-hwe-24.04
  • linux-image-generic-hwe-24.04

Examples (GA)

  • linux-image-6.14.0-37-generic
  • linux-image-generic
  • linux-generic

Verify the meta source

apt info linux-generic-hwe-24.04

Example (first lines)

Package: linux-generic-hwe-24.04
Version: 6.17.0-14.14~24.04.1
Priority: optional
Section: kernel
Source: linux-meta-hwe-6.17
Origin: Ubuntu
2 Likes