From installation to provisioning - upgrading the Ubuntu Desktop installer

Desktop
, ,
1

From installation to provisioning - upgrading the Ubuntu Desktop installer

Background

In Ubuntu 24.04 LTS provisioning is a top priority. The concept of provisioning – preparing and equipping a system for use – is particularly crucial now that we’ve aligned the Ubuntu desktop installer’s backend with that of Ubuntu Server. This alignment not only streamlines our development processes but also introduces features to desktop users that were previously exclusive to server environments. This change enables us to better address the needs of modern OEMs and managed environments – key stakeholders in Ubuntu’s ecosystem and crucial to its longevity. By evolving from installation to provisioning, we aim to offer a more robust, efficient, and versatile experience, ensuring Ubuntu continues to meet the evolving demands of all our users. Finally, it’s important to call out that this project is a work in progress so do expect the final deliverable to look different from what I write about here. Let’s jump in!

Provisioning is a multi-stage process

During our review of stakeholder requirements, we realised our focus was too much on the installer. As I mentioned in the opening, installation is a step within a broad provisioning process. By broadening our perspective, we found opportunities to enhance the overall experience and to set foundations for future improvements. Our aim is not just to refine the process, but to make it efficient and inclusive for everyone. With these considerations in mind, we defined provisioning as having six distinct and generally sequential stages:

  1. Image creation: In the first stage the provisioner creates their live cd image. This is the image that you download from releases.ubuntu.com. Today images are created using a combination of tools but in the future it will be done declaratively with imagecraft.

  2. Image discovery and delivery: Once we have an image, your computer needs to discover and then boot from it. There are various ways to do this: PXE, netboot, mini.iso, USB thumb drive, etc.

  3. Device bootstrap: Once your computer has booted from this image you’re typically in a ‘live cd’ environment. At this stage we can bootstrap your computer for independent booting, including disk partitioning, encryption, bootloader installation, and copying the base OS onto the target device. This is what we previously called the installer.

  4. First boot initialisation: Now that your computer is bootstrapped, it’s ready to boot independently but there is still work to be done before it is usable. For example we need to create the first user, perhaps agree to a EULA if it is direct from a factory OEMs, configure it for managed use if in an enterprise, and so on.

  5. Welcome: Finally we’re on the desktop :partying_face:. Here we should greet the user, tell them what’s new and offer suggestions for getting started.

  6. Post provision hooks: At this stage most people consider provisioning complete. However, I think there are future opportunities to deliver a more polished and complete experience across the lifecycle of a machine. We’re calling this ‘hooks’ where time or event-based triggers could deliver bespoke behaviours. For example, reminders to backup your disk recovery key after seven days if TPM backed FDE is enabled.

Design

Abstract framework out of the way, let’s dive into details. Below you’ll find some earlier work defining user flows through Stages 3 to 5. You’ll want to click on each image for easier viewing in a new tab. Lastly, its important to call out that for Ubuntu 24.04 LTS these stages will differ between OEMs and public images.

The legend


Figure 1. Provisioning flows legend

Stage 3 Device bootstrap

This stage lays the foundation for the device to load the operating system independent of the image created in stage 1. The flow includes partitioning, encryption, installing the bootloader, copying the base operating system files, etc. The design is structured to make this process as user-friendly as possible.


Figure 2. Stage 3 provisioning flow

Stage 4 First boot initialisation

During the first boot an admin or the first user configures the first account and potentially enrols the machine in a remote management solution. For Ubuntu 24.04 LTS this stage will only been used for OEMs.


Figure 3. Stage 4 provisioning flow

Stage 5 Welcome

Welcome is the user’s first glimpse of their new desktop :confetti_ball:. We want to help users get up and running smoothly whilst experienced users can skip this if they want to.


Figure 4. Stage 5 provisioning flow

Automation, customisation and happy paths

We want to provide a robust, well-tested provisioning ‘happy path’. This polished path prioritises user experience and aims to be accessible to everyone – especially those unfamiliar with the myriad of choices and technical terms. Simultaneously, we also recognise the need for automation and customisation.

For those advanced users of Ubuntu Server, you’re likely familiar with autoinstall.yaml. In Ubuntu 24.04 LTS we’re making this feature visible by exposing it in the graphical installer (see Figure 2 where the user chooses manual, guided or automated). Users may point the installer to a local or remote autoinstall file and it will, depending on your configuration, fully or partially provision your machine. This encourages users to iterate on declarative configurations and makes for an efficient and, frankly, pretty cool experience – though I acknowledge it’s not NixOS cool yet! Lastly, these interfaces will respect a whitelabel.yaml to cater for specific aesthetic or branding requirements.

Sneak peak

In addition to reworking user journeys, the team has been evolving our existing designs to match:


Figure 5. A selection of WIP screens from stages 3-5.

Or better yet, here’s an very much WIP prototype of Stage 3 flow.

Why didn’t we do all of this for 23.10?

In theory it would have been great for this to land in 23.10, but practice is rarely straightforward because teams have to juggle shifting priorities and resource constraints. It’s important to note that Subiquity has been the default server installer since Ubuntu 18.04 LTS and has therefore undergone several cycles of testing and refinement. The Flutter-based GUI landed in 23.04 and so it has seen two releases of testing and refinement. As a result, much of this ‘new’ stack is more mature and better tested than it might appear at first glance.

As I mentioned in the opening paragraph, OEMs and managed use cases are vital to the longevity of Ubuntu. The better these experiences are, the healthier Ubuntu will be. So yes, while aspects of this could have been initiated earlier, following mostly well-trod code paths and the development and testing that have taken place over recent cycles gives us confidence in the foundations, and for the rest we are addressing requirements that we cannot ignore.

CONTRIBUTING.md

Your involvement here is invaluable, and so if you want to lend a hand, there are several ways you can:

  1. Report and suggest: We’re hoping to have the new experience in the dailies next week. So, when it lands your feedback is crucial, and if you have a suggestion for a fix or improvement, that’s even better :+1:
  2. Join the discussion: We have been experimenting with a public desktop-dev Matrix space. You’re welcome to join the developer-focused conversation there. My handle is @timhm:matrix.org.
  3. Share your ideas: If you have ideas for enhancements or suggestions for future features, I am listening. Your input helps define the direction of Ubuntu.
  4. Participate in testing and translations: As we approach the release of Noble Numbat, active participation during our testing weeks is important. The more testers we have, the better our release will be. Additionally, contributions to translations are equally valuable. Stay tuned for more specific information on how you can help there.

Closing

Hopefully in this short post I’ve convinced you of the benefit in expanding our perspective from installer to provisioning. I hope you agree that our new structure makes room for future enhancements and is going to be an awesome experience in Noble Numbat. I will follow up to this post soon with an update specifically for flavours and share some prioritisations we’ve made this week. I will link to that post when its available.

Thanks for your attention, enthusiasm and constructive engagement :pray:

Best,

Tim

14 Likes
3

You can find a dev update on this project here.

1 Like
4

FWIW, I have been provisioning my desktop-style machines using the Server ISO and autoinstall (using Foreman). The Ubuntu install is minimal, just defaults. On first boot, cloud-init installs Puppet Agent. Puppet then installs the ubuntu-desktop metapackage, which it knows to do based on the Puppet role of the machine, which was specified by selecting the right host group for the machine in Foreman.

2 Likes
5

This is exactly what we’re doing too. But these changes do look promising and could make the hole process a lot easier.

I can see this being very useful. Instead of building a custom ISO, I could just as easily tell our helpdesk to boot the standard ISO and point to a specific URL for the autoinstall.yaml.

3 Likes
6

I’m really interested in this. Currently we are using a pretty clunky python script I threw together to build our custom ISO from the base Ubuntu ISO.

When can we learn more about how it works? The github is looking pretty sparse right now.

1 Like
7

Exactly this :slight_smile:

I don’t have dates to share because its another team working on this but the sooner the better imo! And yes my hope is that in the future building ISOs will be declarative and straightforward :crossed_fingers:

1 Like
8

Hi tim! As we’re now approaching the release date, I wanted to ask if there was any part of this description that has not been included in the initial release.

1 Like
9

I’m assuming this is the same framework as the one in use with Ubuntu Server? Would this affect the ability to create btrfs subvolumes?

I just had an issue with btrfs on Ubuntu Server not creating subvolumes and just dumped everything on the partition. Any advice on moving the data from / to @?

btrfs subvol snapshot / /@
mount -o subvol=@ /dev/nvme0n1p2 /mnt
cd /mnt
mount -o bind /dev  dev
mount -o bind /sys  sys
mount -o bind /proc proc
mount -o bind /boot/efi boot/efi
chroot .
update-grub

Thankfully, it’s not impossible.

10

Hi ya! Sorry for the delayed response. Unfortunately we didnt’ get stage 4 and stage 5 to the maturity we wanted and so they aren’t included in the final release. Most of stage 3 landed as we wanted it but with a few tweaks and a few more design changes we’ll push to next cycle.

11

Hi Tim. Thanks for the great upgrade.
I’m trying to deploy 24.04 Desktop via iPXE, and set “shutdown: reboot” in user-data (autoinstall.yaml), but the pane of “Installation complete” is still there and waiting for users to click “Restart now”. :rofl:
Any sense about this or could you please share one sample yaml which really works well for Desktop :100:, thanks.

12

It looks like the images in the top post here are all 403’ing. Likely as a result of being hot-linked to Tim’s old Canonical Google account. Now Tim has left Canonical, all those links are broken as his Google account was disabled.

Can someone from Canonical extract the images and upload them in here, rather than hot-link them?

@aaronprisk maybe?

1 Like
13

I should gain ownership of those assets over the next week and can re-host them. Please bear with me a bit tho.

2 Likes
14

" 3. Device bootstrap: Once your computer has booted from this image you’re typically in a ‘live cd’ environment. At this stage we can bootstrap your computer for independent booting, including disk partitioning, encryption, bootloader installation, and copying the base OS onto the target device. This is what we previously called the installer."

Ubuntu now includes snaps. Snaps cannot be installed in a chroot environment. How does the installer “copy the base OS onto the target device” if chroot cannot be used? I have looked through the subiquity code on github and haven’t been able to identify how the initial install is achieved. Do you use debootstrap for the initial base OS transfer then cloud-init config for the first boot initialisation where the snap packages are then installed?

For context, I am the author of the ubuntu zfsbootmenu install script. The script installs ubuntu using debootstrap for the initial system install then installs ubuntu server / desktop using chroot. The chroot approach doesn’t look like it will work going forward with Canonical’s inclusion of snaps. Is there a “base OS” package that is snap free that I can install and then use a cloud-init approach to provision the system on first boot? What is the best practice approach?

1 Like