How to netboot the server installer on amd64

mwhudson · June 5, 2020, 3:45am

amd64 systems boot in either UEFI or legacy (“BIOS”) mode, and many systems can be configured to boot in either mode. The precise details depend on the system firmware, but both modes usually support the “Preboot eXecution Environment” (PXE) specification, which allows the provisioning of a bootloader over the network.

Steps needed

The process for network booting the live server installer is similar for both modes and goes like this:

The to-be-installed machine boots, and is directed to network boot.
The DHCP/BOOTP server tells the machine its network configuration and where to get the bootloader.
The machine’s firmware downloads the bootloader over TFTP and executes it.
The bootloader downloads configuration, also over TFTP, telling it where to download the kernel, RAM Disk and kernel command line to use.
The RAM Disk looks at the kernel command line to learn how to configure the network and where to download the server ISO from.
The RAM Disk downloads the ISO and mounts it as a loop device.
From this point on the install follows the same path as if the ISO was on a local block device.

The difference between UEFI and legacy modes is that in UEFI mode the bootloader is an EFI executable, signed so that is accepted by Secure Boot, and in legacy mode it is PXELINUX. Most DHCP/BOOTP servers can be configured to serve the right bootloader to a particular machine.

Configure DHCP/BOOTP and TFTP

There are several implementations of the DHCP/BOOTP and TFTP protocols available. This document will briefly describe how to configure dnsmasq to perform both of these roles.

Install dnsmasq with:
```
sudo apt install dnsmasq
```

Put something like this in /etc/dnsmasq.conf.d/pxe.conf:

interface=<your interface>,lo
bind-interfaces
dhcp-range=<your interface>,192.168.0.100,192.168.0.200
dhcp-boot=pxelinux.0
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-boot=tag:efi-x86_64,bootx64.efi
enable-tftp
tftp-root=/srv/tftp

Note:
This assumes several things about your network; read man dnsmasq or the default /etc/dnsmasq.conf for many more options.

Restart dnsmasq with:
```
sudo systemctl restart dnsmasq.service
```

Serve the bootloaders and configuration.

We need to make this section possible to write sanely

Ideally this would be something like:

apt install cd-boot-images-amd64
ln -s /usr/share/cd-boot-images-amd64 /srv/tftp/boot-amd64

Mode-independent set up

Download the latest live server ISO for the release you want to install:

wget http://cdimage.ubuntu.com/ubuntu-server/daily-live/current/focal-live-server-amd64.iso

Mount it:

mount ubuntu-19.10-live-server-amd64.iso /mnt

Copy the kernel and initrd from it to where the dnsmasq serves TFTP from:
```
cp /mnt/casper/{vmlinuz,initrd} /srv/tftp/
```

Set up the files for UEFI booting

Copy the signed shim binary into place:

apt download shim-signed
dpkg-deb --fsys-tarfile shim-signed*deb | tar x ./usr/lib/shim/shimx64.efi.signed -O > /srv/tftp/bootx64.efi

Copy the signed GRUB binary into place:

apt download grub-efi-amd64-signed
dpkg-deb --fsys-tarfile grub-efi-amd64-signed*deb | tar x ./usr/lib/grub/x86_64-efi-signed/grubnetx64.efi.signed -O > /srv/tftp/grubx64.efi

GRUB also needs a font to be available over TFTP:

apt download grub-common
dpkg-deb --fsys-tarfile grub-common*deb | tar x ./usr/share/grub/unicode.pf2 -O > /srv/tftp/unicode.pf2

Create /srv/tftp/grub/grub.cfg that contains:

 set default="0"
 set timeout=-1
     
 if loadfont unicode ; then
   set gfxmode=auto
   set locale_dir=$prefix/locale
   set lang=en_US
 fi
 terminal_output gfxterm
     
 set menu_color_normal=white/black
 set menu_color_highlight=black/light-gray
 if background_color 44,0,30; then
   clear
 fi
     
 function gfxmode {
         set gfxpayload="${1}"
         if [ "${1}" = "keep" ]; then
                 set vt_handoff=vt.handoff=7
         else
                 set vt_handoff=
         fi
 }
     
 set linux_gfx_mode=keep
     
 export linux_gfx_mode
     
 menuentry 'Ubuntu 20.04' {
         gfxmode $linux_gfx_mode
         linux /vmlinux $vt_handoff quiet splash
         initrd /initrd
 }

Set up the files for legacy boot

Download pxelinux.0 and put it into place:

wget http://archive.ubuntu.com/ubuntu/dists/eoan/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/pxelinux.0
mkdir -p /srv/tftp
mv pxelinux.0 /srv/tftp/

Make sure to have installed package syslinux-common and then:
```
cp /usr/lib/syslinux/modules/bios/ldlinux.c32 /srv/tftp/
```

Create /srv/tftp/pxelinux.cfg/default containing:

 DEFAULT install
 LABEL install
   KERNEL vmlinuz
   INITRD initrd
   APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://cdimage.ubuntu.com/ubuntu-server/daily-live/current/focal-live-server-amd64.iso

As you can see, this downloads the ISO from Ubuntu’s servers. You may want to host it somewhere on your infrastructure and change the URL to match.

This configuration is very simple. PXELINUX has many, many options, and you can consult its documentation for more.

tomreyn · November 21, 2020, 5:36pm

It would seem that

We need to make this section possible to write sanely

is longing for a better replacement (this may have been forgotten, and made it into the current LTS server manual).

mgedmin · November 22, 2020, 3:06pm

I think this is supposed to be /etc/dnsmasq.d.

ludmata · February 21, 2021, 10:28pm

Hi, after spending the entire evening trying to make pxe server for ubuntu focal server, I think your article is not 100% accurate. I am using HyperV Ver2 Secure boot vritual machines, so after I made the initial config of pxe I struggled with booting ubuntu focal - the most far I could get was “unable to find a medium containing a live file system”. After spending much time experimenting with various configs it downed on me that obviously i need to
change the menu entry in grub.cfg

menuentry 'Ubuntu 20.04' {
        gfxmode $linux_gfx_mode
        linux /boot/casper/vmlinuz ip=dhcp url=http://mirror/focal/ubuntu-20.04.2-live-server-amd64.iso quiet ---
        initrd /boot/casper/initrd
}

matt-causey · April 6, 2021, 11:46pm

Another question I have after working with this a bit. I can see that the ISO gets downloaded 3 times over the network when doing an automated install. Is there a way I can make it not do that?

192.168.1.225 - - [06/Apr/2021:22:09:47 +0000] "GET /ubuntu-20.04.1-live-server-amd64.iso HTTP/1.1" 200 958398464 "-" "Wget"
192.168.1.225 - - [06/Apr/2021:22:13:24 +0000] "GET /ubuntu-20.04.1-live-server-amd64.iso HTTP/1.1" 200 958398464 "-" "Cloud-Init/20.2-45-g5f7825e2-0ubuntu1~20.04.1"
192.68.1.225 - - [06/Apr/2021:22:16:50 +0000] "GET /ubuntu-20.04.1-live-server-amd64.iso HTTP/1.1" 200 958398464 "-" "Cloud-Init/20.2-45-g5f7825e2-0ubuntu1~20.04.1"

rbasak · April 28, 2021, 11:46am

The link to pxelinux.0 is broken. http://archive.ubuntu.com/ubuntu/dists/focal/main/installer-amd64/current/legacy-images/netboot/ might work, but it would be better to make sure that a proper file extracted from the release is made available to users.

quashee · March 28, 2022, 9:57pm

apt install cd-boot-images-amd64

This only works with Ubuntu 21.04 and above.

Gerard · March 8, 2023, 12:45pm

Daily server is now https://cdimage.ubuntu.com/ubuntu-server/daily-live/current/lunar-live-server-amd64.iso

axino · September 29, 2023, 2:58pm

Used for to netboot a jammy server.

The instructions extract the kernel in a vmlinuz file but grub.cfg uses vmlinux (z vs x), so either needs to be fixed.

The signed shim is now in ./usr/lib/shim/shimx64.efi I think - at least ./usr/lib/shim/shimx64.efi.signed doesn’t exist anymore.

Also if you want to run this from the serial console, you’ll have to tweak grub.cfg a bit. This worked for me :

set default="0"
set timeout=-1

if loadfont unicode ; then
  set gfxmode=auto
  set locale_dir=$prefix/locale
  set lang=en_US
fi

serial --unit=0 --speed=115200 --stop=1
terminal_output serial
terminal_input serial

#set menu_color_normal=white/black
#set menu_color_highlight=black/light-gray
#if background_color 44,0,30; then
#  clear
#fi

function gfxmode {
        set gfxpayload="${1}"
        if [ "${1}" = "keep" ]; then
                set vt_handoff=vt.handoff=7
        else
                set vt_handoff=
        fi
}

set linux_gfx_mode=text

export linux_gfx_mode

menuentry 'Ubuntu 22.04' {
        gfxmode $linux_gfx_mode
        linux /vmlinuz console=ttyS0,115200n1 root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://10.1.1.1/jammy-live-server-amd64.iso
        initrd /initrd
}

dannf · February 5, 2024, 6:25pm

There are netboot tarballs now for newer releases, e.g.:

https://releases.ubuntu.com/23.10/ubuntu-23.10-netboot-amd64.tar.gz
https://cdimage.ubuntu.com/releases/mantic/release/ubuntu-23.10-netboot-arm64.tar.gz

We should document those.