So I’ve followed those instructions (mostly) and here’s my experience. The ‘goal’ and measure of success will be:
- Use an external DHCP server that only serves IP’s to LAN1
- Admin and Test VM’s in LAN1
- TFTP/HTTP server in LAN2
- Generation 1 VM (BIOS) in LAN1 can PXE boot and pull image from TFTP server in LAN2
- Generation 2 VM (UEFI with Secure boot enabled) in LAN1 can PXE boot and pull image from TFTP server in LAN2
Pretty straight forward. With this setup, I would be able to PoC a workable solution. pfSense supports the network booting options necessary to pass on the next server, default BIOS file (pxelinux.0) and UEFI64bit file (bootx64.efi).
My PoC setup consists of the following:
- Using MS Hyper-V on Windows 10 Pro to test this all out
- 3 Virtual Network Switches (One “External Switch” (WAN), two “Private Switches” (LAN1 and LAN2))
- 5 Virtual Machines
- 1 Virtual machine with 3 virtual network adapters. Each adapter connected to one of the 3 switches. This Virtual machine is running pfSense and is my FW, DHCP server for LAN1 and router.
- 1 Virtual machine running Ubuntu Desktop 22.04 on LAN1. This is my (for a lack of a better word) admin system that I am using to validate connectivity, ssh/web browse/etc to all the other components.
- 1 Virtual Machine running Ubuntu Server 20.04 setup as per the instructions from https://www.molnar-peter.hu/en/ubuntu-jammy-netinstall-pxe.html minus the DHCP config.
- 1 Virtual Machine setup as a BIOS (Generation 1) system configured to boot from network.
- 1 Virtual Machine setup as a UEFI (Generation 2) system configured to boot from network.
So here are my results so far:
I’m able to get Generation 1 (BIOS) based systems to network boot, present me with a menu and install the OS. I would say that anyone following these instructions should be met with success for BIOS based systems.
Things kinda fell off the rails when it came to trying to get a Generation 2 (UEFI) system to network boot. I’ve documented my journey over on askubuntu.com ( https://askubuntu.com/questions/1406685/22-04-jammy-pxe-booting-help ) but very long story short, all I get is a GRUB> prompt. My setup isn’t reading the /boot/grub/grub.cfg file and I have no idea why.
In trying to resolve this issue:
- I’ve since moved the TFTP server from LAN2 to LAN1 removing the pfSense firewall out of the equation.
- I’ve tried using the EFI files found here: http://archive.ubuntu.com/ubuntu/dists/jammy/main/uefi/grub2-amd64/current/
- I’ve tried using the EFI files found on both the 20.04 and 22.04 ISO’s
- I’ve tried moving the EFI files into the /boot/grub directory and updating the DHCP server
- I’ve tried moving the grub files up into the /boot directory
What I ‘think’ the problem is, if you look at this screenshot https://i.stack.imgur.com/LrQiI.png you can see that my $prefex is set correctly. However last night before I gave up, while at the $GRUB prompt I did an ls comand and while (memdisk) was there, (tftp,172.16.1.3) was not. I don’t know if this is normal/expected but this is where I’m going to focus my efforts today.
Something else that I may try is to enable the DHCP server on the TFTP/HTTP server (and disable it on my pfSense firewall). There is mention of possibly needing to specify option arch code 93 = unsigned integer 16; in the dhcp scope. My understanding is that Option 93 would carry the architecture type (BIOS, UEFI, etc.) and is needed to help the logic of the DHCP if statements to determine which boot file to serve (pxelinux.0 or bootx64.efi).
Sorry for the wall of text but If you have time to take a look and see anything that I might have missed I’d appreciate it.