Groovy to use GRUB2 for booting installer media in any modes on all architectures

Foundations
,
1

With upcoming changes to scripts used to produce all installation media, the bootloaders used to boot will change.

Previously we used ISOLINUX with GFXBOOT when booting in BIOS mode. Grub2 when booting in UEFI mode.

With the above change, only GRUB2 will be used to boot the installation media.

It means that installation experience will be the same, if one boots CD-ROM, USB, under either UEFI or BIOS. The same grub.cfg will be used in all cases, and the same menu will be shown and the same kernel cmdline options will be used.

This will change how some things appear, however the installer will now behave a lot closer to how the installed systems boot.

There are no changes to plymouth, plymouth themes, actual installers.

All existing boot targets will continue to be supported. However, if you notice that Groovy installers stop booting after the above mentioned changes land, then please respond to this thread with details of how the installation media is prepared, how it is booted, and on which hardware configuration.

This change affects all flavours.

7 Likes
2

:frowning: I for one, will miss the bright cheery screens of groovy boots on my BIOS boxes. The menu may have only been on screen briefly, but the background color & logo told me I wrote the correct flavor ISO.

It makes sense though.

I suspect most won’t notice anyway; not using museum aged hardware…

1 Like
3

Is there chance for more broad UEFI support in groovy for AARCH64 devices?

Unlike Debian aarch64 images, ubuntu-20.04-live-server-arm64.iso doesn’t even detected as bootable media on EDK2 Raspberry Pi UEFI firmware and on Snapdragon-based laptops too, because"EFI" folder has been missing from Ubuntu images, while it’s present in Debian images. I can even copy “EFI” folder from Debian aarch64 image to usb drive with ubuntu-20.04-live-server-arm64.iso and it will boot on EDK2 RPi UEFI, but why it’s not present in ubuntu-20.04-live-server-arm64.iso in first place?

4

“This change has now landed.” => this refers to the topic of the post obviously, that AMD64 in Groovy is to switch to using grub2 for booting under both bios and uefi.

5

I’d like to know more.

By default our ARM64 isos boot as either CD-ROM or USB-DISK, under EDK2, ARM64 servers, Snapdragon based laptops, and QEMU-KVM virtual machines with UEFI EDKII firmware that we build and provide for virt-manager.

We also have ready made ARM64 cloud-images that one can simply boot in-place with cloud-config metadata, to access ARM64 Server VM.

For RaspberryPI we provide preinstalled images that use gpu/uboot bootloader, as seen on actual raspberry pi boards.

I will double check that none of these things have regressed?

Why are you using EDK2 Raspberry Pi UEFI firmare? And are you using it ina VM or on the Pi image itself?

Note that our .iso are multiple partitions, and the EFI folder is present correctly, on the EFI parition. You can see all partiitions, and all contents of our ISOs by using qemu-nbd to connect it, and then mount. I.e.

$ sudo qemu-nbd -c /dev/nbd0 ubuntu-20.04-live-server-arm64.iso
$ sudo mkdir /media/isop1 /media/isop2
$ sudo mount /dev/nbd0p1 /media/isop1
$ sudo mount /dev/nbd0p2 /media/isop2
$ tree /media/isop2
/mnt/
└── efi
    └── boot
        └── bootaa64.efi

2 directories, 1 file

Note the ARM64.isos use the generic ARM64 kernel, which does not have raspberry pi support.

The Raspberry Pi images that are preinstalled classic server use a linux-raspi kernel build specifically for all raspberry pi boards with hardware support for all versions of the boards.

6

Once plymouth starts, you do see flavour specific logo at that point. It is slightly later.

Also the new grub menu have Flavour name at the very start, so has a clue as to what one is booting then as well.

7

Last time I checked 20.04 images half year ago - it have 128MB firmware partition that hold both of RPi’s firmwares and /boot, so attempt to just install updates failed due to partition size (old Linux files rename to .bak, new initrd can’t fit, etc.)

Last time I used it due to mentioned issues with rpi -specific images.

I used EDK2 Raspberry Pi UEFI with regular (not rpi-specific) aarch64 image on hardware, not in VM.

Well, it works in general, including GPU accelerated rendering (vc4 driver works). But, sure, I probably missing some functions I don’t need on RPIs.

Interesting, I didn’t know about that. Due to lack of usb-creator packages for aarch64 I can not use usb-creator on my laptop, so I tried to create bootable usb drive in the same way that used to work with amd64 iso - open iso in mc and just copy all files to FAT32 parition on usb drive. I guess it’s expected that mc ignore second partition, so I never seen it and assumed EFI folder is not there for some reason.

8

I’m having trouble with todays daily (lubuntu groovy)

I’ve confirmed my ISO is valid (re-ran zsync anyway as the link on http://iso.qa.ubuntu.com/qatracker/milestones/413/builds/216043/downloads won’t provide md5sum for me to do it myself)

The ISO was written twice to two thumb-drives. On a number of boxes it’s wanting me to download aka https://bugs.launchpad.net/ubuntu-cdimage/+bug/1883040 however it’s done that on boxes not impacted by that bug, which makes me think thumb-drive/squashfs errs related. (Further I noted @leok has recorded a valid test today which means a real issue is unlikely, thumb-drives & me not ruled out)

On the sony crapbook (sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)) the first window appears rather small (about 8x5 cm wide on the ~25x14.5 cm screen) where it used to use the whole screen. photo

2 Likes
9

If I am doing my iso testing correctly, a serious bug is affecting all current daily groovy iso files. We try to describe the bug in this bug report:

Cloning from the iso file to the drive seen as a device, e.g. /dev/sdb does not work.

Please let us know if we should use a new method to create a USB boot drive from these new iso files, and in that case describe that method.

2 Likes
10

It’s best to “burn .iso” onto the usb stick. On Ubuntu, Windows, MacOS X one can use disks to “restore a disk image” that generally works to correctly effectively dd the iso onto the usb stick / sd card.

1 Like
11

I think you can try to install linux-raspi package to try the raspi specific kernel. It might be missing EFI stubs, if it doesn’t boot at all, we could fix to enable raspi flavour to be EFI bootable too.

1 Like
12

MD5SUMS are insecure, one should only validate SHA256SUMS that are available from the same directory where the .iso is.

I will follow up where/why ISO tracker is not pushing SHA256SUMS urls.

1 Like
13

So with my terminology cloning from the iso file to the drive seen as a device, e.g. /dev/sdb should work also with this new boot structure.

Then there is a serious bug that makes us fail to run a live session using the current daily groovy iso files in almost all of our PC computers.

Edit: The current daily Ubuntu Groovy Desktop file works for me in VirtualBox, when booted from the iso file seen as a virtual optical drive.

But it does not work in the same computer (Dell Precision M4800) on real metal, neither in UEFI mode nor BIOS mode, when cloned from the iso file to a USB drive.

2 Likes
14

Thanks to @jibel the iso tracker URLs are updated to point checksums URLs at the SHA256SUMS.

I.e. see

http://iso.qa.ubuntu.com/qatracker/milestones/413/builds/216520/downloads

For an example.

ps. i know the heading says “MD5 checksum” to fix that, we will need to do like a database migration so will take longer.

2 Likes
15

All this is great, but bug 1885414 prevents installation of Groovy on MBR/legacy systems.

2 Likes
16

I hear you, I will try to investigate this further and/or escalate it with Dell.

2 Likes
17

We landed a few fixes. They should all be in the 20200709 build-stamp directories or higher.

Can you please try 20200709 images and let me know if there are any improvements? (fingers crossed)

1 Like
18

Tried with dd and with x86 usbcreator - unfortunately, UEFI on both of RPi3B and Lenovo Yoga C630 WOS does not detect flash drives created from ubuntu-20.04-live-server-arm64.iso as bootable devices. If you decide to try it yourself I would like to remind that only left port of Lenovo Yoga C630 WOS is bootable and Ubuntu stock kernel does not boot on Yoga C630 WOS: Bug #1863939 “Linux 5.4 from focal repo does not boot on Lenovo ...” : Bugs : linux package : Ubuntu

I tried to remove all generic kernels and installed linux-raspi. During linux-raspi installation flash-kernel showed errors:

flash-kernel: installing version 5.4.0-1013-raspi
Installing new vmlinuz.
mv: cannot move ‘/tmp/flash-kernel.vxattpQW/vmlinuz’ to ‘/boot/firmware/vmlinuz’: No such file or directory
dpkg: error processing package flash-kernel (–configure):
installed flash-kernel package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
flash-kernel
E: Sub-process /usr/bin/dpkg returned an error code (1)

By the way, flash-kernel package somehow always get installed with aarch64 kernel updates, and it’s always source of errors on Yoga C630 WOS. Even if I remove it and hold it - it get installed on kernel upgrade.

Anyway, back to linux-raspi. Unlike list time I tested it, this time it boot, but:
RPi integrated Ehternet adapter does not work.
None of USB devices attached via USB hub (Belkin F4U093) is detected. Including Belkin’s Ethernet adapter.

generic kernel dmesg: Ubuntu Pastebin
raspi kernel dmesg: Ubuntu Pastebin

If you need it, I can publish dd image of microSD with working combination of raspi firmwares and UEFI binary build, that works well together (more details).

19

@xnox,

Do you want the dialogue about the boot bug in PC computers to continue here, or is it enough at the bug report #1886148?

20

It looks like yeah, this is causing unstable issues, you can check the family on Groovy Daily QA:

http://iso.qa.ubuntu.com/qatracker/milestones/413/builds

1 Like