I’d be fine with a one-off manual edit, even if that invalidates the measurements; at worst I’d have to enter the recovery key, but the Desktop should come up, so I can snap revert pc-kernel or something like it, to force the regeneration of the correct grub.cfg. Or is there something like a forced reinstall/configure, equivalent to apt reinstall and dpkg-reconfigure?
@av66, can you post an excerpt of the Linux menu entries from your running Desktop?
What if this is an ordering issue? Perhaps /writeable gets mounted first, somehow, and then the read-only fs gets mounted on top of it?
It could be anything really (could as well be ordering but I guess it is mostly just two different implementations fighting over each other), the system you are now running is indeed a FrankenCore.
UbuntuCore boots essentially by mounting your seed partition, then assembling a rootfs from mounting the core snap inside that seed partition and bind mounting single files (or dirs) from /writable over any file that should be written to later in the running system (the list lives in /etc/system-image/writable-paths if you want to check what is actually writable on a core system)
Your mount output looks like it did mount the core snap but did not process the writable side of things (you would see these files in the mount output otherwise), there are also a lot of tmpfs mounts which smells like you even have half the initrd still around … the messed up update-grub results clearly got you in a weird state …
The grub.cfg used by Ubuntu Core is here. I tested running update-grub which overwrote /boot/grub/grub.cfg. On reboot I caught the system in the UEFI Firmware Settings menu, pressed c for the command line and run
chainloader /EFI/ubuntu/kernel.efi snapd_recovery_mode=run
boot
This booted without splash screen, I had to enter the recovery key in the text console and system booted into the full desktop. I downloaded the grub.cfg:
curl -LO https://raw.githubusercontent.com/canonical/snapd/refs/heads/master/bootloader/assets/data/grub.cfg
moved it into the /boot/grub/grub.cfg and rebooted. Now it was with normal splash screen, I had to enter the recovery key again and the next reboot was normal.
If you customized the kernel command line it is written in the /boot/grub/grubenv and you can load it at boot time with load_env command (see the grub.cfg).
My understanding it that grub.cfg is written when snapd runs in installer mode, maybe when it is updated.
Sorry, I didn’t make myself clear. I meant that the installed system goes into an unusable loop. The monitor and keyboard are still completely usable, predictable and reliable, e.g. I can access BIOS and GRUB.
Then perhaps we are misunderstanding each other? Define “unusable”, and keep in mind that said missing ttyS0 might be what’s causing your usability issues.
What I got was usable, but it was not a (recovery) desktop as advertised by the grub menu. The system, as a whole, pretty much stays unusable, because I’m unable to do proper recovery, but that is not the same as the recovery “desktop” being unusable.
Hmm, I somehow cannot make this work. But maybe we are talking about different situations. You seem to have run update-grub without the apt resinstall memtest86+ and ended up with a (second stage) grub menu containing only the “UEFI firmware settings”, like @paddylandau here.
I have since reinstalled the Desktop (“Erase disk and install”), to get to a clean slate state; wiped the TPM via the UEFI menu, offline install, no additional settings. On first boot, I immediatly entered the (first stage) grub menu (Esc) and booted the recovery desktop menu entry. I had also removed all console type devices from the VM to approximate @paddylandau’s setup in the other thread, so the boot splash gave way to the console login prompt. And, sure enough, those messages about the ttyS0 service kept spamming TTY1. BTW, one can switch to TTY2 (CtrlAltF2; “send keys” from virt-viewer’s menu) to evade that, which I did.
Now, though, I cannot login with my password.
As for the issue with the manual boot command not working, this is the (second stage) grub menu after I booted to the normal desktop (to confirm my password was correct), ran sudo apt reinstall memtest86+ and added a “Run Ubuntu Core” menu entry manually:
This is said “Run Ubuntu Core” entry; just a copy from GH, in the hopes that this would preserve the environment variables and act accordingly:
Running that gets me this:
Having been dropped back to the grub menu, I then entered CLI mode and input the command manually, as you suggested, @avb66:
error: no server is specified. on my first attempt. I then dusted off what little understanding I have of grub-shell and poked around to find the /boot partition. Still no luck. 
Also, this seems relevant (from my OP):
I don’t know how the LUKS module went missing and why it’s even supposed to be loaded. From what I learnt in the Streamlining secure boot for 26.10 thread, it shouldn’t even matter, because /boot is ext4 and the bootstrap of LUKS happens by initrd magic.
It is added by /etc/grub.d/20_memtest86+ because it thinks it needs access to the root which is encrypted.
You got errors because the grub.cfg already set $root to the LUKS container, just set it back to the ubuntu-boot partition.
Great, that worked. 
At least, it booted, but I am stuck in a loop. I get the recovery key prompt and the system reboots soon after I enter it; rinse and repeat.
But it’s progress, nonetheless.
Perhaps I should add that the VM is currently running with Secure Boot in setup mode, with an empty key database. But that should only matter for the TPM revealing the LUKS key. Entering the recovery key should eventually drop me to the desktop.
BTW, I just found this while going through the list of bugs on my watchlist. Even though, I had clicked “the button” on that one, which validated it, it was changed to invalid.
I think it should be made clearer that the recovery desktop never comes up, to begin with. One shouldn’t even be on the TTY by my definition of “Desktop”.
I’d have expected issues such as these to be fixable by a quick detour through recovery mode, but as it stands now, FDE/TPM is quite brittle. Be prepared to restore backups anytime.
I was finally able to fix this. At some point it hit me; why not just overwrite grub.cfg on the ubuntu-boot labeled partition from a live session? So I downloaded the file @avb66 linked and put it in /run/media/ubuntu/ubuntu-boot/grub/grub.cfg, but still no luck. After another reboot I just grep’d the boot partition for “Run Ubuntu Core” and found the real one the grub reads when chainloaded from the seed grub. After overwriting that with the copy from GH, I am now finally back to my desktop.
Steps to fix:
wget https://github.com/canonical/snapd/raw/refs/heads/master/bootloader/assets/data/grub.cfg
udisksctl mount -b /dev/disk/by-label/ubuntu-boot
# udisks mounts to /run/media these days
cd /run/media/ubuntu/ubuntu-boot/
for target in ./EFI/ubuntu/ ./grub/; do
sudo cp ~/grub.cfg "$target"
done
reboot
Eureka! 
Now I’m feeling kind of silly for not thinking of this sooner. Thanks to @avb66’s explanation and link, it should have been obvious. But all’s well that ends well. 
Just to make extra sure this doesn’t happen again:
sudo dpkg-divert --local --rename /usr/sbin/update-grub
As a stopgap, until this has trickled down.
I still miss a simple official way to rebuild the content of ubuntu-boot and/or ubuntu-seed. The grub.cfg is just one way it may go wrong. Those two partitions have no user content except kernel command line parameters. It should be technically straightforward to rebuild them, also from the live session.
Maybe those recovery options in grub menu are supposed to do something similar. Then they need to work even if ubuntu-boot is corrupted 
Oh, and it appears that memtest86+ cannot even be used on a TPM2 FDE system because there is no way to add it to the bootloader menu …
As it stands now, they don’t work at all, apparently, even when nothing is broken.
You’re right, hadn’t even occurred to me. But there are worse things, I guess. One can just use the one in the installer’s grub menu, if need be.
I think that you posted the wrong link!
Thanks! Fixed it. I had to use my comment after the change, because there is no comment link to copy on the status change; and my brain must have switched to github mode, where the links are on the left – autopilot gone haywire. 
I’ve seen your comment after mine; thanks for that as well. I’m still not quite at home on LP; didn’t have much occasion to use it until recently. If that doesn’t speak to the good quality of Ubuntu, nothing does. 
This topic was automatically closed 18 hours after the last reply. New replies are no longer allowed.