Ubuntu, UEFI, BIOS, grub, and confusion

kcf.maguire · January 2, 2025, 4:04pm

Hi

I have a mini PC, and ages ago installed Windows 11 onto the internal M.2 drive
AND Ubuntu 22.04 LTS onto an external USB drive. I don’t actually remember in which order I did this. The boot drive, and default boot, was Ubuntu 22.04, i.e. the external USB drive, which was first in the boot order in the AMI BIOS. grub was the bootloader, and offered me Ubuntu or Windows, on rare occasions I wanted to boot into windows I just selected it.

A while ago I physically moved the computer, and after moving it I did not use it for several weeks. Today I did use it, but forgot to connect the USB drive, forgot I had even used a USB drive, and the system would not boot. After a bit of confusion I (stupidly) reset BIOS to factory defaults. It did not help, and was likely unwise.

Then I remembered about the USB boot drive, connected it, and Ubuntu boots fine. The BIOS is in UEFI “boot mode”. As system had not been running for a while, there were loads of Ubuntu updates, whcih installed fine. This included kernel and grub updates.

On next boot I noticed Windows was no longer in boot menu. Curious …

Then in Ubuntu I noticed these errors from the M.2 drive:

Jan  2 13:40:01 u2023 kernel: [    2.063562]  nvme0n1: unable to read partition table
nvme0n1: unable to read partition table
Jan  2 13:40:01 u2023 kernel: [    1.282679] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.328783] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.391818] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.437320] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.485396] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.530219] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.575634] Buffer I/O error on dev nvme0n1, logical block 0, async page read
Jan  2 13:40:01 u2023 kernel: [    1.576149] Dev nvme0n1: unable to read RDB block 0

Looking at previous logs:

Oct 25 12:51:59 u2023 kernel: [    1.184875] nvme nvme0: pci function 0000:01:00.0
Oct 25 12:51:59 u2023 kernel: [    1.188952] nvme nvme0: missing or invalid SUBNQN field.
Oct 25 12:51:59 u2023 kernel: [    1.198526] nvme nvme0: allocated 16 MiB host memory buffer.
Oct 25 12:51:59 u2023 kernel: [    1.207461] nvme nvme0: 8/0/0 default/read/poll queues
Oct 25 12:51:59 u2023 kernel: [    1.209318] nvme nvme0: Ignoring bogus Namespace Identifiers
Oct 25 12:51:59 u2023 kernel: [    1.213962]  nvme0n1: p1 p2 p3 p4 p5

Now I don’t think the M.2 drive is broken, I think I’ve somehow made it inaccessible by resetting the BIOS and losing some important setting(s)? But truly, though I understand Linux quite well, I dont understand UEFI topics well at all, so maybe the M.2 “drive” is indeed simply broken.

How to troubleshoot this?

I don’t have any other device that has a M.2 slot.

TIA for any advice, Kevin

oldfred · January 2, 2025, 4:20pm

Lets see details on your configuration. Please copy & paste the pastebin link to the BootInfo summary report ( do not post report), do not run the auto fix till reviewed. Use often updated ppa version with your USB installer or any working install over somewhat older ISO.
https://help.ubuntu.com/community/Boot-Repair &
https://sourceforge.net/p/boot-repair/home/Home/
I typically recommended updates to UEFI & SSD/NVMe firmware. But that often resets UEFI to defaults & you may need to redo settings. My new system needs no changes, but older one had at least 7 settings in UEFI that I had to redo, so I kept a list. Varies a lot by UEFI firmware vendors.
Report will show, but important that Windows & Ubuntu are both in same boot mode, or both UEFI, or if old system both BIOS.
Windows updates may turn bitlocker or fast startup/hibernation back on which prevents the Linux NTFS driver from seeing the Windows partitions. Make sure Windows fast startup is off.
https://www.elevenforum.com/t/turn-on-or-off-fast-startup-in-windows-11.1212/

kcf.maguire · January 2, 2025, 6:49pm

Hi Fred

Thanks for responding.

In meantime I had temporarily removed the M.2 disk/card, so I ran boot-repair first while it was out, uploaded to pastbin, re-inserted the M.2 and did same again. So the 2 files are:

Ubuntu Pastebin - no M.2 disk in system

Ubuntu Pastebin - WITH M.2 disk in system

Note that the PC has/had not booted into windows for several months.

Obviously I made 2 big mistakes, one booting the system several times (while confused) without the boot disk even attached, brain fail, and secondly when doing so and in the AMI BIOS just accepting a “rewert to defaults” suggestion from BIOS without thinking. I have no recollection of what I might have changed from the defaults, PC installed was in 2023 I think.

Windows & Ubuntu are both in same boot mode

Well, for the several months when I was regularly dual0-booting, I just arrowed down in the grub menu to choose Windows, so I presume both were in same boot mode.

I should also get a borrowed and usable M.2 card to test tomorrow, just to see if I can even read a single block of that device.

oldfred · January 2, 2025, 7:29pm

Report barely sees NVMe drive.

nvme0n1:512GB:nvme:512:512:unknown:AirDisk 512GB SSD:;

But does not show your typical Windows partitions. Either bitlocker is on or some other issue.
Does UEFI settings show M.2 drive correctly?
Is AirDisk the brand, and does it have firmware updates?
UEFI entry for Windows does not look like a typical UEFI boot entry. Most UEFI lose UEFI setting when drive is removed, and again most UEFI find Windows UEFI from ESP, but not Linux entries.
You either have to use efibootmgr to create new UEFI entry or reinstall grub. Your system looks like it does not lose Ubuntu entry, so I would not expect it to then lose Windows when you unplug M.2 drive.

yancek · January 2, 2025, 7:32pm

Not necessarily true if they are on separate physical drives. Need to be the same mode if they are on the same device. The errors you report would indicate the drive is failing. There is no sign of a windows install in your boot repair output other than the UEFI Boot entry which is in the firmware of the motherboard. I don’t know what option you chose in resetting to default but usually a recovery option will set to the way it came from the factory but whatever you did seems to have obliterated windows entriely. You might try your windows recovery disk to see if it gives you any option to repair.

kcf.maguire · January 2, 2025, 7:45pm

Hi

Yeah, I don’t expect to lose anything on Linux.

The M.2 drive is just the bog standard one that came with the mini PC, which was pretty cheap (250 euros or so for the whole system). It might have had bitlocker, but I doubt it. I cannot even dd a single byte from the disk.

Does UEFI settings show M.2 drive correctly?

Well, within the AMI BIOS I can see the disk type and size OK. As I said, I don’t really understand UEFI well, there are minimal settings within the BIOS. Are there other Linux utilities I can try to find out more ?

kcf.maguire · January 2, 2025, 8:31pm

Mmm - I think the M.2 drive is maybe just broken. In grub I did

ls

and I see the 2 disks

(hd0). the working drive, with its various (hd0,gptX) partitions

and (hd1), the probably broken one.

It even gives read errors in grub menu, strangely for sector 0x3b9e12a0 and also 0x0.

Could bitlocker make the drive completely unreadable? I thought it just encrypted the data. The windows installed was Windows 11, and I didn’t pay much attention whether it was encrypting the data or not. But I did now find an old email to myself with a bitlocker recovery key in it …

… now to find out how to use that key!

yancek · January 2, 2025, 8:37pm

The second boot repair output which you indicate was run with the windows drive attached shows on line 33, 1 OS detected and that is Ubuntu. Beginning at line 45, you see the eifbootmgr output which shows an entry for windows as well as ubuntu. That is the only indication of windows.

If you boot Ubuntu and run sudo parted -l and/or sudo fdisk -l, it should show drives and partitions for any working devices attached at the time. If you had a recovery partition for windows and selected an option to restore to factory defaults, it should have done that and you would have had your windows but any changes made while installed would have been overwritten and I believe any data would have also been overwritten. I don’t use windows so can’t verify it .

Do you have a windows recovery or install disk/usb? If so, you should get an option to repair when you boot it. If you don’t have a recovery device you can download the windows iso and write it to a usb to boot and either try to repair or reinstall. Do you know how to access the windows site to download the iso? Boot repair and other Linux tools cannot ‘repair’ windows boot files. What boot repair does to try to boot windows is to add a chainload entry to grub.cfg so if the necessary file(s) are missing or corrupted, there is nothing boot repair can do to fix that.

The link below is to the windows site from which you can download the windows iso. Do NOT bother with the Installation Assistant or the Media Creation tool as they only work on windows. You can download the iso from that site. It is 5.8GB and you can either create a partition on a flash drive or on your Ubuntu drive, format it as ntfs and then loop mount the iso and copy all the files to that partition. If you write it to a usb you would just set the usb to first boot in the BIOS to boot. If you copy it to a partition on your ubuntu drive, you will need to put an entry in the grub.cfg file on Ubuntu. Just did this last week and know it works. The link I posted is for windows 11. You did not post that I can see which release of windows you are using. The example entry below is what you would use if you copy to an ntfs partition on your drive. You need to change the (hd1,9) part to show the correct drive and partition.

menuentry ‘Windows Boot Manager’ {
insmod part_gpt
insmod ntfs
chainloader (hd1,9)/efi/boot/bootx64.efi
}

kcf.maguire · January 2, 2025, 8:49pm

Thanks yancek. I think we were typing responses here simultaneously.

Note I have found an old email with a bitlocker recovery key.

I certainly can make a windows boot disk on USB (btw I think you forgot to add the “link below”) - I used file from

https://www.microsoft.com/en-us/software-download/windows11

kevin@u2023:~/Downloads$ sha256sum Win11_24H2_EnglishInternational_x64.iso 
d5a4c97c3e835c43b1b9a31933327c001766ce314608ba912f2fffc876044309  Win11_24H2_EnglishInternational_x64.iso
kevin@u2023:~/Downloads$ ls -l Win11_24H2_EnglishInternational_x64.iso 
-rw-rw-r-- 1 kevin kevin 5832091648 Jan  2 21:47 Win11_24H2_EnglishInternational_x64.iso

I’d need to shrink a filesystem to use my Ubuntu disk so easier use a flash drive.

yancek · January 2, 2025, 9:18pm

Yes, I forgot to post the link but you have the correct one and it is the one I used to create the bootable ntfs partition. If you have the iso in your user Downloads directory loop mount it after creating a directory as a mount point. mkdir iso; sudo mount -o loop Win11_24H2_English_x64.iso iso. If your flash drive is sdb, use gparted or another partition manager, create a gpt partition table on the device, create a partition and format it as ntfs, then from the Downloads directory create a mount point for that partition, mkdir windows and mount it. sudo mount /dev/sdb1 windows Lastly, do the copying. sudo cp -r iso/* /windows. These commands would all be done in your user Downloads directory.

kcf.maguire · January 2, 2025, 10:05pm

My spare partition is /dev/sda4, so …

root@u2023:~# mkfs.ntfs /dev/sda4
Cluster size has been automatically set to 4096 bytes.
Initializing device with zeroes: 100% - Done.
Creating NTFS volume structures.
mkntfs completed successfully. Have a nice day.

root@u2023:~# mkdir /windows

root@u2023:~# mount /dev/sda4 /windows/

root@u2023:~# mount | grep /windows
/dev/sda4 on /windows type fuseblk (rw,relatime,user_id=0,group_id=0,allow_other,blksize=4096)

root@u2023:~# mount -oloop,ro ~kevin/Downloads/Win11_24H2_English_x64.iso /iso

root@u2023:~# cp -r /iso/* /windows/
root@u2023:~#

and my grub entry is:

menuentry WinRec {
insmod part_gpt
insmod ntfs
chainloader (hd0,4)/efi/boot/bootx64.efi
}

but when I select that from grub menu I get error below (note the case in errors is different from what I see of that tree on linux, where everything is 100% lower-case - don’t know of that’s important.

root@u2023:~$ sudo ls /windows/efi/microsoft/boot/bcd 
/windows/efi/microsoft/boot/bcd

yancek · January 2, 2025, 11:25pm

You should have been running each of the commands from your user Downloads directory so if you did that, you would not have the forward slash in front of windows or iso.

Run this command: sudo mkdir iso windows This should create both directories in the Downloads directory and you should then see both directories created there. While in the Downloads directory after running these commands, run: ls and you should see them. Also, in the loop mount command, you do not put the forward slash in front of iso and you don’t need the path if you are in the Downloads directory.

root@u2023:~# mount -o loop,ro Win11_24H2_English_x64.iso iso

You need to mount sda4: sudo mount /dev/sda4 windows
The same problem with the copy command, do not use a forward slash if you are in the Downloads directory, rather do:
cp -r iso/* windows/

The image you posted means the usb was not created correctly, is not bootable. Try the steps I’ve posted here and don’t change the path, leave it as below.

chainloader (hd0,4)/efi/boot/bootx64.efi

If you get an error similar to file not found you may need to change the (hd0.4) to (hd1,4) as the priority changes in certain situations. I don’t think it will here.

kcf.maguire · January 3, 2025, 10:10am

Hi

Thanks for responding again. Respectfully, as I was working as root at that time,

I loop mounted the ISO file at /iso
I mounted /dev/sda4 (NTFS partition) at /windows

I copied as above - cp -r /iso/* /windows

This does EXACTLY the same as if I’d mounted somewhere else in the directory tree, e.g. in a users Downloads folder, and did the cp without forward slashes there, as is seen below:

root@u2023:~# ls /iso/ /windows/
/iso/:
autorun.inf  boot  bootmgr  bootmgr.efi  efi  setup.exe  sources  support

/windows/:
autorun.inf  boot  bootmgr  bootmgr.efi  efi  setup.exe  sources  support
root@u2023:~# df -i /iso/ /windows/
Filesystem      Inodes IUsed   IFree IUse% Mounted on
/dev/loop16       1063  1063       0  100% /iso
/dev/sda4      4539588  1084 4538504    1% /windows
root@u2023:~# df -k /iso/ /windows/
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop16      5682482 5682482         0 100% /iso
/dev/sda4       10239996 5733176   4506820  56% /windows
root@u2023:~#

(the slight differences in space used are due to the different filesystems)

The chainloader path (hd0,4)/efi/boot/bootx64.efi

is definitely correct and is what I used, and it obviously loads that file correctly, but then does not find the BCD file.

yancek · January 3, 2025, 4:05pm

My suggestion with the mkdir commands was so that they would be in the Downloads directory. Logged in as a user in a terminal in to the Downloads directory one can run: mkdir iso windows and it will create both the iso and windows directory mount points in the Downloads directory. mkdir /iso /windows as a user will give you a permission denied error. If you use sudo it will then create the directories in the / of the filesystem not in the Downloads directory. Either way should work if you use the cp commannds as you show with the forward slashes.

If you have the extracted windows iso on sda4 mounted at /windows, do you see the bcd file when you do: ls /windows/boot That file exists on the partition where I have the windows extracted iso mounted on my system. What is the exact error you see when you try to boot?

kcf.maguire · January 3, 2025, 4:38pm

Hi

Some answers and updates. Note I was typically working as root. Not a big fan of sudo, sign of my age perhaps - sudo had a terrible reputation for poor security going back to 1990s, and if you give user x the right to run every command via sudo, then he might as well be root IMO.

sudo more fitted to limiting access to some very specific commands , each with a good reason).

The screenshot of what i see on boot is above, the error about the \EFI\Microsoft\Boot\BCD.

I subsequently just wrote an ISO image of windows 11 installer to a thumb drive, that boots, and installer says my computer does not meet windows 11 requirements. I thought it was Windows 11, but maybe it was windows 10.

I am 99% sure the old M.2 drive/stick is just broken/unreadable now.

My new M.2 drive arrived and is physically installed. I can see it under Linux, where it does not show as a NVME device, rather as another SATA drive, sda in fact. Not that I really care about that.

My plan now is basically to move my Linux installation, via clean Ubuntu 24.04 installation and then copy of personal files from the external USB drive, onto this new internal M.2 drive. And either just forget about windows or install win10 later to an external drive.

But process has been painful. I will need to take time to learn how the UEFI stuff works in more detail, and what tools there are to troubleshoot.

vidtek99 · January 3, 2025, 6:53pm

Your final post shows a satisfactory resolution. As far as using windows rescue discs I have never had any luck in getting them to actually do what they profess to do, even using bootrec usually throws up errors and doesn’t work nine times out of ten - it has permission issues or missing files, it invariably takes longer messing around with the windows “fixes” than it does to install from scratch and you never know if it’s going to work. Always hit and miss.

If you want to know about UEFI just ask @oldfred , he is the guru of UEFI, booting and grub.

If you go into your bios, you may find a setting which allows you to have varying “profiles” usually meant for overclocking. When I first set up my machine, I use this to save my settings, profiles are preserved even after a “reset to defaults” in your bios. Cheapie mobos may not have them but it’s worth a look.
Cheers Tony.

yancek · January 3, 2025, 7:59pm

I don’t know why you were getting that error when trying to boot the windows installer as I used the same method and had no problem booting it from an external hard drive. I just test booted it again, no problem. With windows mounted at /windows, there is a bcd file in both /windows/efi/microsoft/boot/bcd and /windows/boot/bcd

Maybe the copy wasn’t successful but that’s just a guess. Using the flash drive got you the solution so that’s good.

oldfred · January 3, 2025, 10:25pm

There are two kinds of M.2 drives. SATA & NVMe. When I built my desktop in 2017, motherboard had the M.2 connector which was new to me. But back then NVMe drives were very expensive. So I installed a smaller SSD. A year or two ago I upgraded M.2 drive and found NVMe not all that more expensive, but while supposedly faster for my use I barely notice any difference. But best thing I did was put old M.2 drive in a USB3 to M.2 adapter. Found that so much faster than the many flash drives I have, I now use external SSD for most things. Full install, loopmount ISO, and some backup of data.

kcf.maguire · January 4, 2025, 1:58pm

Hi

Thanks @oldfred

Yes, the old/broken one was the NVMe type, and new one is the SATA type. I cannot notice any difference either, but in fairness the old one had only barely used Windows on it, seemed to work great, until it didn’t work at all

In meantime I also bought one off those adapters. Inserted the old NVMe drive and attach to USB on Ubuntu system and it gives output below. It looks just broken. Unless there is some UEFI/Windows/SecureBoot/bitlocker magic can make the drive utterly inaccessible - which I really doubt.

My recollections return - the mini PC, from NiPoGi via Amazon, did come with Win 11 pre-installed. Strange that now when I try to install Win 11 on it now, it says the hardware is not supported - weird. Win 10 also not, says I need other drivers. I will need to ask the vendor, not a Q for Ubuntu Discourse.

https://www.amazon.de/-/en/dp/B0BY25NQGP?ref=ppx_yo2ov_dt_b_fed_asin_title

My new Ubuntu 24.04 LTS installation onto the now M.2 SATA is looking good.

So I now believe the NVMe drive was already broken, so when I tried to boot the system without my then external drive, the BIOS was then justifiably a little confused/confusing, as the external drive was my default boot device, which was chainloading windows, but not present. The internal NVMe drive was simply broken. So obviously my system could not boot.

Lesson learned here are 3-fold

Be more careful when hit with a unexpected message, dont just accept “reset to factory defaults” too quick.
Hardware does fail, my “it was all right a few months ago” was correct but led to an unhelpful assumption that it was still OK and I’d screwed up on software side.
My own documentation of what I did was not great - my memory is not bad but writing stuff down somewhere is better.

[57504.566599] usb 3-2: new high-speed USB device number 4 using xhci_hcd
[57504.702677] usb 3-2: New USB device found, idVendor=0bda, idProduct=9210, bcdDevice=20.01
[57504.702694] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[57504.702701] usb 3-2: Product: RTL9210B-CG
[57504.702707] usb 3-2: Manufacturer: Realtek
[57504.702713] usb 3-2: SerialNumber: 012345679039
[57504.711741] usb-storage 3-2:1.0: USB Mass Storage device detected
[57504.712246] scsi host4: usb-storage 3-2:1.0
[57505.731954] scsi 4:0:0:0: Direct-Access     Realtek  RTL9210B-CG      1.00 PQ: 0 ANSI: 6
[57505.732701] sd 4:0:0:0: Attached scsi generic sg1 type 0
[57506.171704] sd 4:0:0:0: [sdc] 1000215216 512-byte logical blocks: (512 GB/477 GiB)
[57506.172661] sd 4:0:0:0: [sdc] Write Protect is off
[57506.172668] sd 4:0:0:0: [sdc] Mode Sense: 37 00 00 08
[57506.173648] sd 4:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[57510.392969] usb 3-2: USB disconnect, device number 4
[57510.401492] device offline error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[57510.401512] Buffer I/O error on dev sdc, logical block 0, async page read
[57510.401551] device offline error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[57510.401560] Buffer I/O error on dev sdc, logical block 0, async page read
[57510.401592] device offline error, dev sdc, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[57510.401601] Buffer I/O error on dev sdc, logical block 0, async page read

oldfred · January 4, 2025, 3:29pm

My newish Dell with 11th Gen Intel had to go back for repairs twice. I had made it dual boot with Windows 11 and Kubuntu. First time back from Dell it booted both Windows & Kubuntu without issue. After 2nd repair, it only booted Kubuntu. And none of many suggested Windows repairs would repair the Windows install. Finally did the Dell recovery with a download from Dell. It totally erased drive & wiped my Kubuntu. System was like brand new. Or good backups required. I think now that second Dell repair replaced motherboard and they did not update UEFI with product key Windows expected. Then converted to using Kubuntu on external SSD.