Cannot mount installation ISO on VM

LXD Support
1

Hello,

trying to install a Windows 11 VM as per this tutorial: https://ubuntu.com/tutorials/how-to-install-a-windows-11-vm-using-lxd#1-overview

After adding the installation medium to the VM with this command:

lxc config device add win11 install disk source=/home/wexi/Downloads/win11.lxd.iso boot.priority=10

I then try to start the VM:

lxc start win11 --console=vga

but I get the following error:

Error: Failed setting up device via monitor: Failed adding block device for disk device "install": Failed adding block device: aio=native was specified, but it requires cache.direct=on, which was not specified.
Try `lxc info --show-log win11` for more info

The VM log is empty:

Name: win11
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2024/01/16 00:08 EET
Last Used: 2024/01/16 02:26 EET

Log:

Been trying to find an answer for this but no luck so far. LXD version:

lxd 5.19-8635f82 26200  latest/stable

win11 config:

architecture: x86_64
config:
  limits.cpu: "4"
  limits.memory: 8GiB
  volatile.cloud-init.instance-id: 45da4837-45ac-4314-befe-7f44187c35ec
  volatile.eth0.hwaddr: 00:16:3e:ba:ba:b9
  volatile.last_state.power: STOPPED
  volatile.uuid: 7b12cc92-268e-4442-a712-962b99c07b4a
  volatile.uuid.generation: 7b12cc92-268e-4442-a712-962b99c07b4a
  volatile.vsock_id: "2179293296"
devices:
  install:
    boot.priority: "10"
    source: /home/wexi/Downloads/win11.lxd.iso
    type: disk
  root:
    path: /
    pool: lxd-wd-green
    size: 50GiB
    type: disk
  vtpm:
    path: /dev/tpm0
    type: tpm
ephemeral: false
profiles:
- default
stateful: false
description: ""
2

Tried this also on my laptop which has a pretty recent clean install of Ubuntu 23.10, the problematic host (my desktop workstation) has the same. On my laptop the VM spinned up fine, went thru the Windows install and got it running easily.

My desktop has these differences compared to my laptop:

  • ZFS on a block device, not loopback
  • The Ubuntu installation has been running from 18.04 (or something) and upgraded during the years
  • The desktop host has also had qemu/kvm installed alongside LXD, lxd containers work fine but have not been running VMs
3

Anyone? This is driving me crazy…

4

Hi,

This is strange.

Do you see the same issue when using latest/candidate LXD 5.20, e.g. sudo snap refresh lxd --channel=latest/candidate?

5

I suspect that may be the difference. @sdeziel1 is this something you could look into please?

6

Have you tried creating a temporary dir storage pool using lxc storage create foo dir and then trying the VM on that pool using lxc init ... -s foo?

7

Yup, tried this and the same error

8

Tried this also and it did not work either.

9

What I’m suspecting is that for some reason LXD is trying to access the install device with the wrong parameters (as the error says) but why this is happening is not clear.

10

If I remove the install device, the VM starts but of course I cannot do anything with it :smiley:

11

@wexi79 could you run journalctl -fk while trying to boot VM with the ISO attached? Please share the journal output here, thanks.

12 
tammi 19 16:02:42 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered blocking state
tammi 19 16:02:42 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered disabled state
tammi 19 16:02:42 wz620 kernel: tap1b770cf1: entered allmulticast mode
tammi 19 16:02:42 wz620 kernel: tap1b770cf1: entered promiscuous mode
tammi 19 16:02:42 wz620 kernel: audit: type=1400 audit(1705672962.943:2780): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" profile="unconfined" name="lxd-win11_</var/snap/lxd/common/lxd>" pid=45349 comm="apparmor_parser"
tammi 19 16:02:43 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered blocking state
tammi 19 16:02:43 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered forwarding state
tammi 19 16:02:43 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered disabled state
tammi 19 16:02:44 wz620 kernel: tap1b770cf1: left allmulticast mode
tammi 19 16:02:44 wz620 kernel: tap1b770cf1: left promiscuous mode
tammi 19 16:02:44 wz620 kernel: lxd-wz620: port 5(tap1b770cf1) entered disabled state
13

Thanks, so that’s not the source of the problem. I’ll try to reproduce it here.

14

Could you show the output of findmnt too so we can see what filesystem your ISO file is on?

15 
wexi@wz620:~$ findmnt
TARGET                                                  SOURCE                                      FSTYPE      OPTIONS
/                                                       /dev/mapper/wz620-root                      ext4        rw,relatime,errors=remount-ro
├─/sys                                                  sysfs                                       sysfs       rw,nosuid,nodev,noexec,relatime
│ ├─/sys/kernel/security                                securityfs                                  securityfs  rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/cgroup                                      cgroup2                                     cgroup2     rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/pstore                                      pstore                                      pstore      rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/bpf                                         bpf                                         bpf         rw,nosuid,nodev,noexec,relatime,mode=700
│ ├─/sys/kernel/debug                                   debugfs                                     debugfs     rw,nosuid,nodev,noexec,relatime
│ ├─/sys/kernel/tracing                                 tracefs                                     tracefs     rw,nosuid,nodev,noexec,relatime
│ ├─/sys/fs/fuse/connections                            fusectl                                     fusectl     rw,nosuid,nodev,noexec,relatime
│ └─/sys/kernel/config                                  configfs                                    configfs    rw,nosuid,nodev,noexec,relatime
├─/proc                                                 proc                                        proc        rw,nosuid,nodev,noexec,relatime
│ └─/proc/sys/fs/binfmt_misc                            systemd-1                                   autofs      rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=28751
│   └─/proc/sys/fs/binfmt_misc                          binfmt_misc                                 binfmt_misc rw,nosuid,nodev,noexec,relatime
├─/dev                                                  udev                                        devtmpfs    rw,nosuid,relatime,size=24613148k,nr_inodes=6153287,mode=755,inode64
│ ├─/dev/pts                                            devpts                                      devpts      rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
│ ├─/dev/shm                                            tmpfs                                       tmpfs       rw,nosuid,nodev,inode64
│ ├─/dev/hugepages                                      hugetlbfs                                   hugetlbfs   rw,relatime,pagesize=2M
│ └─/dev/mqueue                                         mqueue                                      mqueue      rw,nosuid,nodev,noexec,relatime
├─/run                                                  tmpfs                                       tmpfs       rw,nosuid,nodev,noexec,relatime,size=4933744k,mode=755,inode64
│ ├─/run/lock                                           tmpfs                                       tmpfs       rw,nosuid,nodev,noexec,relatime,size=5120k,inode64
│ ├─/run/credentials/systemd-sysctl.service             ramfs                                       ramfs       ro,nosuid,nodev,noexec,relatime,mode=700
│ ├─/run/credentials/systemd-sysusers.service           ramfs                                       ramfs       ro,nosuid,nodev,noexec,relatime,mode=700
│ ├─/run/credentials/systemd-tmpfiles-setup-dev.service ramfs                                       ramfs       ro,nosuid,nodev,noexec,relatime,mode=700
│ ├─/run/rpc_pipefs                                     sunrpc                                      rpc_pipefs  rw,relatime
│ ├─/run/snapd/ns                                       tmpfs[/snapd/ns]                            tmpfs       rw,nosuid,nodev,noexec,relatime,size=4933744k,mode=755,inode64
│ │ ├─/run/snapd/ns/cups.mnt                            nsfs[mnt:[4026533113]]                      nsfs        rw
│ │ ├─/run/snapd/ns/lxd.mnt                             nsfs[mnt:[4026533114]]                      nsfs        rw
│ │ ├─/run/snapd/ns/remmina.mnt                         nsfs[mnt:[4026533115]]                      nsfs        rw
│ │ ├─/run/snapd/ns/prospect-mail.mnt                   nsfs[mnt:[4026533534]]                      nsfs        rw
│ │ ├─/run/snapd/ns/rambox.mnt                          nsfs[mnt:[4026533676]]                      nsfs        rw
│ │ ├─/run/snapd/ns/indicator-sound-switcher.mnt        nsfs[mnt:[4026533535]]                      nsfs        rw
│ │ └─/run/snapd/ns/morgen.mnt                          nsfs[mnt:[4026533677]]                      nsfs        rw
│ ├─/run/credentials/systemd-tmpfiles-setup.service     ramfs                                       ramfs       ro,nosuid,nodev,noexec,relatime,mode=700
│ ├─/run/credentials/systemd-resolved.service           ramfs                                       ramfs       ro,nosuid,nodev,noexec,relatime,mode=700
│ └─/run/user/1000                                      tmpfs                                       tmpfs       rw,nosuid,nodev,relatime,size=4933740k,nr_inodes=1233435,mode=700,uid=1000,gid=1000,inode64
│   └─/run/user/1000/doc                                portal                                      fuse.portal rw,nosuid,nodev,relatime,user_id=1000,group_id=1000
├─/snap/gtk2-common-themes/9                            /dev/loop0                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/boot                                                 /dev/sdd1                                   ext4        rw,relatime
├─/snap/bare/5                                          /dev/loop1                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/blender/4228                                    /dev/loop2                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/blender/4300                                    /dev/loop3                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/chromium/2724                                   /dev/loop4                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/chromium/2729                                   /dev/loop5                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/compress-video/29                               /dev/loop6                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/compress-video/30                               /dev/loop7                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core/16091                                      /dev/loop8                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core/16202                                      /dev/loop9                                  squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core18/2796                                     /dev/loop10                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core18/2812                                     /dev/loop11                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core20/2015                                     /dev/loop12                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core20/2105                                     /dev/loop13                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core22/1033                                     /dev/loop14                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/core22/864                                      /dev/loop15                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/cups/1024                                       /dev/loop16                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/cups/980                                        /dev/loop17                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/distrobuilder/1833                              /dev/loop18                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/fakecam/104                                     /dev/loop19                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/firefox/3600                                    /dev/loop20                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/firefox/3626                                    /dev/loop21                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-28-1804/194                             /dev/loop22                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-28-1804/198                             /dev/loop23                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-34-1804/90                              /dev/loop24                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-34-1804/93                              /dev/loop25                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-38-2004/140                             /dev/loop26                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-3-38-2004/143                             /dev/loop27                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-42-2204/132                               /dev/loop28                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gnome-42-2204/141                               /dev/loop29                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gtk2-common-themes/13                           /dev/loop30                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gtk-common-themes/1534                          /dev/loop31                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/gtk-common-themes/1535                          /dev/loop32                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/hunspell-dictionaries-1-7-2004/2                /dev/loop33                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/indicator-sound-switcher/90                     /dev/loop34                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/indicator-sound-switcher/91                     /dev/loop35                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/inkscape/10549                                  /dev/loop36                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/inkscape/10555                                  /dev/loop37                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/kde-frameworks-5-core18/32                      /dev/loop38                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/kde-frameworks-5-core18/35                      /dev/loop39                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/kde-frameworks-5-qt-5-14-core18/4               /dev/loop40                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/lxd/26200                                       /dev/loop42                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/mailspring/540                                  /dev/loop43                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/mailspring/541                                  /dev/loop44                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/mediainfo/22                                    /dev/loop45                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/mediainfo/24                                    /dev/loop46                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/morgen/90                                       /dev/loop47                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/morgen/93                                       /dev/loop48                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/p7zip-desktop/220                               /dev/loop49                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/pdfmixtool/862                                  /dev/loop50                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/pdfmixtool/864                                  /dev/loop51                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/phpstorm/366                                    /dev/loop52                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/phpstorm/368                                    /dev/loop53                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/postman/236                                     /dev/loop54                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/postman/238                                     /dev/loop55                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/prospect-mail/38                                /dev/loop56                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/prospect-mail/41                                /dev/loop57                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/rambox/x14                                      /dev/loop58                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/rambox/x15                                      /dev/loop59                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/remmina/6117                                    /dev/loop60                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/remmina/6134                                    /dev/loop61                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/shotcut/1298                                    /dev/loop62                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/shotcut/1312                                    /dev/loop63                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/slack-term/205                                  /dev/loop64                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/snappy-debug/696                                /dev/loop65                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/snappy-debug/704                                /dev/loop66                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/untrunc-anthwlock/163                           /dev/loop67                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/vysor/x1                                        /dev/loop68                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/snap/youtube-dl/4806                                 /dev/loop69                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
├─/var/snap/firefox/common/host-hunspell                /dev/mapper/wz620-root[/usr/share/hunspell] ext4        ro,noexec,noatime,errors=remount-ro
├─/var/lib/lxcfs                                        lxcfs                                       fuse.lxcfs  rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other
├─/shares/Openshare                                     //192.168.1.50/Openshare                    cifs        rw,relatime,vers=1.0,cache=strict,username=Wexi,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.1.50
├─/var/snap/lxd/common/ns                               tmpfs                                       tmpfs       rw,relatime,size=1024k,mode=700,inode64
│ ├─/var/snap/lxd/common/ns/mntns                       nsfs[mnt:[4026533114]]                      nsfs        rw
│ └─/var/snap/lxd/common/ns/shmounts                    nsfs[mnt:[4026533117]]                      nsfs        rw
├─/shares/Wexi                                          //192.168.1.50/Wexi                         cifs        rw,relatime,vers=1.0,cache=strict,username=Wexi,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.1.50
├─/home/wexi                                            /home/wexi/.Private                         ecryptfs    rw,nosuid,nodev,relatime,ecryptfs_fnek_sig=dec2e62926d12dde,ecryptfs_sig=a8dfb36fac493227,ecryptfs_
│ └─/home/wexi/2TB                                      /dev/mapper/2tbluks                         ext4        rw,relatime
└─/snap/lxd/26570                                       /dev/loop70                                 squashfs    ro,nodev,relatime,errors=continue,threads=single
16

Aha, I moved the ISO to /tmp and it works from there so this is probably related to my home directory being encrypted…

On my laptop I have full disk encryption.

1 Like
17

It sounds like full disk encryption with ecryptfs for your home on top of it :slight_smile: IIRC, ecryptfs is deprecated/abandoned.

18

On my desktop only my home dir is encrypted, full disk encryption is also in place for a 2TB drive where I store my Dropbox data etc… but the main system disk is not encrypted.

19

Try setting io.cache=none or io.cache=unsafe on the disk device in LXD.

See https://documentation.ubuntu.com/lxd/en/latest/reference/devices_disk/#devices-disk

20

This is backed by ecryptfs according to the findmnt output. Can you try moving it under /home/wexi/2TB and see if that works?