Please test autoinstalls for 20.04!

[geschnei]

To follow up on my own question, I managed to have the iso booting with my autoinstall.yml like this:

First I added the .yml to the squashfs:

sudo unsquashfs -d ./squashfs-root -no-progress cd_amd64/casper/installer.squashfs
sudo cp -vf autoinstall.yml squashfs-root/etc/cloud/cloud.cfg.d/90_autoinstall.cfg
sudo rm cd_amd64/casper/installer.squashfs
sudo mksquashfs squashfs-root cd_amd64/casper/installer.squashfs -no-progress -noappend
sudo rm -rf squashfs-root

Then I modified Grubs txt.cfg to add the autoinstall parameter:

label autoinstall
  menu label ^Install Ubuntu Server unattended
  kernel /casper/vmlinuz
  append   initrd=/casper/initrd autoinstall quiet  ---

This runs the autoinstall with the given .yml file.

Just using the content from the /var/log/installer/autoinstall-user-data file of my previous installation resulted in similar errors as @jinxcappa described:

None is not of type ‘string’

I removed the attribute instead of setting it to ‘’, which worked for me. I also had to add version:1 below autoinstall:.

Now I’m also stuck at the point

Command ‘[‘netplan’, ‘apply’]’ returned non-zero exit status 1

The network section is just:

network:
      ethernets:
        ens160: {dhcp4: true}
      version: 2

ens160 is the correct name of the interface, I can confirm this in a second console.

[geschnei]

I looked around using the second console. Running the command netplan apply manually results in the following error message:

/etc/netplan/00-installer-config.yml:2:1: Error in network definition: unknown key ‘ethernets’
ethernets:
^

The file /etc/netplan/00-installer-config.yml looks like this:

# This is the network config written by 'subiquity'
ethernets:
  ens160:
    dhcp4: true
version: 2

Looking at netplan examples it looks like the top level network branch is missing.

I changed the network configuration in the autoinstall.yml to look like this:

network:
  network:
    ethernets:
      ens160: {dhcp4: true}
    version: 2

With this the installation went through.

[s3rj1k]

@mwhudson Is there any example on how to use USB drive for autoinstall?

• formatting it to VFAT
• setting CIDATA label
• putting user-data and meta-data
• booting installer with: ‘initrd=/casper/initrd quiet autoinstall ds=nocloud;s=/ —’

all of the above does not work

[geschnei]

Next problem: Can I add multiple autoinstall configurations? On my previous ISO I had multiple preseed files with corresponding menu entries on the boot screen.

Trying to do the same with the new method I got the message

Confirmation is required to continue.

Add ‘autoinstall’ to your kernel command line to avoid this

when I booted from the second configuration. It seems that the parameter autoinstall is not just a name for the configuration to load, but a keyword.

How can I place multiple autoinstall configurations on a .iso?

[robdumas]

I’m planning a nuke-and-pave for my server now that 20.04 is out and I’m interested in using autoinstall (with either the server or netinstall USB ISOs) to automate the early installation, such as:

• Partitioning the SSD (ESP/boot.ext4/root.xfs)
• Set hostname, network (DHCP)
• Install a few packages (e.g., openssh-server, build-essential, etc.)
• Add a specified user with SSH authorized key and sudo permissions

Once those steps are complete, I should be able to connect to the server with Ansible and provision it fully from there.

Right now, I’m trying to figure out how to access autoinstall in VirtualBox (6.1.6 as of this writing) to model it before trying it on the live server, but I’m feeling a bit lost. Must the autoinstall be served via HTTP or can it be local? Could I put it on another disk plugged into the machine, like a USB drive? I’m thinking here of the way Kickstart will look for a volume labeled “OEMDRV” and then pull the Kickstart config from there. Here’s a rough screenshot of what I’m talking about. Is that possible with autoinstall? How would I use GRUB’s command line to tell the server/netinstall disc to load an autoinstall (and/or cloud-config) file from a second disk that’s plugged in?

[mwhudson]

@s3rj1k

@mwhudson Is there any example on how to use USB drive for autoinstall?

No, we need to write this up. Comment #4 : Bug #1869291 : Bugs : subiquity provides a recipe for this but the original reporter didn’t confirm if it worked for them.

[mwhudson]

@hlinden

Works a treat here, much better than hours of trying to debug d-i statements!

Yay, that’s the sort of response we were hoping for!

@hlinden

Rsyslog implementation would be awesome, I see that is in the pipeline.

Hopefully we can get to it soon. If someone wants to help it would look a bit like this Merge into trunk : trunk.add-journald-reporter : Code : curtin

@hlinden

Any thoughts on ZFS support?

We’re aiming to have some kind of ZFS support in 20.10 / 20.04.2. It will probably not support many options
.

@hlinden

I did some experimentation with adding the curtin zfsroot config but got python exceptions (which I kind of expecting).

One aim I had that I admit I have mostly forgotten is to allow people to use curtin storage features that subiquity itself does not support. I should see if I can make that possible somehow.

@hlinden

Is it possible to run commands in the /target “chroot”, for example I have a template that installs apache2 but I don’t want it to start on default boot until it has been configured by the end customer.

Yes sure, you can run “curtin in-target – whatever” in late commands, this is even documented at FoundationsTeam/AutomatedServerInstalls/ConfigReference - Ubuntu Wiki (although I’m not sure I’ve tried it!).

@hlinden

Lastly, what are the credentials to ssh in to a crashed installer to debug?

The long, random password for the ‘installer’ user is in the cloud-init output on the console but probably a bit hard to find in amongst everything else. Or you can put a user config with ssh keys into the user-data you give the live session.

[mwhudson]

@jinxcappa

I myself am having issues. Im trying to use the autoinstall-user-data file that gets generated by the installer as a template. taking the file right after installation and attempting to use it as a config file for the same server fails in a couple areas.

First off the file is missing the version: 1 key to properly read the file

Oops.

@jinxcappa

Second after adding that the next failure comes from the toggle: key under keyboard: . the set value in the file is null but that throws an error in the installation:

None is not of type 'string'

changing the value from null to '' gets passed that error

Also oops. Both of these should be easy to fix, at least.

@blackline-silverfiel

• Is it feasible to throw a more visible error if, for whatever reason, user-data is not accessible and autoinstall was specified on the kernel command line? Just dropping the user into the non-automated installer is very confusing.

Yes, this is a good idea.

@blackline-silverfiel

• Where should I be looking for more information about what cloud-init is trying and failing to do? /var/log/cloud-init.log mentioned in the cloud-init docs does not appear to contain information about this part of the process, it seems to start by reading 0 bytes from /var/lib/cloud/seed/nocloud/user-data .

There is also /var/log/cloud-init-output.log but I’m not sure it helps with this. It’s always hard to debug why something isn’t happening! Looking in /var/lib/cloud/instance/datasource or the output of ‘cloud-init query’ might help.

@geschnei

I changed the network configuration in the autoinstall.yml to look like this:

network:
  network:
    ethernets:
      ens160: {dhcp4: true}
    version: 2

With this the installation went through.

Oh argh. I’ll fix this one soon so that the double network key is not necessary (but leave some code in so it works).

@geschnei

Next problem: Can I add multiple autoinstall configurations? On my previous ISO I had multiple preseed files with corresponding menu entries on the boot screen.

Trying to do the same with the new method I got the message

Confirmation is required to continue.

Add ‘autoinstall’ to your kernel command line to avoid this

when I booted from the second configuration. It seems that the parameter autoinstall is not just a name for the configuration to load, but a keyword.

You can put ds=nocloud,s=/autoinstall-variant1 on the kernel command line (and put user-data and meta-data files into the /autoinstall-variant1 directory). Or you can have an early-command swap a different yaml file into place depending on a marker you put into the kernel command line.

@robdumas

I’m planning a nuke-and-pave for my server now that 20.04 is out and I’m interested in using autoinstall (with either the server or netinstall USB ISOs) to automate the early installation, such as:

• Partitioning the SSD (ESP/boot.ext4/root.xfs)
• Set hostname, network (DHCP)
• Install a few packages (e.g., openssh-server , build-essential , etc.)
• Add a specified user with SSH authorized key and sudo permissions

All this should be very straightforward.

@robdumas

Right now, I’m trying to figure out how to access autoinstall in VirtualBox (6.1.6 as of this writing) to model it before trying it on the live server, but I’m feeling a bit lost. Must the autoinstall be served via HTTP or can it be local? Could I put it on another disk plugged into the machine, like a USB drive? I’m thinking here of the way Kickstart will look for a volume labeled “OEMDRV” and then pull the Kickstart config from there. Here’s a rough screenshot of what I’m talking about. Is that possible with autoinstall? How would I use GRUB’s command line to tell the server/netinstall disc to load an autoinstall (and/or cloud-config) file from a second disk that’s plugged in?

Yes, you can use a filesystem with the cidata (or CIDATA) label for this. As I replied to someone else earlier Comment #4 : Bug #1869291 : Bugs : subiquity has a recipe for this, but I do need to write this up somewhere.

[s3rj1k]

The issue is that I can’t use another ISO file, only USB drive.
And when I try to do that I am not seeing that installer copies cloud init seed into ‘/var/…’

[mwhudson]

I wrote up an example of using a different volume to provide the config: FoundationsTeam/AutomatedServerInstalls/QuickStart - Ubuntu Wiki

@s3rj1k

The issue is that I can’t use another ISO file, only USB drive.

You can dd an ISO to a USB device just fine?

[s3rj1k]

Does VFAT option works? or only iso9660?

[mwhudson]

VFAT should work, I haven’t tried though and I’m going to bed now sorry…

[tryhard]

I also gave the autoinstall a try and I can’t configure the network:

#cloud-config
autoinstall:
  version: 1
  interactive-sections:
    - network
  apt:
    geoip: true
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://de.archive.ubuntu.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  identity: 
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    realname: ubuntu
    username: ubuntu
  network:
    network:       
      ethernets:
        enp0s3:
          addresses: [192.168.178.52/24]
          gateway4: 192.168.178.1
          nameservers: 
            addresses: [192.168.178.6]
    version: 2
  keyboard: 
    layout: us
  locale: en_US
  ssh:
    allow-pw: yes
    install-server: true

Command ‘[‘netplan’, ‘apply’]’ returned non-zero exit status 1
…
AttributeError: ‘ScrollBarListBox’ object has no attribute ‘_w’

Even a DHCP network configuration is not workinng same error.

[geschnei]

Your version: 2 is incorrectly indented. It should be at the same level as ethernets:.

[tryhard]

Thanks a lot this helps. But after the installation the network is still configured dhcp.

#cloud-config
autoinstall:
  version: 1
  apt:
    geoip: true
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://de.archive.ubuntu.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  identity: 
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    realname: ubuntu
    username: ubuntu
  network:
    network:
      version: 2       
      ethernets:
        enp0s3:
          dhcp4: no
          addresses: [192.168.178.52/24]
          gateway4: 192.168.178.1
          nameservers: 
            addresses: [192.168.178.6]
  keyboard: 
    layout: us
  locale: en_US
  ssh:
    allow-pw: yes
    install-server: true

I added explicitly dhcp4: no. But no luck the network is configured dhcp. enp0s3 is the correct interface name.

[blackline-silverfiel]

Moment of enlightenment: It occurred to me to check /proc/cmdline, and I discovered that the ; character in the cloud-init config string was causing a problem: When typed directly into the grub console (as packer does), it needs to be escaped to avoid being interpreted as a line feed. This is obviously not necessary when executed via KVM. Probably worth noting in the docs for people who forget that grub’s console is a shell-like environment and passing the kernel command line through that environment represents a(nother) level of escaping.

I am now at least to the point that the config is downloading into the VM and failing schema validation ( ), so I think my noteworthy issues are resolved and the rest is just development work. Thanks for the help everyone.

[s3rj1k]

Well, it does not for me, on a real server with 10 disks and 1 USB formated with vfat.

[s3rj1k]

Also there is an issue with user creation, can’t login after autoinstall then using

interactive-sections: 
  - identity

[s3rj1k]

Works only when I repack ISO, put cloud init on it and boot with

autoinstall ds=nocloud;s=/cdrom/nocloud/

[mwhudson]

@tryhard

I added explicitly dhcp4: no . But no luck the network is configured dhcp. enp0s3 is the correct interface name.

Can you check what is in /etc/netplan in the installed system?