Automated Server install quickstart

Server Guide
23

@powersj I should clarify that I have used debian-installer, live-build, mkisofs, etc but at least two of these approaches seem to be unusable with the new autoinstall workflow.

24

@powersj Forgive me for my lack of sysadmin skills, I’ve been writing software instead of managing servers for quite awhile and I’m out of practice. I tried the following tonight and was able to repackage an ubuntu 20.04 iso. However, it looks like I’m not setting the grub configuration correctly since the boot options don’t show up when booting from the iso:

# On an ubuntu 20.04 manual headless install

# Setup Temp Directory
export ISO_TMP=$(mktemp -d)
cd $ISO_TMP

# Get and mount images
wget http://releases.ubuntu.com/20.04/ubuntu-20.04.1-live-server-amd64.iso
mkdir $ISO_TMP/mnt
sudo mount ubuntu-20.04.1-live-server-amd64.iso $ISO_TMP/mnt

# Make build directory and copy iso files
mkdir $ISO_TMP/build
cd $ISO_TMP/mnt
tar cf - . | (cd $ISO_TMP/build; tar xfp - )
chmod ug+w $ISO_TMP/build
cd $ISO_TMP/build

# Modify grub.cfg
sudo vi boot/grub/grub.cfg

# Add the following boot options to the applicable linux menuentries
# autoinstall ds=nocloud;s=/cloud-init

# Create a simple cloud-init (from https://ubuntu.com/server/docs/install/autoinstall-quickstart)
mkdir cloud-init
cat > cloud-init/user-data << 'EOF'
#cloud-config
autoinstall:
version: 1
identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
EOF
touch cloud-init/meta-data

# (Optional-ish) Generate new md5 sums
mv md5sum.txt md5sum.txt.orig
cat md5sum.txt.orig | awk '{print $2}' | xargs md5sum > md5sum.txt
chmod 444 md5sum.txt

# Package a custom iso
sudo mkisofs -o $ISO_TMP/custom.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -J -R -V "My Ubuntu Custom ISO" .
25

@mwhudson can you please look at the post outlining a custom ISO next year? I can confirm that the grub options don’t make it into the live system. I’m not certain if we need to regenerate the grub files and if so how.

We need to add docs to roll a custom ISO for a non-VM use case to this page.

1 Like
26

I’ve built a script to automate creating those kinds of ISO image - you can find it here: https://github.com/covertsh/ubuntu-autoinstall-generator.

3 Likes
27

Since there is no autoinstall with the desktop version. I decided to create a USB Autoinstall for my laptop using the Server. I create the method on top using the ISO on USB and second vfat partition on the USB labeled CIDATA that contains user-data and meta-data. The setup goes well for the most part except that the wifi manager needed are not on the Server by default. So basically I have to do the wpa_supplicant setup manually. Only after that I could update all the packages needed for desktop mode. So it’s not longer autoinstall. Anybody knows if there is a similar autoinstall method for the Ubuntu desktop version? I am still a newbie with this so I might be missing the obvious.

28

Are you able to use a wired network connection to perform the installation?

I have been able to install the desktop version using this user-data config if the laptop is connected via ethernet while installing:

#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-desktop
    # password: ubuntu
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
  packages:
    - ubuntu-desktop-minimal
  network:
    version: 2
    renderer: networkd
    ethernets:
      enp2s0:
        dhcp4: yes
        dhcp6: yes
    wifis:
      wlp4s0:
        dhcp4: yes
        dhcp6: yes
        access-points:
          "my_wifi_ssid":
             password: "my_wifi_password"
  late-commands:
    - sed -i -e 's/networkd/NetworkManager/g' /target/etc/netplan/00-installer-config.yaml

When the install finishes, I let the laptop reboot without the cable connected and it will connect to my WiFi network.

29

I have not tried Ethernet since it does not come with the Laptop I have ASUS ZenBook UX550VE-DB71T, it does come with a Intel Corporation Wireless 8265.

I guess the next step would be to add a USB to LAN device and see if that works. So I am guessing Wifi usually not a standard way of setting up server and more desktop. So the packages are not included?

1 Like
30

I tried a ‘normal’ live server install on my laptop and it detected the ethernet NIC, but not my WiFi card. So I think the server ISO does not include the required firmware packages.

It’s definitely possible to set up a graphical desktop with the autoinstall process as I showed above, but I have not been able to get it to work via WiFi only yet.

1 Like
31

Yes that is also my problem with the live server install. I thought of updating the ISO by adding the wpa_supplicant package using Cubic but I was not successful and it does not seem like the proper way given that it’s an old package.

It seems logical that the desktop version would use the same autoinstall feature.

Thanks for the confirmation that it only works with an Ethernet NIC.

32

I am working on setting up a wifi only enabled automated install from a usb stick with the user-data and meta-data provided by the NoCloud datasource to cloud-init. I’ve observed a few issues. First lesson, if you modify any of the information in /etc/os-release or /etc/lsb-release, then you may possibly break cloud-init when it checks for devices with the cidata label. It’s best to take a look at those files in a non modified running system and make sure if you do modify them, they don’t have any unquoted funny white spaces. Second lesson, cloud-init doesn’t do nearly enough path sanitation. If you give the kernel seedfrom a value that doesn’t end with a os.sep value, then cloud-init just blindly will join this with the filenames, resulting in a failure. Why cloud-init decided to go with a seedfrom value of a directory for NoCloud, I have no clue? Who cares about verbosity on something that no-one other than us poor lonely sys admins will read anyway? Thirdly, I couldn’t get cloud-init to detect a separately mounted vfat volume with the cidata label. I had to resort to methods similar to the script from covertsh and others that puts the user-data and meta-data in a modified unsquashed filesystem. Unfortunately, even when I did that, and cloud-init seemingly loaded my datasource (as evidenced by logging the mydata variable from the _get_data func in /lib/python3/dist-packages/cloudinit/sources/ DataSourceNoCloud.py) when subiquity starts to run, for some unknown reason, the seedfrom value which was only moments earlier recognized, is rejected, and the installer craps out and puts me back in the live installer. I have no hypothesis as of now why that happens. Why and exactly when cloud-init needs to modify files in /var/lib/cloudinit and in /run/cloudinit is beyond me. But, presumably, knowing when these paths are written to would go a long way in easing my pain.

As a side note, I ran the live server installer and manually added the ubuntu-desktop package with --no-install-recommends, and I get what I want, just without wifi. Of course, adding the wpa_supplicant package and its dependents missing from the live server iso works fine, that is, I get internet connectivity but only through this extra unfortunate manual step.

What you might ask do I want to achieve? Well, what I want to achieve for my small office is a lan accessible service that provides custom isos for download. Employees can simply write to usb and plug in to get their base system. No, I don’t want netboot, because I actually do want these usb sticks, not provisioning directly to the eventual host machine.

33

This looks like a pretty handy script. One thing you can do instead is to mount the iso, create an overlay over it and then pack the overlay which will be quicker and use less disk, but will make the script require root. You can also make it more robust using xorriso to extract the arguments needed to recreate the ISO, as done here: https://github.com/CanonicalLtd/subiquity/blob/main/scripts/inject-subiquity-snap.sh#L194

The desktop installer is currently still d-i based. I think it’s possible to install it automatically somehow but I certainly don’t know how.

This is pragmatically the case yes. It would be possible of course to add the needed packages to the ISO and change the installer to enable the wifi bits and so on but very far from trivial.

34

@mwhudson I am trying to test a candidate change to adduser for Private home directories for Ubuntu 21.04 onwards? using the hirsute daily image and autoinstall. I had hoped to use a config like the following to add the PPA with the candidate build during the install but it doesn’t seem to work.

autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
    username: ubuntu
  apt:
    sources:
      adduser-private-homedirs-ppa:
        source: ppa:alexmurray/adduser-private-homedirs

Whilst I can see the PPA is enabled in the apt sources in the installed system, the version of adduser is still the one from the hirsute archive and not the newer one from my PPA. So if I log into the installed system and run and apt upgrade I then have the newer version but I want to try and test that the private home dirs part works when this newer version of adduser is used as part of the installer. Am I doing something wrong?

1 Like
35

As i understand we are talking about CVE and security issues that needs to be solved not after a release but while we are designing a release engineering lifecycle. How does that effect in real world devices such as embedded devices ?

Could you please give me a spesific security issue example (CVE number like we talk in mailing lists) ?

If this is the right place to talk general risks of the related CVE

36

This is what I would expect to happen, yes. The installer works by copying a preinstalled filesystem into the new system, it doesn’t run debootstrap like debian-installer (in some variants anyway) does. You can use inject-subiquity-snap.sh to do this I expect – something like:

  1. snap download subiquity
  2. inject-subiquity-snap.sh -i -f hirsute-live-server-amd64.iso subiquity_*.snap custom.iso
  3. wait until you get a shell
  4. dpkg-deb -x /path/to/your/adduser.deb new_filesystem (or maybe some chroot games if you need maintainer scripts to run)
  5. exit the shell
  6. boot custom.iso
1 Like
37

Thanks @mwhudson - that got me far enough to test what I needed. Cheers.

38

There is still no autoinstall with the desktop version, but I have created a new tool that enables you to use the traditional preseed method for Ubuntu 20.04 desktop. Check it out here if you are still looking for a solution: https://github.com/covertsh/ubuntu-preseed-iso-generator

39

Another question I have after working with this a bit. I can see that the ISO gets downloaded 3 times over the network when doing an automated install. Is there a way I can make it not do that?

40

I’ve found this to be a very useful page, thank you! I am not terribly familiar with KVM, so I expected the terminal output from the test install to go directly to my terminal.

Instead, I got

Unable to init server: Could not connect: Connection refused
Could not access KVM kernel module: Permission denied
qemu-system-x86_64: failed to initialize KVM: Permission denied

until I added the ‘-nographic’ flag and ran the command with sudo. Even then, I found it unhelpful that the screen was blank for a long period of time until I realized I needed to tell the kernel to use the serial console as well. All of which to say, I might suggest you’re expecting these test commands to be run on a desktop system, or use this slightly altered command for “Run the install”

sudo kvm -nographic -no-reboot -m 1024 \
    -drive file=image.img,format=raw,cache=none,if=virtio \
    -cdrom ~/Downloads/ubuntu-20.04-live-server-amd64.iso \
    -kernel /mnt/casper/vmlinuz \
    -initrd /mnt/casper/initrd \
    -append 'autoinstall console=ttyS0 ds=nocloud-net;s=http://_gateway:3003/'

Similarly for “boot the installed system”:

sudo kvm -nographic -no-reboot -m 1024 \
    -drive file=image.img,format=raw,cache=none,if=virtio

Thanks!

41

I am convinced that adding some of what is documented at
https://www.molnar-peter.hu/en/ubuntu-jammy-netinstall-pxe.html
to an improved (and less VM targeted) autoinstall manual would be super helpful.

1 Like
42

Hi, I want to install the system into BTRFS volumes. How do I script that?