Please test autoinstalls for 20.04!

Server
296

How does one put the LUKS encryption key in the autoinstall file in an encrypted form? So far I’ve only managed to do this in plain text. This has been a limitation of the old preseed method, and apparently it’s been carried over to autoinstall.

I’ve tried to omit the "key: " section, but then crutin just crashes.

I’ve also tried to set the “storage” section to interactive, but then it apparently discards my settings for storage altogether. (on the edge channel it also crashes as a bonus lol)

It’d be great if I could tell autoinstall to ask interactively just for the encryption passphrase, and nothing else. Is there a way?

297

Trying to work out why something has not happened is always hard but perhaps we should add some tips to the documentation about this. I’d start by looking at the cloud init logs and seeing if it’s finding a data source at all.

I’d like to solve this with better tooling really. One thing would be to provide a tool to make modifying the kernel command line arguments in the ISO a one liner (I have some code for this but it’s a bit terrible at the moment). The other thing is to provide a better way to create VM images than booting an installer in the VM – it’s really not what the installer is designed for! (This would be a much bigger project).

How would this work? luksFormat needs the unencrypted passphrase, it’s not like a password for authentication where supplying a hash is sufficient. If you simply don’t want to have it in the autoinstall file I guess you can have that reference a keyfile and use an early-command to fetch the passphrase from wherever and put the it into the keyfile (I haven’t tried this mysef).

Neither of those sound great :confused: Can you file bugs?

Not today. I would suggest you use an early-command again but it’s hard to be interactive from an early command as they run in a process without a controlling terminal. I guess in some circumstances you can assume that any interaction will take place on /dev/tty0.

298

Thanks for your response. So you mean there’s absolutely no means to securely autoinstall Ubuntu on an encrypted volume? To be honest, I don’t quite understand the priorities here, I don’t think FDE is a negligible aspect in 2021. We don’t even allow unencrypted workstations at our company. And it’s been like this since 2016 I think. I’m not willing to put disk encryption passwords on a non-authenticated web server, where the autoinstall file resides. Nor on a thumb drive. No one should, it kinda defeats the purpose of encryption.

I’ve tried to look up early-command user input, but couldn’t find much. Do you have any pointers by any chance?

I can file bug reports, but where’s the place for that? In any case, I’m not sure all of those are bugs. It’s more of a design thing. If you designate a section as interactive, but also specify some answers for that section in the autoinstall file, there’s a conflict which autoinstall has to resolve. As of right now, autoinstall does that by simply ignoring any predefined answers for a section if it’s interactive. There’s at least 2 obvious solutions to this:

  • you should be able to select single fields as interactive, not just whole sections, or
  • autoinstall should pre-fill the fields specified in the autoinstall file for interactive sections

Neither happens right now. The 2nd option could be further expanded, because that one you could approach in at least 2 ways:

  • you could just offer defaults which can be overridden interactively, or
  • the predefined fields could be immutable so that the person doing the install could observe those values but couldn’t change them, they’d be “greyed out”.

This can be again either a design choice of autoinstall, or an option in the autoinstall file, like being able to define “soft interactive” and “hard interactive” sections, the former of which has mutable defaults and the latter immutable.

For the record, this is not limited to storage, the same happens with other sections, like identity. I do want to specify the user, because I don’t wanna risk helpdesk messing up the username and or password every now and then. But I do not want to specify the hostname, since that’s obviously machine-specific. And I can’t do that as of now. I either specify all or none of them. If I set identity interactive, my defaults are ignored completely.

It gets even worse if I need more than one user: if I specify one user in user-data/users, and also fill in the identity section with a user and hostname, the users under user-data/users are completely ignored. So in essence, you can’t specify the hostname AND have 2 users at the same time.

Not being able to interactively set hostname during install is particularly painful, because it’s too confusing to change afterwards. You can use hostnamectl set-hostname, but it’s worthless, because all it does is dump your input into /etc/hostname, but that’s not enough, you still need to manually adjust /etc/hosts too, otherwise su/sudo operations will keep throwing name resolution errors. So why bother with hostnamectl in the first place, you just substitute vi/echo/cat with a different command. You might as well just manually edit both files.

299

I’m not sure what you expect a solution to this to look like for a fully automated installation. Creating the encrypted volume requires the passphrase, thus the passphrase must be provided to the installer. If having it on the media or the network are both out, what is left?

There are things that can be done with the TPM and secure boot and so on to make this smoother, like ubuntu core 20 does (Full disk encryption | Ubuntu), but bringing that to more generic systems like arbitrary server installers vs ubuntu core is a lot of engineering work.

No I don’t. I think you could put something like

early-commands:
 - |
 sh -c "bash myscript.sh < /dev/tty1 >& /dev/tty1"

and it would run myscript.sh interactively on tty1. But it might not work very well at all, as there will be other stuff running on tty1. Perhaps we need to design some kind of protocol for an early-command to ask for input.

FWIW, this isn’t the intent. The intent is that the autoinstall-provided values become defaults for the interactive screen. But I can well believe there are bugs around this.

Yeah so option 2 is the intent. Option 1 sounds … challenging to implement, although that’s not really a good argument for not doing it.

300

Are there any plans to make autoinstall work completely offline as described here?

I am also interested in this line of inquiry because I am currently trying to figure out how to bake an ISO with the following packages configured in user-data simply to avoid the need to fetch them from the repos online during the autoinstallation process:

  packages:
    - ntpdate
    - htpdate
    - gcc
    - make
    - perl
    - easy-rsa
301

Hm, it already should work with no network. Where things might go a bit awry is where there is a network, but there is no archive mirror on it. In these cases it’s probably better to not configure the network during install and put the desired config into place in a late-command.

Good timing, just a few days ago I put together a thing for doing pretty much exactly this: A tool to modify live server ISOs. It’s still a work in progress (it seems something isn’t quite right yet in the process of regenerating the apt indices for the pool on the ISO) but it’s definitely something you should look at :slight_smile:

1 Like
302

I’ve fixed this bug, I think, and I used the version of livefs-edit in current git to bake a config like yours and the necessary packages into an ISO and used it to install a VM with no network access.

303

The only word that comes to mind is - Champion!

Will give your livefs-editor a whirl.

Off the bat I can say with certainty that I will have to combine livefs-editor with convertsh’s ubuntu-autoinstall-generator (https://github.com/covertsh/ubuntu-autoinstall-generator) because I already use ubuntu-autoinstall-generator to explode and repack the VirtualBox Guest Additions ISO within Ubuntu Live Server and have autoinstall late-commands to successfully install it.

304

Oh right, I’d forgotten about that. I think livefs-edit mostly subsumes its functionality fwiw with its --add-autoinstall-config action.

306

Hi all,
I’m coming from CentOS with Anaconda Kickstart installs. There it is possible to have so called post-scripts. I need this post-script to automatically configure something in the installed system.
With the Ubuntu Server Installer, the only way I know to issue commands in the target system is a late-command curtin in-target -- command. But with this I can issue single commands only. I need to conditionally execute commands depending on the return value of previous commands, like this:

command -v postconf && postconf -e 'relayhost = [mail.$mydomain]' && postconf -e 'mydestination = ' && postconf -e 'masquerade_exceptions = root' && postconf -e 'mynetworks_style = host'

If I write this as a late command:

late-commands:
    - curtin in-target -- command -v postconf && postconf -e 'relayhost = [mail.$mydomain]' && postconf -e 'mydestination = ' && postconf -e 'masquerade_exceptions = root' && postconf -e 'mynetworks_style = host'

then only command -v postconf will be executed with curtin in-target. The rest will be executed in the install system, not in the target system, causing the auto installation to stop with an error.

How can I execute multiple shell commands conditionally as late-commands, or, even better, execute complete shell scripts as late-command?

307

One additional issue: The final installation of security updates takes very long. I installed from 20.04.1 server media on a VMware VM with SSD storage, and the installation of security updates took 12 minutes. This is the far majority of the whole install time. If I would install the updates with “apt upgrade”, it would not only install security updates, but take only 4 minutes and 30 seconds. Maybe the code used by the installer to install the security updates (/usr/bin/unattended-upgrade, /usr/lib/update-notifier/apt_check.py) is inefficient and should be improved.

308

OK, I found that calling unattended-upgrade with --no-minimal-upgrade-steps takes only 3 minutes. The default is --minimal-upgrade-steps, which took 14 minutes. Maybe it’s possible to change the installer behaviour to call unattended-upgrade with --no-minimal-upgrade-steps? Or make this configurable in user-data?

309

Would be great if this is included. The automatic updates in the installer take forever to complete!

After the system is installed we run automated setup scripts which would also install all the security updates.
And no we don’t want a system with known security vulnerabilities but we would like to have the possibility if needed.

310

seriously… how to get rid of the swap file?

In this thread here a more people trieing to achive this…

no mater what i try i end up with:

cat /proc/swaps
Filename                                Type            Size    Used    Priority
/swap.img                               file            4027388 1292    -2

config

  storage:
    grub:
      reorder_uefi: false
    swap:
      filename: swap.img
      size: 0
    layout:
        name: lvm

also tried

  storage:
    grub:
      reorder_uefi: false
    swap:
      size: 0
    layout:
        name: lvm

or

  storage:
    grub:
      reorder_uefi: false
    swap: {swap: 0}
    layout:
        name: lvm

seems bugged for me ?! Anyone has a workarround?

1 Like
311
  • curtin in-target --target=/target – bash /target/root/advanced_bash.sh

will be work ?
bash script in any variable is need to configure with target prefix ?

312

Late to the party here but have a few suggestions for docs and elsewhere.

I realize that last suggestion is quite nebulous. It stems from a failing late-command using modprobe and my lack of ability to see into the error. To make some concrete suggestions around this, perhaps the documentation could:

  • Better explain conceptually and more prominently that curtin in-target is effectively a way to run chroot’d commands in the target
  • What factors are “different” about this environment from e.g. a normal login to the box after it’s fully spun up
  • Usage of something like /usr/bin/tail -n 250 /var/log/syslog in error-commands to get hopefully some more debugging detail on failed commands since they are executed through systemd-cat vis a vis curtin
1 Like
313

could you please focus to implement possibility to run autoinstall fully offline, even if the network is configured, preffered for IoT/Edge devices with wwan connection

most of us are rebuilding the official iso to our needs and not only applying latest patches, but also including security hardenings

it would be great to have it as optional section

314

if you have the need of an IoT/Edge tailored OS with WWAN support (and adapted data-handling behaviour for WWAN), special security hardening and a rolling release model, have you taken a look at Ubuntu Core yet ? It is designed for particulary this use case and used a lot in IoT/Edge, PLCs, Industrial gateways, Digital Signage and POS systems around the world …

315

Pre-installing the package ifenslave seems to break the installer.

I’ve been working on a solution to preinstall some packages to make it possible to setup offline targets.

I’ve come up with a script that mounts casper/filesystem.squashfs, chroot-install some packages, configures my autoinstall script (nocloud) and repacking everything into a new filesystem.squashfs. It’s heavily inspired on inject-subiquity-snap.sh following mwhudson’s suggestions : Automated Server Install Quickstart.

But preinstalling ifenslave makes the installer to crash.

Script to reproduce the issue : https://gist.github.com/creatldd1/99db8d68e2a97e2737c5bb488be67328.

Check especially the “PARAMETERS” section before running it.

316

is it possible to run ansible playbook in late command? i’ve tried it, and it doesnt’ seem work ,possibly due to it being run as chroot…

1 Like