Cloud-init config lost

equator8848 · September 2, 2024, 3:58pm

Hi, when I build image with the new image

lxc remote add --protocol simplestreams ubuntu-daily https://cloud-images.ubuntu.com/
lxc launch ubuntu-daily:focal base-image-roll-builder-202409010900 --target lxd8

I config instance cloud-init as before

but when instance start, I found that cloud-init is lost. (cloud-init version is 24.1.3-0ubuntu1~20.04.5)

I wait cloud-init fix a bug (Cloud-init status always reporting running · Issue #5304 · canonical/cloud-init · GitHub) for one month. when that bug fixed, I found this new problem.

tomp · September 5, 2024, 9:29am

Was this applied to a fresh instance or had the instance already booted?

Because cloud-init only runs once, it doesn’t re-run when the config is changed.

Try cloud-init status and also cloud-init clean which should get it to rerun on next boot.

equator8848 · September 8, 2024, 1:40am

hi，tomp, this applied to a fresh instance.

I make my new image base latest ubuntu-daily image, when I start the instance use my imgae, I found cloud-init config is lost.

equator8848 · September 8, 2024, 7:41am

I use two official image to start Instance, the instance base new image lost cloud-init config

tomp · September 17, 2024, 3:24pm

Please can you expand a bit on this, im not really following what you mean about the cloud-init config being lost?

Is it being lost from inside the guest or is it being lost from the LXD instance config?

equator8848 · September 19, 2024, 4:54am

hi, tomp. cloud-init config lost means that I config static ip in instance cloud-init config, but after instance start, instance can not get static ip.

If cloud-init not lost, /etc/netplan/50-cloud-init.yaml should find static ip config.

tomp · October 7, 2024, 9:30am

I wonder if @holmanb might be able to help with this?

holmanb · October 10, 2024, 12:17am

Thanks for the ping @tomp.

To reproduce, I set the following configuration on a 20.04 instance:

...
config:
  cloud-init.network-config: |
    version: 2
    ethernets:
      eth0:
        dhcp4: no
        addresses: [172.168.19.25/16]
        nameservers:
          addresses: [1.1.1.1, 8.8.8.8, 9.9.9.9]
        gateway4: 172.16.0.1
...

Then I check the NoCloud seed folder in the instance:

# cat /var/lib/cloud/seed/nocloud-net/network-config 
version: 1
config:
    - type: physical
      name: eth0
      subnets:
          - type: dhcp
            control: auto

It looks like the network configuration isn’t getting passed to the seed directory. I would guess that’s probably an lxd bug?

tomp · October 10, 2024, 7:21am

Which version of LXD were you using?

holmanb · October 10, 2024, 2:40pm

$ lxd --version  
6.1

tomp · October 10, 2024, 2:43pm

Does cloud-init in Focal support the new cloud-init.* keys rather than the older user.* keys?

holmanb · October 10, 2024, 7:18pm

On Focal, cloud-init uses the NoCloud datasource rather than the unix socket, so neither key is used. Cloud-init uses whatever is provided in /var/lib/cloud/seed/nocloud-net.

tomp · October 14, 2024, 7:56am

Hrm, I’m not sure LXD provides that info via /var/lib/cloud/seed/nocloud-net

tomp · October 14, 2024, 7:59am

No, wait, there is support but its generated in the image’s template files rather than in LXD:

github.com

canonical/lxd-imagebuilder/blob/dcd52abfd4abb84f6fb1cc7698b14ab69766e9e8/generators/cloud-init.go#L71-L177


      
          func (g *cloudInit) RunLXD(img *image.LXDImage, target shared.DefinitionTargetLXD) error {
          	templateDir := filepath.Join(g.cacheDir, "templates")
          
          	err := os.MkdirAll(templateDir, 0755)
          	if err != nil {
          		return fmt.Errorf("Failed to create directory %q: %w", templateDir, err)
          	}
          
          	var content string
          	properties := make(map[string]string)
          
          	switch g.defFile.Name {
          	case "user-data":
          		content = `{%- if config_get("cloud-init.user-data", properties.default) == properties.default -%}
          {{ config_get("user.user-data", properties.default) }}
          {%- else -%}
          {{- config_get("cloud-init.user-data", properties.default) }}
          {%- endif %}
          `
          		properties["default"] = `#cloud-config

This file has been truncated. show original

tomp · October 14, 2024, 8:07am

Yes, for instances created from the LXD image server (using images:) I can see LXD is using the templates in those images, e.g.

var/lib/lxd/containers/c1/templates# cat cloud-init-network-config.tpl
{%- if config_get("cloud-init.network-config", "") == "" -%}
{%- if config_get("user.network-config", "") == "" -%}
version: 1
config:
  - type: physical
    name: {% if instance.type == "virtual-machine" %}enp5s0{% else %}eth0{% endif %}
    subnets:
      - type: dhcp
        control: auto
{%- else -%}
{{- config_get("user.network-config", "") -}}
{%- endif -%}
{%- else -%}
{{- config_get("cloud-init.network-config", "") -}}
{%- endif %}

And the example cloud-init config you’re providing is working.

However the new images from CPC (using ubuntu*:) do not include these templates.

But let me try a Focal VM with 4.0 LXD and a Focal container…

tomp · October 14, 2024, 8:17am

So this seems to work OK:

lxc launch ubuntu-daily:20.04 v1 --vm # Launch a Focal VM for testing
lxc shell v1
lxd init --auto # Use LXD bundled with Focal
lxc version # 
Client version: 4.0.10
Server version: 4.0.10

lxc init ubuntu-daily:20.04 c1 # Create container and set cloud-init config

# Note I had to use `user.network-config` with LXD 4.0:
lxc config show c1
architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 20.04 LTS amd64 (daily) (20240821)
  image.label: daily
  image.os: ubuntu
  image.release: focal
  image.serial: "20240821"
  image.type: squashfs
  image.version: "20.04"
  user.network-config: |
    version: 2
    ethernets:
      eth0:
        dhcp4: no
        addresses: [172.168.19.25/16]
        nameservers:
          addresses: [1.1.1.1, 8.8.8.8, 9.9.9.9]
        gateway4: 172.16.0.1
  volatile.base_image: 6d465d8544583b826bad13fd1c92973b38ff454d3c0d3554a81c4d9dd258974b
  volatile.eth0.host_name: veth629ae113
  volatile.eth0.hwaddr: 00:16:3e:15:98:7c
  volatile.idmap.base: "0"
  volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.power: RUNNING
  volatile.uuid: e01aeb28-88e5-4b7b-a173-6fe29336f2e6
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""

lxc start c1
lxc ls
+------+---------+----------------------+------+-----------+-----------+
| NAME |  STATE  |         IPV4         | IPV6 |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+------+-----------+-----------+
| c1   | RUNNING | 172.168.19.25 (eth0) |      | CONTAINER | 0         |
+------+---------+----------------------+------+-----------+-----------+

It worked ok!

tomp · October 14, 2024, 8:20am

tomp:

  user.network-config: |
    version: 2
    ethernets:
      eth0:
        dhcp4: no
        addresses: [172.168.19.25/16]
        nameservers:
          addresses: [1.1.1.1, 8.8.8.8, 9.9.9.9]
        gateway4: 172.16.0.1

The same thing works OK on Focal with LXD 6.1 and a Focal container.

So no LXD regression observed.

But I notice that the CPC image Focal templates only support user.network-config and not cloud-init.network-config