Running lxc/lxd in GCP

LXD Support
1

Anyone running LXD servers or even clusters on GCP (Google Cloud Platform)?

What kind of networking do you use? how did you sort out the routing? How are you coping with the fact that GCP, as well as most other cloud providers, do not really support L2 networking? I have found an old linuxcontainers topic (https://discuss.linuxcontainers.org/t/gcp-lxd-clustering-and-networking/11780) with a tip to use vxlan and static routes and started experimenting with this but am wondering what are my other options?

I found this https://cloud.google.com/compute/docs/containers/ and there it says i quote:

“You can convert your existing systems into LXD images and run them within Compute Engine VM instances for a lift-and-shift migration solution. LXD runs on Ubuntu images.”

any idea what do they mean with that? Is this container or VM images they are talking about? would then one GCP VM run one LXC containter or what?

Any insights are much appreciated.

2

I’ve not used it myself, but if you can get one or more IPs routed to the host VM then you could use the routed NIC type to pass them into an instance.

https://documentation.ubuntu.com/lxd/en/latest/reference/devices_nic/#nic-routed

3

yeah, that works and is a much cleaner solution than vxlan.

With vxlan you can theoretically route the complete subnet to any of the VMs running LXD and avoid setting up specific routes and messing around with them when you migrate the containers around, but I never got it to work properly, it was always breaking down randomly, either I don’t understand the intricacies of L2 enough, or Google is doing some funny bussines in the background.

for posterity, for routed network you do:

`gcloud compute routes create routename --project=yourproject --network=yourVPC --priority=1000 --destination-range=172.18.1.4 --next-hop-instance=vm_running_lxd --next-hop-instance-zone=europe-west3-a`

to set the route, and then in the VM:

# lxc profile show default
config:
description: default profile
devices:
  eth0:
    name: eth0
    nictype: routed
    type: nic
  root:
    path: /
    pool: ee
    type: disk
name: default
used_by: []

# lxc launch images:ubuntu/22.04/cloud testa -d eth0,ipv4.address=172.18.1.4 << EOF
config:
  user.network-config: |
    version: 1
    config:
      - type: nameserver
        address: 169.254.169.254
        search: google.internal
EOF

that last bit I’m still struggling with, so if anyone has any ideas how to set just the nameserver using cloud-init, much appreciated :wink:

1 Like
4

For the network config bit, maybe try version: 2 as that is a simple passthrough to Netplan config as explained in https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html

5

According to https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v1.html#nameserver, the address and search params you have should be provided as list (so inside []). This was noted by @dinmusic, thanks!

2 Likes