Anyone running LXD servers or even clusters on GCP (Google Cloud Platform)?
What kind of networking do you use? how did you sort out the routing? How are you coping with the fact that GCP, as well as most other cloud providers, do not really support L2 networking? I have found an old linuxcontainers topic (https://discuss.linuxcontainers.org/t/gcp-lxd-clustering-and-networking/11780) with a tip to use vxlan and static routes and started experimenting with this but am wondering what are my other options?
I found this https://cloud.google.com/compute/docs/containers/ and there it says i quote:
“You can convert your existing systems into LXD images and run them within Compute Engine VM instances for a lift-and-shift migration solution. LXD runs on Ubuntu images.”
any idea what do they mean with that? Is this container or VM images they are talking about? would then one GCP VM run one LXC containter or what?
Any insights are much appreciated.
I’ve not used it myself, but if you can get one or more IPs routed to the host VM then you could use the
routed NIC type to pass them into an instance.
yeah, that works and is a much cleaner solution than vxlan.
With vxlan you can theoretically route the complete subnet to any of the VMs running LXD and avoid setting up specific routes and messing around with them when you migrate the containers around, but I never got it to work properly, it was always breaking down randomly, either I don’t understand the intricacies of L2 enough, or Google is doing some funny bussines in the background.
for posterity, for routed network you do:
`gcloud compute routes create routename --project=yourproject --network=yourVPC --priority=1000 --destination-range=172.18.1.4 --next-hop-instance=vm_running_lxd --next-hop-instance-zone=europe-west3-a`
to set the route, and then in the VM:
# lxc profile show default
description: default profile
# lxc launch images:ubuntu/22.04/cloud testa -d eth0,ipv4.address=172.18.1.4 << EOF
- type: nameserver
that last bit I’m still struggling with, so if anyone has any ideas how to set just the nameserver using cloud-init, much appreciated
For the network config bit, maybe try
version: 2 as that is a simple passthrough to Netplan config as explained in https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html
According to https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v1.html#nameserver, the
search params you have should be provided as list (so inside
). This was noted by @dinmusic, thanks!