LXD: single host to cleanly separate functionality - best practices and network config?

LXD Support
1

Hi,
First post here so a short background: I have quite some experience in administering (bare metal) Debian servers and clients, but less so in networking. I just dipped my toes in the pool of containerization over the past year, first with podman, then libvirt and now lxd. I am convinced lxd/lxc is what I need to separate functionality in logical clusters instead of mixing everything (mail, files, calendar, sso, http, younameit…), but the wealth of options that lxd offers is intimidating…

I have now installed HomeAssistant OS (a vm with a cluster of docker containers, governed by a supervisor) in an lxd virtual machine on 1 single host with a simple network bridge. This setup means that between the host network and the actual HA core, there is the lxdbr0 bridge, and then a docker container. The basic stuff is working nicely after setting up an additional route to the subnet created inside the lxd instance. I did notice that not all of my iot devices were automatically detected from inside the HA application, but more importantly: I am now seeing issues connecting from the host network to a SIP server inside the lxd instance, and I suspect that there is an issue with UDP ingress.

Long story short, I have 2 questions:

  1. Am I going about this the right way? Meaning: If I stick to a single host with up to 15 containers max, with the requirement that some services need to connect seamlessly to the host network, do I have to design the setup differently (1 bridge for all vs 1 bridge per container vs 1 routed NIC per container vs an OVN)?
  2. Is it to be expected that UDP ingress is blocked, whereas TCP is routed correctly (at least to the sip and sips ports) with a simple network bridge?

Thx!

2

If you want to connect your VM to the physical network, without going to the hassle of setting up an unmanaged bridge on the LXD host and then moving the host’s existing IP configuration to that instance and connecting the physical NIC to the bridge, then you could try using a macvlan NIC type.

https://documentation.ubuntu.com/lxd/en/latest/reference/devices_nic/#nictype-macvlan

This should work OK if you only need to use a single MAC address on the physical network, which by the sounds of it should be sufficient in your case because inside the VM all the docker containers are likely using port forwarding from the VM’s IP into their respective containers, so they all still use the VM’s MAC.

I can’t comment in detail on the SIP issue without knowing more about the setup, but you could use tcpdump to see where the packets are being dropped (on the host, in the VM, etc).