WAN LXD Cluster with OVN

Hello,

I’d like to know if there is some fundamental limit that prevents the deployment of a LXD cluster on different sites, so with nodes communicating over public IPs, with a shared OVN network.

I’m trying with the following setup:

  • node01 and node02 are VMs at site 1 with static public IP 1
  • node03 is a VM at site 2 with static public IP 1

I have followed the tutorial https://documentation.ubuntu.com/lxd/latest/howto/network_ovn_setup/ and I’m at the step of creating the OVN network from a node in site 1.

# lxc network create my-ovn --type=ovn network=UPLINK
Error: Failed getting optimal bridge MTU: Failed getting OVN underlay info: No matching interface found for OVN enscapsulation IP “site 2 public IP”

I can provide more details about the configuration, I don’t exclude configuration errors in setting up the OVN, but first I’d like to make sure if this is possible?

The final goal is to be able to simply move containers from site 1 to site 2 in case of maintenance at site 1 leveraging the evacuate operation and a manual DNS change.

Thanks!

Hi! The current OVN architecture in LXD assumes all cluster members share the same uplink network segment (L2), which is not feasible when cluster nodes are in different locations with public IPs.

For now, the solution is to create two separate LXD clusters and configure them as remotes of each other, then use manual instance migration between the clusters. We are working on a solution that will allow two remote clusters to establish trust and communicate securely, enabling better multi-cluster scenarios.

Hello @maria-seralessandri , thanks for the feedback and the workaround.

Looking forward to test the solution you mention, in the meanwhile do you see some issues in having OVN over a Wireguard mesh?

Yes, it is possible, but you are hitting a classic “networking physics” wall. There is no fundamental “thou shalt not” in the LXD code, but OVN (Open Virtual Network) has very specific expectations about how nodes talk to each other.

The error you’re seeing No matching interface found is LXD’s way of saying: “You told me to use Site 2’s Public IP for the tunnel, but when I look at the local network cards on Site 1, I don’t see that IP anywhere.” Ev Bikes