I’m running into issues with inter-network connectivity between OVN networks in my MicroCloud setup and would appreciate guidance on the recommended approach.
Environment:
MicroCloud 2.1.1 LTS - 3-node cluster
LXD 5.21
Ubuntu 24.04 LTS
OVN overlay networking
Setup: I have two OVN networks that need to communicate:
The peers are created but remain in PENDING state indefinitely. Containers cannot ping across networks. Running ovn-nbctl show reveals that each OVN network has its own logical router (lxd-netX-lr), and there are no routes between them.
Problem 2: Security ACL Configuration Error
When trying to configure ACLs on the OVN network level, I get this error:
lxc network set networkB security.acls=my-acl
Error: Uplink address key "volatile.network.ipv6.address" cannot be empty
when network address key "ipv6.address" is populated
Both networks have ipv6.address: none configured, and the UPLINK has no IPv6 configuration either.
Questions:
What is the recommended LXD/MicroCloud version for stable OVN multi-network setups? Should I upgrade from MicroCloud 2.1.1 LTS / LXD 5.21?
What is the proper way to enable inter-network communication between OVN networks in LXD? Should I:
Use network peering (and if so, how to activate PENDING peers)?
Is the ACL error a known issue in LXD 5.21, and is there a workaround for applying ACLs to OVN networks?
For now, I can work around this by manually creating a transit network in OVN and adding static routes between the logical routers, but I assume LXD should handle this more elegantly.
Current OVN Topology:
Router lxd-netA-lr: 10.10.10.1/24 (external: 192.168.100.100/24)
Router lxd-netB-lr: 10.20.20.1/24 (external: 192.168.100.101/24)
No interconnection between routers
I can reproduce this also using LXD latest/edge. It’s fine to have only IPv4 configured on the uplink.
You can make it work by unsetting the ipv6.address config key from the network using lxc network unset <network> ipv6.address. But I am not sure if this is meant like this by design.
If you don’t have an IPv6 uplink, it might not make sense to have an ipv6.address on the network as you anyway don’t have IPv6 uplink connectivity.
But you might still want to have cross OVN network IPv6 connectivity (via peering).
One UX experience we could make here is that if the user has access to see networks in the target network we could reject the request if the network doesn’t exist.
Currently we allow the pending peer to be created because each end of the peer relationship can be setup by different people without both needing access to each other’s projects.
So we don’t allow validation of the target network name as it may not exist yet or the user may not be allowed to know if it exists or not.
ovn-nbctl set logical_switch lxd-netA-ls-int other_config:stateless=true
ovn-nbctl set logical_switch lxd-netB-ls-int other_config:stateless=true
Result: No change
Discovered Issue: Peer router ports created with /32 masks:
lxd-netA-lr-lrp-peer-netB: 10.10.10.1/32 (instead of /24)
lxd-netB-lr-lrp-peer-netA: 10.20.20.1/32 (instead of /24)
Attempted to fix by recreating ports with /24 masks - no change.
Transit Network Approach: Deleted peering, created manual transit network (10.255.0.0/30) with static routes between routers.
Result: Still no connectivity
Suggest you disable/remove any ACLs you currently have, as well as undoing any custom modifications you’ve made to the ovn northbound DB (as that should not be necessary and puts ovn into an unknown state that might cause issues).
Then setup peering and make sure thats working before adding the ACLs.
The ACLs in LXD will cause all traffic to be dropped by default, including from peer connections, so once you’ve accounted for peering working without ACLs you’ll need to account for that traffic in your ACL rules when you add them.
You can also refer to peer networks using Network subject selectors: