"Mostly" isolated container network

LXD Support
1

Hi lxd community,

I am trying to use LXD to set up containers in an arrangement with 2 containers (c1 and c2) on a host (h0):

  • h0 essentially acts as a router. It is connected to the internet on its enp1s0 interface and runs the nft firewall
  • c1 and c2 have static 10.0.0.x ipv4 addresses and hosts file entries about each other (so DHCP and local DNS are not needed)
  • c1 should be able to reach c2 on port 80 and receive responses to related/established connections.
  • otherwise c1 and c2 cannot communicate with each other on L2 or L3.
  • h0 should route incoming tcp/80 traffic from the internet to c2
  • c1 and c2 can both reach the internet through h0 via NAT

h0, c1, and c2 are all running Debian 12.

I have been searching and tinkering for the past week without success. I have seen references to security.port_isolation, routed nics, multiple bridged networks with firewall rules, etc. I also admit the problem may be my lack of understanding which nft hooks control the traffic above (I am coming from a background of FreeBSD jails and pf).

Any suggestions are most welcome! Thank you!

2

I apologize, I think my question may lack direction. If I may try something more limited:

I currently have both containers on a managed bridge network and have enabled security.port_isolation on both container NICs. Is there any way to selectively allow tcp/80 traffic between them (even if it has to route through the host)?

Thanks again for reading.

3
  1. Skip LXD networks
  2. Assign routed NICs to each container, define an IP address, but no parent.
  3. Netfilter rules under the forward hook control how the containers communicate with each other. Make sure to allow traffic from containers heading outbound and masquerade in postrouting.
1 Like
4

Hi, yes that seems like a good approach as it makes firewalling between instances and host more straightforward, especially if you don’t need DHCP/SLAAC.

One additional thing you could use is to use a static host_name setting on the routed NIC so that the host interface name is predictable, allowing you to use interface references in your firewall rules rather than IP addresses.

https://documentation.ubuntu.com/lxd/en/latest/reference/devices_nic/#nic-routed

5

Thank you, great tip! I had assigned the IPs to variables near the top of my nftables.conf as an alternate approach.

1 Like