How do I limit access to proxy devices to certain external hosts using iptables and ufw?

For a production server, I would like to limit access to certain services to specific allowed hosts on the public internet.

I am using ufw to configure iptables on the lxd host.

I have nat proxy devices configured in lxd to forward external connections to container instances. ip4.firewall is not set.

I have observed that deny rules for connections to ports on the external interface that lxd is listening on have no effect on the lxd proxy devices. Only rules applied to lxdbr0 have any effect at all. It’s as though the external physical interface doesn’t even exist.

Consider a container with the following devices configured:

eth0:
  ipv4.address: <container_ip>
  ipv6.address: <container_ip6>
  nictype: bridged
  parent: lxdbr0
  type: nic
ssh:
  connect: tcp:<container_ip>:22
  listen: tcp:<external_ip>:2201
  nat: "true"
  type: proxy

With these firewall rules, I can log into containers from the host at <allowed_host>:

$ sudo ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 22/tcp on lxdbr0           ALLOW FWD   <allowed_host>               (out)
[ 2] 22                         ALLOW IN    Anywhere
[ 3] Anywhere on lxdbr0         ALLOW IN    Anywhere
[ 4] Anywhere                   ALLOW FWD   Anywhere on lxdbr0
[ 5] 22 (v6)                    ALLOW IN    Anywhere (v6)
[ 6] Anywhere (v6) on lxdbr0    ALLOW IN    Anywhere (v6)
[ 7] Anywhere (v6)              ALLOW FWD   Anywhere (v6) on lxdbr0

If I add a deny rule, as below, where eth0 has <external_ip>:

$ sudo ufw prepend deny in on eth0 from <allowed_ip> to any port 2201 proto tcp
[sudo] password for user:
Rule inserted
user@lxdhost 2024-01-03 16:53:15 ~ $ sudo ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] 2201/tcp on eth0           DENY IN     <allowed_host>
[ 2] 22/tcp on lxdbr0           ALLOW FWD   <allowed_host>               (out)
[ 3] 22                         ALLOW IN    Anywhere
[ 4] Anywhere on lxdbr0         ALLOW IN    Anywhere
[ 5] Anywhere                   ALLOW FWD   Anywhere on lxdbr0
[ 6] 22 (v6)                    ALLOW IN    Anywhere (v6)
[ 7] Anywhere (v6) on lxdbr0    ALLOW IN    Anywhere (v6)
[ 8] Anywhere (v6)              ALLOW FWD   Anywhere (v6) on lxdbr0

it has no effect on connections to the container on the lxdbr0 network from <allowed_host>. (It does not matter if I deny traffic in or out, the non-effect is the same.)

Only manipulating lxdbr0 has any effect.

This is not what I expect. LXD is listening on the external interface! If I am blocking ports on the external interface, LXD should not even be able to receive connections on those blocked ports.

How do you explain what I am observing? My guess is that lxd network is handling the packet before the firewall sees it.

(One other thing that may be relevant: I noticed that I cannot see the listeners for every lxd device in ss -tlpn output, only listeners for interfaces configured very early, perhaps via lxd? The listener above is not one of the ones that is visible in ss -tlpn. The installation was originally the 4.0.x snap, and was upgraded to the 5.0 stable snap.)

Okay, I have (partially) understood what is happening here.

When an instance’s proxy device is configured without NAT, lxd will start a listener on the listen address. In this case, firewall rules that apply to the interface having that address do work as expected.

But if the proxy device has nat="true", the behaviour changes: no listener is started, and lxd behaves as though the bridge is the only interface on the host handling the NAT traffic. Rules applied to the interface actually having the listen address have no effect.

I find this peculiar but it probably has a reasonable technical explanation that I would like to hear.

Anyway, this behaviour should really be documented somewhere. As far as I know it isn’t.

When in nat=true mode LXD uses DNAT rules so the netfilter subsystem will see these packets as being “forwarded” rather than arriving at the host system via the “input” chain.

Try changing “DENY IN” to “DENY FWD” and see if that helps.

1 Like

Sorry, this (DENY FWD on eth0) doesn’t seem to work:

$ sudo ufw status numbered
Status: active

     To                         Action      From
     --                         ------      ----
[ 1] Anywhere                   ALLOW FWD   Anywhere on lxdbr0
[ 2] 67/udp on lxdbr0           ALLOW IN    Anywhere                   # DHCP
[ 3] 547/udp on lxdbr0          ALLOW IN    Anywhere                   # DHCPv6
[ 4] 53 on lxdbr0               ALLOW IN    Anywhere                   # DNS
[ 5] 2201/tcp on eth0           ALLOW IN    <authorised_host1>
[ 6] 22,80,443/tcp on lxdbr0    ALLOW FWD   <authorised_host1>         (out) 
[ 7] 22/tcp on lxdbr0           ALLOW FWD   <authorised_host2>         (out) 
[ 8] 22                         ALLOW IN    Anywhere
[ 9] Anywhere                   DENY FWD    Anywhere on eth0
[10] Anywhere on eth0           DENY FWD    Anywhere                   (out)
[11] Anywhere (v6)              ALLOW FWD   Anywhere (v6) on lxdbr0
[12] 67/udp (v6) on lxdbr0      ALLOW IN    Anywhere (v6)              # DHCP
[13] 547/udp (v6) on lxdbr0     ALLOW IN    Anywhere (v6)              # DHCPv6
[14] 22 (v6)                    ALLOW IN    Anywhere (v6)
[15] 53 (v6) on lxdbr0          ALLOW IN    Anywhere (v6)              # DNS
[16] Anywhere (v6)              DENY FWD    Anywhere (v6) on eth0
[17] Anywhere (v6) on eth0      DENY FWD    Anywhere (v6)              (out)

Connections from <authorised_host1> to port 22 on any host on lxdbr0 (matched by rule 6) continue to work after adding DENY FWD on eth0. Following your logic I would expect any packets to be dropped at eth0 by the DENY FWD rule before they reach lxdbr0.

It seems my options are to

  1. do all my filtering on the bridge interface, or
  2. convert all my proxies to nat=false and filter on the physical interface

unless you have another idea.

Please can you get output of sudo nft list ruleset and sudo iptables-save

Sanitized to remove real IP addresses:

$ sudo iptables-save
# Generated by iptables-save v1.8.4 on Fri Jan  5 12:06:11 2024
*raw
:PREROUTING ACCEPT [2912252:739642636]
:OUTPUT ACCEPT [1298701:197341056]
COMMIT
# Completed on Fri Jan  5 12:06:11 2024
# Generated by iptables-save v1.8.4 on Fri Jan  5 12:06:11 2024
*mangle
:PREROUTING ACCEPT [2912252:739642636]
:INPUT ACCEPT [1352319:342735423]
:FORWARD ACCEPT [1052133:337815463]
:OUTPUT ACCEPT [1298702:197341332]
:POSTROUTING ACCEPT [2347893:534986735]
COMMIT
# Completed on Fri Jan  5 12:06:11 2024
# Generated by iptables-save v1.8.4 on Fri Jan  5 12:06:11 2024
*nat
:PREROUTING ACCEPT [911695:93098218]
:INPUT ACCEPT [330725:25045576]
:OUTPUT ACCEPT [2415:190551]
:POSTROUTING ACCEPT [40607:2462180]
COMMIT
# Completed on Fri Jan  5 12:06:11 2024
# Generated by iptables-save v1.8.4 on Fri Jan  5 12:06:11 2024
*filter
:INPUT DROP [8:362]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [1:52]
:ufw-after-forward - [0:0]
:ufw-after-input - [0:0]
:ufw-after-logging-forward - [0:0]
:ufw-after-logging-input - [0:0]
:ufw-after-logging-output - [0:0]
:ufw-after-output - [0:0]
:ufw-before-forward - [0:0]
:ufw-before-input - [0:0]
:ufw-before-logging-forward - [0:0]
:ufw-before-logging-input - [0:0]
:ufw-before-logging-output - [0:0]
:ufw-before-output - [0:0]
:ufw-logging-allow - [0:0]
:ufw-logging-deny - [0:0]
:ufw-not-local - [0:0]
:ufw-reject-forward - [0:0]
:ufw-reject-input - [0:0]
:ufw-reject-output - [0:0]
:ufw-skip-to-policy-forward - [0:0]
:ufw-skip-to-policy-input - [0:0]
:ufw-skip-to-policy-output - [0:0]
:ufw-track-forward - [0:0]
:ufw-track-input - [0:0]
:ufw-track-output - [0:0]
:ufw-user-forward - [0:0]
:ufw-user-input - [0:0]
:ufw-user-limit - [0:0]
:ufw-user-limit-accept - [0:0]
:ufw-user-logging-forward - [0:0]
:ufw-user-logging-input - [0:0]
:ufw-user-logging-output - [0:0]
:ufw-user-output - [0:0]
-A INPUT -j ufw-before-logging-input
-A INPUT -j ufw-before-input
-A INPUT -j ufw-after-input
-A INPUT -j ufw-after-logging-input
-A INPUT -j ufw-reject-input
-A INPUT -j ufw-track-input
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A FORWARD -j ufw-track-forward
-A OUTPUT -j ufw-before-logging-output
-A OUTPUT -j ufw-before-output
-A OUTPUT -j ufw-after-output
-A OUTPUT -j ufw-after-logging-output
-A OUTPUT -j ufw-reject-output
-A OUTPUT -j ufw-track-output
-A ufw-after-input -p udp -m udp --dport 137 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 138 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 139 -j ufw-skip-to-policy-input
-A ufw-after-input -p tcp -m tcp --dport 445 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 67 -j ufw-skip-to-policy-input
-A ufw-after-input -p udp -m udp --dport 68 -j ufw-skip-to-policy-input
-A ufw-after-input -m addrtype --dst-type BROADCAST -j ufw-skip-to-policy-input
-A ufw-after-logging-forward -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-after-logging-input -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-before-forward -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-forward -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-forward -j ufw-user-forward
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-input -m conntrack --ctstate INVALID -j ufw-logging-deny
-A ufw-before-input -m conntrack --ctstate INVALID -j DROP
-A ufw-before-input -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 12 -j ACCEPT
-A ufw-before-input -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A ufw-before-input -p udp -m udp --sport 67 --dport 68 -j ACCEPT
-A ufw-before-input -j ufw-not-local
-A ufw-before-input -d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT
-A ufw-before-input -d 239.255.255.250/32 -p udp -m udp --dport 1900 -j ACCEPT
-A ufw-before-input -j ufw-user-input
-A ufw-before-output -o lo -j ACCEPT
-A ufw-before-output -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-output -j ufw-user-output
-A ufw-logging-allow -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW ALLOW] "
-A ufw-logging-deny -m conntrack --ctstate INVALID -m limit --limit 3/min --limit-burst 10 -j RETURN
-A ufw-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW BLOCK] "
-A ufw-not-local -m addrtype --dst-type LOCAL -j RETURN
-A ufw-not-local -m addrtype --dst-type MULTICAST -j RETURN
-A ufw-not-local -m addrtype --dst-type BROADCAST -j RETURN
-A ufw-not-local -m limit --limit 3/min --limit-burst 10 -j ufw-logging-deny
-A ufw-not-local -j DROP
-A ufw-skip-to-policy-forward -j DROP
-A ufw-skip-to-policy-input -j DROP
-A ufw-skip-to-policy-output -j ACCEPT
-A ufw-track-output -p tcp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-track-output -p udp -m conntrack --ctstate NEW -j ACCEPT
-A ufw-user-forward -i lxdbr0 -j ACCEPT
-A ufw-user-forward -s <authorised_host1> -o lxdbr0 -p tcp -m multiport --dports 22,80,443 -j ACCEPT
-A ufw-user-forward -s <authorised_host2> -o lxdbr0 -p tcp -m tcp --dport 22 -j ACCEPT
-A ufw-user-forward -i eth0 -j DROP
-A ufw-user-forward -o eth0 -j DROP
-A ufw-user-input -i lxdbr0 -p udp -m udp --dport 67 -j ACCEPT
-A ufw-user-input -i lxdbr0 -p udp -m udp --dport 547 -j ACCEPT
-A ufw-user-input -i lxdbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A ufw-user-input -i lxdbr0 -p udp -m udp --dport 53 -j ACCEPT
-A ufw-user-input -p tcp -m tcp --dport 22 -j ACCEPT
-A ufw-user-input -p udp -m udp --dport 22 -j ACCEPT
-A ufw-user-limit -m limit --limit 3/min -j LOG --log-prefix "[UFW LIMIT BLOCK] "
-A ufw-user-limit -j REJECT --reject-with icmp-port-unreachable
-A ufw-user-limit-accept -j ACCEPT
COMMIT
# Completed on Fri Jan  5 12:06:11 2024

I’m wondering if the rules would need to be here:

# Generated by iptables-save v1.8.4 on Fri Jan  5 12:06:11 2024
*nat
:PREROUTING ACCEPT [911695:93098218]
:INPUT ACCEPT [330725:25045576]
:OUTPUT ACCEPT [2415:190551]
:POSTROUTING ACCEPT [40607:2462180]
COMMIT

(Just FYI, I don’t have nftables installed.)

Yes for proxy devices in nat mode you would expect to see some dnat entries.

Its possible LXD is still using nftables even if you dont have it installed.

Try installing it on your host and see if LXD is adding rules there.

This might explain why ufw is interfering with it.