What are the differences between 'lxc config device add <instance> <device> proxy' and 'lxc network forward'?

I am a long-time user of the 4.x LXD branch. (I had all the relevant documentation bookmarked on the old linuxcontainers.org site, now, sadly, all my links are broken.)

I am setting up new containers. All these are to get proxy devices in order to forward certain external public ports to internal container services. I used to do this with some variant of

lxc config device add <instance_name> <device_name> proxy listen=<type>:<addr>:<port>[-<port>][,<port>] connect=<type>:<addr>:<port> bind=<host/instance_name>

Now, looking at the new Ubuntu documentation, I see there is a new-to-me command:

lxc network forward create <network_name> <listen_address> [configuration_options...]

Two questions: 1. When was ‘network forward’ added to LXD? and 2. What is the difference between these two commands?

The proxy device by default runs a separate process that listens on one socket, then connects to the connect socket and passes the data between the two connections. This allows cross-transport forwarding (for example TCP->unix socket). It also allows for listening on the wildcard address.

Additionally the way the proxy process transports the data between network namespaces of containers means that one can setup the proxy device to connect to the local loopback address inside the container without there being a network connection between the host and the container.

This is very flexible. But it comes at a cost of reduced performance (data is transported through the proxy process) and the source IP of the remote client is lost (unless the proxy protocol mode is supported by the target application).

The proxy device also has a nat mode. When operating in this mode, rather than use a separate process, it instead configures DNAT firewall rules on the host. This has the benefit that it does not lose the remote client IP address and that it is more performant because the data doesn’t leave the kernel and have to go through a userland proxy process.

The downside of nat mode is that it is not as flexible. It cannot listen on the wildcard address, it requires the instance to be directly reachable over the network from the host (and thus it requires the instance NIC to have a static IP), and it cannot do cross-transport forwarding.

This brings us to the network forward feature. This feature is supported for bridge and ovn networks. This feature is similar to proxy in nat mode, in that it sets up DNAT rules that forward packets from the external network to the instance.

The main conceptual difference is that, unlike proxy devices, network forwards are defined at the network level and the target address of a network forward is not associated to a specific instance, which means you can setup a network forward to an internal IP address, and you are then free to “move” that target IP between instances without needing to reconfigure the network forward.

1 Like

So, network forward existed in 4.x already?

Only in the form of the proxy device in nat mode.