Questions regarding "How to increase the network bandwidth" LXD how-to guide

LXD Support
1

Regarding the LXD how-to guide: Production Setup > “How to increase the network bandwidth” at https://documentation.ubuntu.com/lxd/en/stable-5.21/howto/network_increase_bandwidth/

I’ve been trying to figure this out for a few hours tonight.

I believe, unless something else pops up, that I am only stuck on this one part from those instructions:

To make the change permanent, add the following command to your interface configuration in /etc/network/interfaces
up ip link set eth0 txqueuelen 10000

On Ubuntu 22.04 there is no /etc/network/interfaces file. After some research I learned that method was removed & replaced with Netplan/Cloudinit.

My Netplan YAML file is at /etc/netplan/00-installer-config.yaml and includes:

# This is the network config written by 'subiquity'
network:
  ethernets:
    eno1:
      addresses:
      - IP_ADDRESS/29
      critical: true
      dhcp-identifier: mac
      gateway4: IP_ADDRESS
      match:
        macaddress: MAC_ADDRESS
      nameservers:
        addresses:
        - IPADDRESS
        search:
        - DOMAIN.TLD
  version: 2

Looked through https://netplan.readthedocs.io/en/stable/netplan-yaml/ but couldn’t find anything helpful regarding txqueuelen setting for that file.

What is the best / proper way to increase the txqueuelen value to 10000? (in Ubuntu 22.04)

Can a setting for txqueuelen be added into that /etc/netplan/ YAML file above?

2

If I understand the command/docs correctly the up ip link set eth0 txqueuelen 10000 is using the up hook to call the given command in case the interface is up.

There doesn’t seem to be a direct replacement in netplan for those hooks but according to https://netplan.io/faq you might want to check networkd-dispatcher to perform this action:

Instead to achieve this functionality with the networkd renderer users can use networkd-dispatcher. The package provides users and legacy packages hook points when specific network states are reached to aid in reacting to network state

But it looks the LXD docs require an update too.

1 Like
3

netplan doesn’t seem to support it but systemd-networkd does https://www.freedesktop.org/software/systemd/man/latest/systemd.link.html#TransmitQueueLength=
Presumably, an override would do.

Alternatively, a systemd-tmpfiles snippet (see man tmpfiles.d) managing /sys/devices/virtual/net/${IFACE}/tx_queue_len would also do.

It’s also possible to use a udev rule as proposed in https://askubuntu.com/questions/1033299/permanently-set-txqueuelen-in-18-04

4

I was able to get this all figured out with a bunch of additional Googling along with help from both of your comments.

The LXD doc definitely requires an update. I just submitted edits to that page so hopefully they are approved as it would have saved me hours of getting this all sorted out if it was already in that page.

What I ended up doing:

To increase txqueuelen on the eth & lxdbr0 LXD bridge, I added the below lines to /etc/udev/rules.d/60-custom-txqueuelen.rules

KERNEL==“enp5s0f1”, RUN+=“/sbin/ip link set %k txqueuelen 10000”
KERNEL==“lxdbr0”, RUN+=“/sbin/ip link set %k txqueuelen 10000”

Apply those changes via:

sudo udevadm trigger

To increase the receive queue length, edit /etc/sysctl.conf and add the following line:

net.core.netdev_max_backlog = 182757

Apply those changes via:

sudo sysctl -p

Then for the LXD containers, I changed queue.tx.length via the default LXD profile with the following command:

sudo lxc profile device set default eth0 queue.tx.length “10000”

Pull request to add these changes into the documentation is at Update network_increase_bandwidth.md document with details for u18.04+ by JohnHammell · Pull Request #13457 · canonical/lxd · GitHub and is for the doc page at https://documentation.ubuntu.com/lxd/en/stable-5.21/howto/network_increase_bandwidth/

1 Like