Call for testing: NetworkManager YAML settings

Hello folks,

This is Lukas Märdian, a software engineer for the Ubuntu Foundations team. I have been working on integrating NetworkManager’s settings plugin with Netplan’s YAML schema, using libnetplan. We’d like to ask the community to help test our prototype of this integration on a wide variety of networking setups.

Rationale and background information

Netplan.io is the “single source of truth” for controlling Ubuntu’s networking stack, i.e. you can find all the information about your network configuration in /etc/netplan/ (or using sudo netplan get) on Ubuntu Server, Ubuntu Cloud images, Ubuntu Core IoT, etc …
On Ubuntu Desktop OTOH we’re deviating a bit from this approach, as NetworkManager instead stores its “keyfile” settings in /etc/NetworkManager/system-connections/.
This is what we want to unify!

In the past, Netplan used a mostly single-sided “generator” approach, i.e.

  1. setup your configuration in /etc/netplan/ (or using sudo netplan set)
  2. call netplan apply to generate settings and configure the backend(s), like systemd-networkd, NetworkManager, Open vSwitch, …

This is now changing as NetworkManager on Ubuntu Desktop needs to be able to feed back information to Netplan’s YAML config (e.g. a WiFi password that was changed in NetworkManager’s settings UI). Therefore, we need a bidirectional integration where NetworkManager can talk to Netplan and vice versa.

How do you feel about NetworkManager’s YAML backend?
  • Works great, I like!
  • Meh.
  • Boo!

0 voters

Backup, Setup (& Recovery)

WARNING: Always backup your NetworkManager “keyfile” connection profiles from /etc/NetworkManager/system-connections/ and the original .deb packages. In case something goes wrong, you want to be able to roll back those settings. And remember you might need to do that offline, if networking is broken:

$ mkdir ~/NetworkManager.bak && cd ~/NetworkManager.bak/
$ sudo cp -r /etc/NetworkManager/system-connections .
## Download current NetworkManager version .deb packages
$ apt download gir1.2-nm-1.0 libnm0 network-manager network-manager-config-connectivity-ubuntu
$ ls -la . # check backup

Note: NetworkManager will automatically create backups of your original keyfiles on upgrade and strore them in /root/NetworkManager.bak/

Next, install NetworkManager packages from the Netplan Everywhere PPA for Ubuntu “Jammy” 22.04 LTS or above:

$ sudo add-apt-repository ppa:canonical-foundations/networkmanager-netplan
$ sudo apt update
$ sudo apt install network-manager
$ apt list network-manager # check version (should contain "netplan")
Offline recovery (in case something goes wrong)
$ cd ~/NetworkManager.bak
$ sudo rm -r /etc/NetworkManager/system-connections
$ sudo cp -r ./system-connections /etc/NetworkManager/
$ sudo find /run/NetworkManager/system-connections/ -name "netplan-*.nmconnection" -exec rm {} \;
$ sudo find /run/NetworkManager/system-connections/ -name "*.nmmeta" -exec rm {} \;
$ sudo rm /etc/netplan/90-NM-*.yaml
$ sudo netplan generate
## Reboot or continue below with full roll-back

If you want to fully roll-back, continue with:

$ sudo rm /etc/apt/sources.list.d/canonical-foundations-ubuntu-networkmanager-netplan-*.list
$ sudo dpkg -i *.deb # install local, original NetworkManager .deb packages
$ sudo apt update
$ sudo systemctl restart NetworkManager.service

Test plan

Whenever you add, modify or delete a connection in NetworkManager (e.g. via settings UI, D-Bus API, nmcli, …) it will be reflected in Netplan’s YAML configuration (i.e. sudo netplan get).

Below we list a few common cases for testing the NetworkManager + Netplan settings backend integration, as well as some inspiration for additional test cases.

Common case 1: Ethernet Connection

Check /etc/netplan for a .yaml file that already contains an ethernet connection and remove it. Then connect your system to a live ethernet connection via NetworkManager and check for the corresponding YAML to be created and make sure the contents look correct.

$ grep -ri ethernet /etc/netplan
/etc/netplan/90-NM-1b0dfe3a-2815-36aa-8e48-f883bbf16829.yaml:  ethernets:
/etc/netplan/90-NM-1b0dfe3a-2815-36aa-8e48-f883bbf16829.yaml:          ethernet._: ""
$ rm /etc/netplan/90-NM-1b0dfe3a-2815-36aa-8e48-f883bbf16829.yaml

## Plug into ethernet
$ cat /etc/netplan/90-NM-1b0dfe3a-2815-36aa-8e48-f883bbf16829.yaml
network:
  version: 2
  ethernets:
    NM-1b0dfe3a-2815-36aa-8e48-f883bbf16829:
      renderer: NetworkManager
      match:
        name: "enp0s31f6"
      dhcp4: true
      dhcp6: true
      ipv6-address-generation: "stable-privacy"
      wakeonlan: true
      networkmanager:
        uuid: "1b0dfe3a-2815-36aa-8e48-f883bbf16829"
        name: "Wired connection 1"
        passthrough:
          connection.autoconnect-priority: "-999"
          connection.timestamp: "1662490134"
          ethernet._: ""
          ipv6.ip6-privacy: "-1"
          proxy._: ""
          .nmmeta.nm-generated: "true"

Common case 2: Wifi Connection

Using nmcli, create a new connection and look for an equivalent change under /etc/netplan/

$ nmcli con add con-name wifi1 type wifi ifname wifi1 ipv4.method manual ipv4.address 10.10.10.4/24 ipv4.gateway 10.10.10.1 ssid <ssid_name>

## should show your new wifi connection
$ nmcli con show
NAME         UUID                                  TYPE  DEVICE 
XXXXX        dffebd3b-f21e-4e0f-9ca0-59667d6ac3ac  wifi  wlp1s0 
XXXXX        ac0346ad-08f0-48d6-a071-11c7631d4374  wifi  --     
wifi1        f7adb62a-8088-4270-ac09-10755064cbb3  wifi  --

## Note there is a 90-NM-dffebd3.. for another wifi network that I've connected to
$ ls -l /etc/netplan 
total 12
-rw-r--r-- 1 root root 104 Apr 19 04:09 01-network-manager-all.yaml
-rw-r--r-- 1 root root 705 Sep 14 16:43 90-NM-dffebd3b-f21e-4e0f-9ca0-59667d6ac3ac.yaml
-rw-r--r-- 1 root root 676 Sep 12 15:28 90-NM-f7adb62a-8088-4270-ac09-10755064cbb3.yaml

See the new YAML file listed and open it up to look and make sure it looks correct

$ cat /etc/netplan/90-NM-f7adb62a-8088-4270-ac09-10755064cbb3.yaml
network:
  version: 2
  wifis:
    NM-f7adb62a-8088-4270-ac09-10755064cbb3:
      renderer: NetworkManager
      match:
        name: "wifi1"
      addresses:
      - "10.10.10.4/24"
      dhcp6: true
      ipv6-address-generation: "stable-privacy"
      access-points:
        "<ssid_name>":
          networkmanager:
            uuid: "f7adb62a-8088-4270-ac09-10755064cbb3"
            name: "wifi1"
            passthrough:
              ipv4.address1: "10.10.10.4/24,10.10.10.1"
              ipv4.method: "manual"
              ipv6.ip6-privacy: "-1"
              proxy._: ""
      networkmanager:
        uuid: "f7adb62a-8088-4270-ac09-10755064cbb3"
        name: "wifi1"

Insipration for further testing

In addition to standard WiFi and ethernet connections, we’d like to ask the community to test a wide variety of networking setups, as we cannot reproduce all the different setups that any of you might encounter in their daily lifes. Including, but not limited to:

  • Enterprise WiFi (e.g. eduroam)
  • WWAN (gsm/cdma modem connections)
  • WiFi AP mode (Fat, Fit, Cloud - and switching between them)
  • Different types of VPN connections (IPSec, WireGuard, VPNC, …)
  • Composite devices, like bridges or bonds
  • Virtual VLAN and Tunnel devices
  • DHCP address assignment (both IPv4 and IPv6)
  • Static IP assignment (both IPv4 and IPv6)
  • Shared network connectivity, e.g. a connection created with:
    nmcli c add con-name shared type ethernet ifname enx000ec6e241bf ipv4.method shared ipv6.method ignore
    

Closing

Please let us know about your experience using the NetworkManager+Netplan prototype, be it positive or negative. Your feedback is highly appreciated! Try to describe the setup you’ve been testing in detail and present any problems that might have occurred.

Errata (Things already known to be problematic):

  • Connections that rely on an additional NetworkManager plugins (e.g. network-manager-vpnc)
1 Like

It’s appears to work fine. An interesting side effect is that my wifi worked automatically in my NetworkManager + iwd setup.

2 Likes

Could we consider instead going all-NetworkManager instead? NetworkManager is fully capable of handling all the various networking cases we care about, and recently cloud-init got support for generating native NetworkManager keyfiles too.

This would also align us with other distributions (RH/Fedora, [open]SUSE) and let us benefit from the shared development going on in NetworkManager for all these use-cases.

Worked fine here. My setup:

  1. ethernet card
  2. wifi
  3. a handful of OpenVPNs

All got correctly reflected both ways, only spewed warnings all over, but that’s probably by design.

netplan: NM-…: handling NetworkManager passthrough device, settings are not fully supported.

One thing that was unexpected, though — in your “Common case 1”, when I removed the Ethernet connection and sudo netplan apply'd, NM reconnected to the wired network immediately, but netplan was unawares. Only after dis- and reconnecting the cord did things show up.

2 Likes

Actually there’s one issue I found. I had a custom route added to an OpenVPN connection:

[ipv4]
method=auto
never-default=true
route1=1.2.3.4/24,

I can’t get that back with the new setup. Even adding the below to the yaml doesn’t help. It may well be a NM bug, as I was only able to get this behaviour (a gateway-less route configuration) in NM through manual editing of the connection files.

passthrough:
  ipv4.route1: "1.2.3.4/24,"
2 Likes