Bridged networking on Ubuntu Server with systemd-networkd instead network-manager?

I could not get bridged networking working on Ubuntu 22.04.1 LTS Server so far:
Trying to create the bridge automatically by

$ multipass launch --network eno1 
Multipass needs to create a bridge to connect to eno1.                          
This will temporarily disrupt connectivity on that interface.

Do you want to continue (yes/no)? yes
launch failed: Could not create bridge. Could not reach remote D-Bus object: The name org.freedesktop.NetworkManager was not provided by any .service files

fails, because Ubuntu Server uses systemd-networkd and not network-manager to control the network interfaces by default.

Creating the bridge manually with

$ netplan apply

with configuration file:

$ cat /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
  ethernets:
    eno1:
      match:
        macaddress: 74:d0:2b:cb:f3:dc
  bridges:
    br0:
      addresses:
        - 192.168.2.123/24
      interfaces:
        - eno1
      gateway4: 192.168.2.1
      nameservers:
        addresses:
          - 192.168.2.10
      parameters:
        forward-delay: 0
        stp: false
      optional: true
  version: 2

and passing the name of the created bridge:

$ ip a show br0
40: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:93:eb:ae:5e:08 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.123/24 brd 192.168.2.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::b893:ebff:feae:5e08/64 scope link 
       valid_lft forever preferred_lft forever

on invoking

$ multipass launch --network br0
Launched:         enlivened-murrelet                                                

does not attach the instance to the bridge network:

$ multipass list
Name                    State             IPv4             Image
enlivened-murrelet      Running           10.26.22.23      Ubuntu 20.04 LTS

And the kernel log shows:

Aug 26 14:32:50 test-ng systemd-networkd[1012]: br0: Re-configuring with /run/systemd/network/10-netplan-br0.network
Aug 26 14:32:50 test-ng systemd-networkd[1012]: br0: DHCPv6 lease lost
Aug 26 14:32:50 test-ng systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
Aug 26 14:32:50 test-ng sudo[26042]: pam_unix(sudo:session): session closed for user root
Aug 26 14:32:51 test-ng systemd-networkd[1012]: br0: Gained carrier
Aug 26 14:32:51 test-ng systemd-timesyncd[808]: Network configuration changed, trying to establish connection.
Aug 26 14:32:51 test-ng kernel: IPv6: ADDRCONF(NETDEV_CHANGE): br0: link becomes ready
Aug 26 14:32:51 test-ng systemd-timesyncd[808]: Initial synchronization to time server 185.125.190.56:123 (ntp.ubuntu.com).
Aug 26 14:32:52 test-ng systemd-networkd[1012]: br0: Gained IPv6LL
Aug 26 14:33:20 test-ng systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Aug 26 14:40:36 test-ng systemd[2411]: Started snap.multipass.multipass.577ed3e2-c8c4-469e-8ea6-c9d01990b118.scope.
Aug 26 14:40:36 test-ng multipassd[14879]: Using the 'multipass' storage pool.
Aug 26 14:40:46 test-ng systemd[2411]: Started snap.multipass.multipass.72f78346-7d47-4198-b00e-e57e6ad93621.scope.
Aug 26 14:40:46 test-ng multipassd[14879]: Using the 'multipass' storage pool.
Aug 26 14:42:50 test-ng systemd[2411]: Started snap.multipass.multipass.da39fa7d-9ddc-4034-b452-2c38e5d3f348.scope.
Aug 26 14:43:33 test-ng systemd[2411]: Started snap.multipass.multipass.33f7c165-a939-4aea-8f33-438a08e67bab.scope.
Aug 26 14:43:35 test-ng multipassd[14879]: Using the 'multipass' storage pool.
Aug 26 14:43:35 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/proc/14879/mountinfo" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:43:35 test-ng kernel: audit: type=1400 audit(1661525015.790:717): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/proc/14879/mountinfo" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:43:35 test-ng multipassd[14879]: Creating instance with image id: 5567d3311ce96a215e476d2c266be92d026e09fe1a1b210ca27fdf4edb3936e6
Aug 26 14:43:35 test-ng dnsmasq[9884]: read /etc/hosts - 7 addresses
Aug 26 14:43:35 test-ng dnsmasq-dhcp[9884]: read /var/snap/lxd/common/lxd/networks/mpbr0/dnsmasq.hosts/multipass_enlivened-murrelet.eth0
Aug 26 14:43:35 test-ng audit[26256]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_archive-var-snap-lxd-common-lxd-storage-pools-multipass-virtual-machines-multipass_enlivened-murrelet" pid=26256 comm="apparmor_parser"
Aug 26 14:43:35 test-ng kernel: audit: type=1400 audit(1661525015.890:718): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd_archive-var-snap-lxd-common-lxd-storage-pools-multipass-virtual-machines-multipass_enlivened-murrelet" pid=26256 comm="apparmor_parser"
Aug 26 14:43:35 test-ng audit[26262]: AVC apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd_archive-var-snap-lxd-common-lxd-storage-pools-multipass-virtual-machines-multipass_enlivened-murrelet" pid=26262 comm="apparmor_parser"
Aug 26 14:43:35 test-ng kernel: audit: type=1400 audit(1661525015.954:719): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd_archive-var-snap-lxd-common-lxd-storage-pools-multipass-virtual-machines-multipass_enlivened-murrelet" pid=26262 comm="apparmor_parser"
Aug 26 14:43:43 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:43:43 test-ng systemd-udevd[26285]: Using default interface naming scheme 'v249'.
Aug 26 14:43:43 test-ng systemd-networkd[1012]: tapb85e6991: Link UP
Aug 26 14:43:43 test-ng kernel: mpbr0: port 1(tapb85e6991) entered blocking state
Aug 26 14:43:43 test-ng kernel: mpbr0: port 1(tapb85e6991) entered disabled state
Aug 26 14:43:43 test-ng kernel: device tapb85e6991 entered promiscuous mode
Aug 26 14:43:43 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:43:43 test-ng systemd-networkd[1012]: tap1632b249: Link UP
Aug 26 14:43:43 test-ng kernel: br0: port 2(tap1632b249) entered blocking state
Aug 26 14:43:43 test-ng kernel: br0: port 2(tap1632b249) entered disabled state
Aug 26 14:43:43 test-ng kernel: device tap1632b249 entered promiscuous mode
Aug 26 14:43:43 test-ng kernel: br0: port 2(tap1632b249) entered blocking state
Aug 26 14:43:43 test-ng kernel: br0: port 2(tap1632b249) entered forwarding state
Aug 26 14:43:43 test-ng audit[26307]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26307 comm="apparmor_parser"
Aug 26 14:43:43 test-ng kernel: audit: type=1400 audit(1661525023.214:720): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26307 comm="apparmor_parser"
Aug 26 14:43:43 test-ng systemd-networkd[1012]: tapb85e6991: Gained carrier
Aug 26 14:43:43 test-ng systemd-networkd[1012]: mpbr0: Gained carrier
Aug 26 14:43:43 test-ng kernel: mpbr0: port 1(tapb85e6991) entered blocking state
Aug 26 14:43:43 test-ng kernel: mpbr0: port 1(tapb85e6991) entered forwarding state
Aug 26 14:43:43 test-ng systemd-networkd[1012]: tap1632b249: Gained carrier
Aug 26 14:43:43 test-ng multipassd[14879]: Waiting for SSH to be up
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tapb85e6991: Lost carrier
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tapb85e6991) entered disabled state
Aug 26 14:44:00 test-ng kernel: device tap1632b249 left promiscuous mode
Aug 26 14:44:00 test-ng kernel: br0: port 2(tap1632b249) entered disabled state
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap1632b249: Link UP
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap1632b249: Gained carrier
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap1632b249: Link DOWN
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap1632b249: Lost carrier
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 42 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 42 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 42 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 42 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 42 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 42 seen even after reload
Aug 26 14:44:00 test-ng kernel: device tapb85e6991 left promiscuous mode
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tapb85e6991) entered disabled state
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tapb85e6991: Link UP
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tapb85e6991: Link DOWN
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 41 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 41 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 41 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 41 seen even after reload
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 41 seen, reloading interface list
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: ERROR:Unknown interface index 41 seen even after reload
Aug 26 14:44:00 test-ng audit[26428]: AVC apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26428 comm="apparmor_parser"
Aug 26 14:44:00 test-ng kernel: audit: type=1400 audit(1661525040.665:721): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26428 comm="apparmor_parser"
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 43 seen, reloading interface list
Aug 26 14:44:00 test-ng systemd-udevd[26412]: Using default interface naming scheme 'v249'.
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap6c3808f4: Link UP
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tap6c3808f4) entered blocking state
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tap6c3808f4) entered disabled state
Aug 26 14:44:00 test-ng kernel: device tap6c3808f4 entered promiscuous mode
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tap6c3808f4) entered blocking state
Aug 26 14:44:00 test-ng kernel: mpbr0: port 1(tap6c3808f4) entered forwarding state
Aug 26 14:44:00 test-ng networkd-dispatcher[1048]: WARNING:Unknown index 44 seen, reloading interface list
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tapfd19b8f0: Link UP
Aug 26 14:44:00 test-ng kernel: br0: port 2(tapfd19b8f0) entered blocking state
Aug 26 14:44:00 test-ng kernel: br0: port 2(tapfd19b8f0) entered disabled state
Aug 26 14:44:00 test-ng kernel: device tapfd19b8f0 entered promiscuous mode
Aug 26 14:44:00 test-ng kernel: br0: port 2(tapfd19b8f0) entered blocking state
Aug 26 14:44:00 test-ng kernel: br0: port 2(tapfd19b8f0) entered forwarding state
Aug 26 14:44:00 test-ng audit[26450]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26450 comm="apparmor_parser"
Aug 26 14:44:00 test-ng kernel: audit: type=1400 audit(1661525040.789:722): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-multipass_enlivened-murrelet_</var/snap/lxd/common/lxd>" pid=26450 comm="apparmor_parser"
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tap6c3808f4: Gained carrier
Aug 26 14:44:00 test-ng systemd-networkd[1012]: tapfd19b8f0: Gained carrier
Aug 26 14:44:14 test-ng dnsmasq-dhcp[9884]: Ignoring domain multipass for DHCP host name enlivened-murrelet
Aug 26 14:44:20 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:20 test-ng kernel: audit: type=1400 audit(1661525060.429:723): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:20 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:20 test-ng kernel: audit: type=1400 audit(1661525060.497:724): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:20 test-ng multipassd[14879]: Executing '[ -e /var/lib/cloud/instance/boot-finished ]'
Aug 26 14:44:23 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:23 test-ng kernel: audit: type=1400 audit(1661525063.777:725): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:23 test-ng multipassd[14879]: Executing '[ -e /var/lib/cloud/instance/boot-finished ]'
Aug 26 14:44:25 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:25 test-ng kernel: audit: type=1400 audit(1661525065.193:726): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:25 test-ng multipassd[14879]: Executing '[ -e /var/lib/cloud/instance/boot-finished ]'
Aug 26 14:44:26 test-ng audit[14879]: AVC apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:26 test-ng kernel: audit: type=1400 audit(1661525066.557:727): apparmor="DENIED" operation="open" profile="snap.multipass.multipassd" name="/etc/ssh/ssh_config" pid=14879 comm=5468726561642028706F6F6C656429 requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Aug 26 14:44:26 test-ng multipassd[14879]: Executing '[ -e /var/lib/cloud/instance/boot-finished ]'
Aug 26 14:44:26 test-ng multipassd[14879]: Returning setting local.privileged-mounts=true

Installed multipass and lxd version:

$ snap list multipass lxd
Name       Version        Rev    Tracking       Publisher   Notes
lxd        5.0.1-9dcf35b  23541  5.0/stable/…   canonical✓  -
multipass  1.10.1         7689   latest/stable  canonicalâś“  -

3 Likes

Hi @gunterze, sorry you’re having trouble with this. As you found out, Multipass can only create bridges for you through NetworkManager on Linux. We should improve the user experience when that is not the case. In the meantime, I added a piece of documentation covering this.

Another user had a very similar problem and found that NetworkManager and networkd were conflicting somehow. Did you perhaps try to install NetworkManager? Does uninstalling/disabling it help? Let us know what you find!

2 Likes

Manually creating a bridge by netplan apply does not help, using dhcp or static IP/gateway/nameserver configuration, or with or without

      parameters:
        forward-delay: 0
        stp: false

does not make a difference. I can access the host via the new created bridge, but VM instances created by multipass launch --network mybridge does not get attached to the bridge network and journalctl -f shows error messages as listed above.

Yes, I tried to install NetworkManager in a first attempt, but purged it again after realizing the conflict with systemd-networkd. dpkg -l network-manager shows now:

$ dpkg -l network-manager
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name            Version      Architecture Description
+++-===============-============-============-=================================
un  network-manager <none>       <none>       (no description available)

One difference between the manual created bridge on Ubuntu Server:

$ ip a show mybridge
6: mybridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:8e:1a:e8:f7:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.189/24 metric 100 brd 192.168.2.255 scope global dynamic mybridge
       valid_lft 85539sec preferred_lft 85539sec
    inet6 fe80::e08e:1aff:fee8:f7f9/64 scope link 
       valid_lft forever preferred_lft forever

to the bridge generated by multipass on Ubuntu Desktop:

$ ip a show br-enxc025a54b5
4: br-enxc025a54b5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 46:e3:c4:52:74:1e brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.169/24 brd 192.168.2.255 scope global dynamic noprefixroute br-enxc025a54b5
       valid_lft 65442sec preferred_lft 65442sec
    inet6 fe80::95c1:79a0:1157:5929/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

is the attribute noprefixroute, only shown for the second.

For completeness: I also have installed docker, on Ubuntu Server by snap

$ sudo snap list docker
Name    Version   Rev   Tracking       Publisher   Notes
docker  20.10.14  1779  latest/stable  canonicalâś“  -

but also on Ubuntu Desktop by apt:

$ dpkg -l docker*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                      Version                     Architecture Description
+++-=========================-===========================-============-===================>
un  docker                    <none>                      <none>       (no description ava>
ii  docker-ce                 5:20.10.17~3-0~ubuntu-focal amd64        Docker: the open-so>
ii  docker-ce-cli             5:20.10.17~3-0~ubuntu-focal amd64        Docker CLI: the ope>
ii  docker-ce-rootless-extras 5:20.10.17~3-0~ubuntu-focal amd64        Rootless support fo>
un  docker-engine             <none>                      <none>       (no description ava>
un  docker-engine-cs          <none>                      <none>       (no description ava>
ii  docker-scan-plugin        0.17.0~ubuntu-focal         amd64        Docker scan cli plu>
un  docker.io                 <none>                      <none>       (no description ava>

Hi @gunterze, does it help if you add dhcp4: true to your br0 in the netplan config? And if you drop everything else apart from interfaces? So just:

  br0:
    dhcp4: true
    interfaces:
      - eno1

If that doesn’t help, here are a few other things you could try to troubleshoot:

  • Do you see the extra link inside the instance at all (multipass exec <instance> -- ip l)? And, to be sure, multipass exec <instance> -- ip a confirms it didn’t get an IP, correct?
  • Anything relevant in /var/log/cloud-init.log? Perhaps look for the link name in there (e.g. enp6s0).
  • Any firewall on your host? You could try disabling it (it might be blocking DHCP to the instance).
  • You don’t happen to have the bridge that Multipass created still lying around in your server, do you? It shouldn’t matter, but who knows…
1 Like

Using dhcp or manual ip configuration does not make a difference. E.g.

$ cat /etc/netplan/50-custom.yaml

network:
  bridges:
    mybridge:
      dhcp4: true
      interfaces:
        - eno1

creating

$ ip a show mybridge
20: mybridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:8e:1a:e8:f7:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.189/24 metric 100 brd 192.168.2.255 scope global dynamic mybridge
       valid_lft 85838sec preferred_lft 85838sec
    inet6 fe80::e08e:1aff:fee8:f7f9/64 scope link 
       valid_lft forever preferred_lft foreve

listed by

$ multipass networks
Name      Type      Description
docker0   bridge    Network bridge
eno1      ethernet  Ethernet device
mpbr0     bridge    Network bridge for Multipass
mybridge  bridge    Network bridge with eno1

referred on launching

$ multipass launch --network mybridge
Launched: scholarly-gannet 

results in

$ multipass exec scholarly-gannet -- ip a                       
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 52:54:00:1b:9a:f2 brd ff:ff:ff:ff:ff:ff
    inet 10.26.22.120/24 brd 10.26.22.255 scope global dynamic enp5s0
       valid_lft 3547sec preferred_lft 3547sec
    inet6 fd42:51cb:3510:9641:5054:ff:fe1b:9af2/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 3596sec preferred_lft 3596sec
    inet6 fe80::5054:ff:fe1b:9af2/64 scope link 
       valid_lft forever preferred_lft forever
3: enp6s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 52:54:00:16:60:f6 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::5054:ff:fe16:60f6/64 scope link 
       valid_lft forever preferred_lft forever

seeing nothing suspicious in

$ multipass exec scholarly-gannet -- grep enp6s0 /var/log/cloud-init.log
2022-08-30 11:41:06,876 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/address (quiet=False)
2022-08-30 11:41:06,876 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp6s0/address
2022-08-30 11:41:06,877 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/address (quiet=False)
2022-08-30 11:41:06,877 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp6s0/address
2022-08-30 11:41:06,878 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/addr_assign_type (quiet=False)
2022-08-30 11:41:06,878 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/enp6s0/addr_assign_type
2022-08-30 11:41:06,882 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/uevent (quiet=False)
2022-08-30 11:41:06,882 - util.py[DEBUG]: Read 27 bytes from /sys/class/net/enp6s0/uevent
2022-08-30 11:41:06,882 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/address (quiet=False)
2022-08-30 11:41:06,882 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp6s0/address
2022-08-30 11:41:06,882 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/device/device (quiet=False)
2022-08-30 11:41:06,882 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/enp6s0/device/device
2022-08-30 11:41:06,882 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/type (quiet=False)
2022-08-30 11:41:06,882 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/enp6s0/type
2022-08-30 11:41:06,884 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/addr_assign_type (quiet=False)
2022-08-30 11:41:06,884 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/enp6s0/addr_assign_type
2022-08-30 11:41:06,884 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/uevent (quiet=False)
2022-08-30 11:41:06,884 - util.py[DEBUG]: Read 27 bytes from /sys/class/net/enp6s0/uevent
2022-08-30 11:41:06,884 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/address (quiet=False)
2022-08-30 11:41:06,884 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp6s0/address
2022-08-30 11:41:06,884 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/device/device (quiet=False)
2022-08-30 11:41:06,884 - util.py[DEBUG]: Read 7 bytes from /sys/class/net/enp6s0/device/device
2022-08-30 11:41:06,885 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/type (quiet=False)
2022-08-30 11:41:06,885 - util.py[DEBUG]: Read 2 bytes from /sys/class/net/enp6s0/type
{'type': 'physical', 'mac_address': '52:54:00:16:60:f6', 'name': 'enp6s0', 'match': {'macaddress': '52:54:00:16:60:f6'}, 'subnets': [{'type': 'dhcp4', 'metric': 200}]}
2022-08-30 11:41:07,165 - subp.py[DEBUG]: Running command ['udevadm', 'test-builtin', 'net_setup_link', '/sys/class/net/enp6s0'] with allowed return codes [0] (shell=False, capture=True)
2022-08-30 11:41:09,474 - util.py[DEBUG]: Reading from /sys/class/net/enp6s0/address (quiet=False)
2022-08-30 11:41:09,474 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp6s0/address

Just see WARNING/ERROR logs in

$ journalctl -u networkd-dispatcher | tail
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 21 seen, reloading interface list
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: ERROR:Unknown interface index 21 seen even after reload
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 21 seen, reloading interface list
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: ERROR:Unknown interface index 21 seen even after reload
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 21 seen, reloading interface list
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: ERROR:Unknown interface index 21 seen even after reload
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 21 seen, reloading interface list
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: ERROR:Unknown interface index 21 seen even after reload
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 23 seen, reloading interface list
Aug 30 11:40:51 test-ng networkd-dispatcher[964]: WARNING:Unknown index 24 seen, reloading interface list

No firewall:

$ sudo ufw status
Status: inactive

Bridges:

$ ip a show type bridge
5: mpbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:16:3e:b3:30:da brd ff:ff:ff:ff:ff:ff
    inet 10.26.22.1/24 scope global mpbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:51cb:3510:9641::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:feb3:30da/64 scope link 
       valid_lft forever preferred_lft forever
14: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:d7:c2:09:1f brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
20: mybridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether e2:8e:1a:e8:f7:f9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.189/24 metric 100 brd 192.168.2.255 scope global dynamic mybridge
       valid_lft 85443sec preferred_lft 85443sec
    inet6 fe80::e08e:1aff:fee8:f7f9/64 scope link 
       valid_lft forever preferred_lft forever

Already verified, that deinstalling docker and deleting docker0 bridge does not help.

Some log messages of systemd-networkd on creating the bridge may seem suspicious:

Aug 30 11:38:16 test-ng systemd-networkd[943]: eno1: Failed to set master interface: Invalid argument
Aug 30 11:38:16 test-ng systemd-networkd[943]: eno1: Failed
Aug 30 11:38:26 test-ng systemd-networkd[943]: mybridge: netdev ready
Aug 30 11:38:26 test-ng systemd-networkd[943]: eno1: Re-configuring with /run/systemd/network/10-netplan-eno1.network
Aug 30 11:38:26 test-ng systemd-networkd[943]: eno1: Failed to set bridge configurations, ignoring: Operation not supported
Aug 30 11:38:26 test-ng systemd-networkd[943]: mybridge: Link UP
Aug 30 11:38:27 test-ng systemd-networkd[943]: mybridge: Gained carrier
Aug 30 11:38:27 test-ng systemd-networkd[943]: mybridge: DHCPv4 address 192.168.2.189/24 via 192.168.2.1
Aug 30 11:38:29 test-ng systemd-networkd[943]: mybridge: Gained IPv6LL

with

$ cat /run/systemd/network/10-netplan-eno1.network
[Match]
Name=eno1

[Network]
DHCP=ipv4
LinkLocalAddressing=no
Bridge=mybridge

[DHCP]
RouteMetric=100
UseMTU=true

?

I’m running into similar difficulty with Multipass + LXD + Netplan.

Also using Ubuntu 22.04.1 LTS Server, though with different versions of lxd and multipass

$ snap list multipass lxd
Name       Version      Rev    Tracking       Publisher   Notes
lxd        5.6-794016a  23680  latest/stable  canonicalâś“  -
multipass  1.10.1       8071   latest/beta    canonicalâś“  -

My bridge config is substantially similar, and instances created with --network br0 also do not receive an IP from DHCP.

When a static IP is configured on an instance, packet captures show broadcast traffic making it to the instance, and traffic from the instance makes it out to the wire. Inbound traffic (coming off the wire, destined to the instance’s IP/mac) is seen on the host, but does not show up on the instance. For example, ARP requests for the instance’s IP fail, but ARP requests FROM the instance make it out to the switch.

Further, I found that when the qemu driver is in use, single-network-interface instances have normal network connectivity (can get to the internet, etc). When the lxd driver is in use, the same instances cannot reach anything other than the parent host. It seems to me that when lxd is in use, the host does not forward traffic for the host, regardless of what interface is in use.

# multipass get local.driver
qemu
# multipass launch
Launched: justified-angler                                                      
# multipass exec justified-angler -- ping -a 1.1.1.1 -c 2
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=3.55 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=3.82 ms

--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 3.547/3.681/3.816/0.134 ms
  • deleted instance, changed driver, rebooted host (just because)
# multipass get local.driver
lxd
# multipass launch
Launched: neutral-spittlebug                                                    
# multipass exec neutral-spittlebug -- ping -a 1.1.1.1 -c 2
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.

--- 1.1.1.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1007ms

When the lxd driver is in use, the same instances cannot reach anything other than the parent host

Hmm, something is definitely wrong then. Is everything well with the multipass bridge? What do ip link and ip addr say, when your LXD-based instance is running?

instances created with --network br0 also do not receive an IP from DHCP

Probably best to solve basic connectivity first, but how did you check for the extra IP? And what image was the instance based on? I ask because there was a bug on 1.10 where Multipass did not show extra IPs on Jammy-based instances (currently the default), even if the instance got them.

Are you resolve issue ? I have identical problem, before few days it’s worked without problem but today I can’t get LAN IP :frowning:

Just update on this, fixed problem by running iptables -I DOCKER-USER -j ACCEPT.

2 Likes

Thank you for finding and reporting this solution! How did you find it? This command fixed the problem that I was encountering on two hosts where LXD containers using a bridged network could not receive IP addresses from the DHCP server on the host network.

Host 1:
Raspberry Pi 4 8 GB
Ubuntu Server 22.04
LXD 5.11
Docker 23.0.1

Host 2:
AMD Ryzen 3
Ubuntu Server 20.04
LXD 5.11
Docker 22.10.21

1 Like

Prevent issues with LXD and Docker describes some firewall conflicts that users might encounter when running both LXD and Docker on the same host. This advice might also apply to Multipass.

1 Like