LXD networking breaking after a reboot

LXD Support
1

I installed LXD yesterday, with a view to using it with snapcraft to build/update some snaps.

I went through the usual sudo lxd init and selected auto for the ipv4 option and none for the ipv6 option. I do not wish to use ipv6 at all (please do not try to sell me on the advantages of ipv6 over ipv4), because of the network setup I cannot control.

Today, I started my laptop and now none of the LXD containers can reach the network. It worked perfectly fine yesterday, repeatedly. But today, it’s broken. This isn’t the first time this has happened. Typically I’ll just uninstall LXD and forget about it, because I’m unable to get it fixed.

Today, I just deleted all the containers, then restarted the snapd.lxd.daemon process, and created a new container. The container has NO networking. Why!?

Please, what can I do to ensure LXD doesn’t break on reboot?

$ lxc launch ubuntu:22.04 u1                                                                                                                                                                               
Creating u1                                                                                                                                                                                                                                   
Starting u1                                     

$ lxc shell u1                                                                      
root@u1:~# IP --brief address show                                                                                     
lo               UNKNOWN        127.0.0.1/8 ::1/128                                                                    
eth0@if13        UP             10.218.241.156/24 metric 100 fe80::216:3eff:feee:bf6/64            

root@u1:~# curl --connect-timeout 5 -v https://discourse.ubuntu.com
*   Trying 162.213.33.6:443...
* After 2489ms connect time, move on!
* connect to 162.213.33.6 port 443 failed: Connection timed out
*   Trying 162.213.33.126:443...
* After 1244ms connect time, move on!
* connect to 162.213.33.126 port 443 failed: Connection timed out
* Failed to connect to discourse.ubuntu.com port 443 after 3757 ms: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to discourse.ubuntu.com port 443 after 3757 ms: Connection timed out

Exit the container, try from the host:

root@u1:~#                                                                                                                                                                                                                                    
logout                                                                                                                                                                                                                                        
$ curl --connect-timeout 5 -v https://discourse.ubuntu.com                                                                                                                                              
*   Trying 162.213.33.6:443...                                                                                                                                                                                                                
* Connected to discourse.ubuntu.com (162.213.33.6) port 443 (#0)                                                                                                                                                                              
* ALPN: offers h2,http/1.1                                                                                             
* TLSv1.3 (OUT), TLS handshake, Client hello (1):                                                                                                                                                                                             
*  CAfile: /etc/ssl/certs/ca-certificates.crt                                                                          
*  CApath: /etc/ssl/certs                                                                                              
* TLSv1.3 (IN), TLS handshake, Server hello (2):                                                                                                                                                                                              
* TLSv1.2 (IN), TLS handshake, Certificate (11):                                                                                                                                                                                              
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):                                                                                                                                                                                      
* TLSv1.2 (IN), TLS handshake, Server finished (14):                                                                                                                                                                                          
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):                                                                                                                                                                                     
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):                                                            
* TLSv1.2 (OUT), TLS handshake, Finished (20):                                                                                                                                                                                                
* TLSv1.2 (IN), TLS handshake, Finished (20):                                                                                                                                                                                                 
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256                                                                                                                                                                                  
* ALPN: server accepted http/1.1                                                                                                                                                                                                              
* Server certificate:                                                                                                                                                                                                                         
*  subject: CN=discourse.ubuntu.com

etc etc.

I’m using lxd 5.17-e5ead86 from the stable channel on Ubuntu 23.04.

2

Hi,

So your examples here show that:

  1. DHCP is working - your instances are getting IPs from lxdbr0.
  2. DNS is working - your curl request is resolving to an IP using the DNS server on lxdbr0.

What this might be is an issue with the firewall on the host preventing traffic to/from lxdbr0 and the external interface or somehow disabling SNAT.

We do sometimes see reports of users have connectivity issues when Docker is installed on the host as well, or if they are using ufw firewall:

We have some parts of our documentation that give some ways to workaround those issues:

https://documentation.ubuntu.com/lxd/en/latest/howto/network_bridge_firewalld/#prevent-connectivity-issues-with-lxd-and-docker

https://documentation.ubuntu.com/lxd/en/latest/howto/network_bridge_firewalld/#ufw-add-rules-for-the-bridge

If those don’t fix it, then the next step I would use is to run tcpdump -i <external interface> -nn and see if a ping from inside one of the instances makes it out onto the external interface and whether the source IP has been SNATed to the host’s IP on the external network.

3

Thanks for the prompt response @tomp!

I have ejected docker into the sun, made sure the ipv4 forwarding was configured correctly, rebooted, sacrificed a chicken in a pentagram, and it’s working again!

:beers: cheers!

1 Like