MicroCloud/MicroOvn: exposing containers to the outside world

LXD MicroCloud
1

Dear all,

Similar to this topic, I am trying to expose containers in my MicroCloud setup to “the outside world”, i.e. the rest of my (home) network. My setup is, however, slightly different from the one in the aforementioned topic, so I decided to open a new discussion.

My MicroCloud setup consists of three Lenovo ThinkCentre Tiny M910q computers named wiske{1,2,3}, each with a 256GB NVme SSD (for local storage) and a 2TB SATA SSD (for Ceph).

These snaps are currently installed:

$ snap list
Name        Version                 Rev    Tracking       Publisher   Notes
core20      20240416                2318   latest/stable  canonical✓  base
core22      20240408                1380   latest/stable  canonical✓  base
lxd         5.21.1-2d13beb          28463  latest/stable  canonical✓  in-cohort
microceph   0+git.4a608fc           793    quincy/stable  canonical✓  in-cohort
microcloud  1.1-04a1c49             734    latest/stable  canonical✓  in-cohort
microovn    22.03.3+snap0e23a0e4f5  395    22.03/stable   canonical✓  in-cohort
snapd       2.63                    21759  latest/stable  canonical✓  snapd

Given the compact form factor of these Tiny boxes, they only have one physical NIC installed (and I’m wondering if that is the underlying cause of my problems). After doing a fresh Ubuntu 22.04 server install I reconfigured the NIC on each machine as a bridge. This bridge then gets its IP address assigned via DHCP, but I have configured my router to always give each bridge the same address based on its MAC. My local LAN uses 192.168.10.0/24, with the router’s address being 192.168.10.254. The three bridges in the Tiny’s get assigned 192.168.10.{20,22,24}, respectively.

In the microcloud init phase, I set the address for MicroCloud’s internal traffic to 192.168.10.20, configured 192.168.10.254 as the gateway to the uplink network, set the IPv4 range for LXD to 192.168.10.200–192.168.10.230 (my router’s DHCP range is limited to 192.168.10.1–192.168.10.199 so that shouldn’t interfere).

What (mostly) works:

  • I can ping 8.8.8.8 or google.com from the three Tiny machines
  • I can ping 8.8.8.8 or google.com from the containers, however, strangely enough, only the first packet of a ping series (ping -c4) gets through. All subsequent ones never make it.
  • From the Tiny machines, I can ping machines in my local 192.168.10.x network (via IP or DNS entries)
  • From the containers, I can ping machines in my local 192.168.10.x network (via IP or DNS entries), again only the first ICMP packet seems to get through.

These are the current network configs:

$ lxc network show default
name: default
description: ""
type: ovn
managed: true
status: Created
config:
  bridge.mtu: "1442"
  ipv4.address: 10.235.4.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:a8ee:c58f:9b2f::1/64
  ipv6.nat: "true"
  network: UPLINK
  volatile.network.ipv4.address: 192.168.10.200
  volatile.network.ipv6.address: 2a02:redacted:e19e
used_by:
- /1.0/instances/landscape
- /1.0/instances/logitechmediaserver
- /1.0/profiles/default
locations:
- wiske1
- wiske2
- wiske3

and

$ lxc network show UPLINK
name: UPLINK
description: ""
type: physical
managed: true
status: Created
config:
  ipv4.gateway: 192.168.10.254/24
  ipv4.ovn.ranges: 192.168.10.200-192.168.10.230
  ipv4.routes: 192.168.10.200/32
  ipv6.gateway: 2a02:redacted/64
  volatile.last_state.created: "false"
used_by:
- /1.0/networks/default
locations:
- wiske1
- wiske2
- wiske3

I currently have two containers running in the MicroCloud, one with IP address 10.235.4.3 and one with 10.235.4.4, both of which I would like to make available in the 192.168.10.0/24 local LAN so that the computers there can access the services they run.

I tried to follow the instructions in the aforementioned discourse post:

lxc network set UPLINK ipv4.routes=192.168.10.201/32

followed by

lxc network forward create default 192.168.10.201 target_address=10.235.4.4

What I expected to happen was that the 192.168.10.201 address would (magically?) appear on the LAN network and I would be able to contact the container via that address. However, that didn’t happen.

However, when I replace 192.168.10.201 with 192.168.10.200, i.e. the volatile.network.ipv4.address I can access the services in the container from the LAN.

So it seems like I am halfway there: how can I now provide access to the other container (with internal IP address 10.235.4.3? Or should using the .201 address (or others in the range from .201 to .230) have worked? From the earelier topic and this topic I would have thought the following should have worked:

lxc network set UPLINK ipv4.routes=192.168.10.201/32,192.168.10.202/32
lxc network forward create default 192.168.10.201 target_address=10.235.4.4
lxc network forward create default 192.168.10.202 target_address=10.235.4.3

Any help is much appreciated!

2

Have you ensured that nothing else on your network is allocated the IPs in this range (inclusive) as they must be left free for use by OVN networks.

This issue needs to be resolved first as it suggests there’s some conflict:

  • I can ping 8.8.8.8 or google.com from the containers, however, strangely enough, only the first packet of a ping series (ping -c4) gets through. All subsequent ones never make it.
2 Likes