Accessing services inside the microcloud from external world (OVN)

LXD Support
1

Hi

it looks like I have all the components of microcloud up and running as following four commands receive all **online** status from all three nodes(all arm64 running Ubuntu Core 22)

   microcloud cluster list
   microceph cluster list
   microovn cluster list
   lxc cluster list

my next question is, I have lxd container installed in the microcloud and I am able to access the instance and install my snap which has flask server running(it is using port 5001) I want to access this server from machines which are in the same subnet as uplink network of microcloud

route setup inside the container look like this

default via 10.66.61.1 dev eth0 proto dhcp src 10.66.61.2 metric 100 
10.66.61.0/24 dev eth0 proto kernel scope link src 10.66.61.2 metric 100 
10.66.61.1 dev eth0 proto dhcp scope link src 10.66.61.2 metric 100 
192.168.3.254 via 10.66.61.1 dev eth0 proto dhcp src 10.66.61.2 metric 100

My network setup on uplink looks something like this

lxc network show UPLINK
config:
    ipv4.gateway: 192.168.3.254/24
    ipv4.ovn.ranges: 192.168.3.30-192.168.3.100
    volatile.last_state.created: "false"
description: ""
name: UPLINK
type: physical
used_by:
    - /1.0/networks/default
managed: true
status: Created
locations:
    - mtkubuntu
    - ubuntu
    - cobuntu
2

@tomp if you need any further information please let me know.

3

Do you know where this route came from? It looks out of place (and unnecessary given its the same as the default route) for an instance connected to an OVN network?

4

You have some options depending on your setup:

  1. You can use the proxy device on your container. This creates a local listener on the host where the container is running and forwards it into the container. Note: This isn’t related to the OVN or uplink network, and only works for IPs that are bound to the host itself. See also https://www.youtube.com/watch?v=TmGvbXfwJEA
  2. If you want an entire IP from the uplink network routed to the container you can the ipv{n}.routes.external settings. Please note these require that the IPs be marked as routable in LXD’s uplink network by setting ipv{n}.routes.
  3. If you just want to forward certain ports from an IP on the uplink network to the internal IP of the container on the OVN network then you can use network forwards. Because of the way network forwards use a static internal IP for the target, you should also configure the ovn NIC on your container to have static IPs using the ipv{n}.address settings.
5

@tomp ,
Thanks for your reply.
This route probably came while setting up ovn networking during execution of “microcloud init”
The address 192.168.3.254 is address of my physical gateway on uplink network.
Question is am I doing something wrong here?

6

I’m not sure, it shouldn’t be causing any problems, but strange that its been during microcloud setup.
Can you show me your netplan config after a fresh reboot of your container?

7

@tomp,
This is how netplan config of my container instance look like

cat /etc/netplan/50-cloud-init.yaml 
# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    version: 2
ethernets:
    eth0:
        dhcp4: true
8

And the static route for 192.168.3.254 is still there after a reboot?

9

Can you show lxc network show <ovn network> please?

10

yes

root@u2204:~# ip r
default via 10.66.61.1 dev eth0 proto dhcp src 10.66.61.2 metric 100 
10.66.61.0/24 dev eth0 proto kernel scope link src 10.66.61.2 metric 100 
10.66.61.1 dev eth0 proto dhcp scope link src 10.66.61.2 metric 100 
192.168.3.254 via 10.66.61.1 dev eth0 proto dhcp src 10.66.61.2 metric 100
11 
root@mtkubuntu:/home/prashantdhumal# lxc network list
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
|  NAME   |   TYPE   | MANAGED |     IPV4      |           IPV6           | DESCRIPTION | USED BY 
|  STATE  |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| UPLINK  | physical | YES     |               |                          |             | 1       | CREATED |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| br-int  | bridge   | NO      |               |                          |             | 0       |         |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| default | ovn      | YES     | 10.66.61.1/24 | fd42:d57a:2350:958::1/64 |             | 3       | 
CREATED |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| eth0    | physical | NO      |               |                          |             | 1       |         |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| lxdovn1 | bridge   | NO      |               |                          |             | 0       |         |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
| wlp1s0  | physical | NO      |               |                          |             | 0       |         |
+---------+----------+---------+---------------+--------------------------+-------------+---------+---------+
root@mtkubuntu:/home/prashantdhumal# lxc network show default
config:
  bridge.mtu: "1442"
  ipv4.address: 10.66.61.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:d57a:2350:958::1/64
  ipv6.nat: "true"
  network: UPLINK
  volatile.network.ipv4.address: 192.168.3.30
description: ""
name: default
type: ovn
used_by:
- /1.0/instances/a1
- /1.0/instances/u2204
- /1.0/profiles/default
managed: true
status: Created
locations:
- mtkubuntu
- ubuntu
- cobuntu
12

Thanks.

Can you show me the output of lxc config show <instance> --expanded please?

13

lxc config show u2204 --expanded
architecture: aarch64
config:
image.architecture: arm64
image.description: ubuntu 22.04 LTS arm64 (release) (20230914)
image.label: release
image.os: ubuntu
image.release: jammy
image.serial: “20230914”
image.type: squashfs
image.version: “22.04”
volatile.base_image: 0c084cb8769727cd8c0af5daeeeb933992fcc6f1bbe53ec63f24729b13c18aca
volatile.cloud-init.instance-id: 9befb17b-bcc8-4c3c-9f94-1d44bf358da8
volatile.eth0.host_name: vethc009c0f7
volatile.eth0.hwaddr: 00:16:3e:e3:aa:ef
volatile.eth0.last_state.ip_addresses: 10.66.61.2,fd42:d57a:2350:958:216:3eff:fee3:aaef
volatile.idmap.base: “0”
volatile.idmap.current: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.idmap.next: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.last_state.idmap: ‘[]’
volatile.last_state.power: RUNNING
volatile.uuid: e4889342-76c3-4c46-8bf4-2d2308fd569a
volatile.uuid.generation: e4889342-76c3-4c46-8bf4-2d2308fd569a
devices:
eth0:
name: eth0
network: default
type: nic
root:
path: /
pool: remote
type: disk
ephemeral: false
profiles:

  • default
    stateful: false
    description: “”
14

Ah I tested this and it seems to be a behaviour of Ubuntu images and the way it responds to DHCP responses from OVN. Its not causing a problem though.

See https://www.freedesktop.org/software/systemd/man/systemd.network.html

RoutesToDNS=
When true, the routes to the DNS servers received from the DHCP server will be configured. When UseDNS= is disabled, this setting is ignored. Defaults to true.