Hello everyone,
I have a cluster of 3 Ubuntu 22.04 servers using MicroCloud and MicroOVN (no MicroCeph). The network was entirely set up by MicroCloud + MicroOVN. My setup includes:
- Server1 (Master)
- Server2
- Server3
Goal:
I aim to create an interconnected LXC network across a cluster of 3 nodes, where containers can communicate seamlessly with each other and be accessible from any of the three LXC cluster servers. The ultimate objective is to expose services hosted within the containers, allowing external servers or systems to connect to these services as needed.
To achieve this, I expected MicroCloud and MicroOVN to provide a fully functional and interconnected network for the cluster. However, I am currently facing issues connecting to the containers from the host servers within the cluster, which blocks further testing and use of this setup.
The problem is that I have an LXC container running on one of the nodes (server2), and I cannot connect to it via SSH or ping from any of the three servers.
Here are the details of my configuration:
LXC Container
lxc ls
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
+------+---------+--------------------+------+-----------+-----------+-----------------------------+
| u1 | RUNNING | 10.50.227.3 (eth0) | | CONTAINER | 0 | server2 |
Ping Results
ping 10.50.227.3
PING 10.50.227.3 (10.50.227.3) 56(84) bytes of data.
^C
--- 10.50.227.3 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1001ms
LXD Networks
lxc network ls
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| UPLINK | physical | YES | | | | 1 | CREATED |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| br0 | bridge | NO | | | | 0 | |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| br-int | bridge | NO | | | | 0 | |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| default | ovn | YES | 10.50.227.1/24 | fd42:c0ec:8df1:40da::1/64 | Default OVN network | 3 | CREATED |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| ens3 | physical | NO | | | | 1 | |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
| lxdovn1 | bridge | NO | | | | 0 | |
+---------+----------+---------+----------------+---------------------------+---------------------+---------+---------+
“Default” Network
lxc network show default
name: default
description: Default OVN network
type: ovn
managed: true
status: Created
config:
bridge.mtu: "1442"
ipv4.address: 10.50.227.1/24
ipv4.nat: "true"
ipv6.address: fd42:c0ec:8df1:40da::1/64
ipv6.nat: "true"
network: UPLINK
volatile.network.ipv4.address: 10.2.123.1
volatile.network.ipv6.address: fd42:2:1234:1234:216:3eff:fe53:b087
used_by:
- /1.0/instances/u1
- /1.0/instances/u5
- /1.0/profiles/default
locations:
- server1
- server2
- server3
“UPLINK” Network
lxc network show UPLINK
name: UPLINK
description: ""
type: physical
managed: true
status: Created
config:
dns.nameservers: 10.2.123.36
ipv4.gateway: 10.2.123.1/24
ipv4.ovn.ranges: 10.2.123.100-10.2.123.120
ipv6.gateway: fd42:2:1234:1234::1/64
volatile.last_state.created: "false"
used_by:
- /1.0/networks/default
locations:
- server1
- server2
- server3
Server Routes
ip route
default via 10.0.0.62 dev br0 proto static
10.0.0.48/28 dev br0 proto kernel scope link src 10.0.0.49
Network Interfaces
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 50:6b:8d:a0:aa:44 brd ff:ff:ff:ff:ff:ff
altname enp0s3
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether f2:12:6c:97:76:c7 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.49/28 brd 10.0.0.63 scope global br0
valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 52:54:ef:f3:4f:f8 brd ff:ff:ff:ff:ff:ff
inet6 fe80::5054:efff:fef3:4ff8/64 scope link
valid_lft forever preferred_lft forever
5: lxdovn1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 4e:16:a7:19:e1:44 brd ff:ff:ff:ff:ff:ff
inet6 fe80::4c16:a7ff:fe19:e144/64 scope link
valid_lft forever preferred_lft forever
6: br-int: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether 7a:db:29:cd:3b:ab brd ff:ff:ff:ff:ff:ff
inet6 fe80::78db:29ff:fecd:3bab/64 scope link
valid_lft forever preferred_lft forever
7: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
link/ether 52:d0:6a:cf:2b:be brd ff:ff:ff:ff:ff:ff
inet6 fe80::ac83:d0ff:fe77:8aa4/64 scope link
valid_lft forever preferred_lft forever
9: veth28d0d963@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 0a:ff:11:40:aa:31 brd ff:ff:ff:ff:ff:ff link-netnsid 0
Does anyone have an idea why I can’t connect to the container?
Any help is greatly appreciated!