| Project | MicroCloud |
| Status | Drafting |
| Author(s) | @gabrielmougard |
| Approver(s) | @tomp @fnordahl |
| Release | |
| Internal ID | LX060 |
Abstract
Introduce an OVN underlay network for handling east-west traffic.
Rationale
As of the current state, MicroCloud relies on its management network for handling “east-west” traffic, which imposes certain limitations. One of the constraints is the potential for network congestion and reduced performance due to the management network being overburdened with both control and data plane traffic. This setup can lead to inefficiencies, particularly in environments with high network traffic or large-scale deployments. This current setup, while functional for smaller or less demanding environments, might not scale efficiently or meet the performance and flexibility needs of more complex, dynamic, or larger-scale cloud deployments (e.g: the nodes have a 100GB NIC that they would use for the data plane and a 1GB for the management plane).
Specification
To address the current limitations and enhance network efficiency in MicroCloud, we propose to give the option to the user to define a dedicated OVN underlay network for east-west traffic. By doing so, we aim to alleviate network congestion, improve overall performance, and enable more efficient scaling:
- The proposal includes introducing configurable service handlers and dynamic configuration capabilities within the MicroCloud and MicroOVN codebases.
- This will allow for greater flexibility and control over network settings, including the option to specify custom IP addresses for OVN Geneve tunnel encapsulation.
- Such configurability ensures that the network can be optimized according to the specific needs of different deployments.
Manual interactive setup (microcloud init)
We would start by asking the following question:
-
Configure dedicated underlay networking? (yes/no) [default=no]
This question is asked after the OVN management network questions. Ifnois answered, we skip the underlay configuration and we configure MicroCloud as usual. Ifyesis answered, we’ll give to the user a list of all the network interfaces with an IP address among the detected mDNS nodes. The user will need to choose one per node. If done properly, eachInitSystemstruct should have a non-nilOVNGeneveAddrfield containing the chosen IP address of this node’s interface. -
Once the nodes are all characterized, we arrive at the cluster setup phase. We’ll now want to make use of the new feature introduced in MicroCluster which is that we can now pass an
initConfigmap (map[string]string) to the MicroCluster hooks. When creating the MicroOVN cluster (OnBootstraphook), we propose to give the following initConfig map:
{
“<node0_hostname>.ovn-encap-ip”: “<chosen_iface_ip_for_node0>”,
“<node1_hostname>.ovn-encap-ip”: “<chosen_iface_ip_for_node1>”,
…
}
- Now, on the MicroOVN
Bootstrap(initConfig map[string]string)function, if these fields are present, we can set the OVS configuration with the proper interface ip for the Geneve tunnel:
// Configure OVS to either use a custom encapsulation IP for the geneve tunel
// or the hostname of the node.
var ovnEncapIp string
if initConfig != nil {
for k, v := range initConfig {
if k == fmt.Sprintf("%s.ovn-encap-ip", s.Name()) {
ovnEncapIp = v
break
}
}
}
if ovnEncapIp == "" {
ovnEncapIp = s.Address().Hostname()
}
_, err = VSCtl(
s,
"set", "open_vswitch", ".",
fmt.Sprintf("external_ids:system-id=%s", s.Name()),
fmt.Sprintf("external_ids:ovn-remote=%s", sbConnect),
"external_ids:ovn-encap-type=geneve",
fmt.Sprintf("external_ids:ovn-encap-ip=%s", ovnEncapIp),
)
Preseed setup (microcloud init --preseed <PRESEED_FILE.yaml>)
Just like we have the ovn_uplink_interface , we will introduce a new underlay_ip field that is also an IP address in CIDR notation. After we validate that each node has a valid underlay_ip and that the state of the LXD networks correspond to the values entered by the user, we set the OVNGeneveAddr to this IP address (no CIDR notation needed). The cluster setup phase is the same as described in the above section.
Auto setup (microcloud init --auto)
The auto setup just skips the option of having a dedicated OVN underlay network.
Communication with MicroOVN
-
The obvious issue here is what should we do in the case where the installed version of MicroOVN does not support consuming the
initConfigmap in theBootstrapfunction? In such a situation, the configuration given by the MicroCloud user will simply not be applied and will cause no setup error but this will definitely feel wrong. To counter that, and this is an open interrogation, would it be a good idea to add an API extension system (that is queryable through MicroOVN’s REST API) like how it is done in LXD for example? We could then return early in the MicroCloud setup phase with a warning saying that the installed version of MicroOVN does not support that kind of setting. -
If the API extension solution is out of scope for now, what could be a workable solution for this?
API Changes
None.
CLI Changes
- Adding a new question after the OVN uplink configuration to ask if the user desires to configure a custom underlay for OVN. If applicable, a table with a list of network interface with allocated IPs will be shown for the user to choose one per node to describe the underlay.
Database changes
None.