Manage a proxied environment

Re this issue

rather than downgrading the protocol level on the bridge (which may have side-effects for the OVN integration) you can pass the protocol version to use to the ovs-ofctl tool:

sudo openstack-hypervisor.ovs-ofctl show --protocols OpenFlow15 br-ex

@m4t7e0 question on your host networking setup - I’m assuming you’re using Open vSwitch installed on the Ubuntu host to configure all of the bridges that you detailed in your network configuration summary?

If that is the case we provide Open vSwitch as part of the hypervisor snap (as you have discovered) and I wonder how having two sets of userspace daemons running for OVS is impacting on function - I’ve done this when each is in its own discrete network namespace under LXC containers, but not both on the host OS.

changed my configuration, compared to the first post, I set up a new setup from scratch on three other nodes using the experience of the old setup. Now I’m using the standard Linux bridge for the first time, I was able to set up without interruptions. I have encountered the same problems (the four points of my previous post). At the current state, I have reached the br-ex via the command I indicated, but I still have a block for the Ceph part… … now I’m trying to fix the Ceph problem and the change of IP address of the dashboard (during the first deploy, I made a typo, now I can’t access the dash anymore). I think I’ll restart a new deploy from scratch to solve all the problems soon

               |    br-api  (linux bridge) |
               |    192.168.20.X/24
               |    Gateway:
               |    DNS:,
               |    Interfaces: bond0.20
               | br-floating (not assignet) |
               |    Interfaces: bond0.21 
               |    192.168.21.X/24 (for  public network but NO IP associated to this bridge)
               |    br-mgm  (linux bridge) |
               |  192.168.2.X/24
               |    DNS:,,
               |    Interfaces: eno1
               |    br-stor (linux bridge) |
               |  192.168.31.X/24
               |    DNS:,,
               |    Interfaces: bond1.31
               |    bond0   |
               |  Interfaces: enp129s0f0np0, enp225s0f0np0
               |    bond1   |
               |  Interfaces: enp129s0f1np1, enp225s0f1np1

ubuntu@node-01:~$ sudo microceph.ceph status

    id:     2b6bdec7-6575-4c49-b85f-c3bba69ce7f8
    health: HEALTH_OK

    mon: 3 daemons, quorum node-01.dom, node-03.dom, node-02.dom (age 2d)
    mgr: node-01.dom(active, since 5d), standbys: node-03.dom, node-02.dom
    mds: 2/2 daemons up, 1 standby
    osd: 45 osds: 45 up (since 2d), 45 in (since 2d)

    volumes: 2/2 healthy
    pools:   8 pools, 897 pgs
    objects: 8.17k objects, 31 GiB
    usage:   76 GiB used, 65 TiB / 65 TiB avail
    pgs:     897 active+clean

ubuntu@node-01:~$ juju status -m admin/controller

Model       Controller          Cloud/Region     Version  SLA          Timestamp       Notes
controller  sunbeam-controller  sunbeam/default  3.4.2    unsupported  11:30:06+02:00  upgrade available: 3.4.3

SAAS                   Status   Store  URL
cert-distributor       waiting  local  node-01.dom/openstack.cert-distributor
certificate-authority  active   local  node-01.dom/openstack.certificate-authority
cinder-ceph            active   local  node-01.dom/openstack.cinder-ceph
keystone               waiting  local  node-01.dom/openstack.keystone
ovn-relay              active   local  node-01.dom/openstack.ovn-relay
rabbitmq               active   local  node-01.dom/openstack.rabbitmq

App                   Version  Status   Scale  Charm                 Channel        Rev  Exposed  Message
controller                     active       1  juju-controller       3.4/stable      79  no
microceph                      blocked      3  microceph             reef/edge       47  no       (workload) Error in charm (see logs): Command '['microceph', 'cluster', 'join', 'eyJuYW1lIjoibWljcm9jZXBoLzEiLCJzZWNy...
microk8s                       active       3  microk8s              legacy/stable  121  no
openstack-hypervisor           active       1  openstack-hypervisor  2023.2/edge    165  no
sunbeam-machine                active       3  sunbeam-machine       2023.2/edge     14  no

Unit                     Workload  Agent  Machine  Public address  Ports      Message
controller/0*            active    idle   0
microceph/0*             active    idle   0
microceph/1              blocked   idle   2              (workload) Error in charm (see logs): Command '['microceph', 'cluster', 'join', 'eyJuYW1lIjoibWljcm9jZXBoLzEiLCJzZWNy...
microceph/2              blocked   idle   1              (workload) Error in charm (see logs): Command '['microceph', 'cluster', 'join', 'eyJuYW1lIjoibWljcm9jZXBoLzIiLCJzZWNy...
microk8s/0*              active    idle   0   16443/tcp
microk8s/1               active    idle   1   16443/tcp
microk8s/2               active    idle   2   16443/tcp
openstack-hypervisor/0*  active    idle   0
sunbeam-machine/0*       active    idle   0
sunbeam-machine/1        active    idle   1
sunbeam-machine/2        active    idle   2

Machine  State    Address        Inst id               Base          AZ  Message
0        started  manual:               ubuntu@22.04      Manually provisioned machine
1        started  manual:  ubuntu@22.04      Manually provisioned machine
2        started  manual:  ubuntu@22.04      Manually provisioned machine

Offer      Application  Charm      Rev  Connected  Endpoint  Interface    Role
microceph  microceph    microceph  47   2/2        ceph      ceph-client  provider

@m4t7e0 a few things to note about the local mode (rather than MAAS) of deployment you’re using.

The networking is still very simple and flat for this mode - we have plans to improve support for using multiple networks and some fixes have been implemented into the 2024.1 track of the openstack charm to resolve some issues we’ve seen once MicroK8S gets installed (which results in the machine ending up with lots of IP addresses) - but its still only simple flat networking!

@m4t7e0 thanks for the update on your networking configuration - what’s the rationale for using Linux bridges?

I’m exploring the product, having never used it before, and I’m trying to find a way that works with my cluster. I have 10 nodes to use and I’m deploying on a set of 3 nodes to verify that the deployment, refresh, and add node process functions without issues… I encountered a problem when using the deployment with OVS, failing to bind it to br-ex because the interface was used by br-floating… so, simply because I didn’t know what Sunbeam wanted, I switched to Linux bridge configuration. If you don’t see any issues, there might be problems creating the cluster with OVS. In the next setup, I’ll return to OVS (which is definitely more suitable for my use case).

have started configuring 5 nodes for the deployment of "OpenStack 2024.1 (Caracal) - multi-node-maas " following the guide, but I am unsure if I can use the proxy in the deployment. Additionally, MAAS cannot manage the power-on part of the node in my case, so I am trying to understand if this configuration can work for my scenario.