Upgrade a Landscape 19.10 Manual Installation to Landscape 23.03

The upgrade of a Manual Install can be repurposed to create steps to upgrade a Juju install. If you have compartmentalized the services responsible for operating Landscape, this guide will walk you through upgrading an individual Landscape Server (appserver), and then more specifically provide a checklist of tasks and commands for performing a manual upgrade.

Upgrade via Appserver Replacement

This approach is very basic – simply stop the Landscape Server 19.10 appserver, back up the relevant files, then replace it with a Landscape Server 23.03 appserver with the backed-up files restored.

NOTE: for any steps below requiring domain names (such as self-signed cert and such), this guide takes the shortcut of editing /etc/hosts on various machines so that landscape-test.lxd always pointed at the “active” Landscape Server appserver.

Setup

4 LXD containers

  • landscape-app-bionic (bionic) where Landscape 19.10 is running
  • landscape-app-focal (focal) where Landscape 23.03 is running
  • landscape-db (focal) where PostgreSQL is running

1 LXD container that we are managing in Landscape, for demonstration purposes

  • landscape-client (focal) is a machine enrolled in Landscape

Steps

  1. Follow the install directions, doing the PostgreSQL setup touches landscape-db, rabbitmq, and apache, and the Landscape setup touches landscape-app-bionic.

    • NOTE: in the “Configure PostgreSQL” step, this guide uses 10.76.244.0/24 as the <IP-OF-APP> so that any of other LXD machines used in this guide have access to it
  2. Install the landscape-client package on the landscape-client machine and register it with Landscape

  3. Follow the backup directions for Landscape config files, skipping the logs and /etc/postgresql files

  4. Stop landscape-app-bionic

  5. On landscape-app-focal, follow the install directions, starting at “Adding the Landscape package archive and installing the package”, only up to the end of “Configure rabbitmq”

  6. Push the backed up files from step 3 to landscape-app-focal, to the appropriate (identical) locations

  7. Continue following the install directions, starting at “Webserver configuration”, skip the step for creating the sites-available file, and proceed to the end of “Start Landscape Services”

  8. Update /etc/hosts file on landscape-client to point to the IP address of landscape-app-focal

  9. Confirm that client is able to ping server. Confirm that server UI is accessible via browser

Upgrade via Manual Installation

This how-to is intended to guide the user through upgrading a manual installation of Landscape Server running on Ubuntu 18.04 LTS (“bionic”) to a manual installation of Landscape Server running on Ubuntu 20.04 (“focal”) or Ubuntu 22.04 (“jammy”).

Preparing for the upgrade

What you will need:

  • Ubuntu 20.04 LTS (“focal”) or 22.04 LTS (“jammy”) install media.
  • A backup of all of the files in /etc/landscape on your current Landscape Server machine
  • The Server X509 certificate file and key file issued for the FQDN hostname of the application server. These should be on your current Landscape Server machine at /etc/ssl/certs/landscape_server.pem and /etc/ssl/private/landscape_server.key, respectively. (If you are using a self-signed certificate, you will also need /etc/ssl/certs/landscape_server_ca.crt)
  • A copy of your Landscape Server default configuration file. This should be on your current Landscape Server machine at /etc/default/landscape-server

If your PostgreSQL installation is running on the same machine as your Landscape Server installation:

  • A backup of all of the files in /etc/postgresql/<pg-version>/main

If your Apache web server installation is running on the same machine as your Landscape Server installation:

  • A copy of /etc/apache2/sites-available/server-name, the Landscape Apache vhost config file, usually named after the FQDN of the server

Stop Landscape Server

Stop the Landscape Server services on your existing installation to prevent changes from occurring during the migration:

sudo lsctl stop

To prevent maintenance tasks from occurring, you can also move or remove the /etc/cron.d/landscape-server file.

Backing Up and Restoring the Database

NOTE: If your Landscape PostgreSQL database is not running on the same machine as your Landscape Server, it is very likely that you can skip this section.

  1. Backup your database server using pg_dumpall, for example:

    sudo -u postgres pg_dumpall > backup.sql
    
  2. Install PostgreSQL and required libraries on your new machine:

    sudo apt install postgresql-12 python3-apt postgresql-plpython3-12 postgresql-contrib-12 postgresql-12-debversion
    

NOTE: If using Ubuntu 22.04, the PostgreSQL and related package versions should be at version 14

  1. Replace existing files in /etc/postgresql/<pg-version>/main with those previously copied

  2. Copy the backup file from your previous installation and restore it on the new installation:

    sudo -u postgres psql -f backup.sql postgres
    
  3. Run sudo -u postgres -- psql -l and make sure Encoding is UTF8, and Collate and Ctype are C.UTF-8

Installing Landscape Server 23.03

Landscape is distributed in a public PPA.

  1. Add the PPA on your target machine:

    sudo add-apt-repository ppa:landscape/self-hosted-23.03
    
  2. Install the server package:

    sudo apt install landscape-server
    
  3. Replace the files in /etc/landscape with the ones you copied earlier from your previous Landscape installation (and ensure they are owned by the landscape user, and the root group)

  4. Replace /etc/default/landscape-server with the version you copied earlier from your previous installation (ensure it is owned by the landscape user and the root group)

  5. Add /etc/ssl/certs/landscape_server.pem, /etc/ssl/private/landscape_server.key, and /etc/ssl/certs/landscape_server_ca.crt as needed. Ensure that the .key is owned by the landscape user and the www-data group.

If you previously had Apache installed on the same machine as Landscape Server, perform these additional steps.

  1. Install Apache:

    sudo apt install apache2
    
  2. Add the /etc/apache2/sites-available/<server-name> file you copied earlier from your previous installation

  3. Configure apache and restart it:

    for module in rewrite proxy_http ssl headers expires; do sudo a2enmod $module; done
    sudo a2dismod status
    sudo a2dissite 000-default
    sudo a2ensite landscape.conf
    sudo service apache2 restart
    

If you previously had RabbitMQ installed on the same machine as Landscape Server, perform these additional steps.

  1. Install RabbitMQ:

    sudo apt install rabbitmq-server
    
  2. set up the new installation using a password of your choice:

    sudo rabbitmqctl add_user landscape <password>
    sudo rabbitmqctl add_vhost landscape
    sudo rabbitmqctl set_permissions -p landscape landscape ".*" ".*" ".*"
    
  3. Edit /etc/rabbitmq/rabbitmq-env.conf:

    NODE_IP_ADDRESS=127.0.0.1
    
  4. Restart rabbitmq-server

    sudo systemctl restart rabbitmq-server
    
  5. Update the [broker] section of /etc/landscape/service.conf to use your new password

Configuring Landscape Server

Landscape Server will need to check that your database has an up-to-date schema. If it does not, it will perform a schema migration. Depending on what version of Landscape Server you are upgrading from, this may be a backwards-incompatible change. Make sure that you are able to restore your previous setup before performing this step.

On your new Landscape Server installation, run the setup:

sudo setup-landscape-server

Then start the Landscape services:

sudo lsctl restart

Make sure that your Landscape installation is working satisfactorily before performing any deletions of your previous installation.

I’ve managed to do a juju deployed environment migration from bionic/19.10 to focal/23.03 and stumbled upon this: https://bugs.launchpad.net/landscape/+bug/2019828 if anyone else has this problem

Also something to take info consideration while deploying a new environment wit 23.03 is this: https://bugs.launchpad.net/postgresql-charm/+bug/2019196

I’ve used this guide (and the official manual) to upgrade from Landscape 19.10 to 24.04. I created a fresh Ubuntu server 24.04 install and copied all above mentioned config files and databases over from my Ubuntu 18.04 server. I installed Postgresql 16.

All went fine. I only noticed a 100% cpu load on these 3 processes for over 24 hours:

 137086 ?        Rs     0:02 python3 /opt/canonical/landscape/landscape-secrets-service
 137423 ?        Rs     0:00 python3 /opt/canonical/landscape/landscape-hostagent-consumer
 137425 ?        Rs     0:00 python3 /opt/canonical/landscape/landscape-hostagent-messenger

They keep restarting every few seconds.

Landscape secrets service gives this error:

Jun 02 12:48:41 landscape.***.net systemd[1]: Started landscape-secrets-service.service - Landscape's Secrets Management Service.
Jun 02 12:48:42 landscape.***.net landscape-secrets-service[164985]: 2024-06-02 10:48:42.525Z INFO landscape-secrets-service "Starting service.."
Jun 02 12:48:42 landscape.***.net landscape-secrets-service[164985]: Starting service..
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]: Traceback (most recent call last):
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/landscape-secrets-service", line 7, in <module>
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     run()
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/canonical/landscape/scripts/secrets_service.py", line 184, in run
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     secrets_url, service_port = load_secrets_config()
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:                                 ^^^^^^^^^^^^^^^^^^^^^
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/canonical/landscape/scripts/secrets_service.py", line 162, in load_secrets_config
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     config = load_config("secrets")
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:              ^^^^^^^^^^^^^^^^^^^^^^
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/canonical/landscape/setup.py", line 234, in load_config
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     config = setup_config(instance_type)
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/canonical/landscape/setup.py", line 132, in read_config
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     instance_config = config.get(instance_type)
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:   File "/opt/canonical/landscape/canonical/lib/config.py", line 32, in get
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:     return self._sections[section]
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]:            ~~~~~~~~~~~~~~^^^^^^^^^
Jun 02 12:48:45 landscape.***.net landscape-secrets-service[164985]: KeyError: 'secrets'

Any idea what is causing this? I’ve disabled this service and everything still seems to work.

I’ve also disabled the 2 WSL services, since I won’t be using WSL. Will probably work after I add these config options mentioned here.

Just in case, someone has the same problem (like Stefan1 and me), here’s the solution:

I compared the two files /etc/landscape/service.conf (copied from the old system) and /etc/landscape/service.conf.dpkg-dist

I found out, that there is a new section “secrets”:

[secrets]
secrets-url =
secrets-service-url = http://localhost:26155

After adding this to the service.conf, the “landscape-secrets-service” runs smooth. For the other two services, just add the “virtual host to RabbitMQ”, as already mentioned by Stefan1 :slight_smile:

best regards,
Markus

1 Like