Upgrade a Landscape 19.10 Quickstart Installation to Landscape 23.03

rajanpatel · April 18, 2023, 5:34pm

The best approach is to perform new manual installation and copy over the database and configuration files from your Quickstart install. As a matter of convenience, it is possible to perform a quickstart upgrade in-place. The following guide outlines your 2 options for doing so:

Quickstart Upgrade: Option 1

This method is somewhat basic, but could be complicated depending on how much customization has happened after the landscape-server-quickstart install. It is strongly recommended that you backup the database before performing an upgrade this way.

Setup

2 LXD containers for Landscape

landscape-app-bionic (bionic) where Landscape 19.10 is running
landscape-app-focal (focal) where Landscape 23.03 is running

1 LXD container that we are managing in Landscape, for demonstration purposes

landscape-client (focal) is a machine enrolled in Landscape

Steps

Install landscape-server-quickstart on landscape-app-bionic according to the quickstart install instructions
Install landscape-client on landscape-client and register with server
Stop all Landscape services on landscape-app-bionic with this command:
```
sudo lsctl stop
```
Backup all landscape- database tables using pg_dump
Install landscape-server-quickstart on landscape-app-focal from the beta PPA
Stop all Landscape services on landscape-app-focal with this command:
```
sudo lsctl stop
```
Restore database data using this command:
```
psql -d <database> -f <file.sql>
```
Run sudo -u postgres -- psql -l and make sure Encoding is UTF8, and Collate and Ctype are C.UTF-8
Run the setup command to do db schema migrations, if any:
```
sudo setup-landscape-server
```
Start Landscape services using this command:

sudo lsctl start

Made minor edits to /etc/apache2/sites-available landscape conf to correct the domain name, it should match what domain name you use for the Landscape Server
If you are using a self-signed SSL certificate, copy the new SSL cert to landscape-client. It may be prudent to copy the old SSL certificate from landscape-app-bionic to landscape-app-focal, because the Quickstart creates one based on the machine’s hostname. There is a chance this hostname could be different when you were setting it up landscape-app-focal.
Restart the landscape-client LXD machine to ensure everything is working

Quickstart Upgrade: Option 2

This method leverages do-release-upgrade and is not recommended, because configuration files differ from one Ubuntu LTS to the next. For those motivated to go this route, it is possible:

Setup

1 LXD container for Landscape

landscape-app-bionic (bionic)

Steps

Install landscape-server-quickstart on landscape-app-bionic according to the quickstart install instructions
Backup all landscape- database tables using pg_dump
Perform a PPA purge with this command:
```
sudo ppa-purge ppa:landscape/19.10
```
The postgres package blocks do-release-upgrade because it is not marked for removal through that process:
```
sudo apt remove postgresql-10-debversion
```
Perform the release upgrade:
```
sudo do-release-upgrade
```

Add the Landscape 23.03 PPA:

sudo add-apt-repository ppa:landscape/self-hosted-23.03

Perform the Quickstart install

sudo apt install landscape-server-quickstart

jerquiaga · March 6, 2023, 10:22pm

I’m trying to follow this, but can’t figure out how to backup the DBs via pg_dump from a quickstart install. I don’t remember it asking for a Postgres username/password.

Also, the in-place upgrade steps don’t seem to work. I get an error from Postgres when it tries to update the schema and then dies.

mdscunningham · March 6, 2023, 11:17pm

You can dump the DB after switching to the postgres user. This allows for the simplest backup option using pg_dumpall. The output file will need to be sent to a location where the postgres user has write permissions, but that avoids the need to look up or process any credentials. I would suggest trying something like the following:

sudo -u postgres -- pg_dumpall -f /var/lib/postgresql/landscape.sql

If you wanted to get the credentials that Landscape uses internally, you can get those from the /etc/landscape/service.conf file, but those users are limited in their actions and what they can access. I do not believe these are able to use the pg_dump commands, so they will likely not work for this purpose. These passwords will likely also be base64 encoded, so you would also need to decode them before you could use them.

[stores]
user = landscape
password = b64:<encoded password>
host = localhost
main = landscape-standalone-main
account-1 = landscape-standalone-account-1
...

The best option is likely to just switching to the postgres user and using either pg_dump or pg_dumpall.

jerquiaga · March 7, 2023, 12:36am

Thanks, that was helpful and I was able to use pg_dumpall to backupt the DBs. Following the first set of outlined steps, I transferred the backup to the new server and used the following to restore:

sudo -u postgres -- psql -f /var/lib/postgresql/landscape.sql

That doesn’t appear to cause any errors, but on step 8 I get:

2023-03-07 00:31:29.826Z INFO landscape-setup "Bootstrapping from service.conf file ..."
2023-03-07 00:31:29.850Z ERROR landscape-setup "connection to server at \"localhost\" (127.0.0.1), port 5432 failed: FATAL:  password authentication failed for user \"landscape_superuser\"\nconnection to server at \"localhost\" (127.0.0.1), port 5432 failed: FATAL:  password authentication failed for user \"landscape_superuser\"\n"

scahartner · March 10, 2023, 5:19am

I did the upgrade, but ran into this issue. Any pointers on how to repair my server setup.

Setting up landscape-server (23.03+1-0landscape0) ...

========================================================================
Attempting schema upgrade as requested.
WARNING: this could take several minutes or more.

2023-03-10 05:14:32.412Z INFO landscape-setup “Bootstrapping from service.conf file …”
2023-03-10 05:14:32.430Z INFO landscape-setup “Skipping configuration migration …”
2023-03-10 05:14:32.430Z INFO landscape-setup “Checking Landscape databases …”
2023-03-10 05:14:32.443Z INFO landscape-setup “Checking database schema …”
2023-03-10 05:14:32.809Z INFO landscape-setup “Schema configuration output:\n\nLoading site configuration…\nWARNING: PostgreSQL has max_prepared_transactions set to 0, not using two-phase commit.\nSetting up database schema
s (will timeout after 86400 seconds) …\nSchema patch version: 483\n”
2023-03-10 05:14:32.809Z INFO landscape-setup “Checking package database initial data …”
2023-03-10 05:14:32.828Z INFO landscape-setup “Package database already initialized.”
2023-03-10 05:14:32.828Z INFO landscape-setup “Renaming stock hash-id stores …”
2023-03-10 05:14:32.828Z INFO landscape-setup “Stock package database not loaded, ignoring stock hash-id stores.”
Job for landscape-msgserver.service failed because the control process exited with error code.
See “systemctl status landscape-msgserver.service” and “journalctl -xe” for details.
invoke-rc.d: initscript landscape-msgserver, action “start” failed.

landscape-msgserver.service - LSB: Enable Landscape message processing
Loaded: loaded (/etc/init.d/landscape-msgserver; generated)
Active: failed (Result: exit-code) since Fri 2023-03-10 05:14:38 UTC; 12ms ago
Docs: man:systemd-sysv-generator(8)
Process: 5726 ExecStart=/etc/init.d/landscape-msgserver start (code=exited, status=1/FAILURE)

Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: ImportError: cannot import name ‘UBUNTU_PRO_INFO’ from ‘landscape.message_schemas’ (/usr/lib/python3/dist-packages/landscape/message_schemas/init.py)
Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: Failed to load application: File “/opt/canonical/landscape/canonical/landscape/message/handlers/configure.zcml”, line 220.4-224.3
Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: File “/opt/canonical/landscape/configs/standalone/message-server.zcml”, line 8.4-8.53
Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: File “/opt/canonical/landscape/configs/standalone/message-server-configure.zcml”, line 3.0-3.49
Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: File “/opt/canonical/landscape/canonical/landscape/message/configure.zcml”, line 4.4-4.35
Mar 10 05:14:37 dc0b-landscape.th.local landscape-msgserver[5773]: ImportError: cannot import name ‘UBUNTU_PRO_INFO’ from ‘landscape.message_schemas’ (/usr/lib/python3/dist-packages/landscape/message_schemas/init.py)
Mar 10 05:14:38 dc0b-landscape.th.local landscape-msgserver[5726]: …fail!
Mar 10 05:14:38 dc0b-landscape.th.local systemd[1]: landscape-msgserver.service: Control process exited, code=exited, status=1/FAILURE
Mar 10 05:14:38 dc0b-landscape.th.local systemd[1]: landscape-msgserver.service: Failed with result ‘exit-code’.
Mar 10 05:14:38 dc0b-landscape.th.local systemd[1]: Failed to start LSB: Enable Landscape message processing.
dpkg: error processing package landscape-server (–configure):
installed landscape-server package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of landscape-server-quickstart:
landscape-server-quickstart depends on landscape-server (>= 23.03+1-0landscape0); however:
Package landscape-server is not configured yet.

dpkg: error processing package landscape-server-quickstart (–configure):
dependency problems - leaving unconfigured
Errors were encountered while processing:
landscape-server
landscape-server-quickstart
E: Sub-process /usr/bin/dpkg returned an error code (1)

jerquiaga · March 10, 2023, 7:12pm

Just in case this is helpful to someone - maybe it was left unsaid because people expect experience with Postgres, but to get the first upgrade option to work I needed to do the following to backup the databases on the original server (landscape-app-bionic):

sudo -u postgres -- pg_dump -d landscape-standalone-account-1 -f /var/lib/postgresql/landscape-standalone-account-1.sql
sudo -u postgres -- pg_dump -d landscape-standalone-knowledge -f /var/lib/postgresql/landscape-standalone-knowledge.sql
sudo -u postgres -- pg_dump -d landscape-standalone-main -f /var/lib/postgresql/landscape-standalone-main.sql
sudo -u postgres -- pg_dump -d landscape-standalone-package -f /var/lib/postgresql/landscape-standalone-package.sql
sudo -u postgres -- pg_dump -d landscape-standalone-resource-1 -f /var/lib/postgresql/landscape-standalone-resource-1.sql
sudo -u postgres -- pg_dump -d landscape-standalone-session -f /var/lib/postgresql/landscape-standalone-session.sql

Then, on the new server (landscape-app-focal) I needed to drop and re-create the DBs, then restore them:

Drop and re-create DBs:
sudo -u postgres – psql
DROP DATABASE “landscape-standalone-account-1”;
DROP DATABASE “landscape-standalone-knowledge”;
DROP DATABASE “landscape-standalone-main”;
DROP DATABASE “landscape-standalone-package”;
DROP DATABASE “landscape-standalone-resource-1”;
DROP DATABASE “landscape-standalone-session”;

CREATE DATABASE "landscape-standalone-account-1";
CREATE DATABASE "landscape-standalone-knowledge";
CREATE DATABASE "landscape-standalone-main";
CREATE DATABASE "landscape-standalone-package";
CREATE DATABASE "landscape-standalone-resource-1";
CREATE DATABASE "landscape-standalone-session";
\q

Restore DBs:
sudo -u postgres – psql -d landscape-standalone-account-1 -f /var/lib/postgresql/landscape-standalone-account-1.sql
sudo -u postgres – psql -d landscape-standalone-knowledge -f /var/lib/postgresql/landscape-standalone-knowledge.sql
sudo -u postgres – psql -d landscape-standalone-main -f /var/lib/postgresql/landscape-standalone-main.sql
sudo -u postgres – psql -d landscape-standalone-package -f /var/lib/postgresql/landscape-standalone-package.sql
sudo -u postgres – psql -d landscape-standalone-resource-1 -f /var/lib/postgresql/landscape-standalone-resource-1.sql
sudo -u postgres – psql -d landscape-standalone-session -f /var/lib/postgresql/landscape-standalone-session.sql

mdscunningham · March 10, 2023, 11:21pm

Since the dumpall copies the entirety of the PostgreSQL contents, it is likely pulling in the landscape users and their respective passwords. You might also be able to get around this by copying over the passwords from the previous service.conf on the source, so that the users/passwords line up with what is already in the database.

Edit: That said, I see that you found a workaround by copying just the individual databases, which also avoids the conflict of existing landscape database users.

kevinnasto · March 11, 2023, 3:55pm

This error is because the version of landscape-client that is running on the server (landscape-client is a dependency of landscape-server) is an older one. It should be the latest but it’s not (maybe a different version is installed than the ppa one)

scahartner · March 14, 2023, 4:53am

Thanks. Uninstalling landscape-client and re-running the installer fixed my issue.

scahartner · March 21, 2023, 12:23am

After the upgrade the /opt/canonical/landscape/scripts/update_security_db.sh fails consistently. I raised this on the forums under.

mdscunningham · March 27, 2023, 9:41pm

Looking at the output from your manual script run (thanks for including that output with the bash debug) it looks like the usn database import failed. This seems a little odd though since the curl command to download the database returned a successful exit code (note the test for 0 not equal 0).

sudo -u landscape bash -x /opt/canonical/landscape/scripts/update_security_db.sh

# Download the usn database
++ curl -f -y 120 -Y 1 -L --max-redirs 10 --connect-timeout 60 --output /var/lib/landscape/usndb.json.bz2-new https://usn.ubuntu.com/usn-db/database.json.bz2
+ output='  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.3M  100 27.3M    0     0  39.0M      0 --:--:-- --:--:-- --:--:-- 38.9M'
+ '[' 0 -ne 0 ']'

# Rename the downloaded file and initiate the import
+ mv -f /var/lib/landscape/usndb.json.bz2-new /var/lib/landscape/usndb.json.bz2
+ cd /opt/canonical/landscape
+ set -o pipefail
+ bzcat /var/lib/landscape/usndb.json.bz2
+ ./process-usns /dev/stdin

# The import appears to fail as if the downloaded file was invalid json
<11>Mar 16 00:20:24 update-security-db: Traceback (most recent call last):
<11>Mar 16 00:21:11 update-security-db:   File "./process-usns", line 7, in <module>
<11>Mar 16 00:21:11 update-security-db:     canonical.landscape.scripts.usn.run()
<11>Mar 16 00:21:11 update-security-db:   File "/opt/canonical/landscape/canonical/landscape/scripts/batch.py", line 84, in __call__
<11>Mar 16 00:21:11 update-security-db:     code = self.run()
<11>Mar 16 00:21:11 update-security-db:   File "/opt/canonical/landscape/canonical/landscape/scripts/usn.py", line 37, in run
<11>Mar 16 00:21:11 update-security-db:     changeset = update_from_usn_tool_db(db)
<11>Mar 16 00:21:11 update-security-db:   File "/opt/canonical/landscape/canonical/landscape/model/package/usn.py", line 274, in update_from_usn_tool_db
<11>Mar 16 00:21:11 update-security-db:     search_client.update_usns(
<11>Mar 16 00:21:11 update-security-db:   File "/opt/canonical/landscape/canonical/landscape/model/package/client.py", line 40, in query
<11>Mar 16 00:21:11 update-security-db:     return self._query(method, params)
<11>Mar 16 00:21:11 update-security-db:   File "/opt/canonical/landscape/canonical/landscape/model/package/client.py", line 62, in _query
<11>Mar 16 00:21:11 update-security-db:     raise PackageSearchRequestError(loads(error.body)["Error"])
<11>Mar 16 00:21:11 update-security-db:   File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
<11>Mar 16 00:21:11 update-security-db:     return _default_decoder.decode(s)
<11>Mar 16 00:21:11 update-security-db:   File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
<11>Mar 16 00:21:11 update-security-db:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
<11>Mar 16 00:21:11 update-security-db:   File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
<11>Mar 16 00:21:11 update-security-db:     raise JSONDecodeError("Expecting value", s, err.value) from None
<11>Mar 16 00:21:11 update-security-db: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
+ '[' 1 -ne 0 ']'

Testing this on my own focal and jammy installs of Landscape 23.03, it seemed to succeed without issue, so I am at a little bit of a loss unless your proxy is somehow modifying the file in transit, making it invalid and breaking the import (honestly just a guess).

Additionally, the item that was mentioned in the forum post about a service being masked does not appear to be relevant, as the debug output is showing a test that failed. The state of the update_security_db.service is returned as “not-found” and this is compared against the string “masked” which would give a false (non-zero) result. This is the same that I see in my setup where the script executed successfully, so this at least looks “normal” so to speak.

++++ service=update_security_db.service
+++++ systemctl -p LoadState --value show update_security_db.service
++++ state=not-found
++++ '[' not-found = masked ']'

I would suggest downloading the usn database from the link: https://usn.ubuntu.com/usn-db/database.json.bz2, decompressing this and checking to see if it shows as valid json. Also comparing a copy of this downloaded outside the proxy, to the copy downloaded on the Landscape server behind the proxy, and see if there are any differences between these that would explain the failed import.

rajanpatel · April 4, 2023, 11:12am

Could it be possible that curl needs system-wide proxy configurations in order to work in your environment?

export http_proxy="http://user:pwd@127.0.0.1:1234"
export https_proxy="http://user:pwd@127.0.0.1:1234"

Please alert us if this resolves your issue, and we can include proxy configuration steps.

ianmac51 · August 24, 2023, 10:21am

I have what appears to be the exact same problem

quick start install
reinstalled on two clean VM’s
uname -a
Linux hlulpv01.openans.co.uk 5.15.0-79-generic #86-Ubuntu SMP Mon Jul 10 16:07:21 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
cat /etc/os-release
PRETTY_NAME=“Ubuntu 22.04.3 LTS”
NAME=“Ubuntu”
VERSION_ID=“22.04”
VERSION=“22.04.3 LTS (Jammy Jellyfish)”
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
…
only allowed 2 links
UBUNTU_CODENAME=jammy

From @mdscunningham comment

Manually download the file on the server using curl and it uses the system-wide proxy OK
Manually decompress the file database.json.bz2 OK
Note json is minified
Unminify
python3 -m json.tool < usndb.json > usndb_pretty.json
looks ok
run the script manually.
sudo -u landscape bash -x /opt/canonical/landscape/scripts/update_security_db.sh

The final sections show

++ curl -f -y 120 -Y 1 -L --max-redirs 10 --connect-timeout 60 --output /var/lib/landscape/usndb.json.bz2-new https://usn.ubuntu.com/usn-db/database.json.bz2
- output=’ % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  100 29.3M 100 29.3M 0 0 21.3M 0 0:00:01 0:00:01 --:–:-- 21.3M’
- ‘[’ 0 -ne 0 ‘]’
- mv -f /var/lib/landscape/usndb.json.bz2-new /var/lib/landscape/usndb.json.bz2
- cd /opt/canonical/landscape
- set -o pipefail
- bzcat /var/lib/landscape/usndb.json.bz2
- ./process-usns /dev/stdin
- pipe_to_syslog update-security-db
- tag=update-security-db
  ++ get_logger_arguments
  ++ echo /dev/log
  ++ grep -q :
  ++ ‘[’ -n /dev/log ‘]’
  ++ ‘[’ /dev/log ‘!=’ /dev/log ‘]’
  ++ echo ‘’
- args=
- logger -s -p user.error -t update-security-db
  <11>Aug 24 10:03:28 update-security-db: Traceback (most recent call last):
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/./process-usns”, line 7, in
  <11>Aug 24 10:05:07 update-security-db: canonical.landscape.scripts.usn.run()
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/canonical/landscape/scripts/batch.py”, line 84, in call
  <11>Aug 24 10:05:07 update-security-db: code = self.run()
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/canonical/landscape/scripts/usn.py”, line 37, in run
  <11>Aug 24 10:05:07 update-security-db: changeset = update_from_usn_tool_db(db)
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/canonical/landscape/model/package/usn.py”, line 274, in update_from_usn_tool_db
  <11>Aug 24 10:05:07 update-security-db: search_client.update_usns(
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/canonical/landscape/model/package/client.py”, line 40, in query
  <11>Aug 24 10:05:07 update-security-db: return self._query(method, params)
  <11>Aug 24 10:05:07 update-security-db: File “/opt/canonical/landscape/canonical/landscape/model/package/client.py”, line 62, in _query
  <11>Aug 24 10:05:07 update-security-db: raise PackageSearchRequestError(loads(error.body)[“Error”])
  <11>Aug 24 10:05:07 update-security-db: File “/usr/lib/python3.10/json/init.py”, line 346, in loads
  <11>Aug 24 10:05:07 update-security-db: return _default_decoder.decode(s)
  <11>Aug 24 10:05:07 update-security-db: File “/usr/lib/python3.10/json/decoder.py”, line 337, in decode
  <11>Aug 24 10:05:07 update-security-db: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  <11>Aug 24 10:05:07 update-security-db: File “/usr/lib/python3.10/json/decoder.py”, line 355, in raw_decode
  <11>Aug 24 10:05:07 update-security-db: raise JSONDecodeError(“Expecting value”, s, err.value) from None
  <11>Aug 24 10:05:07 update-security-db: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
- ‘[’ 1 -ne 0 ‘]’
- alert_admin update_security_db.sh
- echo ‘Error running /opt/canonical/landscape/scripts/update_security_db.sh: 0’
  Error running /opt/canonical/landscape/scripts/update_security_db.sh: 0
- echo ‘Check out the syslog output for script update_security_db.sh.’
  Check out the syslog output for script update_security_db.sh.
- exit 1
- release_lock update_security_db.sh
- get_distributed_lock update_security_db.sh --release
- local command=/opt/canonical/landscape/get-distributed-lock
- /opt/canonical/landscape/get-distributed-lock update_security_db.sh --release
- rm -f /var/lock/update_security.lock
File exists
ls -l /var/lib/landscape/usndb.json.bz2
-rw-rw-r-- 1 landscape landscape 30783874 Aug 24 10:03 /var/lib/landscape/usndb.json.bz2
Decompress and file looks ok
cp /var/lib/landscape/usndb.json.bz2 .
bzip2 -d usndb.json.bz2
python3 -m json.tool < usndb.json > usndb_pretty.json
tail usndb_pretty.json
“action”: “In general, a standard system update will make all the necessary changes.\n”,
“cves”: [
“CVE-2018-20102”,
“CVE-2018-20103”,
“CVE-2018-20615”
],
“id”: “3858-1”,
“isummary”: “Several security issues were fixed in HAProxy.\n”
}
}

Did this ever get resolved?

ianmac51 · August 24, 2023, 3:23pm

Hi,

The following resolved this issue for me

Change the proxy in /etc/environment to ignore localhost

export http_proxy=http://my.proxy:port
export https_proxy=http://my.proxy:port

Add below to fix

export NO_PROXY=“127.0.0.1,localhost”

@rajanpatel Thanks that was it

Upgrade a Landscape 19.10 Quickstart Installation to Landscape 23.03

Quickstart Upgrade: Option 1

Setup

Steps

Quickstart Upgrade: Option 2

Setup

Steps

======================================================================== Attempting schema upgrade as requested. WARNING: this could take several minutes or more.

Add below to fix

========================================================================
Attempting schema upgrade as requested.
WARNING: this could take several minutes or more.