This is a follow-up to Package changes between two Ubuntu images where I discussed finding all the package changes between two cloud or server images given the image’s package version manifests.
I have recently completed some work on the ubuntu-cloud-image-changelog tool to allow for JSON output which can be used in automation.
A common use case is to determine what CVEs will I no longer be vulnerable to or Launchpad bugs will no longer affect me if I move to this new cloud image.
With the new JSON output option a user could generate the changelog diff between two images and as it is JSON can use a JSON parsing tool like jq to parse the CVEs addressed by moving to the new image.
This is best explained/demonstrated in full using a demo script
#!/bin/bash -eux
if ! command -v ubuntu-cloud-image-changelog &> /dev/null
then
echo "ubuntu-cloud-image-changelog could not be found. 'sudo snap install ubuntu-cloud-image-changelog' or 'python3 -m pip install ubuntu-cloud-image-changelog' to install"
exit
fi
if ! command -v jq &> /dev/null
then
echo "jq could not be found. 'sudo apt install jq' to install"
exit
fi
# Download some package version manifests to compare and generate changelog for
wget http://cloud-images.ubuntu.com/releases/jammy/release-20220420/ubuntu-22.04-server-cloudimg-amd64.manifest -O 20220420-ubuntu-22.04-server-cloudimg-amd64.manifest
wget http://cloud-images.ubuntu.com/releases/jammy/release-20221117/ubuntu-22.04-server-cloudimg-amd64.manifest -O 20221117-ubuntu-22.04-server-cloudimg-amd64.manifest
changelog_txt="20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff.txt"
changelog_json="20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff.json"
# generate the changelog for all changes between the two manifests
ubuntu-cloud-image-changelog generate --from-manifest 20220420-ubuntu-22.04-server-cloudimg-amd64.manifest \
--to-manifest 20221117-ubuntu-22.04-server-cloudimg-amd64.manifest \
--from-series jammy \
--to-series jammy \
--image-architecture amd64 \
--highlight-cves \
--notes "Changelog diff for Ubuntu 22.04 jammy base cloud image from serial 20220420 to 20221117" \
--output-json-pretty \
--output-json ${changelog_json} \
| tee ${changelog_txt}
# list all unique CVEs addressed by moving to this new image
jq --raw-output '.. | .cves? |select(length>0)[] | "\(.cve) \(.cve_priority)"' ${changelog_json} | sort --unique | tee "20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff-unique-CVEs.txt"
# list all unique high priority CVEs addressed by moving to this new image
jq --raw-output '.. | .cves? |select(length>0)[] | select(.cve_priority=="high") | "\(.cve) \(.cve_priority)"' ${changelog_json} | sort --unique | tee "20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff-unique-high-priority-CVEs.txt"
# list all unique launchpad bugs fixed by moving to this new image
jq --raw-output '.. | .launchpad_bugs_fixed? |select(length>0)[]' ${changelog_json} | sort --unique | tee "20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff-unique-launchpad-bugs.txt"
# list all packages that were updated by moving to this new image
jq --raw-output '.summary.deb.diff[]' ${changelog_json} | sort --unique | tee "20220420-20221117-ubuntu-22.04-server-cloudimg-amd64-changelog-diff-updated-deb-packages.txt"
I have uploaded the demo script, manifests, and output to https://people.canonical.com/~philroche/20221122-ubuntu-cloud-image-changelog/
- demo.sh
- 20220420-ubuntu-22.04-server-cloudimg-amd64.manifest
- 20221117-ubuntu-22.04-server-cloudimg-amd64.manifest
- text output
- JSON output
- unique CVEs referenced
- unique high priority CVEs referenced
- unique launchpad bugs addressed
- updated deb packages
The interesting output to highlight from running this script is the list of high priority CVEs referenced in the changelogs
CVE-2022-1015 high
CVE-2022-1966 high
CVE-2022-21499 high
CVE-2022-2585 high
CVE-2022-2586 high
CVE-2022-2588 high
CVE-2022-2602 high
CVE-2022-29581 high
CVE-2022-29799 high
CVE-2022-29800 high
CVE-2022-34918 high
CVE-2022-3515 high
CVE-2022-3602 high
CVE-2022-42919 high
The JSON output is verbose and gives a lot of information about the changes in packages between the two images including
- Summary of deb and snap packages added, updated and removed
- All source package changelog entries between the two package versions listed in the manifests
- All launchpad bugs fixed as part of a package update
- All CVEs referenced as part of a package update
- This also includes the CVE priority, assigned by the Ubuntu security team and CVE description
- Mappings between binary package installed to the source package that built that binary
With the updated tool you can now also generate the JSON schema used to build the JSON output
ubuntu-cloud-image-changelog schema
{
"title": "ChangelogModel",
"type": "object",
"properties": {
"summary": {
"$ref": "#/definitions/Summary"
},
"diff": {
"$ref": "#/definitions/Diff"
},
"added": {
"$ref": "#/definitions/Added"
},
"removed": {
"$ref": "#/definitions/Removed"
},
"notes": {
"title": "Notes",
"type": "string"
},
"from_series": {
"title": "From Series",
"type": "string"
},
"to_series": {
"title": "To Series",
"type": "string"
},
"from_manifest_filename": {
"title": "From Manifest Filename",
"type": "string"
},
"to_manifest_filename": {
"title": "To Manifest Filename",
"type": "string"
}
},
"required": [
"summary",
"diff",
"added",
"removed",
"from_series",
"to_series",
"from_manifest_filename",
"to_manifest_filename"
],
"definitions": {
"SnapSummary": {
"title": "SnapSummary",
"type": "object",
"properties": {
"added": {
"title": "Added",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"removed": {
"title": "Removed",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"diff": {
"title": "Diff",
"default": [],
"type": "array",
"items": {
"type": "string"
}
}
}
},
"DebSummary": {
"title": "DebSummary",
"type": "object",
"properties": {
"added": {
"title": "Added",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"removed": {
"title": "Removed",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"diff": {
"title": "Diff",
"default": [],
"type": "array",
"items": {
"type": "string"
}
}
}
},
"Summary": {
"title": "Summary",
"type": "object",
"properties": {
"snap": {
"$ref": "#/definitions/SnapSummary"
},
"deb": {
"$ref": "#/definitions/DebSummary"
}
},
"required": [
"snap",
"deb"
]
},
"FromVersion": {
"title": "FromVersion",
"type": "object",
"properties": {
"source_package_name": {
"title": "Source Package Name",
"type": "string"
},
"source_package_version": {
"title": "Source Package Version",
"type": "string"
},
"version": {
"title": "Version",
"type": "string"
}
}
},
"ToVersion": {
"title": "ToVersion",
"type": "object",
"properties": {
"source_package_name": {
"title": "Source Package Name",
"type": "string"
},
"source_package_version": {
"title": "Source Package Version",
"type": "string"
},
"version": {
"title": "Version",
"type": "string"
}
}
},
"Cve": {
"title": "Cve",
"type": "object",
"properties": {
"cve": {
"title": "Cve",
"type": "string"
},
"url": {
"title": "Url",
"type": "string"
},
"cve_description": {
"title": "Cve Description",
"type": "string"
},
"cve_priority": {
"title": "Cve Priority",
"type": "string"
},
"cve_public_date": {
"title": "Cve Public Date",
"type": "string"
}
},
"required": [
"cve",
"url",
"cve_description",
"cve_priority",
"cve_public_date"
]
},
"Change": {
"title": "Change",
"type": "object",
"properties": {
"cves": {
"title": "Cves",
"default": [],
"type": "array",
"items": {
"$ref": "#/definitions/Cve"
}
},
"log": {
"title": "Log",
"default": [],
"type": "array",
"items": {
"type": "string"
}
},
"package": {
"title": "Package",
"type": "string"
},
"version": {
"title": "Version",
"type": "string"
},
"urgency": {
"title": "Urgency",
"type": "string"
},
"distributions": {
"title": "Distributions",
"type": "string"
},
"launchpad_bugs_fixed": {
"title": "Launchpad Bugs Fixed",
"default": [],
"type": "array",
"items": {
"type": "integer"
}
},
"author": {
"title": "Author",
"type": "string"
},
"date": {
"title": "Date",
"type": "string"
}
},
"required": [
"package",
"version",
"urgency",
"distributions",
"author",
"date"
]
},
"DebPackage": {
"title": "DebPackage",
"type": "object",
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"from_version": {
"$ref": "#/definitions/FromVersion"
},
"to_version": {
"$ref": "#/definitions/ToVersion"
},
"cves": {
"title": "Cves",
"default": [],
"type": "array",
"items": {
"$ref": "#/definitions/Cve"
}
},
"launchpad_bugs_fixed": {
"title": "Launchpad Bugs Fixed",
"default": [],
"type": "array",
"items": {
"type": "integer"
}
},
"changes": {
"title": "Changes",
"default": [],
"type": "array",
"items": {
"$ref": "#/definitions/Change"
}
},
"notes": {
"title": "Notes",
"type": "string"
}
},
"required": [
"name",
"from_version",
"to_version"
]
},
"SnapPackage": {
"title": "SnapPackage",
"type": "object",
"properties": {
"name": {
"title": "Name",
"type": "string"
},
"from_version": {
"$ref": "#/definitions/FromVersion"
},
"to_version": {
"$ref": "#/definitions/ToVersion"
}
},
"required": [
"name",
"from_version",
"to_version"
]
},
"Diff": {
"title": "Diff",
"type": "object",
"properties": {
"deb": {
"title": "Deb",
"type": "array",
"items": {
"$ref": "#/definitions/DebPackage"
}
},
"snap": {
"title": "Snap",
"type": "array",
"items": {
"$ref": "#/definitions/SnapPackage"
}
}
},
"required": [
"deb",
"snap"
]
},
"Added": {
"title": "Added",
"type": "object",
"properties": {
"deb": {
"title": "Deb",
"type": "array",
"items": {
"$ref": "#/definitions/DebPackage"
}
},
"snap": {
"title": "Snap",
"type": "array",
"items": {
"$ref": "#/definitions/SnapPackage"
}
}
},
"required": [
"deb",
"snap"
]
},
"Removed": {
"title": "Removed",
"type": "object",
"properties": {
"deb": {
"title": "Deb",
"type": "array",
"items": {
"$ref": "#/definitions/DebPackage"
}
},
"snap": {
"title": "Snap",
"type": "array",
"items": {
"$ref": "#/definitions/SnapPackage"
}
}
},
"required": [
"deb",
"snap"
]
}
}
}
Schema above also available @ https://people.canonical.com/~philroche/20221122-ubuntu-cloud-image-changelog/schema.json
All source is available @ https://github.com/CanonicalLtd/ubuntu-cloud-image-changelog and can be installed as a snap @ https://snapcraft.io/ubuntu-cloud-image-changelog or from PyPi @ https://pypi.org/project/ubuntu-cloud-image-changelog/