| Index | FO066 |
|---|---|
| Title | APT deb822 sources by default |
| Status | Approved |
| Authors | @juliank |
| Type | Standard |
| Created | 2021-05-13 |
Abstract
APT’s deb822 sources format is an alternative, often shorter, representation of sources.list that also interacts better with new features in apt. We need to port tools to understand it and then switch the default.
Rationale
APT introduced deb822 .sources file as an alternative to sources.list files a couple of years ago. The new format is more succinct and is substantially easier to read, especially when options come into play. To cite the sources.list manual page:
The format for two one-line-style entries using the deb and deb-src types is:
deb [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...] deb-src [ option1=value1 option2=value2 ] uri suite [component1] [component2] [...]Alternatively the equivalent entry in deb822 style looks like this:
Types: deb deb-src URIs: uri Suites: suite Components: [component1] [component2] [...] option1: value1 option2: value2
Most important in terms of options, the Signed-By option allows the specification of a keyring that can sign the repository. We would like to migrate to requiring Signed-By for all sources, and in the old syntax this becomes unwieldy.
Recently APT gained support for specifying signing keys directly in the Signed-By option, allowing third-party repositories to be delivered as a single .sources file that just needs to be dropped into sources.list.d as opposed to a sources.list entry and a separate keyring file that needs to be managed.
This is a two cycle process: In the first cycle we need to prepare the foundations for supporting it, and in the second cycle we need to switch the default.
Specification
1. Support for deb822 format in remaining software in 23.10
Support for deb822 format needs to be implemented in python-apt’s aptsources module, which requires breaking the API slightly, and then porting the existing code over. To avoid a flag day, a compatibility layer is added that renders multivalue fields as text in the way they appear in the file (with spaces). This works reasonably well for reading, but may confuse some use cases if you use them (e.g. the edior dialog in software-properties does not modify the item, but builds a new sources.list line and then creates an item from that; which stops working).
- software-properties (critical)
- Ubuntu-release-upgrader (critical)
- command-not-found (critical)
While most software should be trivial, software-properties needs some thought because it exposes an interface to edit sources that needs changes. Presenting deb822 sources in the URI poses a challenge because they can be arbitrary number of lines and don’t have a human-readable description.
Probably the interface should use some sort of more separation between entries so that it’s clearer were a multi-line entry starts and ends, and only show a selected number of fields.
With the entries being longer and combined more, the code potentially needs to change to enable/disable components and suites on existing entries rather than comment/uncomment.
Other software can go at its own pace in the next cycle or perhaps not at all:
- aptdaemon (or kill the feature, do we even use it?)
- ansible (not critical)
- apt-clone (not critical)
- apturl (not critical)
- Curtin
- Cloud-init
The porting essentially revolves around having multiple URIs and Suites instead of singular.
Further software may need to gain support for the format that is yet unknown and might only be discovered once the default has been switched. People working on this need to run with the new default sources list documented in 2.
2. Replacing the default /etc/apt/sources.list in 24.04
The default default sources list should be /etc/apt/sources.list.d/ubuntu.sources instead of /etc/apt/sources.list. A default ubuntu.sources matching the current sources.list is shown in appendix A. Note that it already includes Signed-By. add-apt-repository / software-properties should start writing deb822 at the same time.
The caveat with moving to a sources file like that is that it appears in a different order relative to other files in sources.list.d. This will affect the order in which apt update downloads files, and third-party repositories with an earlier name in the alphabet will take precedence if they ship the same version of a package (identical debs), but that is a minor cosmetic concern. There is no precedence for enumerating files in that directory, so going with 00ubuntu.sources or similar would be a larger diversion from the common naming schemes.
2.1 Places that need changing for the transition
We have identified multiple places that write sources.list that need to be unified.
livecd-rootfs: live-build/functionsconfigure_universe()live-build:scripts/build/lb_chroot_archivesubiquity:d-i/source/apt-setup/generators/50mirror.ubuntu(update only needed if community flavors retain ubiquity in 24.04; this is obsoleted for Ubuntu Desktop)cloud-init:templates/sources.list.ubuntu.tmplcurtin:curtin/commands/apt_config.py(already supports deb822?)subiquity: does not generatesources.listfor the target system but does manipulate it during install
3. Upgrades to 24.04
On upgrades to 24.04 and later versions, ubuntu-release-upgrader should remove Ubuntu entries from sources.list and write them /etc/apt/sources.list.d/ubuntu.sources, adding Signed-By fields to them in the process.
The ubuntu-release-upgrader tarball will have to vendorize the apt.sources module for this to work when upgrading from 22.04 to 24.04, or alternatively, the new module needs to be ported back to 22.04, or actually the code can just write the new sources using Python formatting itself.
Further Information
Outlook: Mandatory Signed-By
In an interim release (24.10) following the first LTS with deb822 sources by default (24.04), APT will start issuing warnings about sources that do not have Signed-By set. After an LTS with warnings for missing Signed-By fields, APT will start enforcing Signed-By for all sources, and trusted.gpg.d becomes obsolete.
Due to ubuntu-release-upgrader adding the Signed-By field when migrating the main list to deb822, and it disabling third-party sources, users should not face any errors, but it might be worthwhile figuring out which key signed which repository, and then rewriting the sources.list files despite commenting them out, but this only concerns 24.10 or so.
Add-apt-repository needs to be changed so it writes the key into the Signed-By field of the sources file directly instead of creating a key in trusted.gpg.d (this has shipped in 23.10).
Note that 23.10 already has these messages as notices (which are similar to warnings except they start with N: instead of W: and only appear when connected to a tty), and only for sources that are in the deb822 format.
Appendix A: Default sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
## Ubuntu distribution repository
##
## The following settings can be tweaked to configure which packages to use from Ubuntu.
## Mirror your choices (except for URIs and Suites) in the security section below to
## ensure timely security updates.
##
## Types: Append deb-src to enable the fetching of source package.
## URIs: A URL to the repository (you may add multiple URLs)
## Suites: The following additional suites can be configured
## <name>-updates - Major bug fix updates produced after the final release of the
## distribution.
## <name>-backports - software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
## Components: Aside from main, the following components can be added to the list
## restricted - Software that may not be under a free license, or protected by patents.
## universe - Community maintained packages. Software from this repository is
## ENTIRELY UNSUPPORTED by the Ubuntu team. Also, please note
## that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
## multiverse - Community maintained of restricted. Software from this repository is
## ENTIRELY UNSUPPORTED by the Ubuntu team, and may not be under a free
## licence. Please satisfy yourself as to your rights to use the software.
## Also, please note that software in multiverse WILL NOT receive any
## review or updates from the Ubuntu security team.
##
## See the sources.list(5) manual page for further settings.
Types: deb
URIs: http://archive.ubuntu.com/ubuntu
Suites: kinetic kinetic-updates
Components: main universe
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
## Ubuntu security updates. Aside from URIs and Suites,
## this should mirror your choices in the previous section.
Types: deb
URIs: http://security.ubuntu.com/ubuntu
Suites: kinetic-security
Components: main universe
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
Updates to noble switch to ubuntu.sources
WIP