Foundations Team Updates - Thursday 13 Jul 2023

Foundations
1

The previous status is here: Foundations Team Updates - Thursday 6 Jul 2023

2 Likes
2

Boot flow design

  • In release v2023.10 U-Boot SPL has been changed to let main U-Boot or EDK II be loaded as FIT image from file. This corresponds to a suggestion in the EBBR specification. I evaluated necessary changes to support such a scenario.

Vendor packages

  • package vendor packages into ppa

OpenSBI

1 Like
3

Rust

Distro

  • Currently still investigating armhf time_t checking failure (batch 43)

Ubuntu ROCK

1 Like
4

APT

  • Spent significant time analysing and fixing yet another phasing bug (LP: #2025462). At the same time also got another bug report over separate channels where the new version of ubuntu-desktop installed firefox deb which got even worse with this change (previously only upgrade was affected, but not dist-upgrade). It turned out we looked at the wrong version when trying to not mark packages for upgrade, oops.

    The fixes landed in apt 2.7.2 which is currently awaiting migration in mantic-proposed and will be
    SRUed to stable releases next week.

    Unfortunately the fixes are not exactly complete. While we now would not try to upgrade ubuntu- desktop and install the firefox deb, if anything else pulled in the phased update, we’d still mark it for install and then firefox, only to revert the ubuntu-desktop update later.

    Fixing this correctly is hard because the very thing that would fix it correctly, marking the ubuntu-desktop for keep early is also what caused the bug where ubuntu-desktop was removed. One can potentially work around this by iterating over all new automatically installed packages and marking them “keep” again, however we must make sure we don’t break any Recommends doing so and we do not have the infrastructure to check that in the depcache yet.

    So this is what leads me to the next point…

  • A new solver research project is occupying my mind and spare time. It is heavily inspired by apk and unexpectedly to some sat solvers: https://salsa.debian.org/apt-team/solver3, compiled draft paper: https://magenta.jak-linux.org/bin/solver3.pdf

    I think we have a reasonable model now. It’s a fairly significant change in direction for APT solving, as it will by default do less, but it should end up being more predictable and much easier to understand.

    Here we can now reliably mark phased updates for keep back before marking anything for upgrading without risking stuff getting removed, as it only removes stuff if it installs anything that conflicts with installed stuff (for some subset).

  • Fixed snapshot support crashes discovered during the tech demo. Need to revisit that once more and do a full test in a docker image or similar and see if there’s still gaps to fill. Appreciating any testing that could be done.

  • deb822 sources without a Signed-By field now cause apt to display a notice about the missing Signed-By field. Note that notices are only shown in interactive use (if stdout is a tty)

  • Started discussion about removing the chroot handling of phased updates in 24.04/24.10.

  • Started an EPIC for apt 2.8 technical debt work, specifically so far I gathered the transition of apt’s internal use from apt-key to gpgv directly in there, as well as the addition of SHA3 support and with that also the switch to libnettle from libgcrypt.

python-apt

  • Wrote a complicated solution to having aptsources.distro detect partial matches for the distro in a deb822 source entry by presenting an exploded view of the deb822 sources file, where each entry essentially corresponds to a legacy sources.list file (which in turn corresponds to a release file).

    Modifying the exploded view explodes the underlying deb822 structure “for real”, and then when saving the file, it merges any sections that were exploded before as much as possible (preserving the order).

    This is slightly suboptimal because we do not track the state across runs, so if you explode in one tool invocation and then want to undo the change later, you remain with exploded entries.

    The solution does not seem to be enough for software-properties to actually make changes, but software-properties has a complex daemon setup that complicates all of these matters. It makes it recognize sources correctly though, that’s a win.

Secure boot

  • Upstream developers already started working on a branch to add LoadImage and friends to shim, which I reviewed the grub PoC patch for it with suggestions on how to improve it for submission.

    I do think we still want to go ahead with our approach, but ultimately I missed the time this pulse.

  • Evaluating which modules we can remove from our signed grub images with grub 2.12. We’d like to drop all the graphics modules, as the grub menu these days is more a means of entering a recovery mode, and png, jpg readers, and graphics drivers and whatnot all pose security risks. This will then degrade the experience for users on Ubuntu releases without flicker free boot, as they will see the Ubuntu splash later, and it will degrade the experience for people using os-prober where the menu is always shown.

  • We also discussed how we should get rid of os-prober and that it would make a whole lot of sense for the desktop to offer you alternative boot entries at reboot but that’s more of a desktop topic. Discussions with GNOME designers about such a feature happened in the past apparently.

  • For shim-review, spent a couple minutes looking at Chrome OS’s new boot loader, crdyboot, written in Rust, which implements its own boot verification outside of the existing shim framework. I must say I do not particularly enjoy the idea of the kernel not being measured by shim, so I won’t be going much further ahead with this review.

  • During our latest meeting we also rediscovered how Debian signs their kernel modules with their kernel signing key instead of using an ephemeral key, which means that the kernel modules can be loaded across different kernel builds (since you can also force override the magic even if you change the ABI).

    I filed a release critical bug in Debian about this, and added a question to the shim-review template to make sure nobody submits shim with such a broken design in the future.

  • Added the appropriate mechanism to copy ESM updates to the ubuntu-uefi-team-tools uefi-release script, and then used it to copy the ESM grub to the security PPA for @eslerm to continue with releasing next week

Misc

  • Submitted a talk about APT sandboxing and a lightning talk about our 64-bit time_t efforts to All Systems Go! which takes place in Berlin this year. My main interest really is the summit the day before, but the conference being directly after makes it very convenient to do as well.

  • Started working on the vim merge, but shifted focus to other work when I hit multiple test suite failures.

1 Like
5

Distro

Netplan

Misc

  • Candidate interviews
1 Like
6

Distro

  • diffoscope: investigated why the build is failing on all architectures. The root cause is the new version of android-platform-tools. Problem reported in #1040916
  • sudo: fixed a few problems with my MP
  • glib2.0: helped investigating a regression affecting keyfiles generation that lead to netplan’s tests failures. #3047

Netplan

  • Refactoring parts of the frontend to accommodate future changes related to netplan diff PR#379
  • Tested the solution proposed by #3047 and verified it restores the old glib behavior and fixes netplan build
  • Worked on a workaround for a problem reported on network-manager related to some special settings one can set to the macaddress property which causes network-manager to fail when adding a new connection PR#376
  • Code review PR#377
1 Like
7
  • More s390x autopkgtest wrangling, and managed to fill the autopkgtest-web workers’ disks :man_facepalming: (sorry @bdmurray)
  • While monitoring the queues, added some with autopkgtests running out of memory to big_packages; r-cran-datawizard, r-cran-parameters, r-cran-insight, kmc and cppcheck
  • Proprosed-migration: build-essential migrated
  • Reviewed and sponsored jtreg6 for @vpa1977
  • Patch Pilot shift:
  • TIL Universe merges: dh-r, dh-r again, and fpc, then investigated and solved a long-standing issue in one of fpc’s tests (patch forwarded to Debian, so next upload should be sync’able)
  • Started review of dotnet8 preview packaging
1 Like
8

autopkgtest

  • Submitted an MP for autopkgtest-cloud modifying the regex used in the filter-amqp tool.
  • Removed some debvm tests from the queues as they would have ran with the “bad” version of debvm.
  • Submitted an RT regarding bos01.scalingstack being unusable for s390x. Worked with IS to troubleshoot the issues.
  • Worked with @andersson123 to replace some lxd-armhf workers which had gone AWOL. The instances had literally disappeared and didn’t leave a goodbye note. Then merged his MP updating the service-bundle.

Ubuntu

  • Uploaded a fix for bug 2025093 which is about debvm’s autopkgtests failing due to a missing testdep as the autopkgtests keep looping.
  • Investigation into a Xubuntu crash report having “DistroRelease: Xubuntu 23.04” which is not by default retraceable by apport.
  • Submitted a pull request for apport adding a --gdb-sandbox argument to the crash-digger command. Maybe its something I should have done years ago when adding the gdb sandbox to apport-retrace?
  • Special sru-review and a copy-package of a glibc update for focal-proposed.

Misc

  • Worked on the migration of the apport retracing service from a server in 3FP to a PS5 system.
  • Attended Developer Membership Board meeting.
  • Administrative work.
  • Conducted an interview with a candidate.
1 Like
9

OpenJDK

  • Reworked my first Debian merge task for the openjdk-lts package. Learning the git-ubuntu workflow the hard way :frowning:
  • Technical debt - tracked down a JTREG failure to its root cause.

OpenJDK FIPS

  • Reviewed initial package structure of the openjdk-fips package

Misc

  • Mandatory HR training videos
  • Knowledge sessions with Vladimir
  • More VISA doc work
  • Adoptium Workgroup meetings
1 Like
10
  • subiquity
    • PR: 1708 - converting the autoinstall tutorial to RTD/Diataxis
    • LP: #2026225 - find and file issue around kernel handling in curtin on Mantic Desktop
  • cloud-init PR: 4238 - doc typo fix
  • patch pilot
    • The sponsor report had some surprises in terms of stuff showing as red. Proposed a fix for “Date Queued” showing as older than it should be. Also reviewed Robie’s python3 MP. Fixed report has at worst yellow (3 wk old) items.
    • ec2-hibinit-agent sponsor MPs (LP: #2023924, LP: #2024505): review and upload for f, j, k, l
    • Review cdebconf merge
    • LP: #2026199 - review adduser merge, upload
  • +1 - full report later. Some highlights:
    • ocaml-sexplib0 cluster: ppx-import vs frama-c - fixed with retests
    • ocaml-sexplib0 cluster: ocaml-sedlex - fixed with retests
    • ocaml-sexplib0 cluster: utop vs ocaml-dune (LP: #2027333)
      • control file fixes for dh-ocaml compat sent to Debian and uploaded.
    • ocaml-sexplib0 cluster: ocaml-qcheck vs ocaml-dune (LP: #2027522)
      • implement autopackage test fix for compat with new ocaml-dune, sent to Debian and uploaded.
    • nbs: libarcus3 vs cura-engine (LP: #2026769, LP: #2026778)
      • pull in a fix from upstream and a fix from Debian, uploaded.
1 Like
11

Go

  • New release 1.19.11 and 1.20.6

    • Updated the Debian package, and synced back to Ubuntu
    • Updated the Go snap

  • Rebuild the Go packages against golang-1.21 in PPA. Only 79 packages FTBFS. Looks like very optimistic.

  • golang-defaults proposed migration.

    After retrying the autopkgtest multiple times for ycmd and vim-youcompleteme packages, they finally passed. But still puzzle about why they kept failing previously and why I can’t reproduce locally.

armhf time_t

  • Still working another batch of packages.
1 Like
12

dotnet

  • triaged autopkgtest failure with security team for 6.0.120 and 7.0.109 release
  • still trying to integrate ms smoke-tests
  • fixed miss-configured watch file for dotnet6 (not released yet)
  • attended weekly .NET Source-Build Partner Sync Meeting

dash

  • LP: #2026730 – Merge dash 0.5.12-6 from debian

casper

  • submitted Merge Proposal for LP: #1990621(PXE Boot contains wrong suggested link to ISO for live file system)

misc

  • ubuntu packaging guide
    • meeting
    • writing articles
    • followup to IS request for redirect
  • ubuntu-mir PR #23 (Modernize Process States Overview) was merged :tada:
1 Like
13

Distro

Misc

  • discussions on initrd compression on ubuntu-devel@
  • trying to find how to make apt/dpkg less verbose during package installs

armhf time_t

1 Like
14
  • Verified fix for left-over packages on pre-installed images (LP: #1925265)
  • Verified fix for missing I2C devices on lunar (LP: #2019320)
  • Verified fix for v3 camera module on lunar (LP: #2024434)
  • Updated the patch for default user groups in user-setup, and proposed merge to ubiquity (LP: #1923363); not sure if ubiquity will wind up being the first-time setup in Mantic, but just in case …
  • Did some major clean-up on the classic branch of the pi-gadget so it can now cross-build without requiring sudo (by virtue of setting up its own apt configuration & state)
    • … to be merged once testing is complete on all archs & relevant releases
  • Investigated the failure of first-time setup on the Ubuntu Pi desktop images (LP: #2025068) which turned out to be … missing livecd-rootfs hacks (because of course it was)
    • … working on adding a package to oem-config to call the first time setup (so it can be seeded instead of using a livecd-rootfs hack)
  • Writing second article on the state of the various desktop flavours on the Pi
  • Pi meetings
1 Like
15

Distro

  • Invalid PEP440 package version (LP: #1991606):
    • Verified SRUs for dput and gpgme1.0
    • Checked failing autopkgtests
  • Triaging bug #2026824
  • initramfs-tools: Helped fixing the autopkgtest for the next SRU (LP: #2027636)

armhf time_t

Apport

1 Like
16

Distro

  • libsereal-decoder-perl proposed-migration

Rustc

  • Reviewed and almost sponsored 1.68.2 (xnox beat me to it)

Glibc

  • Planification for next cycle work on the testing story

armhf

  • Setup armhf time_t testing environment
  • Finished up a batch, and picked up one of @waveform’s leftovers.
1 Like
17

Distro

  • Worked on systemd autopkgtest and merged 253.5-1 from Debian
  • Picked up an armhf batch

I will be out until next Friday.

2 Likes