Ubuntu Server team update - 15 July 2019

Project Discussion Server
1

Hi everyone, below you will find the updates of the Ubuntu Server team members from the last week. If you are interested in discussing a topic please start a thread in the Server area of this Discourse site.

2 Likes
2

Packaging

  • php 7.2 to 7.3 transition
    • Worked through a listing of 51 packages from the transitions page. Am able to successfully build 34 of the 51 packages.
    • Four are failing during build and will need further investigation
    • The rest failed because they have co-dependency on one or more of the other packages, so as long as I upload things in order they should build too.
  • Kafka
    • For Maven packages there is a mh_make script to generate the initial packaging for a Java codebase, but no equivalent for Gradle codebases. I studied mh_makeā€™s code and started cutting code on a gh_make equivalent, but mh_make calls into some Java code for dependency resolution that I need to experiment with to see if I can reuse it or will need to redo it.
    • Looked into a gradle->pom generator plugin for gradlew. This might be an alternate solution, that would make gradle packages resemble maven packages close enough to get the initial packaging done. Need to try this out and see what it does in practice.
  • LP: #1790657 SRU for librabbitmq ā€œamqp-tools server parameter unusableā€
    • Completed packaging & SRU template; ready for review
  • LP: #1804542 SRU for bind9 - ā€œerror msg during name resolutionā€
    • Packaged patch; need to complete packaging & define test case
  • LP: #1823441 SRU for xen - ā€œFTBFSā€
    • Packaged patch; need to complete packaging & define test case

Development

  • git-ubuntu
    • Implemented a fix to a Coverity flagged issue, with docs and thorough test cases. Waiting on review by Robbie.
  • ppa-dev-tools
    • Implemented functionality to list the source and binary packages in a specified ppa. (Was needed for extracting a list of 400+ php packages processed in the php 7.0->7.1 transition.)
    • Implemented version support
  • usmerges tool
    • Implemented code to query Ubuntu seeds files to identify which ubuntu-server subscribed packages would be highest priority.
    • Implemented code to use uscan on watch files from git-ubuntu checked out packages, to identify latest upstream releases
  • Ubuntu Maintainersā€™ Handbook
    • Copyedits
    • Elaborated on purpose/use of deb-src lines in sources.list

Miscellaneous

  • Looked into gpgv codebase, to see whatā€™d be involved in fixing a longstanding issue with unavailable signer keys. Code looks straightforward enough, but a reproducible testcase would be the logical next action for this.
3

Packaging/Bugs

  • squid FTBFS #1835831. Not done yet, a workaround was uploaded by someone else in the meantime.
  • openldap ssl/start_tls certificate validation verification for (now invalid) bug #1835181
  • merged zeromq3 (still in eoan proposed due to a test failure in pyzmq, looks like a very flaky test)
  • sponsored iproute2 (5.1.0-1ubuntu1 and 5.1.0-1ubuntu2)
  • apache SSL regression (#1836329) (ongoing)

Misc

ubuntu advantage tools

  • Addressed review feedback for the trusty upload
  • still need to streamline the newly added test cases to the SRU bug #1832757
  • tested procedure to upgrade Precise to Trusty with ESM enabled and filed a paper cut bug: #693
1 Like
4

VIRTUALIZATION

  1. QEMU HW mitigations support (ARCH_CAPABILITIES)
    LP: #1828495

New QEMU version for Ubuntu Disco, supporting CascadeLake/IceLake, IA32_ARCH_CAPABILITIES MSR, thus the following HW mitigations:

  • IBRS_ALL (enhanced IBRS support)
  • SKIP_L1DFL_VMENTRY (L1D flush is needed on VMENTRY)
  • RDCL_NO (HW is vulnerable to Rogue Data Cache Load)
  • Foreshadow-NG (OS) vuln. (L1 terminal fault, OS)
  • Foreshadow-NG (VMM) vuln. (L1 terminal fault, VMM)

Is already in disco-proposed (1:3.1+dfsg-2ubuntu3.3). Anyone willing to test it can install packages from the -proposed pocket and open bugs if needed. We havenā€™t provided official instructions yet, but, you can follow the MDS instructions (https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/MDS) to enable specific CPU capabilities (disabled by default). Weā€™re still working on the libvirt support for those.

  1. QEMU s390x secure boot toleration feature (by @paelzer)

reviewed merge requests - #1 | #2 | #3 - for s390x secure Linux boot toleration QEMU patches. Using s390-tools from Eoan makes it possible to burn IPL stages with signed kernels. This toleration patches allow QEMU to IPL those burned ā€œscsiā€ (vda) disks.

UBUNTU HA

  1. CTDB NFS HA Enablement
    LP: #722201 | DOC | PPA | Disco MR | Debian MR

The merge request was done against Ubuntu Disco, bringing all fixes done in Eoan to Disco. Iā€™m waiting code to be uploaded and accepted by the SRU team in order to provide verification tests in public bug (PPA contains the CTDB NFS HA capable samba/ctdb package).

  1. Pacemaker and Corosync
    Eoan Migrations | kronosnet

Corosync

  • Corosync merges are blocked by regressions. Corosync depends on libknet1 now, included in [universe] pocket. Weā€™ve asked for a MIR and it has been accepted. Weā€™re waiting an archive admin to migrate packages to [main]. This will satisfy merge done for Corosync. It is already merged to latest upstream version (3.0.1-2).

Corosync and Pacemaker

  • Pacemaker merges are blocked by a alleged regression for armhf architecture. Explanations of what could be happening are here and it is possible that we force-badtest to armhf for corosync/pacemaker since armhf container limits are stepping in our way here.

** OTHER **

1 Like
5

Hi,
I missed most of last week being away, but that means I did not post about two weeks ago yet, so Iā€™ll mostly summarize the week before.

Virtualization

  1. QEMU HW mitigations support (ARCH_CAPABILITIES) LP: #1828495
    Did some reviews for @rafaeldtinoco and prepared and ran the regression test suit on all supported architectures.
  2. Identified bug 1836299 which was then fixed
    and confirmed to be good now in retests.
  3. extended virt-tests for the new qemu in Eoan
  4. extended virt-tests with a testcase that defines type host-model guests and migrates them through updates and releases helping to identify issues with the lack of feature support between versions.
    5.packaging and testing of secure boot toleration for s390x qemu in bug 1830243 into Eoan (will work on SRUs next week)
  5. packaging and testing of support for new s390x cpu types in bug 1830238 and bug
    1828038
  6. fix qemu bug around md-clear bit positioning (due to backport applying it to the wrong line)
  7. debugged a migration issue in qemu 4.0 with copy-storage which led to file ownership breakage on migrations. it turned out that this is special to my testbed which runs on the same machine in two containers (not really supported) sharing the same filesystem. libvirt would usually detect the shared FS like gpfs on FS-magic and -not- update the ownership, but in this case it doesnā€™t. Added a workaround to test automation and discussed with the LXD Team about potential probing of shared FS between containers.
  8. analyze configurability for 1745114 (include if exists support for apparmor), but after tests and discussions detecting that at build time wonā€™t be so easy (runtime works fine). Discussed with jjohanson about the lack of this configuration check but the item itself is punted to next cycle for reevaluation.
  9. coordinated testing and uploading of libvirt CVE fixes with mdeslaur
  10. finalize qemu 4.0 and upload to Eoan
  11. write and submit upstream a fix for libvirt fro .vhd images (in the meantime accepted and pushed)
  12. further qemu/xen experiments to cut the last few dependencies holding xen back in main (completed, handed over to kernel for testing)
  13. discuss new required nova-compute-qemu dependencies with jamespage
  14. debug and fix an issue in virt-tests an issue with xenial libvirt not ablte to spawn multiple graphics devices
  15. TCP+SCRAM libvirt auth bug that needed analysis but turned out to be works as intended (insecure)
  16. fix 1836066 for qemu in Eoan (minor change about vector facility name)
  17. combine branches several intended SRUs for 1830243 1832622 and 1828495 to kick off regression tests for them.
  18. identified, debugged and fixed qemu FTBFS in eoan. Upstream still hasnā€™t settled on a final fix, but all Distributions seem to carry the same interim change for now.

DPDK

  1. DPDK 18.11.2 merge for deb_dpdk and Debian
  2. work dpdk 18.11.2 into Eoan (sync + migation massage)
  3. move deb_dpdk to Salsa (from gerrit) for better CI and review
  4. work DKD 17.11.6 into deb_dpdk
  5. prepared 18.11.2 and 17.11.6 SRUs in MPs for bug 1836365
  6. Started basic testing on these SRUs, but I need the systems that currently do virt testing for the next step

Misc

  1. nspr merge for eoan. Needed an analysis for a build error for Delta that we had. Started the discussion about that with Debian and upstream as well as adding the insight to the patch header.
  2. prepared a merge for chrony 3.5 (waiting for review)
  3. help vmware for their open-vm-tools test builds to better cover most Ubuntu releases
  4. strongswan discussed with Debian maintainer about the many changes we still ahve as Delta and if more could be accepted in Debian as well (yes for most \o/)
  5. upstream report for sssd 1821927 to get things rolling
  6. help upstream systemd for s390x tests in qemu TCG
  7. file dkms bug 1836144 to ease debugging on autopkgtest fails on dkms one day
  8. postgres MRE for xenial was still ongoing, I needed to resolve a few more test errors to let it pass SRU
  9. numad crash on sparese nodeids unblocked 1832915 by MP review coming in, upload to Eoan and prep SRU branches and MPs

MP and MIR Reviews

  1. iproute2 - MP review and help on proposed migration
  2. review for Ruby 2.3 update by @ahasenack
  3. MIR review of ec2-instance-connect 1835114
  4. worked on the probert MIR to get it completed
  5. review and sponsoring on rsyslogd on bug 1827253 for sdeziel
  6. review zeromq3 for @ahasenack
  7. review for lasso for dmirtiish in bug 1833299
  8. edk2 MIR 1570617 completed, but needed some help to get completed in Eoan (eventually an qemu upload)
  9. review iproute uploads for Disco by @rafaeldtinoco (and sponsor them)
  10. MIR re-review on usbguard
  11. uvtool MP for new features in Eoan needed to resplit changes for rbasak to better review
6

Very short week this week, as I was travelling Tuesday to Friday for partner visits.

cloud-init

curtin

Miscellaneous

  • cloud-init/curtin triage between flights on Tuesday
1 Like
7

Short week due to travel Tues/Friday for partner visits.

Cloud-init

  • Worked on an alternative threadpool based branch for async cloud-init config module running. This avoids a dependency on systemd directly; it also allows to enable asycn on more than just mkfs/mount modules; notably growpart, resizefs and others. WIP branch here

Curtin

1 Like
8

Performance

  • As a step towards automated performance measurements on the clouds, Iā€™m extending pycloudlib, improving its support for LXD (merge proposal) and KVM via multipass
  • Performance metrics on devices (Raspbery PIs, DragonBoard 410c) made more solid

curtin

  • Merge proposal for curtin enabling the vmtests on arm64; ongoing work in this direction
  • Initial work towards enabling the vmtests in ppc64

Subiquity

  • Testing of the new ā€œreusing existing partitionsā€ feature
  • Review of a new relevant test case definition

Infrastructure

  • Updated UTAH on the main ISO testing Jenins node
  • Fixed some bugs in UTAH which were causing failures in the ISO testing jobs with the updated version, partially because of a bug in ubiquity
  • Documented the incident for future reference
  • Connected two more Jenkins worker nodes to our CI and testing infrastructure
1 Like