Phased updates in APT in 21.04

Your original comment said you wanted explanation of how to tune the behavior in apt configs, which is definitely covered in the top post. Do you have specific questions about what’s written there?

I agree that this post doesn’t currently explain what phased updates ARE, which was also one of your concerns.

You describe the wiki page as “stale”, but it hasn’t changed because neither have the mechanics or definition of phased updates. It does contain a lot of information that is irrelevant and distracting to end users, because it is a developer spec, not end-user documentation. I agree that we should have some more user-facing documentation of the nature of this feature.

2 Likes

I’m sorry I don’t understand what you want. The post describes all the details relevant to end users / system administratrors.

It started with “what changed”, then “what are phased updates even”, then “how can I control that”, then “what if i configured update-manager already”, and then “advanced caveats for people working with chroots”.

What it specifically does not include is details as to how phased updates work, because that’s not relevant for people not writing package managers. It’s documented in the historical spec on the wiki, and of course in the DebianRepository/Format specification in the Debian wiki. But this seems completely out of scope for user-facing communication.

If you have a specific question, please ask it. As far as I can tell, everything you talked about is addressed in the initial post in as much detail as is relevant for end users.

1 Like

We became aware that some packages currently or in the past days failed to install due to this feature. For example, if you had ubuntu-release-upgrader-core installed, and wanted to install ubuntu-release-upgrader-gtk, the phasing was only applied to ubuntu-release-upgrader core, and hence it tried to install ubuntu-release-upgrader-gtk 21.04.11, but ubuntu-release-upgrader-core 21.04.10.

There will be a stable release update shortly to fix those issues by applying phasing to not-yet-installed packages as well.

4 Likes

I stumbled across this when investigating, why one raspberry finds updates and the other does not…
To be able to decide how to configure this, I miss this information:

What packages are phased?
All? Non-Security? Decided by maintainer?

Usually phasing applies only to the updates pocket, and happens automatically by a script - in absence of the original blog post, see the LWN article from 2013 for more information about how this works: https://lwn.net/Articles/563966/.

If the version in -security is the same as in -updates, phasing from updates may also apply to the security update. This depends on the concrete implementation and the order of the entries in sources.list.

The Phased-Update-Percentage field is stored once per version: It seems that in previous implementation in update-manager, the first entry of a version determines the value. In APT, all repositories are looked at and the last repository with a Phased-Update-Percentage field for a given version determines the stored value.

I think we now have the ability to also set the Phased-Update-Percentage field in the security pocket on demand for larger security updates, but I’m not sure.

To find out how phased updates apply to your specific system, it’s best to look at apt-cache policy for a package that is currently phasing (see https://people.canonical.com/~ubuntu-archive/phased-updates.html for a list) .

2 Likes

Revisiting my Feb 2021 prediction after one year: Well, I was wrong about that (and happy to be wrong).

AskUbuntu has had but a few questions about Phased Updates, and the explanations in this thread were enough to answer those questions.

Update: Sept 2022 – LOTS of questions about phased updates after a batch of SRUs, lots of confused users, but the information in this thread is still enough to answer those questions.

2 Likes

I’m curious to know when the libclutter-gst-3.0-0 with the webcam fix for Cheese will reach my machine. apt policy says

libclutter-gst-3.0-0:
  Installed: 3.0.27-2build2
  Candidate: 3.0.27-2build2
  Version table:
     3.0.27-2ubuntu1 1 (phased 50%)
        500 http://fi.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
 *** 3.0.27-2build2 500
        500 http://fi.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

https://wiki.ubuntu.com/PhasedUpdates says the phasing will be automatically increased by 10% every 6 hours if there are no reported regressions. It also says

A computer is in the testing set if Phased-Update-Percentage ✕ 2^128 ≥ md5(machine id + package name + package version).

So if I compute md5(machine-id + package name + package version), divide by 2^128, and multiply by 100%, I should know what the target phase is for my machine, and then I can estimate the number of hours remaining, right?

$ printf "%s" $(cat /etc/machine-id) libclutter-gst-3.0-0 3.0.27-2ubuntu1|md5sum
70842b420e10c434b1de82ffae8400f2  -
$ python
>>> '{:.1%}'.format(0x70842b420e10c434b1de82ffae8400f2/2**128)
'43.951674%'

Since 50% >= 43.951674%, my machine should be in the testing set already, no?

Where did I go wrong?

1 Like

Don’t trust the wiki, it describes the initial design from a long time ago. The string itself changed, and we actually seed a RNG since quite a long time, even in the Python code in update-manager.

To be concrete, what happens in APT is that we take the string <source package>-<source version>-<machine id>, seed a std::minstd_rand RNG and then use std::uniform_int_distribution<unsigned int> dist(0, 100) to create a value between 0 and 100 which we then compare with the percentage:

   std::string seedStr = std::string(Ver.SourcePkgName()) + "-" + Ver.SourceVerStr() + "-" + machineID;
   std::seed_seq seed(seedStr.begin(), seedStr.end());
   std::minstd_rand rand(seed);
   std::uniform_int_distribution<unsigned int> dist(0, 100);

   return dist(rand) > Ver.PhasedUpdatePercentage();

This might yield different values from update-manager due to differences in seeding and RNG implementation between Python and gcc’s libstdc++.
However apt’s one is implemented a level lower and can’t be overriden by update-manager anyway, so it’s safe.

Or between different C++ standard libraries, even, I suppose. I don’t know how much the behavior of minstd_rng and others are standardized.

I don’t know for sure why we stopped using md5sum, that was decades ago, but it certainly simplifies the math rather than having to use ** 128 (and rng seeding presumably is faster than md5 too?).

3 Likes

So I’ve got this configuration:

$ cat /etc/apt/apt.conf.d/99-phased-updates 
# dont take part of phased updates
# Disable phased updates: https://people.canonical.com/~ubuntu-archive/phased-updates.html
# https://discourse.ubuntu.com/t/phased-updates-in-apt-in-21-04/20345
#
# To have all your machines phase the same, set the same string in this field
# If commented out, apt will use /etc/machine-id to seed the random number generator
#APT::Machine-ID "aaaabbbbccccddddeeeeffff";
#APT::Machine-ID "n0c0ffeen0deadbeef50l337";

# Always include phased updates (Default = 1)
APT::Get::Always-Include-Phased-Updates "false";

# Never include phased updates
APT::Get::Never-Include-Phased-Updates "true";

But sometimes, it fails, downgrading them manually, then retrying upgrade/dist-upgrade, doesn’t install them anymore, but initially they get installed anyways. Something is weird, but I can’t tell what.

The GUI for user update upgrades is not installed and removed.

I feel like I am missing something rather basic that everyone else in this thread seems to have grokked without effort. Can someone tell me how it is possible that these two statements (apparently) do different things?

APT::Get::Always-Include-Phased-Updates = true
APT::Get::Never-Include-Phased-Updates = false

Based on my amateur understanding of boolean logic, “true” and “false” are opposite values, as are “always” and “never”. So an inverse relationship resolves to equality, thus Always: True == Never: False. So as described, these should mean the same thing.

I understand (or rather, believe) that this is a tri-state system with three possible values:

  1. This machine installs updated packages as soon as they are published to the repos
  2. This machine participates in the phased update system (the default)
  3. This machine only installs updated packages after they have been fully phased-in

So I suppose my question is: Which exact apt settings and values map to each of these three states?

  1. Always=true
  2. Both unset/false
  3. Never=true

If both are true the behavior is unspecified. The default for each option is false.

Never=false does not imply Always=true. Always true should imply never=false and never=true should imply always=false but in practice you can obviously set both to true and a random one would be respected or it’s an error.

There’s no error reporting where the code was added, so one takes preference, but I don’t recall which one.

1 Like

Shouldn’t this be “To never include phased updates, set APT::Get::Never-Include-Phased-Updates to true.”?

While we are on the topic of “phased updates”, how do Update-Manager::Always-Include-Phased-Updates; and Update-Manager::Never-Include-Phased-Updates; correspond to these 3 states?

The APT ones take precedent but if an apt option is missing it falls back to the legacy name.

1 Like

The addition of phased updates is causing a lot of confusion and a number of questions on ask ubuntu.

Much of this confusion and questioning arises because the user is notified only with the message “the following updates have been held back”…

Many of us have been associating this message for years with problems in the package system.

Can a more informative message be added along the lines of “the following packages are waiting to be installed due to phased updates” ?

10 Likes

just to follow up here, on IRC in the #ubuntu support channel this is currently the most asked issue as well …

8 Likes

Hey Julian, can you take a quick look at this bullet point again? I think I would have expected the english sentence “never include phased updates is false” to mean something like “sometimes include phased updates” or “always include phased updates”. Was false the right word in the sentence?

Thanks

You are right, but I can’t edit the post anymore.

I am aware and weighing the options, which are

  • do not show the list at all
  • show separate lists (new strings, still stuff most won’t care about)
  • keep as is

I’m leaning towards not showing phased updates to not break existing translations and , but I need to make sure that works reliable - that the “silent” flag gets cleared if I remove the keep flag again.

2 Likes

Hi, can we have some clarity on this please? You say sarnold is “right”, but all they’ve done is point out an inconsistency. They (rightly) did not suggest a correction. What is the correct setting? I know you say you can’t edit the post any more, but that’s really not acceptable given that this post is linked from the official release notes of Ubuntu 22.04. If this post cannot be edited, can a new one be created containing the correct content and linked from the release notes instead of this post?