Phased updates in APT in 21.04

Foundations
#20

Usually phasing applies only to the updates pocket, and happens automatically by a script - in absence of the original blog post, see the LWN article from 2013 for more information about how this works: https://lwn.net/Articles/563966/.

If the version in -security is the same as in -updates, phasing from updates may also apply to the security update. This depends on the concrete implementation and the order of the entries in sources.list.

The Phased-Update-Percentage field is stored once per version: It seems that in previous implementation in update-manager, the first entry of a version determines the value. In APT, all repositories are looked at and the last repository with a Phased-Update-Percentage field for a given version determines the stored value.

I think we now have the ability to also set the Phased-Update-Percentage field in the security pocket on demand for larger security updates, but I’m not sure.

To find out how phased updates apply to your specific system, it’s best to look at apt-cache policy for a package that is currently phasing (see https://people.canonical.com/~ubuntu-archive/phased-updates.html for a list) .

2 Likes
#21

Revisiting my Feb 2021 prediction after one year: Well, I was wrong about that (and happy to be wrong).

AskUbuntu has had but a few questions about Phased Updates, and the explanations in this thread were enough to answer those questions.

Update: Sept 2022 – LOTS of questions about phased updates after a batch of SRUs, lots of confused users, but the information in this thread is still enough to answer those questions.

2 Likes
#22

I’m curious to know when the libclutter-gst-3.0-0 with the webcam fix for Cheese will reach my machine. apt policy says

libclutter-gst-3.0-0:
  Installed: 3.0.27-2build2
  Candidate: 3.0.27-2build2
  Version table:
     3.0.27-2ubuntu1 1 (phased 50%)
        500 http://fi.archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
 *** 3.0.27-2build2 500
        500 http://fi.archive.ubuntu.com/ubuntu jammy/main amd64 Packages
        100 /var/lib/dpkg/status

https://wiki.ubuntu.com/PhasedUpdates says the phasing will be automatically increased by 10% every 6 hours if there are no reported regressions. It also says

A computer is in the testing set if Phased-Update-Percentage ✕ 2^128 ≥ md5(machine id + package name + package version).

So if I compute md5(machine-id + package name + package version), divide by 2^128, and multiply by 100%, I should know what the target phase is for my machine, and then I can estimate the number of hours remaining, right?

$ printf "%s" $(cat /etc/machine-id) libclutter-gst-3.0-0 3.0.27-2ubuntu1|md5sum
70842b420e10c434b1de82ffae8400f2  -
$ python
>>> '{:.1%}'.format(0x70842b420e10c434b1de82ffae8400f2/2**128)
'43.951674%'

Since 50% >= 43.951674%, my machine should be in the testing set already, no?

Where did I go wrong?

1 Like
#23

Don’t trust the wiki, it describes the initial design from a long time ago. The string itself changed, and we actually seed a RNG since quite a long time, even in the Python code in update-manager.

To be concrete, what happens in APT is that we take the string <source package>-<source version>-<machine id>, seed a std::minstd_rand RNG and then use std::uniform_int_distribution<unsigned int> dist(0, 100) to create a value between 0 and 100 which we then compare with the percentage:

   std::string seedStr = std::string(Ver.SourcePkgName()) + "-" + Ver.SourceVerStr() + "-" + machineID;
   std::seed_seq seed(seedStr.begin(), seedStr.end());
   std::minstd_rand rand(seed);
   std::uniform_int_distribution<unsigned int> dist(0, 100);

   return dist(rand) > Ver.PhasedUpdatePercentage();

This might yield different values from update-manager due to differences in seeding and RNG implementation between Python and gcc’s libstdc++.
However apt’s one is implemented a level lower and can’t be overriden by update-manager anyway, so it’s safe.

Or between different C++ standard libraries, even, I suppose. I don’t know how much the behavior of minstd_rng and others are standardized.

I don’t know for sure why we stopped using md5sum, that was decades ago, but it certainly simplifies the math rather than having to use ** 128 (and rng seeding presumably is faster than md5 too?).

3 Likes
#24

So I’ve got this configuration:

$ cat /etc/apt/apt.conf.d/99-phased-updates 
# dont take part of phased updates
# Disable phased updates: https://people.canonical.com/~ubuntu-archive/phased-updates.html
# https://discourse.ubuntu.com/t/phased-updates-in-apt-in-21-04/20345
#
# To have all your machines phase the same, set the same string in this field
# If commented out, apt will use /etc/machine-id to seed the random number generator
#APT::Machine-ID "aaaabbbbccccddddeeeeffff";
#APT::Machine-ID "n0c0ffeen0deadbeef50l337";

# Always include phased updates (Default = 1)
APT::Get::Always-Include-Phased-Updates "false";

# Never include phased updates
APT::Get::Never-Include-Phased-Updates "true";

But sometimes, it fails, downgrading them manually, then retrying upgrade/dist-upgrade, doesn’t install them anymore, but initially they get installed anyways. Something is weird, but I can’t tell what.

The GUI for user update upgrades is not installed and removed.

#25

I feel like I am missing something rather basic that everyone else in this thread seems to have grokked without effort. Can someone tell me how it is possible that these two statements (apparently) do different things?

APT::Get::Always-Include-Phased-Updates = true
APT::Get::Never-Include-Phased-Updates = false

Based on my amateur understanding of boolean logic, “true” and “false” are opposite values, as are “always” and “never”. So an inverse relationship resolves to equality, thus Always: True == Never: False. So as described, these should mean the same thing.

I understand (or rather, believe) that this is a tri-state system with three possible values:

  1. This machine installs updated packages as soon as they are published to the repos
  2. This machine participates in the phased update system (the default)
  3. This machine only installs updated packages after they have been fully phased-in

So I suppose my question is: Which exact apt settings and values map to each of these three states?

#26
  1. Always=true
  2. Both unset/false
  3. Never=true

If both are true the behavior is unspecified. The default for each option is false.

Never=false does not imply Always=true. Always true should imply never=false and never=true should imply always=false but in practice you can obviously set both to true and a random one would be respected or it’s an error.

There’s no error reporting where the code was added, so one takes preference, but I don’t recall which one.

1 Like
#27

Shouldn’t this be “To never include phased updates, set APT::Get::Never-Include-Phased-Updates to true.”?

#28

While we are on the topic of “phased updates”, how do Update-Manager::Always-Include-Phased-Updates; and Update-Manager::Never-Include-Phased-Updates; correspond to these 3 states?

#29

The APT ones take precedent but if an apt option is missing it falls back to the legacy name.

1 Like
#30

The addition of phased updates is causing a lot of confusion and a number of questions on ask ubuntu.

Much of this confusion and questioning arises because the user is notified only with the message “the following updates have been held back”…

Many of us have been associating this message for years with problems in the package system.

Can a more informative message be added along the lines of “the following packages are waiting to be installed due to phased updates” ?

11 Likes
#31

just to follow up here, on IRC in the #ubuntu support channel this is currently the most asked issue as well …

9 Likes
#32

Hey Julian, can you take a quick look at this bullet point again? I think I would have expected the english sentence “never include phased updates is false” to mean something like “sometimes include phased updates” or “always include phased updates”. Was false the right word in the sentence?

Thanks

#33

You are right, but I can’t edit the post anymore.

#34

I am aware and weighing the options, which are

  • do not show the list at all
  • show separate lists (new strings, still stuff most won’t care about)
  • keep as is

I’m leaning towards not showing phased updates to not break existing translations and , but I need to make sure that works reliable - that the “silent” flag gets cleared if I remove the keep flag again.

2 Likes
#35

Hi, can we have some clarity on this please? You say sarnold is “right”, but all they’ve done is point out an inconsistency. They (rightly) did not suggest a correction. What is the correct setting? I know you say you can’t edit the post any more, but that’s really not acceptable given that this post is linked from the official release notes of Ubuntu 22.04. If this post cannot be edited, can a new one be created containing the correct content and linked from the release notes instead of this post?

#36

I did a lot of reading and concluded and summarized that in order to configure that one’s machine only installs updated packages after they have been fully phased-in, one should configure:

Update-Manager::Never-Include-Phased-Updates true;
// APT config takes precedence over Update Manager.
APT::Get::Never-Include-Phased-Updates true;

In (a newly created) file:

/etc/apt/apt.conf.d/99-Phased-Updates

But when I run:

sudo apt-get update
sudo apt-get full-upgrade

then I still get the message:

The following packages have been kept back:
dpkg

I suppose that this is to be expected or is this an indication that not fully phased in updates can still be tried out on the system?

#37

So here’s my recipe for 22.04 LTS to get rid of phased upates (configuration, and command to run to downgrade to Ubunut without jammy-updates, to make sure never not any phased update lands on your machine)

https://github.com/alexmyczko/autoexec.bat/blob/master/config.sys/ubuntu-remove-phased-updates

HTH

1 Like
#38

Well, yes, that’s the expected behavior that it keeps back all phased updates when you tell it to keep back all phased updates.

1 Like
#39

@tar The link to the actual code that checks if the phased updates should be ignored is very helpful, thank you. Here is the snippet of code that it boils down to:

if (_config->FindB(“APT::Get::Never-Include-Phased-Updates”,
_config->FindB(“Update-Manager::Never-Include-Phased-Updates”, false)))
return true;

It means that if at least one of the two config options is set to true, then the containing IsIgnoredPhasedUpdate method returns true, i.e. phased updates are ignored by apt.
Which in its turn means one’s machine only installs updated packages after they have been fully phased-in.

@ juliank Thanks for confirming the expected behavior.

1 Like