Archive.ubuntu.com infrastructure change broke do-release-upgrade

[felixf]

Ubuntu Support Template

Ubuntu Version: 24.04
Desktop Environment (if applicable): -
Problem Description:
When trying to run do-relase-upgrade -d, there’s an error message:

Could not find the release announcement
The server may be overloaded.

I did debug this a bit, and found out that this seems to be due to a change in archive.ubuntu.com’s infrastructure:

The code that produces this error message uses Python’s urllib.request.urlopen() to load http://archive.ubuntu.com/ubuntu/dists/resolute/main/dist-upgrader-all/current/ReleaseAnnouncement?lang=C; this results in a 403 Forbidden HTTPError. When using curl to access this URL, or a browser, it works fine.

This is easy to reproduce in Python:

$ python3
>>> from urllib.request import urlopen, Request
>>> urlopen("http://archive.ubuntu.com/ubuntu/dists/resolute/main/dist-upgrader-all/current/ReleaseAnnouncement?lang=C")
Traceback (most recent call last):
[...]
urllib.error.HTTPError: HTTP Error 403: Forbidden

Now if I add a User-Agent header to the request, it works fine:

>>> r = Request("http://archive.ubuntu.com/ubuntu/dists/resolute/main/dist-upgrader-all/current/ReleaseAnnouncement?lang=C")
>>> r.add_header("User-Agent", "curl/1.2.3")
>>> urlopen(r).read()
b"= Welcome to Ubuntu 26.04 LTS 'Resolute Racc[...]"

So for some reason (maybe to fight off the DDoS attack last Friday?), archive.ubuntu.com’s config seems to have been changed to only accept certain user agents. Since I did a similar upgrade on Friday morning, and it worked fine, I guess the config change has been made since them (which is why I guess it’s a mitigation).

I’m not sure where to report this, so I’ve decided to post it here.

Relevant System Information: Nothing relevant.

Screenshots or Error Messages:

Could not find the release announcement
The server may be overloaded.

What I’ve Tried:
Add some debugging print()s to the do-release-upgrade script, and tried to reproduce the problem in regular a python3 REPL.

Before Posting: I tried searching for the error message, but couldn’t find anything relevant.

[g-schick]

Best practice for a successful release-upgrade to 26.04

1. Be Patient Wait until the supported release-upgrade window opens. Your system will prompt you. 25.10 users: Your window opens a few weeks. 24.04 users: Your window opens in August. You’ll see bad advice telling you to use do-release-upgrade -d. Don’t do it…unless you want to be a tester and know how to troubleshoot and report bugs properly. Using the 26.04 installer won’t upgrade your system. The installer will overwrite your system, erasing all data and configuration. 2. Backup your da…

[guiverc]

That is a known problem, and a BLOCKER BUG keeping the 25.10 upgrade to 26.04 closed for now.

Bug #2151246 “upgrades from all releases fail to fetch release a...” : Bugs :...

From that bug report, the description text gives an example

root@j:~# do-release-upgrade
Checking for a new Ubuntu release
Could not find the release announcement
The server may be overloaded.

It’ll be fixed soon.

[rubi1200]

And from the link @g-schick posted please read this:

@ian-weisser

You’ll see bad advice telling you to use do-release-upgrade -d. Don’t do it…unless you want to be a tester and know how to troubleshoot and report bugs properly.

Thanks

[felixf]

Thanks @guiverc for linking to that bug, I didn’t find that one.

[guiverc]

Thank you @felixf for finding the problem, and reporting it here !!!

As it’s a KNOWN issue, all we can do currently is click the ‘affects me too’ button on the bug report (ie. add heat).

Your diligence in finding the issue, researching it & reporting is to be commended, so once again thank you !

( All I did was copy/paste a URL from a matrix room where I saw it mentioned )

[felixf]

It seems the issue is resolved, upgrading is working again.