Release 26.04 LTS without the ISO Tracker

Pre-Release Discussion
1

Abstract

The ISO Tracker has served as the backbone of Ubuntu’s image testing and release validation for over 15 years. However, it has grown increasingly unreliable, difficult to maintain, and nearly impossible to deploy in modern environments (read as: charms). For Ubuntu 26.04 LTS, we propose to release without the ISO Tracker entirely.

Instead, image testing and validation will be coordinated using Test Observer (you’ll hear about it from us soon), QA Jenkins, and lightweight manual aggregation tools (like shared dashboards or spreadsheets). This will help us decouple the release process from legacy infrastructure, simplify the workflow, and learn what parts of the tracker are truly essential – paving the way for its complete replacement in the future cycles.

Rationale

The ISO Tracker is an aging PHP/Drupal application built more than a decade ago and held together with duct tape and goodwill. It has accumulated years of technical debt, database hotfixes, and code rot that make deployment and maintenance increasingly hard.

We had initially planned to “charm” the ISO Tracker to modernize its hosting and reduce manual maintenance. However, this turned out to be a significant engineering challenge:

  • The ISO tracker codebase is outdated and poorly modularized.
  • The database is brittle and prone to failure.
  • Charming and redeploying it would require a near rewrite of large sections of code and schema handling.

Given these realities, continuing to rely on the ISO Tracker was no longer a sustainable option. We have therefore decided to take a more decisive step: release Ubuntu 26.04 LTS without it.

This will give us an opportunity to:

  • Identify which parts of the ISO Tracker process are actually required.
  • Evaluate modern replacements (Test Observer, Jenkins dashboards, Launchpad-based gating).
  • Simplify our release process and shed legacy dependencies.

While this is a bold move, the timing is right – Test Observer and QA Jenkins already cover much of what we need, and the remaining gaps can be bridged with lightweight coordination tools for the upcoming cycle.

Specification

Immediate Approach

For the 26.04 LTS cycle, the ISO Tracker will not be used for adding testcases, image validation, or reporting bugs. Instead:

  • A temporary shared spreadsheet/dashboard (details to be decided) will act as the central view for all reported results – serving as a lightweight replacement for the ISO tracker’s “Mark image as ready”.
  • Release Management will also maintain its own testing coordination thread on Discourse, using data from automated testing for structured reporting.
  • Longer term goal for the Release Management team: Image test results will be collected via Test Observer, which already provides a lot of what we need – Release Management will work in the coming cycles to ensure that it is ready to collect image validation results.

Automated Testing and Validation

  • Automated testing of daily and milestone images will continue through QA Jenkins jobs.
  • LP & Jenkins will remain the source of truth for:
    • Image build success/failure.
    • Smoke test and automated test results.
      • Jenkins instance (internal).
      • Other tests:
        • We’ve now automated all the Desktop ISO testing.
        • Same for Raspberry Pi testing – hours of manual testing is now 90 minutes of automation.
      • We’ll ask Flavours to write automated tests that we can run on our infra as well.
  • The release team will manually confirm image readiness based on the aggregation of results from our automated testing and flavored approved automated testing.

Coordination and Communication

  • Calls for testing, milestone coordination, and release readiness updates will be handled via:
    • Discourse (main threads for milestones and final release)
    • Matrix (for snapshots – if flavours and other folks are interested)
  • Flavour leads will update testing status during the milestone weeks.
  • The release team will maintain a lightweight internal dashboard or Google Sheet (details to be decided and announced at a later time) that aggregates:
    • Jenkins validation results,
    • Test Observer reports, and
    • Flavour readiness confirmations.

Risk & Mitigation

Whilst this is a bold move for an LTS cycle, we’ll have this process without the ISO tracker exercised throughout the cycle during various milestones:

  • Snapshots (end of each month)
  • 24.04.4 LTS release
  • 26.04 LTS Beta release

We’ll do a careful internal retrospective after each cycle and make sure we’re ready to do the 26.04 LTS final release without the ISO tracker.

8 Likes
2

I just hope any type of spreadsheet/dashboard that will be used will be publicly available as ISO tracker is today.

1 Like
3

Absolutely – that’s the idea. Everyone will be able to see the dashboard/spreadsheet and all the flavour leads will be able to add/remove/validate the images there. :smiley:

1 Like
4

Sounds good to me for Ubuntu Cinnamon!

2 Likes
5

Ack from Kubuntu.

(Post must be at least 20 characters)

2 Likes
6

Ack from Ubuntu Studio

Ack from Edubuntu

/me laughs in long flavor name

3 Likes
7

One feature of the ISO tracker that the community loved that I’d hate to see be lost is the ability to keep track of how many tests each user has done. There was a leaderboard on the old tracker to gamify the system, and it was very neat to be able to see the insane numbers of tests some of our top contributors like leok and irihapeti did.

That being said, I think this is a good move. The ISO tracker has been a massive pain almost every release. If we have to lose a leaderboard so that we can test things more efficiently, so be it IMO.

Ack from Lubuntu.

5 Likes
8

My concern here is the timing. If I understand the article correctly, we’re replacing the ISO Tracker with Test Observer. But Test Observer isn’t ready, so we’re replacing the ISO Tracker with a spreadsheet. Why the rush? Is a shared spreadsheet better than the ISO Tracker, even if it is aging infrastructure? Or am I mistaken, and Test Observer will launch sometime during the 26.04 cycle?

Ack from Xubuntu.

9

One feature of the ISO tracker that the community loved that I’d hate to see be lost is the ability to keep track of how many tests each user has done. There was a leaderboard on the old tracker to gamify the system, and it was very neat to be able to see the insane numbers of tests some of our top contributors like leok and irihapeti did.

Immediately speaking, that won’t be available in whatever software we end up using for 26.04 (or the one after that). We might be able to just use Launchpad for all the things and if we do that, we should be able to regain that metric.

That said, I’ve noted your use case and will try and make sure it’s not fully lost.

That being said, I think this is a good move. The ISO tracker has been a massive pain almost every release. If we have to lose a leaderboard so that we can test things more efficiently, so be it IMO.

Yep – agreed. Thank you!

1 Like
10

Two things – Test Observer will be public pretty soon, hopefully in less than a month. And for the other, we had to charm it to move it to another cloud, which is a complex task for not much benefit. So because of that, we decided to pull the plug a bit early.