GNOME doesn’t have formal testplans written down that they follow for those updates if that’s what you mean? They basically rely on community testing the same way as we do when we release a new Ubuntu version
A bit of a FYI
I note that gnome-maps is in the proposal for an exception. Thanks for that @rbasak and team
gnome-maps is a good example of the exception benefit for focal users.
Upstream state that the fix for this failure (where the application no longer launches) is covered by their current 3.36.7 microrelease - focal’s version is currently 3.36.1
Its fortunate in this case that the fix for the failure is a relatively small patch which is the process of being evaluated and hopefully rolled out as part of the standard SRU process.
So, it’s been a year. What is stopping us from completing this MRE documentation?
- We don’t have an algorithm for updating this list other than “Desktop team member asks the SRU team to consider adding something to the list”, but that shouldn’t be a blocker.
- I don’t believe we’ve settled the “How do we handle
gnome-shelluploads with respect to packaged extensions” question.
- There doesn’t seem to have been a clamour for a Vala MRE process, so we can probably just drop that.
- What should the test-plan be for SRU verification.
I think the next thing to do here is to drop everything which is not yet decided from the list, and point the GNOME MRE documentation on the Wiki to the list. As I understand it, that means that
gnome-shell and the GNOME games get removed for now.
This makes the cases which are clearcut clear, so is an improvement on the status quo.
For 4. - In the past the test plan has been poke around and check that things haven’t caught fire , and while I think we can do something better, for now I’d be happy with that.
This is explicitly not expected to be the final state -
gnome-shell obviously belongs on the list, once we’ve decided how to handle the reverse-dependencies, and I’m inclined to include the GNOME games. This is intended to be the easiest thing to do now that makes the situation better while we work out what to do with the difficult cases.
Thanks Chris for posting an update on that topic. Speaking from a desktop perspective, moving forward with the simplified-list-without-the-items-in-discussion seems fine, better than the current blocked situation.
Not having vala included in the GNOME MRE was previous agreed on yes. There is always the option to go through the MRE process specifically for it if we believe that it should also have an exception.
On the gnome-shell side, we removed the universe extensions before the LTS, https://bazaar.launchpad.net/~ubuntu-archive/+junk/sync-blacklist/revision/888.1.1
Was there anything else needed to make gnome-shell an acceptable candidate?
On the testplan, it would be nicer to have a description for each component but that requires work. What about starting with a ‘ensure it is still working by poking at it as you can’ and then we will try to update the wiki for each SRU we do from now one which isn’t covered yet?
I have some notes from a discussion I had out-of-band with Seb on 30 June 2021. At that time I tried to distill a set of remaining steps that could conclude this matter. I’ll paste them here for reference - some of these are already addressed.
Refine how the list is built
Rather than having a static list describing the components it would be better to have a definition pointing to an upstream set (gnome core + sdk + …?) and on top a manual list of exceptions we want to see added.
This way if GNOME is adding new components they should be covered by the exception without having to discuss with the SRU team again.
Decide what we do with the gnome-shell rdepends
We want to avoid breaking components provided in the Ubuntu archive when doing a SRU
Special case for the games
Those are only games, they are not something users rely on so the regression potential is limited. Desktop would like to see them included in the exception even if they are not core component and don’t have testsuites, so they can get updated at lower cost.
It is agreed that if a regression is found it is fine to revert or delete the SRU.
Would that be acceptable to the SRU team?
Describing the SRU verification steps
The wiki page describing the exception should include the verifications to be done with the updates. We probably want domain specific steps for some components at least
SRU team ack
The SRU team needs to sign off the list once they agree with what is there
Vala is a special case, we will keep out of the current MRE discussion and create a separate topic for it later if needed
I found a simpler base list that we can use. Debian provides a tracker for each GNOME release based on what GNOME itself decides to release. It is simpler than trying to verify multiple files in a GNOME git repo and the GNOME git repo doesn’t handle build-depends well. The Debian tracker just provides a prettier view of the GNOME release metadata.
This allows the sets to be updated over time as GNOME makes changes to what they release.
For Ubuntu 22.04 LTS, it’s https://people.debian.org/~fpeters/gnome/debian-gnome-42-status.html
For Ubuntu 20.04 LTS, it’s https://people.debian.org/~fpeters/gnome/debian-gnome-3.36-status.html
Ignore the unrelated Telepathy section.
From that base list, I propose these exceptions:
- everything that the gnome-games apt package directly depends on
Starting with Ubuntu 22.04 LTS, we removed most GNOME Shell extensions but left ones that require some system integration. We are now doing a basic smoke test of those extensions as part of our SRUs. Here’s a recent example.
As a shared critical component with Ubuntu Budgie, we also ensure that new Mutter SRUs are tested on Ubuntu Budgie (so far, the testing has been done directly by Ubuntu Budgie testers).
By the way, years ago, GNOME used to include a lot more optional apps in their release set but they reduced the set to a more basic set that they recommend to be installed by default. That’s how evolution and the GNOME games fell out of the set.
Diff from previous work
Here’s a diff of just the base set for 22.04 LTS compared to the earlier proposed list. It’s a little noisy because the earlier proposed list included the exceptions.
--- old -accerciser adwaita-icon-theme at-spi2-atk at-spi2-core atk1.0 +atkmm1.6 baobab +gnome-calls +fonts-cantarell cheese -clutter-gst +clutter-1.0 +clutter-gst-3.0 clutter-gtk -clutter +cogl +dconf dconf-editor devhelp eog -epiphany +epiphany-browser evince evolution-data-server -evolution -evolution-ews -file-roller folks +gcab gcr gdk-pixbuf gdm3 -gedit +geocode-glib +gexiv2 +gfbgraph gjs +glib2.0 glib-networking -glib -gnome-2048 +glibmm2.4 +gmime gnome-autoar gnome-backgrounds gnome-bluetooth +gnome-bluetooth3 gnome-boxes gnome-builder gnome-calculator gnome-calendar gnome-characters -gnome-chess gnome-clocks gnome-color-manager +gnome-connections +gnome-console gnome-contacts gnome-control-center gnome-desktop gnome-disk-utility gnome-font-viewer -gnome-games-app -gnome-getting-started-docs gnome-initial-setup gnome-keyring -gnome-klotski gnome-logs -gnome-mahjongg gnome-maps gnome-menus gnome-music -gnome-nibbles gnome-online-accounts -gnome-online-miners gnome-photos gnome-remote-desktop -gnome-robots -gnome-screenshot gnome-session gnome-settings-daemon gnome-shell gnome-shell-extensions gnome-software -gnome-sudoku gnome-system-monitor -gnome-taquin -gnome-terminal -gnome-tetravex +gnome-text-editor gnome-tour gnome-user-docs gnome-user-share gnome-video-effects gnome-weather gobject-introspection +libgom grilo grilo-plugins gsettings-desktop-schemas gsound -gst-plugins-base -gtk+-3 +gspell +gssdp +gtk4 +gtk+3.0 gtk-doc -gtkmm-3 -gtksourceview gtk-vnc -gvfs-daemon -iagno +gtkmm3.0 +gtksourceview4 +gtksourceview5 +gupnp +gupnp-av +gupnp-dlna +gvfs json-glib +jsonrpc-glib +libadwaita-1 +libchamplain libdazzle -libgweather -libhandy-0 -libhandy +libgdata +libgee-0.8 +libgnomekbd +libgsf +libgtop2 +libgweather4 +libgxps +libhandy-1 +libmediaart +libnma libnotify +libpeas librsvg libsecret -libsoup +libsigc++-2.0 +libsoup2.4 +libsoup3 +mm-common mutter nautilus +orca +pango1.0 +pangomm +phodav +pyatspi pygobject -quadrapassel +librest rygel -seahorse simple-scan -startup-notification -sushi -swell-foop -tali +gnome-sushi +sysprof totem totem-pl-parser -tracker-2 -tracker-miners-2 +tracker tracker-miners -tracker +vala +vte2.91 +xdg-desktop-portal-gnome yelp yelp-tools yelp-xsl +zenity + +Special additions: +evolution +evolution-ews +file-roller +gedit +gnome-terminal +gnome-tweaks +seahorse +everything that the gnome-games apt package directly depends on + +Special removals: +vala
The Debian tracker parses files like this:
Robie suggested that it may be interesting to diff those files to see changes over time.
The seemingly doubled gnome-bluetooth entry is that:
The seemingly doubled gnome-bluetooth entry
I asked the Debian tracker maintainer to use the Debian package names instead of upstream’s names and he already made that update for the 42 tracker.
So I updated my diff from comment 42 now too.
That looks like a good set to me. It includes some things that I don’t think we particularly care about (particularly
gnome-console), but GNOME obviously cares about them and exercising the MRE exception for a package is an indication that we do care about it .
Having a diff over time would be nice, but I don’t think that’s a blocker.
I think this is a good methodology and list to use for the MRE documentation.
I think the next step will be to update the MRE wiki page to point to the Debian tracker and add the extra documented exceptions?
I’ve now updated the GNOME MRE wiki page. Please check that what I’ve recorded matches your understanding of the consensus.
Thanks @raof! This looks good to me, though there are still some steps that I think we need to do to complete this. Maybe for the desktop team?
I’d like to be able to quickly determine if a particular package is part of the list or not, in order to actually process these SRUs. Right now that seems tricky.
Changes to the list expansion are expected and I assume a single member of the SRU team will decide if they make sense or not as MRE SRUs are reviewed. However this isn’t possible without being able to see the full expansion that was originally approved (ie. as it stands right now) and how it has changed. This is what I mean when I say I want an audit trail. Otherwise, we’d be in the odd position where upstream can effectively change our list without us noticing.
Right now it looks like specific test plans are being proposed individually in SRU bugs of MRE uploads. I suggest these are moved (following SRU team approval) to the wiki page. That way they wouldn’t have to be reviewed every time, which should reduce bikeshedding, save some copying and pasting and and make the review process smoother.
For (1) and (2) - maybe what we want is to our own script to parse the upstream metadata and map to source package names, and then commit the output (per release) to
ubuntu-archive-tools? We don’t expect the list to change within a release, only between releases.
I guess the stretch goal here is for
sru-review to grow support for emitting “This package falls under the
$PACKAGE MRE; see
$URL for details” on the relevant packages…
Please give both the script and the lists a once-over; I’m happy with what it generates, but may well have missed something.
Integrating a check for when a package is on this list (or other MRE lists) into the SRU tooling is still a stretch goal
Update update: I’ve generated the list for 22.10. This has required small updates to the script (to handle newly-duplicated packages)
Do we have a diff from previous one?
EDIT, here it is:
--- jammy 2022-11-01 17:26:19.000000000 +0100 +++ kinetic 2022-11-01 17:25:42.000000000 +0100 @@ -1,8 +1,6 @@ adwaita-icon-theme aisleriot -at-spi2-atk at-spi2-core -atk1.0 atkmm1.6 baobab cheese @@ -31,7 +29,6 @@ gedit geocode-glib gexiv2 -gfbgraph gjs glib-networking glib2.0 @@ -40,7 +37,6 @@ gnome-2048 gnome-autoar gnome-backgrounds -gnome-bluetooth gnome-bluetooth3 gnome-boxes gnome-builder @@ -112,7 +108,6 @@ json-glib jsonrpc-glib libadwaita-1 -libchamplain libdazzle libgdata libgee-0.8 @@ -126,10 +121,12 @@ libmediaart libnma libnotify +libpanel libpeas librest librsvg libsecret +libshumate libsigc++-2.0 libsoup2.4 libsoup3 @@ -150,12 +147,13 @@ swell-foop sysprof tali +template-glib totem totem-pl-parser tracker tracker-miners vte +xdg-desktop-portal-gnome yelp yelp-tools yelp-xsl -zenity
Thanks for working on moving things forward Chris, we will review the script and the list.
On the SRU verification would it work for the SRU team if the instructions are stored in https://wiki.ubuntu.com/DesktopTeam/TestPlans/ (where ‘sourcename’ is the name of the Ubuntu source package formatted wikistyle, GnomeTextEditor for gnome-text-editor is an existing example).
We would create the pages from now on as we SRU packages which don’t have one yet.
I created a category for our current test cases. We commit to keep adding new ones as we prepare SRUs: https://wiki.ubuntu.com/CategoryDesktopTestPlans
raof, thank you for your work on the script! The output matches what I expect.
Just one note - we (SRU team) can definitely consider them case by case per SRU. But it would be better if each could be reviewed approved by the SRU team, if we could note that somewhere such that it is invalidated if there are significant changes. Then we could provide better consistency since if a test plan is already “SRU team approved” it won’t need to be reviewed again, and you wouldn’t be surprised with requests for changes or amendments if a different SRU team member reviews your upload. But then this needs to be done in a way that a future SRU team member can confirm that the test plan is the same one that was previously approved by a previous SRU team member.
I don’t mind what mechanism we use, but could we arrange that please? Since there will be a collection, perhaps we could review the first time a particular test plan is used, and then leave some kind of approval note?
I added a section to the bottom of https://wiki.ubuntu.com/DesktopTeam/TestPlans/gjs with an idea for how pages could be marked as reviewed. But it’s a decision for the SRU Team how those pages should be marked and reviewed.