GNOME doesn’t have formal testplans written down that they follow for those updates if that’s what you mean? They basically rely on community testing the same way as we do when we release a new Ubuntu version
A bit of a FYI
I note that gnome-maps is in the proposal for an exception. Thanks for that @rbasak and team
gnome-maps is a good example of the exception benefit for focal users.
Upstream state that the fix for this failure (where the application no longer launches) is covered by their current 3.36.7 microrelease - focal’s version is currently 3.36.1
Its fortunate in this case that the fix for the failure is a relatively small patch which is the process of being evaluated and hopefully rolled out as part of the standard SRU process.
1 Like
So, it’s been a year. What is stopping us from completing this MRE documentation?
- We don’t have an algorithm for updating this list other than “Desktop team member asks the SRU team to consider adding something to the list”, but that shouldn’t be a blocker.
- I don’t believe we’ve settled the “How do we handle
gnome-shelluploads with respect to packaged extensions” question. - There doesn’t seem to have been a clamour for a Vala MRE process, so we can probably just drop that.
- What should the test-plan be for SRU verification.
I think the next thing to do here is to drop everything which is not yet decided from the list, and point the GNOME MRE documentation on the Wiki to the list. As I understand it, that means that gnome-shell and the GNOME games get removed for now.
This makes the cases which are clearcut clear, so is an improvement on the status quo.
For 4. - In the past the test plan has been 
poke around and check that things haven’t caught fire , and while I think we can do something better, for now I’d be happy with that.
This is explicitly not expected to be the final state - gnome-shell obviously belongs on the list, once we’ve decided how to handle the reverse-dependencies, and I’m inclined to include the GNOME games. This is intended to be the easiest thing to do now that makes the situation better while we work out what to do with the difficult cases.
1 Like
Thanks Chris for posting an update on that topic. Speaking from a desktop perspective, moving forward with the simplified-list-without-the-items-in-discussion seems fine, better than the current blocked situation.
Not having vala included in the GNOME MRE was previous agreed on yes. There is always the option to go through the MRE process specifically for it if we believe that it should also have an exception.
On the gnome-shell side, we removed the universe extensions before the LTS, https://bazaar.launchpad.net/~ubuntu-archive/+junk/sync-blacklist/revision/888.1.1
Was there anything else needed to make gnome-shell an acceptable candidate?
On the testplan, it would be nicer to have a description for each component but that requires work. What about starting with a ‘ensure it is still working by poking at it as you can’ and then we will try to update the wiki for each SRU we do from now one which isn’t covered yet?
I have some notes from a discussion I had out-of-band with Seb on 30 June 2021. At that time I tried to distill a set of remaining steps that could conclude this matter. I’ll paste them here for reference - some of these are already addressed.
Refine how the list is built
Rather than having a static list describing the components it would be better to have a definition pointing to an upstream set (gnome core + sdk + …?) and on top a manual list of exceptions we want to see added.
This way if GNOME is adding new components they should be covered by the exception without having to discuss with the SRU team again.
Decide what we do with the gnome-shell rdepends
We want to avoid breaking components provided in the Ubuntu archive when doing a SRU
Special case for the games
Those are only games, they are not something users rely on so the regression potential is limited. Desktop would like to see them included in the exception even if they are not core component and don’t have testsuites, so they can get updated at lower cost.
It is agreed that if a regression is found it is fine to revert or delete the SRU.
Would that be acceptable to the SRU team?
Describing the SRU verification steps
The wiki page describing the exception should include the verifications to be done with the updates. We probably want domain specific steps for some components at least
SRU team ack
The SRU team needs to sign off the list once they agree with what is there
Vala
Vala is a special case, we will keep out of the current MRE discussion and create a separate topic for it later if needed
I found a simpler base list that we can use. Debian provides a tracker for each GNOME release based on what GNOME itself decides to release. It is simpler than trying to verify multiple files in a GNOME git repo and the GNOME git repo doesn’t handle build-depends well. The Debian tracker just provides a prettier view of the GNOME release metadata.
This allows the sets to be updated over time as GNOME makes changes to what they release.
For Ubuntu 22.04 LTS, it’s https://people.debian.org/~fpeters/gnome/debian-gnome-42-status.html
For Ubuntu 20.04 LTS, it’s https://people.debian.org/~fpeters/gnome/debian-gnome-3.36-status.html
Ignore the unrelated Telepathy section.
From that base list, I propose these exceptions:
Additions
- evolution
- evolution-ews
- file-roller
- gedit
- gnome-terminal
- gnome-tweaks
- seahorse
- everything that the gnome-games apt package directly depends on
Removals
- vala
GNOME Shell
Starting with Ubuntu 22.04 LTS, we removed most GNOME Shell extensions but left ones that require some system integration. We are now doing a basic smoke test of those extensions as part of our SRUs. Here’s a recent example.
Mutter
As a shared critical component with Ubuntu Budgie, we also ensure that new Mutter SRUs are tested on Ubuntu Budgie (so far, the testing has been done directly by Ubuntu Budgie testers).
Apps Note
By the way, years ago, GNOME used to include a lot more optional apps in their release set but they reduced the set to a more basic set that they recommend to be installed by default. That’s how evolution and the GNOME games fell out of the set.
Diff from previous work
Here’s a diff of just the base set for 22.04 LTS compared to the earlier proposed list. It’s a little noisy because the earlier proposed list included the exceptions.
--- old
-accerciser
adwaita-icon-theme
at-spi2-atk
at-spi2-core
atk1.0
+atkmm1.6
baobab
+gnome-calls
+fonts-cantarell
cheese
-clutter-gst
+clutter-1.0
+clutter-gst-3.0
clutter-gtk
-clutter
+cogl
+dconf
dconf-editor
devhelp
eog
-epiphany
+epiphany-browser
evince
evolution-data-server
-evolution
-evolution-ews
-file-roller
folks
+gcab
gcr
gdk-pixbuf
gdm3
-gedit
+geocode-glib
+gexiv2
+gfbgraph
gjs
+glib2.0
glib-networking
-glib
-gnome-2048
+glibmm2.4
+gmime
gnome-autoar
gnome-backgrounds
gnome-bluetooth
+gnome-bluetooth3
gnome-boxes
gnome-builder
gnome-calculator
gnome-calendar
gnome-characters
-gnome-chess
gnome-clocks
gnome-color-manager
+gnome-connections
+gnome-console
gnome-contacts
gnome-control-center
gnome-desktop
gnome-disk-utility
gnome-font-viewer
-gnome-games-app
-gnome-getting-started-docs
gnome-initial-setup
gnome-keyring
-gnome-klotski
gnome-logs
-gnome-mahjongg
gnome-maps
gnome-menus
gnome-music
-gnome-nibbles
gnome-online-accounts
-gnome-online-miners
gnome-photos
gnome-remote-desktop
-gnome-robots
-gnome-screenshot
gnome-session
gnome-settings-daemon
gnome-shell
gnome-shell-extensions
gnome-software
-gnome-sudoku
gnome-system-monitor
-gnome-taquin
-gnome-terminal
-gnome-tetravex
+gnome-text-editor
gnome-tour
gnome-user-docs
gnome-user-share
gnome-video-effects
gnome-weather
gobject-introspection
+libgom
grilo
grilo-plugins
gsettings-desktop-schemas
gsound
-gst-plugins-base
-gtk+-3
+gspell
+gssdp
+gtk4
+gtk+3.0
gtk-doc
-gtkmm-3
-gtksourceview
gtk-vnc
-gvfs-daemon
-iagno
+gtkmm3.0
+gtksourceview4
+gtksourceview5
+gupnp
+gupnp-av
+gupnp-dlna
+gvfs
json-glib
+jsonrpc-glib
+libadwaita-1
+libchamplain
libdazzle
-libgweather
-libhandy-0
-libhandy
+libgdata
+libgee-0.8
+libgnomekbd
+libgsf
+libgtop2
+libgweather4
+libgxps
+libhandy-1
+libmediaart
+libnma
libnotify
+libpeas
librsvg
libsecret
-libsoup
+libsigc++-2.0
+libsoup2.4
+libsoup3
+mm-common
mutter
nautilus
+orca
+pango1.0
+pangomm
+phodav
+pyatspi
pygobject
-quadrapassel
+librest
rygel
-seahorse
simple-scan
-startup-notification
-sushi
-swell-foop
-tali
+gnome-sushi
+sysprof
totem
totem-pl-parser
-tracker-2
-tracker-miners-2
+tracker
tracker-miners
-tracker
+vala
+vte2.91
+xdg-desktop-portal-gnome
yelp
yelp-tools
yelp-xsl
+zenity
+
+Special additions:
+evolution
+evolution-ews
+file-roller
+gedit
+gnome-terminal
+gnome-tweaks
+seahorse
+everything that the gnome-games apt package directly depends on
+
+Special removals:
+vala
2 Likes
The Debian tracker parses files like this:
https://download.gnome.org/teams/releng/42.2/versions
Robie suggested that it may be interesting to diff those files to see changes over time.
2 Likes
The seemingly doubled gnome-bluetooth entry is that:
core:gnome-bluetooth:42.0:
core:gnome-bluetooth:3.34.5:
1 Like
The seemingly doubled gnome-bluetooth entry
I asked the Debian tracker maintainer to use the Debian package names instead of upstream’s names and he already made that update for the 42 tracker.
So I updated my diff from comment 42 now too.
3 Likes
That looks like a good set to me. It includes some things that I don’t think we particularly care about (particularly gnome-console), but GNOME obviously cares about them and exercising the MRE exception for a package is an indication that we do care about it 
.
Having a diff over time would be nice, but I don’t think that’s a blocker.
I think this is a good methodology and list to use for the MRE documentation.
I think the next step will be to update the MRE wiki page to point to the Debian tracker and add the extra documented exceptions?
I’ve now updated the GNOME MRE wiki page. Please check that what I’ve recorded matches your understanding of the consensus.
Thanks @raof! This looks good to me, though there are still some steps that I think we need to do to complete this. Maybe for the desktop team?
-
I’d like to be able to quickly determine if a particular package is part of the list or not, in order to actually process these SRUs. Right now that seems tricky.
-
Changes to the list expansion are expected and I assume a single member of the SRU team will decide if they make sense or not as MRE SRUs are reviewed. However this isn’t possible without being able to see the full expansion that was originally approved (ie. as it stands right now) and how it has changed. This is what I mean when I say I want an audit trail. Otherwise, we’d be in the odd position where upstream can effectively change our list without us noticing.
-
Right now it looks like specific test plans are being proposed individually in SRU bugs of MRE uploads. I suggest these are moved (following SRU team approval) to the wiki page. That way they wouldn’t have to be reviewed every time, which should reduce bikeshedding, save some copying and pasting and and make the review process smoother.
For (1) and (2) - maybe what we want is to our own script to parse the upstream metadata and map to source package names, and then commit the output (per release) to ubuntu-archive-tools? We don’t expect the list to change within a release, only between releases.
I guess the stretch goal here is for sru-review to grow support for emitting “This package falls under the $PACKAGE MRE; see $URL for details” on the relevant packages…
I’ve now done the script to parse the upstream metadata and output a list of packages covered by the GNOME MRE, and committed the results (16.04, 18.04, 20.04, and 22.04).
Please give both the script and the lists a once-over; I’m happy with what it generates, but may well have missed something.
Integrating a check for when a package is on this list (or other MRE lists) into the SRU tooling is still a stretch goal 
Update update: I’ve generated the list for 22.10. This has required small updates to the script (to handle newly-duplicated packages)
2 Likes
Do we have a diff from previous one? /me lazy
EDIT, here it is:
--- jammy 2022-11-01 17:26:19.000000000 +0100
+++ kinetic 2022-11-01 17:25:42.000000000 +0100
@@ -1,8 +1,6 @@
adwaita-icon-theme
aisleriot
-at-spi2-atk
at-spi2-core
-atk1.0
atkmm1.6
baobab
cheese
@@ -31,7 +29,6 @@
gedit
geocode-glib
gexiv2
-gfbgraph
gjs
glib-networking
glib2.0
@@ -40,7 +37,6 @@
gnome-2048
gnome-autoar
gnome-backgrounds
-gnome-bluetooth
gnome-bluetooth3
gnome-boxes
gnome-builder
@@ -112,7 +108,6 @@
json-glib
jsonrpc-glib
libadwaita-1
-libchamplain
libdazzle
libgdata
libgee-0.8
@@ -126,10 +121,12 @@
libmediaart
libnma
libnotify
+libpanel
libpeas
librest
librsvg
libsecret
+libshumate
libsigc++-2.0
libsoup2.4
libsoup3
@@ -150,12 +147,13 @@
swell-foop
sysprof
tali
+template-glib
totem
totem-pl-parser
tracker
tracker-miners
vte
+xdg-desktop-portal-gnome
yelp
yelp-tools
yelp-xsl
-zenity
3 Likes
Thanks for working on moving things forward Chris, we will review the script and the list.
On the SRU verification would it work for the SRU team if the instructions are stored in https://wiki.ubuntu.com/DesktopTeam/TestPlans/ (where ‘sourcename’ is the name of the Ubuntu source package formatted wikistyle, GnomeTextEditor for gnome-text-editor is an existing example).
We would create the pages from now on as we SRU packages which don’t have one yet.
1 Like
I created a category for our current test cases. We commit to keep adding new ones as we prepare SRUs: https://wiki.ubuntu.com/CategoryDesktopTestPlans
raof, thank you for your work on the script! The output matches what I expect.
2 Likes
Having a place to keep test plans would be great - thanks @seb128 and @jbicha!
Just one note - we (SRU team) can definitely consider them case by case per SRU. But it would be better if each could be reviewed approved by the SRU team, if we could note that somewhere such that it is invalidated if there are significant changes. Then we could provide better consistency since if a test plan is already “SRU team approved” it won’t need to be reviewed again, and you wouldn’t be surprised with requests for changes or amendments if a different SRU team member reviews your upload. But then this needs to be done in a way that a future SRU team member can confirm that the test plan is the same one that was previously approved by a previous SRU team member.
I don’t mind what mechanism we use, but could we arrange that please? Since there will be a collection, perhaps we could review the first time a particular test plan is used, and then leave some kind of approval note?
I added a section to the bottom of https://wiki.ubuntu.com/DesktopTeam/TestPlans/gjs with an idea for how pages could be marked as reviewed. But it’s a decision for the SRU Team how those pages should be marked and reviewed.
2 Likes