The seemingly doubled gnome-bluetooth entry
I asked the Debian tracker maintainer to use the Debian package names instead of upstream’s names and he already made that update for the 42 tracker.
So I updated my diff from comment 42 now too.
3 Likes
That looks like a good set to me. It includes some things that I don’t think we particularly care about (particularly gnome-console), but GNOME obviously cares about them and exercising the MRE exception for a package is an indication that we do care about it 
.
Having a diff over time would be nice, but I don’t think that’s a blocker.
I think this is a good methodology and list to use for the MRE documentation.
I think the next step will be to update the MRE wiki page to point to the Debian tracker and add the extra documented exceptions?
I’ve now updated the GNOME MRE wiki page. Please check that what I’ve recorded matches your understanding of the consensus.
Thanks @raof! This looks good to me, though there are still some steps that I think we need to do to complete this. Maybe for the desktop team?
-
I’d like to be able to quickly determine if a particular package is part of the list or not, in order to actually process these SRUs. Right now that seems tricky.
-
Changes to the list expansion are expected and I assume a single member of the SRU team will decide if they make sense or not as MRE SRUs are reviewed. However this isn’t possible without being able to see the full expansion that was originally approved (ie. as it stands right now) and how it has changed. This is what I mean when I say I want an audit trail. Otherwise, we’d be in the odd position where upstream can effectively change our list without us noticing.
-
Right now it looks like specific test plans are being proposed individually in SRU bugs of MRE uploads. I suggest these are moved (following SRU team approval) to the wiki page. That way they wouldn’t have to be reviewed every time, which should reduce bikeshedding, save some copying and pasting and and make the review process smoother.
For (1) and (2) - maybe what we want is to our own script to parse the upstream metadata and map to source package names, and then commit the output (per release) to ubuntu-archive-tools? We don’t expect the list to change within a release, only between releases.
I guess the stretch goal here is for sru-review to grow support for emitting “This package falls under the $PACKAGE MRE; see $URL for details” on the relevant packages…
I’ve now done the script to parse the upstream metadata and output a list of packages covered by the GNOME MRE, and committed the results (16.04, 18.04, 20.04, and 22.04).
Please give both the script and the lists a once-over; I’m happy with what it generates, but may well have missed something.
Integrating a check for when a package is on this list (or other MRE lists) into the SRU tooling is still a stretch goal 
Update update: I’ve generated the list for 22.10. This has required small updates to the script (to handle newly-duplicated packages)
2 Likes
Do we have a diff from previous one? /me lazy
EDIT, here it is:
--- jammy 2022-11-01 17:26:19.000000000 +0100
+++ kinetic 2022-11-01 17:25:42.000000000 +0100
@@ -1,8 +1,6 @@
adwaita-icon-theme
aisleriot
-at-spi2-atk
at-spi2-core
-atk1.0
atkmm1.6
baobab
cheese
@@ -31,7 +29,6 @@
gedit
geocode-glib
gexiv2
-gfbgraph
gjs
glib-networking
glib2.0
@@ -40,7 +37,6 @@
gnome-2048
gnome-autoar
gnome-backgrounds
-gnome-bluetooth
gnome-bluetooth3
gnome-boxes
gnome-builder
@@ -112,7 +108,6 @@
json-glib
jsonrpc-glib
libadwaita-1
-libchamplain
libdazzle
libgdata
libgee-0.8
@@ -126,10 +121,12 @@
libmediaart
libnma
libnotify
+libpanel
libpeas
librest
librsvg
libsecret
+libshumate
libsigc++-2.0
libsoup2.4
libsoup3
@@ -150,12 +147,13 @@
swell-foop
sysprof
tali
+template-glib
totem
totem-pl-parser
tracker
tracker-miners
vte
+xdg-desktop-portal-gnome
yelp
yelp-tools
yelp-xsl
-zenity
3 Likes
Thanks for working on moving things forward Chris, we will review the script and the list.
On the SRU verification would it work for the SRU team if the instructions are stored in https://wiki.ubuntu.com/DesktopTeam/TestPlans/ (where ‘sourcename’ is the name of the Ubuntu source package formatted wikistyle, GnomeTextEditor for gnome-text-editor is an existing example).
We would create the pages from now on as we SRU packages which don’t have one yet.
1 Like
I created a category for our current test cases. We commit to keep adding new ones as we prepare SRUs: https://wiki.ubuntu.com/CategoryDesktopTestPlans
raof, thank you for your work on the script! The output matches what I expect.
2 Likes
Having a place to keep test plans would be great - thanks @seb128 and @jbicha!
Just one note - we (SRU team) can definitely consider them case by case per SRU. But it would be better if each could be reviewed approved by the SRU team, if we could note that somewhere such that it is invalidated if there are significant changes. Then we could provide better consistency since if a test plan is already “SRU team approved” it won’t need to be reviewed again, and you wouldn’t be surprised with requests for changes or amendments if a different SRU team member reviews your upload. But then this needs to be done in a way that a future SRU team member can confirm that the test plan is the same one that was previously approved by a previous SRU team member.
I don’t mind what mechanism we use, but could we arrange that please? Since there will be a collection, perhaps we could review the first time a particular test plan is used, and then leave some kind of approval note?
I added a section to the bottom of https://wiki.ubuntu.com/DesktopTeam/TestPlans/gjs with an idea for how pages could be marked as reviewed. But it’s a decision for the SRU Team how those pages should be marked and reviewed.
2 Likes
What else is needed to complete this discussion?
I’ve refreshed the MRE list for lunar here. In doing so, the script needed additional handling of the libgweather->libgweather4 source transition, and I realised that the previous kinetic list included libgweather, which is not actually in the kinetic archives.
So, with that script refresh, the two diffs are:
--- a/kinetic
+++ b/kinetic
@@ -115,7 +115,7 @@ libgnomekbd
libgom
libgsf
libgtop2
-libgweather
+libgweather4
libgxps
libhandy
libmediaart
--- kinetic 2023-05-24 11:36:05.447903500 +1000
+++ lunar 2023-05-24 11:28:12.252708977 +1000
@@ -109,7 +109,6 @@
jsonrpc-glib
libadwaita-1
libdazzle
-libgdata
libgee-0.8
libgnomekbd
libgom
Also, I’m still on the hook for some SRU tooling to automatically detect when an SRU is for a package with an MRE.
Hm. I was going to say that I think we’ve not yet satisfied Robbie’s “we can tell that the test plan is the same as the approved one” requirement, but on second thoughts, test plans on the Wiki satisfy that (in the same way that all the other test plans on the wiki do) - if the last edited tags match the review time and reviewer.
So, I think what is needed to complete the discussion here is for us, on the SRU team, to go through the test plans as we process GNOME MRE SRUs and approve them or request changes until we can approve them.
To get that started: a review of the Mutter test plan:
In general, I’d like something more structured than “verify things continue to work well”
As a Wayland compositor, Mutter is particularly hardware-dependent and a critical piece of infrastructure. I think we should call out specific testing on whatever set of platforms we think are appropriate (especially since around half the bugs on this particular mutter SRU are explicitly "mutter does $BAD_THING on $SPECIFIC_HARDWARE).
- As a first cut of hardware we explicitly want to support, I suggest testing on:
- A mesa driver (if we’re only doing one, i915 is probably the right choice; if we can do two, both i915 and amdgpu)
- The most recent stable NVIDIA driver series (bonus points: most recent and oldest)
- RPi? Definitely for the current SRU, as it contains an explicit fix for RPi behaviour; I’m not clear exactly how much the desktop team supports RPi-on-the-desktop, though.
The current SRU should probably have explicit testing on multi-monitor setups. I’d be inclined to request it as a part of the generic testing plan, too.
What other aspects of Mutter are high risk, or low cost to test? Scaling factors != 1.0 & 2.0? Mixed scaling? Video playback? Are gnome-shell extensions tied into Mutter, or is that just gnome-shell/gjs?
It should be fairly quick to exclude any really serious breakage on any given platform - just log in, bring up the spread, run a video player, run glmark2-es2-wayland (on Wayland sessions) should verify that a given platform hasn’t been critically regressed. I would therefore expect the major problem in testing to be access to hardware; what does the Desktop team have access to?
1 Like
And a review of the GNOME Shell test plan.
This looks pretty good to me. Again, I’d prefer something more concrete than “verify that things continue to work well” for Test Case 1, but I understand that it’s pretty easy to test and maybe low-value to precisely specify, so I’m OK with that.
It wasn’t immediately obvious to me where the list of extensions was for Test Case 2; I’ve edited the wiki page to make this more obvious (to me, at least!).
1 Like
In reviewing a gjs SRU just now, I noticed that it didn’t link to the test plan @jbicha wrote up above and nor was that linked from the main GNOME exception documentation. So I’ve adjusted these.
I’ve also written up my idea of how we’ve agreed test plans and approvals will work. @jbicha please could you review?
On the gjs test plan itself, it looks fine to me but I’m not particularly familiar with the area. @raof please could you review it?
One further thought.
Right now, gjs has 1.76.2 in lunar-proposed, and I’m accepting 1.72.4 into jammy-proposed.
Which one is newer? Could, for example, 1.72.4 contain fixes for bugs that are only fixed in a 1.76.3 that is not yet prepared for Lunar? If so, then we might be introducing fixes into Jammy that users will then face a regression for if they upgrade to Lunar, which is something we usually try to avoid by asking that later stable releases be fixed first (or at the same time).
This may or may not be the case specifically right now, but generally, is this kind of situation something we should plan against in how we process these SRUs?
For the gjs test plan, it looks broadly sane.
Again, I’d be a fan of a test plan with more structure than “make sure GNOME Shell still works correctly” - for example, is it worth checking the journal to identify any new warnings generated to check that they’re harmless?
Would it additionally be worth checking that the supported Shell extensions still work? They obviously have the potential to be affected by any change in the JS environment, but it’s not clear to me how likely they are to hit things that the default Shell doesn’t, so I don’t know if testing them is worthwhile.
FWIW I’ve also marked the GNOME Shell test plan as approved. I still think the Mutter test plan should call out some explicit hardware testing, and there are still some open questions there.