A while ago we (aka the rocks team) took over the maintainership of the Docker Snap. Yes, snap and not a rock, but you have to admit it does kind of make sense for us to work on it too. The snap is not really being developed atm – we’re not looking at adding new features / improvements outside of direct customer requests. This, however, leaves us with plenty of things which need to be worked on and kept up to date – there is a lot of maintenance and… let’s call it meta-features to work on and that’s what I want to write about.
The maintenance part is quite straightforward to be honest. The happy path is, roughly: every month look at what are the most up-to-date versions of all the components, e.g. moby, containerd, runc, nvidia-container-toolkit, etc etc. Check compatibility, apply all version updates, run all the tests which ofc will come back green and run the progressive release of the snap. Basically just a version bump and update.
That was the boring bit, and, at least in this case, ‘boring’ is good! It means things work fine and the version bumps were just fully compatible fixes, patches and improvements. But, in software as in life, just because you can’t smell smoke does not mean there is no fire… Spoilers: there will not be a dramatic critical bug found in prod two paragraphs on! The snap was fine, and is fine and will be fine. Sorry for immediately unloading Chekhov’s gun. 
The point of the smoke analogy is, however, that in order for things to be the good kind of boring, we need to be sure all the procedures are in place and easy to follow, the tests work, cover all use cases, are reliable and easy to run, ideally run automatically, security reports are easy to inspect and the list goes on. These are not the docker-snap features per se so I’m calling them meta-features, or maybe even MX – the Maintainer eXperience, and that is where there is a lot of work to be done on docker snap.
I shan’t spin this into a long and heroic story of dragging you through the details of the implementation of that test or another, especially because a big part of this work is either still in-flight, or not started yet. Instead let me just do a couple (literally) of points of high-level overview (yes, humans like writing lists too, and yes, I also use em-dashes and by-god I’ve used them when the LLMs were still just a glint in the eye of Markov):
(1) I’ve mentioned the nvidia-container-toolkit before. That’s because one of the features of the docker snap is that it integrates well with NVIDIA-enabled hosts and lets the container workloads make use of all the blazingly fast substrate they reside on. GitHub runners don’t have NVIDIA cards, and they are also themselves containers so instead Canonical has a testing centre with a bunch of real machines which we can request to run our tests on. In GitHub CI we then use Testflinger to run our tests on the remote machines provisioned by MAAS and get the results right there in the CI. Neat and now fully enabled due to a recent docker-snap PR. An explicit nod of the hat is warranted towards the Testflinger and MAAS teams here: I’ve watched this CI run when all of a sudden I got notified that the entire backend, maas, proxies and all would be restarted. The job took a bit longer, but finished green and correct! Testflinger backend authed and picked up the job which its predecessor left mid-way, new baby-faced MAAS provisioned a new box and the whole thing just kept going through what was effectively a data centre restart!
(2) When a PR lands in docker-snap, we trigger a bunch of ‘smoke’ tests in CI, but those only cover the architectures which can run on GitHub runners natively (aka amd64 and arm64), and on a couple of releases of Ubuntu, namely 24.04 and 22.04. There is another broader layer of tests which not only tests more arches but also, ubuntu-cloud and ubuntu-core or, at least, that’s what it was supposed to do. Turns out, to no one’s surprise, writing this kind of automated testing (7 architectures, 3 OS flavours and 3 releases thereof) is a little bit tricky, especially because 5/7 of the tests need to be emulated. The previous setup was trying to run all of these tests back to back on one poor GitHub runner which was barely enough to run the amd64 suite, but gave up the moment we wanted to test more arches. The new setup splits this up into multiple jobs per release-flavour-arch. Each uses the excellent image-garden snap to spin up each of the required environments and runs that. This way we get another great property for free: if e.g. core26 tests fail, as they currently have all right to do since core26 is still in development, they don’t take all the other tests down with them. The PR is still in review, but spread tests are triggered manually by a maintainer. One such maintainer (hi 
) could just choose a different gh base branch for the testing workflow trigger, and presto(!); we have working spread tests for everything we could even want to test on (except NVIDIA hardware, but that’s covered above).