I stumbled upon a jobb listing recently (Director of Chaos Engineering), which made me have a look at the QA work at Canonical. I have many years of experience in both programming (been programming since the age of seven), open source (involved since 2004) and quality assurance (8 years at developing aeronautical software in different capacities).
So, this topic is both a bit of a presentation myself (don’t worry, if you read this far you’ve already made it past that bit) and some thoughts on how QA can be improved in general in the open source community.
After reading through all topics here, I have noticed a few things. First of all, the issue here is the same as in any other distribution, i.e., dependency on upstream packages. Secondly, it is hard to find information, e.g., the information is here, on the wiki, on launchpad, etc. but it is difficult to find (requires lots of reading and scrolling and following links).
This first issue is tricky to solve and will probably take years of work before it is thoroughly resolved, but there are a few steps that would be a great starting point:
- Creating easy-to-use QA processes for any open source project and promoting it everywhere, especially upstream.
- Visualise QA in open source, with badges, certifications or the like.
- Talk about QA in a positive sense. Often QA is seen as a road block (especially by managers) to deliveries, but in reality it reduces time spent on fixing bugs. Start talking about QA as a positive thing will help!
- Automate and visualise tests of every single package that are currently in the Ubuntu deliveries. Just a huge colormap (with like 30.000 points) for instance would be super intuitive. Each color would define the current state, e.g., green = passing, red = failed, purple = no tests available, etc. Clicking on an item in the map should link directly to the upstream development repository/website.
The second issue is (perhaps) easier to remedy:
- Visualise how the ubuntu dependency tree look on a general level and point people to it. This will help when discussing where bugs should be fixed.
- Use more modern tools. I totally agree with @merlijn-sebrechts about making it fun (and easy to use). That will make QA, bug fixing, etc. much more attractive to the community!
I have a tonne of more specific ideas too about QA in open source, like automation, mutational testing, requirement- and test-driven development, AI-based test scripting, etc. but maybe that’s enough for now!
What do you guys think?