Hey there,
I’ve been spending some time in the past week to add additional manual testcases for the new Desktop Canary Installer.
I’ve been using the ISO tracker to provide ISO testing feedback but didn’t really contribute back to the testcases before. It was interesting figuring out some of the details so I’m going to share some of my notes here.
First I learnt that even if as an admin I could add and edit testcases through the webUI, those are actually maintained in a Vcs and there is no automatic synchronization between those. Things added to the website will not end up in the Vcs (and might lead to conflicting test IDs), changes to the Vcs need to be manually pushed to the service.
- Contribute testcase updates
So first step to contribute updates is
$ git clone https://git.launchpad.net/ubuntu-manual-tests
Tests are stored are ‘one xml file per test’ in the testcases/ directory, but the modern way to define those is to write a template in definitions/
In my case I created
I based the file on the existing desktop testcases but adapted them since the new installer is slightly different, I also used the opportunity to refresh some of the content.
The format is rather simple. First you define individual ‘test’ entries, then you have ‘case’ elements where you can list steps defined by the tests. It allows writing the description once and reusing it. The challenge is to split the test entries in a way that let you insert specific actions where needed without rewriting the sections.
- Generating the testcases from the template
There is a script (re)-generating the testcases from the definition
$ ./tools/test_case_gen -o testcases/image/ definitions/canary_desktop_cases.xml
$ git status
you can open the generated testcases in your webbrowser (the formatting will be different from the website though)
- Proposing your changes
You can then propose the changes for review
$ git add .
$ git commit
$ git push git+ssh://<your-launchpad-id>@git.launchpad.net/~<your-launchpad-id>/ubuntu-manual-tests
then open the create-merge-proposal url displayed in the git output, type a description and submit
- Uploading the testcases
If you have admin access to the website you can use that script to upload your testcases
$ perl ./tools/qa_tracker_update.pl iso <testcases_filenames>
you need to edit the script to provide you Ubuntu SSO email/password and will be prompted for your 2fa token.
I’ve proposed a merge request to prompt for credential for those who don’t want to record their password in cleartext.
- The next steps
It would be nice if people could go through the new testcases and review for errors in the instructions or details missing.
Improvement to the description or the english used are also welcome.
You can report bugs against https://bugs.launchpad.net/ubuntu-manual-tests/+filebug or contribute directly by proposing a change.
I’ve worked on new testcases for the initial setup and the minimal install option but there is probably more we could add.
Thanks for reading!