Welcome again to our series on the autopkgtest overhaul. This week, I want to dive into the functioning of autopkgtest-website, formerly autopkgtest-frontend. The website, available on autopkgtest.ubuntu.com is how most of our users interact with the autopkgtest service. I am happy to report that the design and functionality of the website will not be changing with this overhaul, all changes will be confined to under-the-hood functionality.
So how does the website work? In its current iteration, the website is a flask app being served over CGI using apache. The application plugs into the test queue on one side and swift storage on the other, allowing users to add tests to the queue and view logs and other artifacts. The dispatcher will post surface-level information about a test such as the package being tested, information about the release and architecture of the test, and any triggers to a RabbitMQ exchange. This exchange is then consumed by the website and written to a locally-kept sqlite database which backs all of the tables of results you can see for each package.
In the current iteration of the service, the website sits behind a proxy which will redirect requests to view logs to a separate object storage instance, for example for viewing test logs. Due to new cloud policies regarding external visibility into object storage, this is not possible to carry over to the new iteration. The current plan is to introduce a new page to the website which will embed the requested logs into the website directly, instead of forwarding the user to the object storage endpoint.
In the next cycle, we plan to move the sqlite database from out of the website charm and turn it into a standalone service. This will improve the maintainability of the database, as well as make it easier to expand the service, and move toward having better disaster recovery and high availability. We are also planning to drop the apache layer entirely and have a native flask app. This will allow us to separate the webserver component from the static server side rendered frontend.
So, what is the current progress of the new website charm? I am happy to report that thanks to @paride’s tireless efforts the charm is in a state where it will start the website and make it available for use with almost all features present. Throughout the week I have also gotten the dispatcher ready to process tests across almost all available arches. The next step is to orchestrate all of these services together so the new deployment has a full stack for users to request tests, run them, and view their results. Once the stack is deployed and working, we will make it publicly available under a staging domain for people to stress test and report any crashes for.