Recent stability improvements
One of our goals for this cycle is to improve Netplan’s code quality and stability. We also want to continuously test our code to keep the quality while we work to add new features.
Below is a list of things we’ve been working on to achieve these goals.
Increasing the compiler warning level
The compiler warning level was increased from 1 to 2 in our meson.build file. With this change, our code is now compiled with the flags -Wall and -Wextra.
Because we treat warnings as errors, this change forced us to fix a few number of issues in our code, such comparisons between signed and unsigned variables, missing members in static struct initializations and other things. PR#380
Testing Netplan’s C code with cmocka
As you might know, the Netplan’s parser and configuration generation code are written in C.
Until recently, this code was tested only through our bindings for Python, which is good enough to test if it works as intended.
Some time ago, we introduced unit tests using the cmocka framework. One of the main wins with cmocka is that we can also check if the code has memory issues such as memory leaks and out of bound memory accesses by compiling it with GCC’s Address Sanitizer. PR#298
Using ASAN to detect memory issues automatically
As already mentioned, during tests, we compile and run our code with GCC’s Address Sanitizer. Apart from the unit tests, we also run the Netplan’s generator against each configuration example YAML in the examples/ directory. Even though we still don’t have 100% of code coverage in this particular test, it’s helping us to catch some issues automatically. PR#321
Static Analysis with Coverity
More recently we started a periodically code static analysis with Coverity, which helped us to identify and fix a number of issues (PR#383).
Coverity is a powerful tool that can detect a big number of issues in C code. It’s available for free for open source projects.
Configuration Fuzzing
We’ve been experimenting with fuzzing Netplan’s parser with random valid YAML configuration. To do that, we use a JSON schema with a fake data generator to create random but still valid Netplan YAML configuration. We then send it through Netplan’s parser and see what happens. For this test, Netplan is compiled with ASAN so any memory issues will cause the generator to crash so we can detect the problem.
While this project is not ready yet, it already helped us to find and fix a few issues such as memory leaks and crashes. We have plans to use this technique to continuously brute force the Netplan’s parser as part of our CI workflows.