Checkbox 4.4.0 stable release

Project Discussion Checkbox
1

Hello all!

A new version of Checkbox is now available as a Debian package (in the Stable PPA ) and as Snaps (in the stable channel)!

This release packs a bunch of quality-of-life improvements and expands testing coverage across the board – from deeper CUDA and GPU validation (including ARM64 support!) to new tests for OP-TEE, IOMMU, and even wake-on-LAN. We’ve also streamlined several shell scripts with Python equivalents and added more detailed reporting, like showing boot IDs during reboot/suspend checks and exact commands run by stress-ng. Plus, there’s a shiny new full desktop plan for CE-OEM and a bit of extra validation for AMD PMF drivers.

Reminder: if you update Checkbox you will not be able to resume testing session that you begun before updating. If this is undesirable to you, consider pinning Checkbox and finishing all sessions that are ongoing with the following command:

# for snaps:
$ sudo snap refresh --hold=forever checkbox checkbox22
# for debs:
$ sudo apt-mark hold checkbox-ng checkbox-provider-resource checkbox-provider-certification-client checkbox-provider-base

Remember to eventually run the following to re-enable automatic updates:

# for snaps
$ sudo snap refresh --unhold checkbox checkbox22
# for debs
$ sudo apt-mark unhold checkbox-ng checkbox-provider-resource checkbox-provider-certification-client checkbox-provider-base

New in this release

  • Much needed qol for manage.py
  • Add extensive cuda testing for more machines
  • Add new stock report to export submission.json
  • Moved release_info to checkbox support
  • Show the current boot id at the beginning of reboot checks and suspend checks
  • Add new collectors and remove fetch-on-checkpoint mechanism
  • Add optee tests for optee os 4.x
  • Automate wake-on-lan tests
  • Simplify enabling intel gpu for dss
  • [ce-oem] add ce-oem full desktop plan
  • Replace check_cuda.sh with simple in-line commands
  • Replace check_dss.sh with a simple wrapper and add dss/purge
  • Add ipmitool as a suggests dependency of base
  • Update canonical-certification-precheck instructions for gpgpu
  • Add arm64 to gpgpu provider
  • Run ethernet/ping_interface unless hidden manifest set (e.g. lab runs)
  • Bin/cpuid.py add additional arl cpuid string resolves #1752
  • Add readme file for cases and environ variable for ce-oem-provider
  • Replace check_notebook.sh with appropriate python script
  • Add amd pmf driver loading validation
  • [ce-oem] implement video codec test encoder psnr scenario for carmel project
  • Proviers/base: add requires for bluetooth4/beacon_eddystone_url_*
  • Added tests for iommu config
  • Add the keyboard mouse random executable
  • Add ‘json’ as a new special formatter for list
  • Stress-ng-test.py provide the exact command executed for each stressor

Notable Bugfixes

  • Default value is “" not ["”]
  • Use text mode to capture subprocess outputs to avoid buffering issues
  • Implement textual fallback parser if json fails to read
  • Check different ishtp modules according to the release
  • Revised serial test function call for rpmsg-tty test
  • Add optee manifest to optee test cases
  • Fix edid test case that can not find edid on a+n platforms (oex86-680)
  • Ignore nmcli failing to rescan if it is already rescanning
  • Remove libsystemd dependency
  • Fix output to match the script
  • Fix daily builds workflow syntax
  • Exceptions are not iteratable
  • Patched missing popen
  • Unify the cold boot stress method with onetime cold boot
  • Check kernel version is the same when looking for iwlwifi microcode errors
  • Turn dangerous_grade_core_image into a hidden manifest
  • Fixed usb tests for xenial snap
  • Fixed bug with timeout
  • Restore netplan config even when exception is raised
  • Log to error logger all problems
  • Add has_socket_can_fd manifest for all can fd test cases
  • Fix ntfs sd card mount failure by adding sync and udevadm settle
  • Add lunar lake id for thermal test
  • Fix jpeg validation to accept a broader range of jpeg files
  • Add suspend stats related check
  • Out-of-band management network interfaces should be considered cable interfaces
  • Update canonical-certiciation-precheck for new usb device rules
  • [ce-oem] give user the dbus permission when running at remote mode
  • Os release should always provide codename
  • Various - update platform constraints on some jobs to not run on either power arch (ppc64le and ppc64el)
  • Removed custom implementation and using cv2
  • Fix kernel config bug
  • [checkbox-ce-oem] add default value for rs485-settings in class serial init
  • Fix race condition in reboot_check_test’s renderer test and add lsusb/lspci timeout
  • Fix os resource script
  • Fix manifest always no
  • Eth hotplug check cable and routable before pinging
  • Fix slugify when str is empty/none
  • Make sure os resource job gets the proper version on devices running ubuntu core
  • Use process tree to detect if the subprocess child is gone
  • Update cuda-samples snap arguments
  • Slugify helper function should replace hyphens
  • Prevent the ‘state doesn’t be changed’ issue with nv driver
  • Update snapd_task_timeout for checkbox-ce-oem snap
  • Add --oom-avoid-bytes option to stress-ng tests to prevent frequent oom
  • Fix clear old sessions for local
  • Prettier output for the wifi_test
  • Remove keyboard mouse test program
  • Fix black photo issue on intel mipi cameras
  • Backward compatible hidden check