Checkbox 4.2.0 stable release

Project Discussion Checkbox
1

Hello all!

Checkbox v4.2.0 is now available as a Debian package (in the Stable PPA ) and as Snaps (in the stable channel)!

With this release we are starting the work to update our submission format to be more automation friendly and we have created a new feature that is very useful to do partial re-runs of test plans in an automated way: match. We are also proud to finally release in the stable docs our new tutorial on Checkbox test authoring.

Reminder: if you update Checkbox you will not be able to resume testing session that you begun before updating. If this is undesirable to you, consider pinning Checkbox and finishing all sessions that are ongoing with the following command:

# for snaps:
$ sudo snap refresh --hold=forever checkbox checkbox22
# for debs:
$ sudo apt-mark hold checkbox-ng checkbox-provider-resource checkbox-provider-certification-client checkbox-provider-base

Remember to eventually run the following to re-enable automatic updates:

# for snaps
$ sudo snap refresh --unhold checkbox checkbox22
# for debs 
$ sudo apt-mark unhold checkbox-ng checkbox-provider-resource checkbox-provider-certification-client checkbox-provider-base

New in this release

  • New match feature
  • Expose Checkbox version in JSON output (submission.json)
  • Add manifest units to expand output
  • Add image-url test job and get-image-url.sh script
  • Add additional gpu tests
  • Develop the scenario of gst_v4l2_audio_video_synchronization
  • Make all rtcwake cmd can select the RTC_DEVICE_FILE from config file
  • Develop gst_v4l2_video_decoder_performance_fakesink scenario
  • Update possible certification status values for Job units and modify its default
  • Provide a retry decorator to automatically retry Python functions in Checkbox jobs
  • Add info to rejected jobs list in submission.json
  • Add manifest entry for dangerous grade images
  • Refactor DSS provider tests
  • Add UEFI Secure Boot with unsigned kernel case
  • Support both networkd and NetworkManger in netplan
  • Add a new function to find which renderer is running the process

Notable Bugfixes

  • Fix hardcoded strings
  • Remove gtk from camera multiple resolutions
  • Add multi head test for 24.04
  • Fix Unexpected non-empty line
  • Drop the desktop recommand from sru and sru-server
  • Remove manual encryption job from automated UC24 test plan
  • Add an “Initial Stress Tests” category to distinguish tests that are prepared for stress tests
  • EDID test fails at maximum resolution
  • Fixed missing CPUID for cpuid turin
  • change STRESS_S3_ITERATIONS and STRESS_S4_ITERATIONS to 30
  • Improved dkms status handling
  • Simply the testing command and replace ‘and’ with ‘&&’ for suspend_advanced_auto
  • Reduce the length of template-summary
  • Add new steps to the interpolation
  • Enter suspend by ‘systemctl suspend’ in suspend.sh
  • Fix gpgpu setup script
  • Update SPI detect test cases
  • Skip EDID test if there’s no desktop session running
  • Handle HIDRAW devices that have no parents in the udev stack
  • Handle warnings in dkms status
  • Fix grep command for KVM_IMAGE
  • Wrong environment variable in storage watcher
  • Handle no installed dkms modules
  • Update audio playback/capture manifest definitions
  • Fix case typo in 0x906a3 CPU ID
  • Add LED manifest entries for interactive LED checks
  • Fix the insertion of USB3 Gen2x1 device could not being detected
  • fix black format issue
  • Fix race condition timeout decorator
  • Add manifest entries for special key jobs
  • Rebased description to purpose/steps
  • Remove stress/cpu_stress_ng_test from SRU
  • restructure the setup and teardown functions for network.py scripts
  • desktop_session in bootstrap for zapper testplan
  • Add cert-blocker status based on coverage guide for IoT test plans
  • Fixed unicodedecode exception for reading fwts session log
  • Removing manual suspend-key-led-oops-check-cert test plan in odm automated test
  • Fix for Clocktest Failure
  • Fix not seeing debug file after 24.04
  • Sd card not tested
  • Fix wifi nmcli backup/restore /delete on 24.04 Desktop
  • Fixed issue with usb string comparison
  • Ordering of storage tests
  • add depends for suspend/{index}suspend_after_switch_to_card{product_slug}_auto
  • Add parser to parse wifi interface name which is not start with phy
  • Remove hibernate from iot test plan
  • Update gadget interface validation test plan
  • Use the retry decorator in WiFi tests
  • Fix the combination of backup_loc path
  • Switch from mobilebroadband to wwan nested parts for cert test plans
  • Sd card not tested
  • Robot/Keyboard test doesn’t work at first attempt
  • Add the wake on lan manual test to the desktop test plan per customer’s request
  • Using “New USB device found” for all usbs
  • Correct the camera job cmd with right camera tests
  • Fix snap refresh verification job dependency
  • Use wwan-automated nested part in SRU test plan
  • Add cpuid for turin
  • Fix touchscreen tests on ARM64 via remote
  • Fix wifi netplan gateway parser
  • Fix missing subprocess.PIPE in reboot_check_test.py
  • fix dock/all-init-thunderbolt3-storage-manual is not set user to root
  • Fix the parser so that it doesn’t loop forever
  • Add reference to manifest entries in expand subcommand help
  • Save submission url to file
  • Mock os.makedirs in launcher return codes
  • Install AMD GPU setup script in deb package
  • Append to the submission log
  • Changed resizing method from NEAREST to AREA
  • Update the list of rejected jobs if match function is used
  • Ignore common file extensions in ignored files warning
  • Flush stdout in retry functions
  • Executing a first run without ignoring the output on disk tests
  • Add internal microphone manifest and update some audio jobs requirements
  • Fix suspend time integrated gpu
  • Add plugin and template_id to resource and attachment results in submission.json
  • Fix netplan config uc16
  • [checkbox-ceoem] Modify optee test to allow different xtest snap
  • Prevent mandatory jobs from being added to the list of rejected jobs
  • Restore all jobs in the job state map
  • Added the cpuid to detect the Hygon C86-4G 7490 CPU
  • Add manifest gate for obex tests
  • Fixing index & type errors in prime_offload_tester.py
  • Add the missing sort for lsusb in reboot_check_test.py and fix unused device comparison result
1 Like