Skip to content

Commit

Permalink
Refine Canary test plan (BugFix) (#716)
Browse files Browse the repository at this point in the history
* Refine Canary test plan

Remove a few jobs from the Canary test plan.

Some jobs are meant to be run on desktop images and do not run well (or
at all) on Ubuntu Core:

- the graphics-related ones
- i2c/i2c-device-detect (although it could technically run on a desktop
image, it requires a connected I²C device connected to run properly)

On the other hand, some jobs are meant to run on UC and not on desktop
images:

- ubuntu_core_features
- kernel-snap/booted-kernel-matches-current-grub
- parts_meta_info_attachment

A lot of the snap-related tests make requests to the snap store, and
it's preferable to avoid this to avoid false negative in the case where
the snap store is not responding.

The watchdog tests are meant to be run only on devices where the
watchdog device is enabled and properly setup. Watchdog is not part of
the coverage for laptops and desktops.

This commit also renames the pxu file to be aligned with the convention
used for other units in this provider.

Fix CHECKBOX-837

* Remove location/status job from canary test plan

Remove location job (and its after-suspend sibling) as it relies on a
snap (locationd) that we do not support anymore and that is therefore
not installed. As a result, the jobs will always be skipped.

* Remove disk/storage_device_.* test from canary test plan

Stress test that takes a long time to run and does not bring any benefit
in terms of testing Checkbox proper.
  • Loading branch information
pieqq authored Sep 12, 2023
1 parent 7bdf634 commit fa62fd0
Showing 1 changed file with 1 addition and 22 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
id: canary
unit: test plan
_name: Checkbox release self-test
_name: Checkbox release self-test (canary test plan)
_description:
This test plan is meant to run on Checkbox versions that are candidates for
a release. The jobs included here are explicitly listed so the same jobs are
Expand All @@ -13,7 +13,6 @@ bootstrap_include:
com.canonical.certification::bootloader
com.canonical.certification::cpuinfo
com.canonical.certification::device
com.canonical.certification::graphics_card
com.canonical.certification::interface
com.canonical.certification::net_if_management
com.canonical.certification::model_assertion
Expand All @@ -31,7 +30,6 @@ include:
com.canonical.certification::lsusb_attachment
com.canonical.certification::rtc
com.canonical.certification::sleep
com.canonical.certification::parts_meta_info_attachment
com.canonical.certification::dkms_info_json
com.canonical.certification::udev_json
com.canonical.certification::package
Expand Down Expand Up @@ -71,15 +69,9 @@ include:
com.canonical.certification::cpu/topology
com.canonical.certification::disk/detect
com.canonical.certification::disk/stats_.*
com.canonical.certification::disk/storage_device_.*
com.canonical.certification::ethernet/detect
com.canonical.certification::ethernet/ping_.*
com.canonical.certification::graphics/.*driver_version_.*
com.canonical.certification::i2c/i2c-bus-detect
com.canonical.certification::i2c/i2c-device-detect
com.canonical.certification::ubuntu_core_features
com.canonical.certification::kernel-snap/booted-kernel-matches-current-grub
com.canonical.certification::location/status
com.canonical.certification::memory/info
com.canonical.certification::ipv6_detect
com.canonical.certification::ipv6_link_local_address_.*
Expand All @@ -89,14 +81,6 @@ include:
com.canonical.certification::power-management/cold-reboot
com.canonical.certification::power-management/post-cold-reboot
com.canonical.certification::snappy/snap-list
com.canonical.certification::snappy/snap-search
com.canonical.certification::snappy/snap-install
com.canonical.certification::snappy/snap-refresh-automated
com.canonical.certification::snappy/snap-revert-automated
com.canonical.certification::snappy/snap-reupdate-automated
com.canonical.certification::snappy/snap-remove
com.canonical.certification::snappy/test-store-install-beta
com.canonical.certification::snappy/test-store-install-edge
com.canonical.certification::snappy/test-snap-confinement-mode
com.canonical.certification::socketcan/modprobe_vcan
com.canonical.certification::socketcan/send_packet_local_sff_virtual
Expand All @@ -109,15 +93,10 @@ include:
com.canonical.certification::clevis-encrypt-tpm2/detect-ecc-capabilities
com.canonical.certification::clevis-encrypt-tpm2/ecc
com.canonical.certification::usb/storage-detect
com.canonical.certification::watchdog/detect
com.canonical.certification::watchdog/systemd-config
com.canonical.certification::watchdog/trigger-system-reset-auto
com.canonical.certification::watchdog/post-trigger-system-reset-auto
com.canonical.certification::suspend/suspend_advanced_auto
com.canonical.certification::after-suspend-audio/detect-playback-devices
com.canonical.certification::after-suspend-audio/detect-capture-devices
com.canonical.certification::after-suspend-audio/alsa-loopback-automated
com.canonical.certification::after-suspend-ethernet/detect
com.canonical.certification::after-suspend-ethernet/ping_.*
com.canonical.certification::after-suspend-location/status
com.canonical.certification::after-suspend-usb/storage-detect

0 comments on commit fa62fd0

Please sign in to comment.