From 93b2e024a9bda6579fed0fb98a899f6ca9c5a715 Mon Sep 17 00:00:00 2001 From: Massimiliano Date: Tue, 3 Sep 2024 15:14:40 +0200 Subject: [PATCH] Rebased description to purpose/steps (bugfix) (#1450) * move to _purpose from _description in 6LoWPAN * correct the dock test definitions * fix the acpi tests * fix the benchmarks job definitions * fix the Bluetooth job definitions * fix the camera job definitions * fix the codecs job definitions * fix the cpu job definitions * fix the cpu disk definitions * remove the problematic newline * Apply suggestions from code review Applied Max's suggestions (except one) Co-authored-by: Massimiliano * Apply suggestions from code review Co-authored-by: Massimiliano * Applied suggestion in bluetooth * Apply suggestions from code review Co-authored-by: Pierre Equoy --------- Co-authored-by: Maciej Kisielewski Co-authored-by: Fernando Bravo <39527354+fernando79513@users.noreply.github.com> Co-authored-by: ferbraher Co-authored-by: Pierre Equoy --- providers/base/units/6lowpan/jobs.pxu | 4 +- providers/base/units/acpi/jobs.pxu | 6 +- providers/base/units/benchmarks/jobs.pxu | 57 +-- providers/base/units/bluetooth/jobs.pxu | 186 ++++----- providers/base/units/camera/jobs.pxu | 56 ++- providers/base/units/codecs/jobs.pxu | 25 +- providers/base/units/cpu/jobs.pxu | 23 +- providers/base/units/disk/encryption.pxu | 29 +- providers/base/units/disk/jobs.pxu | 44 +-- providers/base/units/dock/jobs.pxu | 462 ++++++++++++----------- 10 files changed, 458 insertions(+), 434 deletions(-) diff --git a/providers/base/units/6lowpan/jobs.pxu b/providers/base/units/6lowpan/jobs.pxu index 99772bf87..50211c20e 100644 --- a/providers/base/units/6lowpan/jobs.pxu +++ b/providers/base/units/6lowpan/jobs.pxu @@ -7,5 +7,5 @@ command: grep -E "$config=(y|m)" /boot/config-"$(uname -r)" || exit 1 done _summary: kernel config options for 6LoWPAN -_description: - Checks the kernel config options for 6LoWPAN / IEEE802.15.4 support +_purpose: Checks the kernel config options for 6LoWPAN / IEEE802.15.4 support + diff --git a/providers/base/units/acpi/jobs.pxu b/providers/base/units/acpi/jobs.pxu index 9578bc951..7d287ae99 100644 --- a/providers/base/units/acpi/jobs.pxu +++ b/providers/base/units/acpi/jobs.pxu @@ -1,7 +1,7 @@ id: acpi/oem_osi -_summary: test ACPI OEM _OSI strings -_description: - This checks if the depreciated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables +_summary: Test ACPI OEM _OSI strings +_purpose: + This checks if the deprecated OEM _OSI strings are still used by checking the ACPI DSDT and SSDT tables plugin: shell category_id: acpi requires: diff --git a/providers/base/units/benchmarks/jobs.pxu b/providers/base/units/benchmarks/jobs.pxu index 0233576cf..340176028 100644 --- a/providers/base/units/benchmarks/jobs.pxu +++ b/providers/base/units/benchmarks/jobs.pxu @@ -10,7 +10,7 @@ estimated_duration: 15.0 user: root command: hdparm -t /dev/{name} _summary: Raw read timing benchmark of {name} using hdparm -_description: This test runs hdparm timing of device reads as a benchmark for {name} +_purpose: This test runs hdparm timing of device reads as a benchmark for {name} unit: template template-resource: device @@ -24,7 +24,7 @@ estimated_duration: 10.0 user: root command: hdparm -T /dev/{name} _summary: Cached read timing benchmark of {name} using hdparm -_description: This test runs hdparm timing of cache reads as a benchmark for {name} +_purpose: This test runs hdparm timing of cache reads as a benchmark for {name} plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -33,8 +33,8 @@ depends: graphics/xorg-version requires: package.name == 'gtkperf' command: python3 -c 'import re,sys,subprocess; (s, o) = subprocess.getstatusoutput("gtkperf -a"); [sys.exit(1) for i in [s] if s]; m = re.search("Total time:\s+(.*)\n", o); [print(i.group(1)+" Seconds") for i in [m] if m]' estimated_duration: 30.000 -_description: - Run gtkperf to make sure that GTK based test cases work +_purpose: Run gtkperf to make sure that GTK based test cases work +_summary: Ensures that GTK based test cases are operational by running gtkperf. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -42,7 +42,8 @@ id: benchmarks/graphics/render-bench requires: package.name == 'render-bench' command: /usr/bin/time -f "%e Seconds" render_bench > /dev/null 2>&1 estimated_duration: 52.000 -_description: Run Render-Bench XRender/Imlib2 benchmark +_summary: Run Render-Bench XRender/Imlib2 benchmark +_purpose: Execute the Render-Bench XRender/Imlib2 benchmark to assess graphics performance. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -52,7 +53,8 @@ requires: 'arm' in cpuinfo.type command: glmark2-es2 2>&1 | sed -e :a -e '$!N;s/\n/ /;ta' | sed -E 's/.*(Score:\s+[0-9]+).*/\1/' -_description: Run GLmark2-ES2 benchmark +_purpose: Run GLmark2-ES2 benchmark +_summary: Execute the GLmark2-ES2 benchmark on ARM devices. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -63,14 +65,16 @@ requires: command: glmark2 2>&1 | sed -e :a -e '$!N;s/\n/ /;ta' | sed -E 's/.*(Score:\s+[0-9]+).*/\1/' estimated_duration: 306.000 -_description: Run GLmark2 benchmark +_purpose: Run GLmark2 benchmark +_summary: Execute GLmark2 benchmark and extract score. plugin: shell category_id: com.canonical.plainbox::benchmarks id: benchmarks/network/wifi_time_to_reconnect requires: device.category == 'WIRELESS' command: wifi_time2reconnect.py -_description: Check the time needed to reconnect to a WIFI access point +_purpose: Check the time needed to reconnect to a Wi-Fi access point +_summary: Measure Wi-Fi reconnection time. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -78,7 +82,8 @@ id: benchmarks/system/cpu_on_idle estimated_duration: 10.0 requires: package.name == 'sysstat' command: iostat -x -m 1 10 | python3 -c 'import sys, re; lines="".join(sys.stdin.readlines()); l=[float(n) for n in (re.findall("idle\n.*?(\S+)\n", lines))]; print(sum(l)/len(l),"%")' -_description: CPU utilization on an idle system. +_purpose: CPU utilization on an idle system. +_summary: Measure CPU utilization on an idle system. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -86,7 +91,8 @@ id: benchmarks/system/disk_on_idle estimated_duration: 10.0 requires: package.name == 'sysstat' command: iostat -x -m 1 10 | python3 -c 'import sys, re; lines="".join(sys.stdin.readlines()); l=[float(n) for n in (re.findall("util\n.*?(\S+)\n", lines))]; print(sum(l)/len(l),"%")' -_description: Disk utilization on an idle system. +_purpose: Disk utilization on an idle system. +_summary: Measure disk utilization on an idle system. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -96,9 +102,8 @@ requires: cpuinfo.platform == 'x86_64' command: gputest_benchmark.py fur --width 1920 --height 1080 -f estimated_duration: 75.000 -_description: - Run a stress test based on FurMark (OpenGL 2.1 or 3.2) - Fullscreen 1920x1080 no antialiasing +_purpose: Run a stress test based on FurMark (OpenGL 2.1 or 3.2) in fullscreen 1920x1080 no anti-aliasing +_summary: Conduct a FurMark stress test in fullscreen 1920x1080 without anti-aliasing. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -108,9 +113,8 @@ requires: cpuinfo.platform == 'x86_64' command: gputest_benchmark.py fur estimated_duration: 75.000 -_description: - Run a stress test based on FurMark (OpenGL 2.1 or 3.2) - Windowed 1024x640 no antialiasing +_purpose: Run a stress test based on FurMark (OpenGL 2.1 or 3.2) windowed 1024x640 no anti-aliasing +_summary: Conduct a FurMark-based OpenGL stress test at 1024x640 without anti-aliasing. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -120,9 +124,8 @@ requires: cpuinfo.platform == 'x86_64' command: gputest_benchmark.py gi --width 1920 --height 1080 -f estimated_duration: 75.00 -_description: - Run GiMark, a geometry instancing test (OpenGL 3.3) - Fullscreen 1920x1080 no antialiasing +_purpose: Run GiMark, a geometry instancing test (OpenGL 3.3) fullscreen 1920x1080 no anti-aliasing +_summary: Run GiMark OpenGL 3.3 benchmark at 1920x1080 fullscreen without anti-aliasing. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -132,9 +135,9 @@ requires: cpuinfo.platform == 'x86_64' command: gputest_benchmark.py gi estimated_duration: 75.500 -_description: - Run GiMark, a geometry instancing test (OpenGL 3.3) - Windowed 1024x640 no antialiasing +_purpose: + Run GiMark, a geometry instancing test (OpenGL 3.3) in a windowed 1024x640 resolution without anti-aliasing. +_summary: Run GiMark OpenGL 3.3 geometry test at 1024x640 windowed without anti-aliasing. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -144,9 +147,8 @@ requires: cpuinfo.platform == 'x86_64' command: gputest_benchmark.py tess --width 1920 --height 1080 -f estimated_duration: 75.000 -_description: - Run a tessellation test based on TessMark (OpenGL 4.0) - Fullscreen 1920x1080 no antialiasing +_purpose: Run a tessellation test based on TessMark (OpenGL 4.0), fullscreen 1920x1080 with no anti-aliasing. +_summary: Run TessMark tessellation test at 1920x1080 fullscreen without anti-aliasing. plugin: shell category_id: com.canonical.plainbox::benchmarks @@ -155,6 +157,5 @@ requires: package.name == 'gputest' cpuinfo.platform == 'x86_64' command: gputest_benchmark.py tess -_description: - Run a tessellation test based on TessMark (OpenGL 4.0) - Windowed 1024x640 no antialiasing +_purpose: Run a tessellation test based on TessMark (OpenGL 4.0) Windowed 1024x640 no anti-aliasing +_summary: Run TessMark tessellation test at 1024x640 windowed without anti-aliasing. diff --git a/providers/base/units/bluetooth/jobs.pxu b/providers/base/units/bluetooth/jobs.pxu index b553f5960..ce0060723 100644 --- a/providers/base/units/bluetooth/jobs.pxu +++ b/providers/base/units/bluetooth/jobs.pxu @@ -50,29 +50,30 @@ command: echo "BT hardware not available" exit 1 fi -_description: - Automated test to store bluetooth device information in checkbox report +_purpose: Automated test to store Bluetooth device information in the Checkbox report +_summary: Store Bluetooth device information for reports. plugin: manual category_id: com.canonical.plainbox::bluetooth id: bluetooth/browse-files depends: bluetooth/detect-output estimated_duration: 120.0 -_description: - PURPOSE: - This test will check that bluetooth connection works correctly - STEPS: - 1. Enable bluetooth on any mobile device (PDA, smartphone, etc.) - 2. Click on the bluetooth icon in the menu bar - 3. Select 'Setup new device' - 4. Look for the device in the list and select it - 5. In the device write the PIN code automatically chosen by the wizard - 6. The device should pair with the computer - 7. Right-click on the bluetooth icon and select browse files - 8. Authorize the computer to browse the files in the device if needed - 9. You should be able to browse the files - VERIFICATION: - Did all the steps work? +_purpose: + This test will check that the Bluetooth connection works correctly +_steps: + 1. Enable Bluetooth on any mobile device (PDA, smartphone, etc.) + 2. Click on the Bluetooth icon in the menu bar + 3. Select 'Setup new device' + 4. Look for the device in the list and select it + 5. On the device, enter the PIN code automatically chosen by the wizard + 6. The device should pair with the computer + 7. Right-click on the Bluetooth icon and select browse files + 8. Authorize the computer to browse the files on the device if needed + 9. You should be able to browse the files +_verification: + Did all the steps work? +_summary: + Verify the functionality of browsing files over a Bluetooth connection. plugin: manual category_id: com.canonical.plainbox::bluetooth @@ -80,18 +81,19 @@ id: bluetooth/file-transfer depends: bluetooth/detect-output estimated_duration: 120.0 _description: - PURPOSE: - This test will check that you can transfer information through a bluetooth connection - STEPS: - 1. Enable bluetooth on a target device (e.g. another laptop) and DUT (device under test) - 2. Launch bluetooth settings from menu bar - 3. Select target device from Device options for connection establishing - 4. Confirm PIN code on both DUT and target device - 5. Target device is connected - 7. Click target device in device list - 8. Click 'Send Files...' and choose a file to send in DUT - VERIFICATION: - Were file sent to target device ~/Downloads folder correctly? +_purpose: + This test will check that you can transfer information through a bluetooth connection +_steps: + 1. Enable bluetooth on a target device (e.g., another laptop) and DUT (device under test) + 2. Launch bluetooth settings from menu bar + 3. Select target device from Device options for connection establishing + 4. Confirm PIN code on both DUT and target device + 5. Target device is connected + 7. Click target device in device list + 8. Click 'Send Files...' and choose a file to send in DUT +_verification: + Were files sent to the target device's ~/Downloads folder correctly? +_summary: Verify the capability to transfer files through a Bluetooth connection. plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -120,15 +122,17 @@ command: fi exit $EXIT_CODE _description: - PURPOSE: - This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, - to see if you can hear audio from it. - STEPS: - 1. Enable and pair the bluetooth headset - 2. Click "Test" to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, - please select the device and change it manually in the "Sound Settings" - VERIFICATION: - Did you hear the tone? +_purpose: + This test will check the High Fidelity Playback (A2DP) capability of your Bluetooth device, + to see if you can hear audio from it. +_steps: + 1. Enable and pair the Bluetooth headset + 2. Click "Test" to play a brief tone on your Bluetooth device, if it failed to set the Mode to A2DP, + please select the device and change it manually in the "Sound Settings" +_verification: + Did you hear the tone? +_summary: + Verify Bluetooth device's High Fidelity Playback (A2DP) capability. plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -137,20 +141,21 @@ depends: bluetooth/detect-output estimated_duration: 120.0 command: audio_bluetooth_loopback_test.sh _description: - PURPOSE: - This test will check the Telephony Duplex capability of your bluetooth device, - to see if you can record and hear audio from it simultaneously. - STEPS: - 1. Enable the bluetooth headset - 2. Click on the bluetooth icon in the menu bar - 3. Select "Setup new device" - 4. Look for the device in the list and pair it - 5. Click the sound icon - 6. Click "Sound Settings" - 7. Select device and ensure Mode is set to "Telephony Duplex (HSP/HFP)" - 8. Click "Test" to record and play for five seconds in the bluetooth device - VERIFICATION: - Did you hear the sound you recorded in the bluetooth with a slight intended delay? +_purpose: + This test will check the Telephony Duplex capability of your bluetooth device, + to see if you can record and hear audio from it simultaneously. +_steps: + 1. Enable the bluetooth headset + 2. Click on the bluetooth icon in the menu bar + 3. Select "Setup new device" + 4. Look for the device in the list and pair it + 5. Click the sound icon + 6. Click "Sound Settings" + 7. Select device and ensure Mode is set to "Telephony Duplex (HSP/HFP)" + 8. Click "Test" to record and play for five seconds in the bluetooth device +_verification: + Did you hear the sound you recorded in the bluetooth with a slight intended delay? +_summary: Check the Telephony Duplex capability of a Bluetooth device by recording and hearing audio simultaneously. plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -159,17 +164,18 @@ depends: bluetooth/detect-output estimated_duration: 120.0 command: keyboard_test.py _description: - PURPOSE: - This test will check that you can use a BlueTooth HID device - STEPS: +_purpose: + This test will check that you can use a Bluetooth HID device +_steps: 1. Enable either a BT mouse or keyboard - 2. Click on the bluetooth icon in the menu bar + 2. Click on the Bluetooth icon in the menu bar 3. Select 'Setup new device' 4. Look for the device in the list and select it 5. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks 6. For keyboards, commence the test to launch a small tool. Enter some text into the tool and close it. - VERIFICATION: +_verification: Did the device work as expected? +_summary: Verify the functionality of a Bluetooth HID device by conducting actions and confirming expected behaviors. plugin: shell category_id: com.canonical.plainbox::bluetooth @@ -196,11 +202,12 @@ command: sleep 3 fi bluetooth_transfer_stress.sh "$BTDEVADDR" -_description: +_purpose: This is an automated Bluetooth file transfer test. It sends a large file to the device specified by the BTDEVADDR environment variable, then retrieves it again using Bluetooth and verifies the checksum to ensure the transfer didn't corrupt the file. +_summary: Conduct an automated Bluetooth file transfer stress test ensuring file integrity post-transfer. plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -216,11 +223,13 @@ command: bt_connect.py --mouse _purpose: This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart mouse. _steps: - 1. Enable a Bluetooth smart mouse, and put it into paring mode. - 2. Commence the test to do the auto-pairing, you will be asked to select targeting mouse from the list. - 3. After it's paired and connected, perform actions such as moving the pointer, right and left button clicks and double clicks. + 1. Enable a Bluetooth Smart mouse, and put it into pairing mode. + 2. Commence the test to do the auto-pairing; you will be asked to select the targeting mouse from the list. + 3. After it's paired and connected, perform actions such as moving the pointer, right and left button clicks, and double clicks. _verification: Did the Bluetooth Smart mouse work as expected? +_summary: + Test the functionality of a Bluetooth Smart mouse using HID Over GATT Profile. plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -236,11 +245,12 @@ command: bt_connect.py --keyboard _purpose: This test will check that you can use a HID Over GATT Profile (HOGP) with your Bluetooth Smart keyboard. _steps: - 1. Enable a Bluetooth Smart keyboard, and put it into paring mode. - 2. Commence the test to do the auto-pairing, you will be asked to select targeting keyboard from the list. + 1. Enable a Bluetooth Smart keyboard, and put it into pairing mode. + 2. Commence the test to do the auto-pairing, you will be asked to select the targeting keyboard from the list. 3. After it's paired and connected, enter some text with your keyboard. _verification: Did the Bluetooth Smart keyboard work as expected? +_summary: Verify HOGP keyboard functionality with Bluetooth Smart. unit: template template-resource: device @@ -285,7 +295,7 @@ id: bluetooth/bluez-internal-rfcomm-tests_{bluez-internal-rfcomm-test} template-id: bluetooth/bluez-internal-rfcomm-tests_bluez-internal-rfcomm-test category_id: bluetooth_bluez5_selftests _summary: BlueZ-{bluez-internal-rfcomm-test} -_description: +_purpose: Runs a specific test from the rfcomm test suite plugin: shell user: root @@ -336,7 +346,7 @@ id: bluetooth/bluez-internal-uc-tests_{bluez-internal-uc-test} template-id: bluetooth/bluez-internal-uc-tests_bluez-internal-uc-test category_id: bluetooth_bluez5_selftests _summary: BlueZ-{bluez-internal-uc-test} -_description: +_purpose: Runs a specific test from the user channel test suite plugin: shell user: root @@ -353,8 +363,8 @@ id: bluetooth/bluez-internal-bnep-tests_{bluez-internal-bnep-test} template-id: bluetooth/bluez-internal-bnep-tests_bluez-internal-bnep-test category_id: bluetooth_bluez5_selftests _summary: BlueZ-{bluez-internal-bnep-test} -_description: - Runs a specific test from the bnep test suite +_purpose: + Runs a specific test from the bnep test suite. plugin: shell user: root flags: also-after-suspend @@ -383,7 +393,7 @@ imports: from com.canonical.plainbox import manifest requires: manifest.has_bt_adapter == 'True' _summary: Bluetooth keyboard manual test _purpose: - Check bluetooth input device works + Check Bluetooth input device works _steps: 1. Run the following command to start bluetoothctl console: sudo bluetoothctl @@ -395,21 +405,21 @@ _steps: default-agent 4. Put controller in pairable mode: pairable on - 5. Scan nearby bluetooth device: + 5. Scan nearby Bluetooth device: scan on - 6. Stop Scanning after bluetooth keyboard is found: + 6. Stop Scanning after Bluetooth keyboard is found: scan off - 7. Pair bluetooth - pair [MAC address of bluetooth keyboard] - 8. Enter PIN Code on bluetooth keyboard if needed - 9. Trust the bluetooth keyboard - trust [MAC address of bluetooth keyboard] - 10. Connect to bluetooth keyboard: - connect [MAC address of bluetooth keyboard] + 7. Pair Bluetooth + pair [MAC address of Bluetooth keyboard] + 8. Enter PIN Code on Bluetooth keyboard if needed + 9. Trust the Bluetooth keyboard + trust [MAC address of Bluetooth keyboard] + 10. Connect to Bluetooth keyboard: + connect [MAC address of Bluetooth keyboard] 11. Quit bluetoothctl console quit - 12. Use bluetooth keyboard to input - **for headless please check the input outside the bluetooth console by using: + 12. Use Bluetooth keyboard to input + **For headless, please check the input outside the Bluetooth console by using: $ sudo cat /dev/tty1 _verification: Confirm characters from Bluetooth input device are displayed correctly @@ -451,8 +461,7 @@ command: exit 1 flags: also-after-suspend _summary: Bluetooth OBEX send -_description: - This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable +_purpose: This is an automated Bluetooth file transfer test. It sends an image to the device specified by the BTDEVADDR environment variable plugin: user-interact-verify category_id: com.canonical.plainbox::bluetooth @@ -482,7 +491,7 @@ command: pactl set-default-sink "$bt_sink" bt_source=$(pactl list sources short | awk '/bluez_source/{print $2}') if [ "$bt_source" = "" ]; then - echo "Please check your bluetooth support HSP/HFP profile" + echo "Please check your bluetooth supports the HSP/HFP profile" exit 1 fi pactl set-default-source "$bt_source" @@ -492,11 +501,12 @@ command: fi exit $EXIT_CODE _purpose: - This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, - to check if you can recording sounds + This test will check the Headset Head Unit (HSP/HFP) capability of your Bluetooth device, + to check if you can record sounds. _steps: - 1. Enable and pair the bluetooth headset - 2. Click "Test", then speak into your Bluetooth microphone. - 3. After a few seconds, your speech will be played back to you. + 1. Enable and pair the bluetooth headset. + 2. Click "Test", then speak into your Bluetooth microphone. + 3. After a few seconds, your speech will be played back to you. _verification: - Did you hear your speech played back? + Did you hear your speech played back? +_summary: Verify Bluetooth HSP/HFP profile capability for recording and playback. diff --git a/providers/base/units/camera/jobs.pxu b/providers/base/units/camera/jobs.pxu index 0c1c42bf6..279697972 100644 --- a/providers/base/units/camera/jobs.pxu +++ b/providers/base/units/camera/jobs.pxu @@ -36,13 +36,12 @@ estimated_duration: 120.0 depends: camera/detect command: camera_test.py display -d /dev/{name} -_description: - PURPOSE: - This test will check that the {product_slug} camera works - STEPS: - 1. Click on Test to display a video capture from the camera for ten seconds. - VERIFICATION: - Did you see the video capture? +_purpose: + This test will check that the {product_slug} camera works +_steps: + 1. Click on Test to display a video capture from the camera for ten seconds. +_verification: + Did you see the video capture? unit: template template-resource: device @@ -61,12 +60,12 @@ depends: camera/detect command: camera_test.py led -d /dev/{name} _purpose: - This test will check that the {product_slug} camera LED works + This test will check that the {product_slug} camera LED works _steps: - 1. Select Test to activate camera - 2. Camera LED should light for a few seconds + 1. Select Test to activate camera + 2. Camera LED should light for a few seconds _verification: - Did the camera LED light? + Did the camera LED light? flags: also-after-suspend unit: template @@ -83,13 +82,12 @@ estimated_duration: 120.0 depends: camera/detect command: camera_test.py still -d /dev/{name} -_description: - PURPOSE: - This test will check that the {product_slug} works - STEPS: - 1. Click on Test to display a still image from the camera for ten seconds. - VERIFICATION: - Did you see the image? +_purpose: + This test will check that the {product_slug} works +_steps: + 1. Click on Test to display a still image from the camera for ten seconds. +_verification: + Did you see the image? unit: template template-resource: device @@ -105,7 +103,7 @@ estimated_duration: 1.2 depends: camera/detect command: camera_test.py resolutions -d /dev/{name} -o "$PLAINBOX_SESSION_SHARE" -_description: +_purpose: Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format. @@ -136,16 +134,16 @@ category_id: com.canonical.plainbox::camera id: camera/camera-quality_{name} template-id: camera/camera-quality_name flags: also-after-suspend -_summary: Webcam brisque score for {product_slug} +_summary: Webcam BRISQUE score for {product_slug} estimated_duration: 20s depends: camera/detect requires: lsb.release >= '22.04' command: camera_quality_test.py -d {name} -o "$PLAINBOX_SESSION_SHARE" -_description: +_purpose: Takes a picture and computes the quality based on a No-Reference image quality assessment algorithm called BRISQUE. This test will timeout and fail - if the quality has not been compute within 120 seconds. + if the quality has not been computed within 120 seconds. unit: template template-resource: device @@ -179,7 +177,7 @@ depends: camera/detect-rpi requires: cpuinfo.platform == 'armv7l' command: camera_test_rpi.py --device /dev/{name} -_description: +_purpose: Takes multiple pictures based on the resolutions supported by the camera and validates their size and that they are of a valid format. user: root @@ -212,7 +210,7 @@ plugin: shell category_id: com.canonical.plainbox::camera id: camera/roundtrip-qrcode_{{ name }} template-id: camera/roundtrip-qrcode_name -_summary: Test video output and camera {{ name }} by displaying and reading a qrcode +_summary: Test video output and camera {{ name }} by displaying and reading a QR code estimated_duration: 5.0 depends: {%- if category == 'MMAL' %} @@ -231,10 +229,10 @@ requires: {% endif -%} command: roundtrip_qr.py {{ name }} -_description: - Generates a QRcode representing a random string of ascii letters. This is - written to tty1 using ascii escape codes. Either the PiCamera python module or - a gstreamer pipeline is used to capture an image of the display. An attempt - to decode a QRcode in the image is then made and data compared against the +_purpose: + Generates a QR code representing a random string of ASCII letters. This is + written to tty1 using ASCII escape codes. Either the PiCamera python module or + a GStreamer pipeline is used to capture an image of the display. An attempt + to decode a QR code in the image is then made and data compared against the random string. user: root diff --git a/providers/base/units/codecs/jobs.pxu b/providers/base/units/codecs/jobs.pxu index 7ad2d4b6c..f161969d3 100644 --- a/providers/base/units/codecs/jobs.pxu +++ b/providers/base/units/codecs/jobs.pxu @@ -6,14 +6,14 @@ requires: package.name == 'totem' package.name == 'ubuntu-sounds' command: totem /usr/share/sounds/ubuntu/stereo/system-ready.ogg -_description: - PURPOSE: - This test will verify your system's ability to play Ogg Vorbis audio files. - STEPS: - 1. Click Test to play an Ogg Vorbis file (.ogg) - 2. Please close the player to proceed. - VERIFICATION: - Did the sample play correctly? +_purpose: + This test will verify your system's ability to play Ogg Vorbis audio files. +_steps: + 1. Click Test to play an Ogg Vorbis file (.ogg) + 2. Please close the player to proceed. +_verification: + Did the sample play correctly? +_summary: Verify the system's ability to play Ogg Vorbis audio files. plugin: user-interact-verify category_id: com.canonical.plainbox::codecs @@ -23,12 +23,11 @@ requires: package.name == 'totem' package.name == 'alsa-utils' command: totem /usr/share/sounds/alsa/Noise.wav -_description: - PURPOSE: +_purpose: This test will verify your system's ability to play Wave Audio files. - STEPS: +_steps: 1. Select Test to play a Wave Audio format file (.wav) 2. Please close the player to proceed. - VERIFICATION: +_verification: Did the sample play correctly? - +_summary: Verify the system's ability to play Wave Audio files. diff --git a/providers/base/units/cpu/jobs.pxu b/providers/base/units/cpu/jobs.pxu index 5148020eb..ce1d8fbb6 100644 --- a/providers/base/units/cpu/jobs.pxu +++ b/providers/base/units/cpu/jobs.pxu @@ -49,7 +49,7 @@ depends: cpu/scaling_test command: [[ -e "${PLAINBOX_SESSION_SHARE}"/scaling_test.log ]] && cat "${PLAINBOX_SESSION_SHARE}"/scaling_test.log _summary: Attach CPU scaling capabilities log -_description: +_purpose: Attaches the log generated by cpu/scaling_test to the results submission. plugin: attachment @@ -60,7 +60,7 @@ depends: after-suspend-cpu/scaling_test command: [[ -e "${PLAINBOX_SESSION_SHARE}"/scaling_test_after_suspend.log ]] && cat "${PLAINBOX_SESSION_SHARE}"/scaling_test_after_suspend.log _summary: Attach CPU scaling capabilities log -_description: +_purpose: Attaches the log generated by cpu/scaling_test to the results submission. plugin: shell @@ -97,7 +97,7 @@ command: checkbox-support-fwts_test -t maxfreq -l "$PLAINBOX_SESSION_SHARE"/maxfreq_test_after_suspend.log _summary: Test that the CPU can run at its max frequency -_description: +_purpose: Use the Firmware Test Suite (fwts cpufreq) to ensure that the CPU can run at its maximum frequency. @@ -109,7 +109,7 @@ depends: cpu/maxfreq_test command: [ -e "$PLAINBOX_SESSION_SHARE"/maxfreq_test.log ] && cat "$PLAINBOX_SESSION_SHARE"/maxfreq_test.log _summary: Attach CPU max frequency log -_description: +_purpose: Attaches the log generated by cpu/maxfreq_test to the results submission. plugin: attachment @@ -120,7 +120,7 @@ depends: after-suspend-cpu/maxfreq_test command: [ -e "$PLAINBOX_SESSION_SHARE"/maxfreq_test.log ] && cat "$PLAINBOX_SESSION_SHARE"/maxfreq_test.log _summary: Attach CPU max frequency log -_description: +_purpose: Attaches the log generated by cpu/maxfreq_test to the results submission. plugin: shell @@ -131,7 +131,7 @@ estimated_duration: 300.0 command: clocktest _summary: Tests the CPU for clock jitter -_description: +_purpose: Runs a test for clock jitter on SMP machines. plugin: shell @@ -145,7 +145,7 @@ user: root command: cpu_offlining.py _summary: Test offlining of each CPU core -_description: +_purpose: Attempts to offline each core in a multicore system. plugin: shell @@ -167,8 +167,9 @@ id: cpu/frequency_governors estimated_duration: 22.0 user: root command: nice -n -20 frequency_governors_test.py --debug -_description: +_purpose: This test checks that CPU frequency governors are obeyed when set. +_summary: Verify CPU frequency governors compliance. unit: template template-resource: cpuinfo @@ -267,7 +268,7 @@ command: [ -e "${PLAINBOX_SESSION_SHARE}"/fwts_cstates_results_after_suspend.log ] && cat "${PLAINBOX_SESSION_SHARE}"/fwts_cstates_results_after_suspend.log _summary: Attach C-States test log -_description: +_purpose: Attaches the FWTS desktop diagnosis results log to the submission. plugin: shell @@ -292,5 +293,5 @@ command: cpufreq_test.py -q requires: cpuinfo.scaling == 'supported' _summary: cpufreq scaling test -_description: - Comprehensive testing of cpu scaling capabilities and directives via cpufreq. +_purpose: + Comprehensive testing of CPU scaling capabilities and directives via cpufreq. diff --git a/providers/base/units/disk/encryption.pxu b/providers/base/units/disk/encryption.pxu index d4c5df8c3..ed4835890 100644 --- a/providers/base/units/disk/encryption.pxu +++ b/providers/base/units/disk/encryption.pxu @@ -1,4 +1,3 @@ - id: disk/encryption/detect category_id: com.canonical.plainbox::disk plugin: shell @@ -9,7 +8,7 @@ requires: executable.name == 'dmsetup' executable.name == 'cryptsetup' _summary: Test that Full Disk Encryption is in use -_description: +_purpose: Examine the system to detect if one of the standard full disk encryption implementations is in use command: @@ -23,11 +22,6 @@ estimated_duration: 2.0 id: disk/encryption/check-fde-tpm _summary: Disk decryption after TPM change _description: - Check that the data partition cannot be decrypted (and therefore the device - cannot boot) if PCR7 value is modified. -category_id: com.canonical.plainbox::disk -estimated_duration: 45m -plugin: manual _purpose: The device partition is encrypted using TPM master key. To unseal the master key from TPM, PCR7 (Platform Configuration Register 7) needs to be identical @@ -35,16 +29,19 @@ _purpose: device boots, it checks PCR7 to unseal TPM and retrieves master key from TPM to decrypt its data partition. If TPM PCR7 is modified (e.g. by flashing the BIOS), the device won't be able to get the master key and decrypt its data - partition. + partition. This test verifies the system's resilience against unauthorized + modifications by ensuring it cannot boot if the PCR7 value is altered. _steps: - NOTE!!!! YOU'LL HAVE TO RE-INSTALL THE IMAGE AFTER THIS TEST + NOTE: YOU'LL HAVE TO RE-INSTALL THE IMAGE AFTER THIS TEST. 1. Install the image and make sure it boots and you can log in. - 2. Make sure the BIOS is set up properly (e.g. TPM enabled, UEFI boot mode) - 3. Based on your project's situation, choose one of the following methods to clean TPM: - a. Turn the device off and upgrade/downgrade the BIOS or modify Secure Boot state - b. Clean TPM via BIOS menu - c. Install checkbox, execute "checkbox-[project name].checkbox-cli run com.canonical.certification::tpm2.0_3.0.4/tpm2_takeownership" - 4. Start or reboot the device + 2. Ensure the BIOS is set up correctly (e.g., TPM enabled, UEFI boot mode). + 3. Depending on the DUT, choose one of these methods to clean TPM: + a. Turn the device off and upgrade/downgrade the BIOS or modify Secure Boot state. + b. Clean TPM via BIOS menu. + c. Install checkbox, execute "checkbox-[project name].checkbox-cli run com.canonical.certification::tpm2.0_3.0.4/tpm2_takeownership". + 4. Start or reboot the device. _verification: Mark this test as "Passed" if the device cannot boot anymore. - +category_id: com.canonical.plainbox::disk +estimated_duration: 45m +plugin: manual diff --git a/providers/base/units/disk/jobs.pxu b/providers/base/units/disk/jobs.pxu index 61dac7cc2..41c25cf68 100644 --- a/providers/base/units/disk/jobs.pxu +++ b/providers/base/units/disk/jobs.pxu @@ -5,7 +5,7 @@ flags: also-after-suspend requires: executable.name == 'lsblk' _summary: Gathers information about each disk detected -_description: +_purpose: Uses lsblk to gather information about each disk detected on the system under test. command: disk_info.py @@ -33,7 +33,7 @@ user: root estimated_duration: 10.0 command: disk_stats_test.sh {name} _summary: Disk statistics for {product_slug} -_description: +_purpose: This test checks disk stats, generates some activity and rechecks stats to verify they've changed. It also verifies that disks appear in the various files they're supposed to. @@ -57,8 +57,8 @@ requires: block_device.smart == 'True' and block_device.name == '{name}' _summary: Test SMART capabilities for {product_slug} -_description: - This tests the SMART capabilities for {product_slug} (Note that this test may not work against hardware RAID) +_purpose: + This test assesses the SMART capabilities for {product_slug}. (Note that this test may not work against hardware RAID) user: root command: disk_smart.py -b /dev/{name} -s 130 -t 530 @@ -73,7 +73,7 @@ flags: also-after-suspend estimated_duration: 65.0 requires: _summary: Disk performance test for {product_slug} -_description: Verify that disk storage performs at or above baseline performance +_purpose: Verify that disk storage performs at or above baseline performance user: root environ: DISK_READ_PERF DISK_NVME_READ_PERF DISK_MDADM_READ_PERF command: disk_read_performance_test.sh {name} @@ -90,7 +90,7 @@ estimated_duration: 375.0 user: root requires: _summary: Disk I/O stress test for {product_slug} -_description: Take the path of the storage device and test is it a block device +_purpose: Take the path of the storage device and test if it is a block device. command: storage_test.py {name} unit: template @@ -105,7 +105,7 @@ user: root requires: block_device.rotation == 'no' and block_device.name == '{name}' _summary: Filesystem TRIM check for {product_slug} -_description: Take the path of the storage device and test its TRIM capabilities +_purpose: Take the path of the storage device and test its TRIM capabilities command: fstrim_test.py --device-file {name} unit: template @@ -120,15 +120,15 @@ user: root requires: executable.name == 'stress-ng' executable.name == 'uuidgen' -_summary: Disk stress_ng test for {product_slug} -_description: Disk stress_ng test for {product_slug} +_summary: Disk stress-ng test for {product_slug} +_purpose: Disk stress-ng test for {product_slug} command: if [ -n "$STRESS_NG_DISK_TIME" ] then - echo "Found STRESS_NG_DISK_TIME env var, stress_ng disk running time is now: $STRESS_NG_DISK_TIME seconds" + echo "Found STRESS_NG_DISK_TIME env var, stress-ng disk running time is now: $STRESS_NG_DISK_TIME seconds" stress_ng_test.py disk --device {name} --base-time "$STRESS_NG_DISK_TIME" else - echo "STRESS_NG_DISK_TIME env var is not found, stress_ng disk running time is default value" + echo "STRESS_NG_DISK_TIME env var is not found, stress-ng disk running time is default value" stress_ng_test.py disk --device {name} --base-time 240 fi @@ -145,7 +145,7 @@ requires: block_device.state != 'removable' and block_device.name == '{name}' executable.name == 'bc' _summary: Check of CPU load imposed by {product_slug} -_description: Check to ensure CPU load of {product_slug} is not too high +_purpose: Check to ensure CPU load of {product_slug} is not too high command: disk_cpu_load.sh {name} plugin: user-interact @@ -161,16 +161,16 @@ depends: input/accelerometer user: root command: hdd_parking.py -_description: - PURPOSE: - This test checks that a systems drive protection mechanism works properly. - Which also known as free fall sensor checking in terms of laptop PC. - STEPS: - 1. Click on Test - 2. Move the system under test around, ensuring it is raised and lowered at some point. - VERIFICATION: - The verification of this test is automated. Do not change the - automatically selected result. +_purpose: + This test checks that a system's drive protection mechanism (free fall + sensor in laptops) works properly. +_steps: + 1. Click on Test + 2. Move the system under test around, ensuring it is raised and lowered at some point. +_verification: + The verification of this test is automated. Do not change the + automatically selected result. +_summary: Test the system's hard drive protection mechanism. unit: template template-resource: device diff --git a/providers/base/units/dock/jobs.pxu b/providers/base/units/dock/jobs.pxu index 47f2f3ace..8a862790e 100644 --- a/providers/base/units/dock/jobs.pxu +++ b/providers/base/units/dock/jobs.pxu @@ -29,12 +29,12 @@ category_id: dock depends: dock/cold-plug plugin: manual estimated_duration: 30.00 -_summary: Docking test +_summary: Test the power button functionality on docking stations. _purpose: - This test will test the power button on docking station + This test will test the power button on the docking station. _steps: Skip this test if your docking station has no power button. - 1. press the power button on docking station. + 1. Press the power button on the docking station. _verification: Did the power management prompt pop up? @@ -114,14 +114,14 @@ imports: from com.canonical.plainbox import manifest requires: manifest.has_usbc_video == 'True' plugin: manual -_summary: Display connected via DisplayPort using an USB Type-C port +_summary: Display connected via DisplayPort using a USB Type-C port _purpose: This test will check the connection of a screen using a "USB Type-C to DisplayPort" adapter. _steps: 1. Connect a display (if not already connected) to the USB Type-C port on - your system using a "USB Type-C to DisplayPort" adapter - 2. Switch display modes between in your Display Settings, check if it can be - set to mirrored, extended, displayed on external or onboard only + your system using a "USB Type-C to DisplayPort" adapter. + 2. Switch display modes in your Display Settings, check if it can be + set to mirrored, extended, displayed on external or onboard only. _verification: Was the desktop displayed correctly on the screen connected using a "USB Type-C to DisplayPort" adapter in every mode? @@ -200,17 +200,18 @@ _steps: _verification: Was the stretched desktop displayed correctly across all the screens? + id: dock/monitor-triple-head category_id: dock-display depends: dock/cold-plug estimated_duration: 120.0 -_summary: Tripple monitors test while docked +_summary: Triple monitors test while docked plugin: manual _purpose: This test verifies that triple-monitor output works using the dock. You will need three external monitors to perform this test. Depends on the specification of the device to use 4K monitor or FHD monitor. _steps: - Skip this test if your video card or the dock do not support multiple monitors. - 1. If your dock provides more than one monitor outputs, connect three monitors + Skip this test if your video card or the dock does not support multiple monitors. + 1. If your dock provides more than one monitor output, connect three monitors 2. Open the "Displays" tool (open the dash and search for "Displays") 3. Turn off internal monitor 4. Configure your output to provide one desktop across all the monitors @@ -245,7 +246,7 @@ requires: manifest.has_audio_playback == 'True' package.name == 'alsa-base' command: cat /proc/asound/cards -_description: Test to detect audio devices +_purpose: Test to detect audio devices id: dock/audio-playback-hdmi category_id: dock-audio @@ -346,7 +347,7 @@ command: _purpose: Dock USB Type-C HDMI audio interface verification _steps: - 1. Plug an external HDMI device with sound on a USB Type-C port using a "USB Typce-C to HDMI" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) + 1. Plug an external HDMI device with sound to a USB Type-C port using a "USB Type-C to HDMI" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Go to the Sound settings and make sure the correct Output is selected 3. Click the Test button _verification: @@ -380,13 +381,13 @@ command: fi exit $EXIT_CODE _purpose: - Dock USB Type-C Displayport audio interface verification + Dock USB Type-C DisplayPort audio interface verification _steps: - 1. Plug an external Displayport device with sound on a USB Type-C port using a "USB Typce-C to Displayport" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) + 1. Plug an external DisplayPort device with sound into a USB Type-C port using a "USB Type-C to DisplayPort" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Go to the Sound settings and make sure the correct Output is selected 3. Click the Test button _verification: - Did you hear the sound from the Displayport device? + Did you hear the sound from the DisplayPort device? id: dock/audio-playback-thunderbolt3 category_id: dock-audio @@ -506,10 +507,7 @@ estimated_duration: 2.0 _summary: Collect audio-related system information depends: dock/cold-plug command: alsa_info --no-dialog --no-upload --output "${PLAINBOX_SESSION_SHARE}"/alsa_info.log -_description: - Collect audio-related system information. This data can be used to - simulate this computer's audio subsystem and perform more detailed tests - under a controlled environment. +_purpose: Collect audio-related system information. This data can be used to simulate this computer's audio subsystem and perform more detailed tests under a controlled environment. plugin: attachment id: dock/audio-alsa-info-attachment @@ -518,8 +516,7 @@ depends: dock/audio-alsa-info-collect estimated_duration: 1.0 _summary: Attach audio hardware data log to results command: [ -e "${PLAINBOX_SESSION_SHARE}"/alsa_info.log ] && cat "${PLAINBOX_SESSION_SHARE}"/alsa_info.log -_description: - Attaches the audio hardware data collection log to the results. +_purpose: Attaches the audio hardware data collection log to the results. plugin: manual id: dock/audio-external-lineout @@ -528,14 +525,14 @@ depends: dock/cold-plug estimated_duration: 30.0 _summary: Line-out connection test _purpose: - Check that external line-out connection works correctly - (Skip this test if dock does not have a line out connector) + Check that the external line-out connection works correctly + (Skip this test if the dock does not have a line-out connector) _steps: - 1. Insert cable to speakers (with built-in amplifiers) on the dock's line-out port + 1. Insert a cable into speakers (with built-in amplifiers) on the dock's line-out port 2. Open system sound preferences, 'Output' tab, select 'Line-out' on the connector list. Click the Test button - 3. On the system sound preferences, select 'Internal Audio' on the device list and click 'Test Speakers' to check left and right channel + 3. In the system sound preferences, select 'Internal Audio' on the device list and click 'Test Speakers' to check the left and right channel _verification: - 1. Do you hear a sound in the speakers? The internal speakers should *not* be muted automatically + 1. Do you hear sound in the speakers? The internal speakers should *not* be muted automatically 2. Do you hear the sound coming out on the corresponding channel? plugin: user-interact-verify @@ -602,6 +599,7 @@ _verification: Verification is automatic, no action is required. The test times out after 30 seconds (and fails in that case). + id: dock/audio-microphone-plug-detection category_id: dock-audio plugin: user-interact @@ -619,7 +617,7 @@ command: pulse_active_port_change.py sources fi _purpose: - Check that system detects a microphone being plugged in + Check that the system detects a microphone being plugged in (Skip this test if the dock does not have a microphone connector) _steps: 1. Prepare a microphone with a standard 3.5mm jack @@ -680,14 +678,13 @@ _steps: Skip this test if your dock station does not support MAC address passthrough. 1. Find Passthrough MAC Address in BIOS setup 2. Check "Passthrough MAC Address" is enabled - 2. Connect dock to system - 3. Boot to OS - 4. Run "ip a" in terminal to get dock ethernet MAC address + 3. Connect dock to system + 4. Boot to OS + 5. Run "ip a" in terminal to get dock ethernet MAC address _verification: - 1. Is the Passthrough MAC Address the same as MAC address found in step 4? - 2. If BIOS doesn't have Passthrough MAC Address in BIOS setup, a bug should be filed. - 3. If dock ethernet MAC address is not the same as BIOS Passthrough MAC Address, a - bug should be filed. + 1. Is the Passthrough MAC Address the same as the MAC address found in step 5? + 2. If BIOS doesn't have a Passthrough MAC Address option, a bug should be filed. + 3. If the dock ethernet MAC address is not the same as the BIOS Passthrough MAC Address, a bug should be filed. ### USB Tests ### @@ -714,13 +711,13 @@ _summary: USB drive insertion test command: removable_storage_watcher.py insert usb _purpose: This test will check that the system correctly detects the insertion of - a USB storage device plugged on the dock. + a USB storage device plugged into the dock. NOTE: Make sure the USB storage device has a partition before starting the test. _steps: 1. Click "Test" and insert a USB storage device, preferably a HDD, - in one of the dock's port. - Although a USB pen drive may be used it might cause performance + in one of the dock's ports. + Although a USB pen drive may be used, it might cause performance related tests to fail. (Note: this test will time-out after 20 seconds.) 2. Do not unplug the device after the test. @@ -791,8 +788,7 @@ user: root estimated_duration: 45.0 _summary: USB drive storage test command: removable_storage_test.py -s 268400000 usb -_description: - This test is automated and executes after the dock/usb_insert test is run. +_purpose: This test is automated and executes after the dock/usb_insert test is run. plugin: shell id: dock/usb3-storage-automated @@ -802,8 +798,8 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the dock/usb3_insert test is run. +_purpose: + This test is automated and executes after the dock/usb3-insert test is run. ### USB-C tests ### @@ -830,6 +826,7 @@ _steps: _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: Verify system detection of a USB 3 storage device inserted into a USB Type-C connector using a Type-C to Type-A adapter. plugin: user-interact category_id: dock-usb @@ -853,6 +850,7 @@ _steps: _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: Verify system detection of a USB 3 storage device insertion on a dock's USB Type-C port. plugin: user-interact category_id: dock-usb @@ -866,14 +864,15 @@ estimated_duration: 10.0 command: removable_storage_watcher.py -m 500000000 remove usb _purpose: This test will check that the system correctly detects the removal of - a USB 3 storage device connected to a USB Type-C port using a "USB Type-C + a USB 3.0 storage device connected to a USB Type-C port using a "USB Type-C to Type-A" adapter. _steps: - 1. Click "Test" and remove the USB 3 device. - (Note: this test will time-out after 20 seconds.) + 1. Click "Test" and remove the USB 3.0 device. + (Note: this test will time out after 20 seconds.) _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: Check the system's detection of a USB 3.0 storage device removal through a USB Type-C to Type-A adapter. plugin: user-interact category_id: dock-usb @@ -890,10 +889,12 @@ _purpose: a USB 3 storage device connected to a USB Type-C port. _steps: 1. Click "Test" and remove the USB 3 device. - (Note: this test will time-out after 20 seconds.) + (Note: this test will time out after 20 seconds.) _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: + Verify the detection of a USB 3 storage device removal from a USB Type-C port. plugin: shell category_id: dock-usb @@ -906,9 +907,9 @@ depends: dock/usb-c/c-to-a-adapter/insert user: root estimated_duration: 45.0 command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the usb-c/c-to-a-adapter/insert test - is run. +_purpose: + This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run. +_summary: Automated test for USB-C to A adapter with storage functionality. plugin: shell category_id: dock-usb @@ -921,9 +922,9 @@ depends: dock/usb-c/insert user: root estimated_duration: 45.0 command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the usb-c/insert test - is run. +_purpose: + This test is automated and executes after the usb-c/insert test is run. +_summary: Automate storage testing after USB-C/insert test completion. plugin: user-interact category_id: dock-usb @@ -944,11 +945,14 @@ _purpose: _steps: 1. Click "Test" and insert a USB 3 storage device in a USB Type-C port on the dock using a "USB Type-C to Type-A" adapter. - (Note: this test will time-out after 20 seconds.) + (Note: this test will time out after 20 seconds.) 2. Do not unplug the device after the test. _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: + Verify correct detection of USB 3 storage device insertion via a Type-C to + Type-A adapter on the dock after suspend. plugin: user-interact category_id: dock-usb @@ -961,18 +965,19 @@ requires: estimated_duration: 10.0 command: removable_storage_watcher.py -m 500000000 insert usb _purpose: - This test will check that the system correctly detects the insertion of - a USB 3 storage device in a USB Type-C connector on the dock after suspend - NOTE: Make sure the USB storage device has a partition before starting - the test. + This test will check that the system correctly detects the insertion of + a USB 3 storage device in a USB Type-C connector on the dock after suspend. + NOTE: Make sure the USB storage device has a partition before starting + the test. _steps: - 1. Click "Test" and insert a USB 3 storage device in a USB Type-C port - on the dock. - (Note: this test will time-out after 20 seconds.) - 2. Do not unplug the device after the test. + 1. Click "Test" and insert a USB 3 storage device in a USB Type-C port + on the dock. + (Note: this test will time-out after 20 seconds.) + 2. Do not unplug the device after the test. _verification: - The verification of this test is automated. Do not change the - automatically selected result. + The verification of this test is automated. Do not change the + automatically selected result. +_summary: Verify the device detects a USB 3 storage device insertion at a USB Type-C dock port after system suspend. plugin: user-interact category_id: dock-usb @@ -985,15 +990,17 @@ requires: estimated_duration: 10.0 command: removable_storage_watcher.py -m 500000000 remove usb _purpose: - This test will check that the system correctly detects the removal of - a USB 3 storage device connected to a USB Type-C port using a "USB Type-C - to Type-A" adapter after suspend. + This test will check that the system correctly detects the removal of + a USB 3 storage device connected to a USB Type-C port using a "USB Type-C + to Type-A" adapter after suspend. _steps: - 1. Click "Test" and remove the USB 3 device. - (Note: this test will time-out after 20 seconds.) + 1. Click "Test" and remove the USB 3 device. + (Note: this test will time-out after 20 seconds.) _verification: - The verification of this test is automated. Do not change the - automatically selected result. + The verification of this test is automated. Do not change the + automatically selected result. +_summary: + Ensure the system detects USB 3 storage device removal via a "USB Type-C to Type-A" adapter after suspend. plugin: user-interact category_id: dock-usb @@ -1014,6 +1021,7 @@ _steps: _verification: The verification of this test is automated. Do not change the automatically selected result. +_summary: Check for system's correct detection of a USB 3 storage device removal after suspend. plugin: shell category_id: dock-usb @@ -1026,9 +1034,9 @@ depends: dock/usb-c/c-to-a-adapter/insert user: root estimated_duration: 45.0 command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the usb-c/c-to-a-adapter/insert test - is run. +_purpose: + This test is automated and executes after the usb-c/c-to-a-adapter/insert test is run. +_summary: Verify automated execution following the usb-c/c-to-a-adapter/insert test. plugin: shell category_id: dock-usb @@ -1041,9 +1049,8 @@ depends: dock/usb-c/insert user: root estimated_duration: 45.0 command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the usb-c/insert test - is run. +_purpose: This test is automated and executes after the usb-c/insert test is run. +_summary: Perform automated USB-C storage test after usb-c/insert test. ### ACPI ### @@ -1055,11 +1062,12 @@ depends: dock/cold-plug estimated_duration: 1.0 command: if [ "$(upower -i /org/freedesktop/UPower/devices/battery_BAT0 | grep "state" | awk '{print $2}')" = "discharging" ] ; then exit 1 ; fi -_description: +_purpose: This test will make sure the laptop is being charged while connected to the dock. ### Hot Plug ### + id: dock/hot-plug category_id: dock-hotplug plugin: manual @@ -1071,7 +1079,7 @@ _steps: 1. Unplug docking station. 2. Replug in docking station. _verification: - Is the system working with problem (e.g. system hang) after docking station hot plugged in? + Is the system working without problems (e.g., system hang) after docking station is hot-plugged in? plugin: user-interact-verify category_id: dock-hotplug @@ -1080,13 +1088,14 @@ depends: dock/hot-plug command: network_check.py estimated_duration: 10 _description: - PURPOSE: - This test will check wired connection after hotplug - STEPS: - 1. Disable WiFi connection - 2. Click "Test" to verify that it's possible to establish a HTTP connection - VERIFICATION: - Did a notification show and was the connection correctly established? +_purpose: + This test will check wired connection after hotplug +_steps: + 1. Disable WiFi connection + 2. Click "Test" to verify that it's possible to establish a HTTP connection +_verification: + Did a notification show, and was the connection correctly established? +_summary: Verify wired connection functionality after hotplug. plugin: user-interact id: dock/hotplug-usb-insert @@ -1120,13 +1129,13 @@ _summary: USB3 drive insertion test command: removable_storage_watcher.py -m 500000000 insert usb _purpose: This test will check that the system correctly detects the insertion of - a USB 3.0 storage device after dock hot plugged in. + a USB 3.0 storage device after the dock is hot-plugged in. NOTE: Make sure the USB storage device has a partition before starting the test. _steps: 1. Click "Test" and insert a USB 3.0 storage device, preferably a HDD, - in one of the dock's USB 3.0 port. Although a USB 3.0 pen drive may - be used it might cause performance related tests to fail. + in one of the dock's USB 3.0 ports. Although a USB 3.0 pen drive may + be used, it might cause performance-related tests to fail. (Note: this test will time-out after 20 seconds.) 2. Do not unplug the device after the test. _verification: @@ -1141,7 +1150,7 @@ user: root estimated_duration: 45.0 _summary: USB drive storage test command: removable_storage_test.py -s 268400000 usb -_description: +_purpose: This test is automated and executes after the dock/usb_insert test is run. plugin: shell @@ -1152,7 +1161,7 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test is automated and executes after the dock/usb3_insert test is run. plugin: user-interact @@ -1198,7 +1207,7 @@ requires: package.name == 'alsa-base' package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0' package.name in ['pulseaudio-utils', 'pipewire'] -_summary: HDMI audio test +_summary: HDMI audio test via USB Type-C DisplayPort plugin: user-interact-verify flags: also-after-suspend estimated_duration: 30.0 @@ -1216,13 +1225,13 @@ command: fi exit $EXIT_CODE _purpose: - Dock USB Type-C Displayport audio interface verification + Dock USB Type-C DisplayPort audio interface verification _steps: - 1. Plug an external Displayport device with sound on a USB Type-C port using a "USB Typce-C to Displayport" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) + 1. Plug an external DisplayPort device with sound into a USB Type-C port using a "USB Type-C to DisplayPort" adapter on Dock (use only one HDMI/DisplayPort/Thunderbolt interface at a time for this test) 2. Go to the Sound settings and make sure the correct Output is selected 3. Click the Test button _verification: - Did you hear the sound from the Displayport device? + Did you hear the sound from the DisplayPort device? id: dock/hotplug-playback-thumderbolt3 category_id: dock-hotplug @@ -1233,7 +1242,7 @@ requires: package.name == 'alsa-base' package.name == 'gir1.2-gst-plugins-base-0.10' or package.name == 'gir1.2-gst-plugins-base-1.0' package.name in ['pulseaudio-utils', 'pipewire'] -_summary: HDMI audio test +_summary: HDMI audio test for Thunderbolt3 docks plugin: user-interact-verify flags: also-after-suspend estimated_duration: 30.0 @@ -1251,11 +1260,11 @@ command: fi exit $EXIT_CODE _purpose: - Dock Thunderbolt3 audio interface verification + Verify the Dock Thunderbolt3 audio interface _steps: - 1. Plug an external Thunderbolt3 device with sound (Use only one HDMI/DisplayPort/Thunderbolt3 interface at a time for this test) - 2. Go to the Sound settings and make sure the correct Output is selected - 3. Click the Test button + 1. Plug an external Thunderbolt3 device with audio support (Use only one HDMI/DisplayPort/Thunderbolt3 interface at a time for this test). + 2. Go to the Sound settings and ensure the correct Output is selected. + 3. Click the Test button. _verification: Did you hear the sound from the HDMI device? @@ -1329,15 +1338,17 @@ _steps: _verification: Did you hear the sound from the HDMI device? + ### Network ### + plugin: shell id: dock/network-before-suspend category_id: dock-network depends: ethernet/detect estimated_duration: 1.2 _summary: Network status before suspend -_description: Record the current network before suspending. +_purpose: Record the current network before suspending. command: set -o pipefail; gateway_ping_test.py | tee "$PLAINBOX_SESSION_SHARE"/network_before_suspend.txt plugin: shell @@ -1349,11 +1360,9 @@ imports: from com.canonical.plainbox import manifest requires: manifest.has_audio_playback == 'True' package.name == 'alsa-base' -_description: Record mixer settings before suspending. +_purpose: Record mixer settings before suspending. command: audio_settings.py store --file="$PLAINBOX_SESSION_SHARE"/audio_settings_before_suspend - - id: dock/networking-gateway-ping-after-suspend category_id: dock-network plugin: shell @@ -1361,24 +1370,24 @@ depends: suspend/suspend_advanced_auto ethernet/detect command: gateway_ping_test.py estimated_duration: 2.00 _summary: Network gateway ping after suspend -_description: Tests whether the system has a working Internet connection. +_purpose: Tests whether the system has a working Internet connection. plugin: user-interact id: dock/usb-insert-after-suspend category_id: dock-usb depends: suspend/suspend_advanced_auto estimated_duration: 10.0 -_summary: USB drive insertion after resuming +_summary: Verify that USB drive insertion is detected after system resumes from suspend command: removable_storage_watcher.py insert usb _purpose: This test will check that the system correctly detects the insertion of - a USB storage device plugged on the dock after suspend. + a USB storage device plugged into the dock after resuming from suspend. NOTE: Make sure the USB storage device has a partition before starting the test. _steps: 1. Click "Test" and insert a USB storage device, preferably a HDD, - in one of the dock's port. - Although a USB pen drive may be used it might cause performance + in one of the dock's ports. + Although a USB pen drive may be used, it might cause performance related tests to fail. (Note: this test will time-out after 20 seconds.) 2. Do not unplug the device after the test. @@ -1413,10 +1422,10 @@ _purpose: NOTE: Make sure the USB storage device has a partition before starting the test. _steps: - 1. Click "Test" and insert a USB 3.0 storage device, preferably a HDD, - in one of the dock's USB 3.0 port. Although a USB 3.0 pen drive may - be used it might cause performance related tests to fail. - (Note: this test will time-out after 20 seconds.) + 1. Click "Test" and insert a USB 3.0 storage device, preferably an HDD, + in one of the dock's USB 3.0 ports. Although a USB 3.0 pen drive may + be used, it might cause performance-related tests to fail. + (Note: this test will time out after 20 seconds.) 2. Do not unplug the device after the test. _verification: The verification of this test is automated. Do not change the @@ -1456,16 +1465,16 @@ _verification: The verification of this test is automated. Do not change the automatically selected result. + plugin: shell id: dock/usb-storage-automated-after-suspend category_id: dock-usb depends: dock/usb-insert-after-suspend user: root estimated_duration: 45.0 -_summary: USB drive storage test after resuming +_summary: USB drive storage test after resuming from suspend command: removable_storage_test.py -s 268400000 usb -_description: - This test is automated and executes after the dock/usb-insert test is run. +_purpose: This test is automated and executes after the dock/usb-insert test is run. plugin: shell id: dock/usb3-storage-automated-after-suspend @@ -1475,22 +1484,21 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test after resuming command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: - This test is automated and executes after the dock/usb3-insert test is run. +_purpose: This test is automated and executes after the dock/usb3-insert test is run. plugin: user-interact-verify id: dock/usb-HID-after-suspend category_id: dock-usb depends: suspend/suspend_advanced_auto estimated_duration: 20.0 -_summary: USB HID device test after resuming +_summary: USB HID device test after resuming from suspend command: keyboard_test.py _purpose: - This test will check that you can use a USB HID device + This test will check that you can use a USB HID device after resuming from suspend. _steps: - 1. Enable either a USB mouse or keyboard - 2. For mice, perform actions such as moving the pointer, right and left button clicks and double clicks - 3. For keyboards, click the Test button to lauch a small tool. Type some text and close the tool. + 1. Enable either a USB mouse or keyboard. + 2. For mice, perform actions such as moving the pointer, right and left button clicks, and double-clicks. + 3. For keyboards, click the Test button to launch a small tool. Type some text and close the tool. _verification: Did the device work as expected? @@ -1561,13 +1569,13 @@ estimated_duration: 120 plugin: manual _summary: Display connected via DisplayPort using an USB Type-C port after resuming _purpose: - This test will check the connection of a screen using a "USB Type-C to DisplayPort" adapter after resuming. + This test will check the connection of a screen using a "USB Type-C to DisplayPort" adapter after resuming from suspend. _steps: Skip this test if the dock does not have a USB Type-C port. 1. Connect a display (if not already connected) to the USB Type-C port on - your system using a "USB Type-C to DisplayPort" adapter - 2. Switch display modes between in your Display Settings, check if it can be - set to mirrored, extended, displayed on external or onboard only + your system using a "USB Type-C to DisplayPort" adapter. + 2. Switch display modes within your Display Settings, checking if it can be + set to mirrored, extended, displayed on external, or onboard only. _verification: Was the desktop displayed correctly on the screen connected using a "USB Type-C to DisplayPort" adapter in every mode? @@ -1577,15 +1585,15 @@ category_id: dock-display depends: suspend/suspend_advanced_auto estimated_duration: 120 plugin: manual -_summary: Display connected via HDMI using an USB Type-C port after resuming +_summary: Display connected via HDMI using a USB Type-C port after resuming _purpose: This test will check the connection of a screen using a "USB Type-C to HDMI" adapter after resuming. _steps: Skip this test if the dock does not have a USB Type-C port. 1. Connect a display (if not already connected) to the USB Type-C port on - your system using a "USB Type-C to HDMI" adapter - 2. Switch display modes between in your Display Settings, check if it can be - set to mirrored, extended, displayed on external or onboard only + your system using a "USB Type-C to HDMI" adapter. + 2. Switch display modes in your Display Settings, checking if it can be + set to mirrored, extended, displayed on external or onboard only. _verification: Was the desktop displayed correctly on the screen connected using a "USB Type-C to HDMI" adapter in every mode? @@ -1595,15 +1603,15 @@ category_id: dock-display depends: suspend/suspend_advanced_auto estimated_duration: 120 plugin: manual -_summary: Display connected via VGA using an USB Type-C port after resuming +_summary: Display connected via VGA using a USB Type-C port after resuming _purpose: This test will check the connection of a screen using a "USB Type-C to VGA" adapter after resuming. _steps: Skip this test if the dock does not have a USB Type-C port. 1. Connect a display (if not already connected) to the USB Type-C port on - your system using a "USB Type-C to VGA" adapter - 2. Switch display modes between in your Display Settings, check if it can be - set to mirrored, extended, displayed on external or onboard only + your system using a "USB Type-C to VGA" adapter. + 2. Switch display modes within your Display Settings, check if it can be + set to mirrored, extended, or displayed on external or onboard only. _verification: Was the desktop displayed correctly on the screen connected using a "USB Type-C to VGA" adapter in every mode? @@ -1625,16 +1633,17 @@ _steps: _verification: Was the stretched desktop displayed correctly across all the screens? + id: dock/monitor-triple-head-after-suspend category_id: dock-display depends: suspend/suspend_advanced_auto estimated_duration: 120.0 -_summary: Tripple monitors test while docked after suspend +_summary: Triple monitors test while docked after suspend plugin: manual _purpose: This test verifies that triple-monitor output works using the dock after suspend. You will need three external monitors to perform this test. Depends on the specification of the device to use 4K monitor or FHD monitor. _steps: - Skip this test if your video card or the dock do not support multiple monitors. + Skip this test if your video card or the dock does not support multiple monitors. 1. If your dock provides more than one monitor outputs, connect three monitors 2. Open the "Displays" tool (open the dash and search for "Displays") 3. Turn off internal monitor @@ -1682,9 +1691,7 @@ command: else exit 1 fi -_description: - Tests that the systems wireless hardware can connect to a router using WPA - security and the 802.11b/g protocols after the system has been suspended. +_purpose: Tests that the system's wireless hardware can connect to a router using WPA security and the 802.11b/g protocols after the system has been suspended. plugin: shell id: dock/wireless-connection-after-suspend-open-bg @@ -1707,8 +1714,8 @@ command: else exit 1 fi -_description: - Tests that the systems wireless hardware can connect to a router using no +_purpose: + Tests that the system's wireless hardware can connect to a router using no security and the 802.11b/g protocols after the system has been suspended. plugin: shell @@ -1732,9 +1739,7 @@ command: else exit 1 fi -_description: - Tests that the systems wireless hardware can connect to a router using WPA - security and the 802.11n protocol after the system has been suspended. +_purpose: Tests that the system's wireless hardware can connect to a router using WPA security and the 802.11n protocol after the system has been suspended. plugin: shell id: dock/wireless-connection-after-suspend-open-n @@ -1757,8 +1762,8 @@ command: else exit 1 fi -_description: - Tests that the systems wireless hardware can connect to a router using no +_purpose: + Tests that the system's wireless hardware can connect to a router using no security and the 802.11n protocol after the system has been suspended. plugin: shell @@ -1783,9 +1788,8 @@ command: else exit 1 fi -_description: - Tests that the systems wireless hardware can connect to a router using WPA - security and the 802.11ac protocol after the system has been suspended. +_purpose: + Tests that the systems wireless hardware can connect to a router using WPA security and the 802.11ac protocol after the system has been suspended. plugin: shell id: dock/wireless-connection-after-suspend-open-ac @@ -1823,7 +1827,7 @@ requires: manifest.has_audio_playback == 'True' package.name == 'alsa-base' depends: dock/suspend-undock-resume dock/audio-before-suspend -_description: Verify that mixer settings after suspend are the same as before suspend. +_purpose: Verify that mixer settings after suspend are the same as before suspend. command: audio_settings.py store --file="$PLAINBOX_SESSION_SHARE"/audio_settings_after_suspend diff "$PLAINBOX_SESSION_SHARE"/audio_settings_before_suspend "$PLAINBOX_SESSION_SHARE"/audio_settings_after_suspend @@ -1832,7 +1836,7 @@ plugin: shell id: dock/record-playback-after-suspend-undock-resume category_id: dock-audio estimated_duration: 10.0 -_summary: Record playback after resuming +_summary: Record playback after resuming from suspend depends: dock/suspend-undock-resume imports: from com.canonical.plainbox import manifest requires: @@ -1850,8 +1854,8 @@ command: else audio_test.py fi -_description: - This will check to make sure that your audio device works properly after a suspend and resume. This may work fine with speakers and onboard microphone, however, it works best if used with a cable connecting the audio-out jack to the audio-in jack. +_purpose: + This will check to make sure that your audio device works properly after a suspend and resume. This may work fine with speakers and onboard microphone, however, it works best if used with a cable connecting the audio-out jack to the audio-in jack. plugin: user-interact id: dock/usb3-insert-after-suspend-undock-resume @@ -1883,7 +1887,7 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test is automated and executes after the dock/usb3_insert test is run. plugin: user-interact @@ -1957,7 +1961,7 @@ command: removable_storage_watcher.py insert firewire _purpose: This test will check the system can detect the insertion of a FireWire HDD on the dock _steps: - (Skip this test id there is no FireWire connector on the dock) + (Skip this test if there is no FireWire connector on the dock.) 1. Click 'Test' to begin the test. This test will timeout and fail if the insertion has not been detected within 20 seconds. 2. Plug a FireWire HDD into an available FireWire port. @@ -1973,7 +1977,7 @@ _summary: FireWire HDD storage test depends: dock/cold-plug dock/firewire-insert user: root command: removable_storage_test.py -s 268400000 firewire -_description: +_purpose: This is an automated test which performs read/write operations on an attached FireWire HDD @@ -1985,9 +1989,9 @@ _summary: FireWire HDD removal test depends: dock/firewire-insert command: removable_storage_watcher.py remove firewire _purpose: - This test will check the system can detect the removal of a FireWire HDD + This test will check if the system can detect the removal of a FireWire HDD. _steps: - 1. Click 'Test' to begin the test. This test will timeout and fail if + 1. Click 'Test' to begin the test. This test will time out and fail if the removal has not been detected within 20 seconds. 2. Remove the previously attached FireWire HDD from the FireWire port on the dock. @@ -2003,15 +2007,15 @@ _summary: eSATA HDD insertion test depends: dock/cold-plug command: removable_storage_watcher.py insert ata_serial_esata _purpose: - This test will check the system can detect the insertion of an eSATA HDD on the dock + This test will check the system can detect the insertion of an eSATA HDD on the dock. _steps: - (Skip this test if there are no eSATA connectors on the dock) + (Skip this test if there are no eSATA connectors on the dock.) 1. Click 'Test' to begin the test. This test will timeout and fail if the insertion has not been detected within 20 seconds. 2. Plug an eSATA HDD into an available eSATA port on the dock. _verification: The verification of this test is automated. Do not change the automatically - selected result + selected result. plugin: shell id: dock/esata-storage-test @@ -2021,7 +2025,7 @@ _summary: eSATA HDD storage test user: root depends: dock/esata-insert command: removable_storage_test.py -s 268400000 ata_serial_esata -_description: +_purpose: This is an automated test which performs read/write operations on an attached eSATA HDD @@ -2106,9 +2110,9 @@ requires: manifest.has_thunderbolt3 == 'True' flags: also-after-suspend command: checkbox-support-usb_read_write _summary: Thunderbolt3 storage test -_description: +_purpose: This is an automated test which performs read/write operations on an attached - Thunderbolt HDD + Thunderbolt3 HDD plugin: user-interact id: dock/thunderbolt3-remove @@ -2139,18 +2143,18 @@ estimated_duration: 45.0 imports: from com.canonical.plainbox import manifest requires: manifest.has_thunderbolt3 == 'True' command: removable_storage_test.py -s 268400000 scsi -_summary: Daisy-chain testing for Thunderbolt 3 storage and display device +_summary: Daisy-chain testing for Thunderbolt 3 storage and display devices _description: _purpose: This test will check if your system can support daisy-chaining of a storage - and a monitor over Thunderbolt 3 port + and a monitor over Thunderbolt 3 ports. _steps: - 1. Connect your Thunderbolt monitor to your systems + 1. Connect your Thunderbolt monitor to your system. 2. Connect and mount your Thunderbolt HDD to another Thunderbolt 3 port of the - monitor (you can do this with HDD first as well) - 3. Click 'Test' to perform the storage test on the Thunderbolt HDD + monitor (you can do this with the HDD first as well). + 3. Click 'Test' to perform the storage test on the Thunderbolt HDD. _verification: - 1. The verification for storage is automated, please select the result combine + 1. The verification for storage is automated; please select the result combined with the result for the display. 2. Was the desktop displayed correctly on the Thunderbolt-connected screen? @@ -2168,7 +2172,7 @@ _steps: 1. Unplug docking station. 2. Replug in docking station. _verification: - Is the system working with problem (e.g. system hang) after docking station hot plugged in? + Is the system working without a problem (e.g., system hang) after docking station is hot-plugged in? plugin: user-interact-verify category_id: dock-hotplug @@ -2177,31 +2181,32 @@ depends: suspend/suspend_advanced_auto command: network_check.py estimated_duration: 10 _description: - PURPOSE: - This test will check wired connection after hotplug - STEPS: +_purpose: + This test will check the wired connection after hotplug +_steps: 1. Disable WiFi connection - 2. Click "Test" to verify that it's possible to establish a HTTP connection - VERIFICATION: + 2. Click "Test" to verify that it's possible to establish an HTTP connection +_verification: Did a notification show and was the connection correctly established? +_summary: Check the wired connection functionality after hotplug following a system suspend. plugin: user-interact id: dock/hotplug-usb-insert-after-suspend category_id: dock-hotplug depends: dock/hot-plug-after-suspend estimated_duration: 10.0 -_summary: USB drive insertion after resuming +_summary: USB drive insertion after resuming from suspend command: removable_storage_watcher.py insert usb _purpose: This test will check that the system correctly detects the insertion of - a USB storage device plugged on the dock after suspend. + a USB storage device plugged into the dock after suspend. NOTE: Make sure the USB storage device has a partition before starting the test. _steps: - 1. Click "Test" and insert a USB storage device, preferably a HDD, - in one of the dock's port. - Although a USB pen drive may be used it might cause performance - related tests to fail. (Note: this test will time-out after 20 + 1. Click "Test" and insert a USB storage device, preferably an HDD, + in one of the dock's ports. + Although a USB pen drive may be used, it might cause performance- + related tests to fail. (Note: this test will time out after 20 seconds.) 2. Do not unplug the device after the test. _verification: @@ -2230,16 +2235,16 @@ _verification: The verification of this test is automated. Do not change the automatically selected result. + plugin: shell id: dock/hotplug-usb-storage-automated-after-suspend category_id: dock-hotplug depends: dock/hotplug-usb-insert-after-suspend user: root estimated_duration: 45.0 -_summary: USB drive storage test after resuming +_summary: USB drive storage test after resuming from suspend command: removable_storage_test.py -s 268400000 usb -_description: - This test is automated and executes after the dock/usb-insert test is run. +_purpose: This test is automated and executes after the dock/usb-insert test is run. plugin: shell id: dock/hotplug-usb3-storage-automated-after-suspend @@ -2249,7 +2254,7 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test is automated and executes after the dock/usb3_insert test is run. plugin: user-interact @@ -2310,30 +2315,31 @@ depends: dock/suspend-dock-resume command: network_check.py estimated_duration: 10 _description: - PURPOSE: - This test will check wired connection after suspend, undocked, docked, resume. - STEPS: - 1. Disable WiFi connection - 2. Click "Test" to verify that it's possible to establish a HTTP connection - VERIFICATION: - Did a notification show and was the connection correctly established? +_purpose: + This test will check wired connection after suspend, undocked, docked, resume. +_steps: + 1. Disable WiFi connection + 2. Click "Test" to verify that it's possible to establish a HTTP connection +_verification: + Did a notification show and was the connection correctly established? +_summary: Verify wired connection functionality after suspend, undocking, docking, and resuming. plugin: user-interact id: dock/usb-insert-after-suspend-dock-resume category_id: suspend-dock-resume depends: dock/suspend-dock-resume estimated_duration: 10.0 -_summary: USB drive insertion after resuming +_summary: USB drive insertion after resuming from suspend using a dock command: removable_storage_watcher.py insert usb _purpose: This test will check that the system correctly detects the insertion of - a USB storage device plugged on the dock after suspend. + a USB storage device plugged into the dock after suspending. NOTE: Make sure the USB storage device has a partition before starting the test. _steps: - 1. Click "Test" and insert a USB storage device, preferably a HDD, - in one of the dock's port. - Although a USB pen drive may be used it might cause performance + 1. Click "Test" and insert a USB storage device, preferably an HDD, + in one of the dock's ports. + Although a USB pen drive may be used, it might cause performance- related tests to fail. (Note: this test will time-out after 20 seconds.) 2. Do not unplug the device after the test. @@ -2354,9 +2360,9 @@ _purpose: NOTE: Make sure the USB storage device has a partition before starting the test. _steps: - 1. Click "Test" and insert a USB 3.0 storage device, preferably a HDD, - in one of the dock's USB 3.0 port. Although a USB 3.0 pen drive may - be used it might cause performance related tests to fail. + 1. Click "Test" and insert a USB 3.0 storage device, preferably a hard disk drive (HDD), + in one of the dock's USB 3.0 ports. Although a USB 3.0 pen drive may + be used, it might cause performance related tests to fail. (Note: this test will time-out after 20 seconds.) 2. Do not unplug the device after the test. _verification: @@ -2371,8 +2377,7 @@ user: root estimated_duration: 45.0 _summary: USB drive storage test after resuming command: removable_storage_test.py -s 268400000 usb -_description: - This test is automated and executes after the dock/usb-insert test is run. +_purpose: This test is automated and executes after the dock/usb-insert test is run. plugin: shell id: dock/usb3-storage-automated-after-suspend-dock-resume @@ -2382,7 +2387,7 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test is automated and executes after the dock/usb3_insert test is run. plugin: user-interact @@ -2454,25 +2459,26 @@ category_id: dock plugin: shell user: root command: mac_passthrough.py -_description: Check dock MAC address pass-through function, please make sure following items before running this test: - 1. Pass-through mac address is flashed in BIOS - 2. MAC address pass-through is enabled in BIOS settings - 3. Dock is connected +_purpose: Check dock MAC address pass-through function, please make sure the following items before running this test: + 1. Pass-through MAC address is flashed in BIOS. + 2. MAC address pass-through is enabled in BIOS settings. + 3. Dock is connected. +_summary: Verify the functionality of dock MAC address pass-through. id: dock/all-init category_id: dock plugin: manual estimated_duration: 300.00 _purpose: - This test is to setup the initial status for all I/O ports on docking. + This test is to set up the initial status for all I/O ports on the docking station. Note: - Try to meet the limitation in the spec of the dock. - If there's no port as description below, you can just skip it. + Try to meet the limitations in the spec of the dock. + If there's no port as described below, you can just skip it. _steps: 1. If the laptop is currently undocked, dock it now. - 2. Connect a Ethernet cable to the docking station. + 2. Connect an Ethernet cable to the docking station. 3. Connect a DP monitor to the docking station. - 4. Connect a HDMI monitor to the docking station. + 4. Connect an HDMI monitor to the docking station. 5. Connect a Thunderbolt monitor to the docking station. 6. Connect a Thunderbolt3 storage to the docking station. 7. Connect a USB Type-C storage to the docking station. @@ -2480,6 +2486,8 @@ _steps: 9. Connect a headset to the docking station. _verification: Mark it Pass if you have connected all I/O ports as mentioned above. +_summary: + Prepare all I/O ports on the dock by connecting all devices it supports. id: dock/all-reboot category_id: dock @@ -2497,26 +2505,30 @@ category_id: dock plugin: manual estimated_duration: 300.00 _purpose: - This test is to check if the device is still working after hotplug. + This test checks if the device is still working after hotplug. _steps: 1. Unplug the docking station. 2. Wait for 30 seconds. 3. Plug in the docking station. _verification: Mark it Pass if you have hotplugged the docking station. +_summary: + Verify the device functionality after hotplug actions with the docking station. id: dock/all-suspend category_id: dock plugin: manual estimated_duration: 300.00 _purpose: - This test is to check if the device is still working after suspend. + This test is to check if the device is still functional after suspend. _steps: 1. Suspend the system. 2. Wait for 30 seconds. 3. Resume the system. _verification: - Mark it Pass if you have suspended and resumed the system. + Mark it as Pass if you have successfully suspended and resumed the system. +_summary: + Test the system's functionality after suspend and resume. id: dock/all-poweroff category_id: dock @@ -2525,11 +2537,13 @@ estimated_duration: 300.00 _purpose: This test is to check if the device is still working after power-off. _steps: - 1. Poweroff the system. + 1. Power off the system. 2. Wait for 30 seconds. - 3. Poweron the system. + 3. Power on the system. _verification: Mark it Pass if you have powered off and powered on the system. +_summary: + Test device functionality after powering off and on. id: dock/all-init-monitor-multi-head category_id: dock-display @@ -2580,7 +2594,7 @@ _purpose: _steps: 1. Launch the test to play sound on every monitor. _verification: - Does every external monitor can play sound? + Can every external monitor play sound? If no sound output, try: 1. Go to Settings > Sound > Output > Test > Try Front Left and Front Right. 2. Repeat Step 1 for each monitor. @@ -2729,9 +2743,9 @@ _purpose: NOTE: Make sure the USB storage device has a partition before starting the test. _steps: - 1. Click "Test" and insert a USB 3 storage device in a USB Type-C port + 1. Click "Test" and insert a USB 3 storage device into a USB Type-C port on the dock. - (Note: this test will time-out after 20 seconds.) + (Note: this test will time out after 20 seconds.) 2. Do not unplug the device after the test. _verification: The verification of this test is automated. Do not change the @@ -2741,6 +2755,8 @@ _siblings: {"id": "dock/all-hotplug-usb-c/insert"}, {"id": "dock/all-suspend-usb-c/insert"}, {"id": "dock/all-poweroff-usb-c/insert"}] +_summary: + Verify the correct detection of a USB 3 storage device insertion into a USB Type-C port on the dock. id: dock/all-init-usb-c/storage-automated plugin: shell @@ -2753,7 +2769,7 @@ depends: dock/all-init-usb-c/insert user: root estimated_duration: 45.0 command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test executes automatically after the usb-c/insert test is run. _siblings: [{"id": "dock/all-reboot-usb-c/storage-automated", @@ -2764,6 +2780,7 @@ _siblings: "depends": "dock/all-suspend-usb-c/insert"}, {"id": "dock/all-poweroff-usb-c/storage-automated", "depends": "dock/all-poweroff-usb-c/insert"}] +_summary: Automatically test USB-C storage functionality after the usb-c/insert test. id: dock/all-init-usb-c/remove plugin: user-interact @@ -2793,6 +2810,7 @@ _siblings: "depends": "dock/all-suspend-usb-c/insert"}, {"id": "dock/all-poweroff-usb-c/remove", "depends": "dock/all-poweroff-usb-c/insert"}] +_summary: Check the system's detection of USB 3 storage device removal on a USB Type-C port. plugin: user-interact id: dock/all-init-usb3-insert @@ -2828,7 +2846,7 @@ user: root estimated_duration: 45.0 _summary: USB3 drive storage test command: removable_storage_test.py -s 268400000 -m 500000000 usb --driver xhci_hcd -_description: +_purpose: This test is automated and executes after the dock/usb3_insert test is run. _siblings: [{"id": "dock/all-reboot-usb3-storage-automated",