Skip to content

Commit

Permalink
Merge pull request 'migrate_to_testing_wiki' (#8) from tcooper/testin…
Browse files Browse the repository at this point in the history 
…g-wiki:migrate_to_testing_wiki into main

Reviewed-on: https://git.resf.org/testing/wiki/pulls/8
Checked the browser pages in development box and the migration has worked correctly.
There are a number of updates required to the migrated files but I view that as a separate issue to be addressed later.
This PR is ready for merge.
  • Loading branch information
Alan Marshall committed Apr 26, 2024
2 parents 700cbb7 + eb7fd1e commit b1cbdac
Show file tree
Hide file tree
Showing 66 changed files with 4,390 additions and 0 deletions.
2 changes: 2 additions & 0 deletions docs/documentation/.pages
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
---
nav:
- ... | index.md
- QA:Test Cases: qa_test_cases.md
- Wiki Development Guides: dev_guides
...
2 changes: 2 additions & 0 deletions docs/documentation/QA/.pages
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
---
hide: true
44 changes: 44 additions & 0 deletions docs/documentation/QA/Testcase_Basic_Graphics_Mode.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: QA:Testcase Basic Graphics Mode
author: Trevor Cooper
revision_date: 2022-06-23
rc:
prod: Rocky Linux
ver: 8
level: Final
---

!!! error "REFERENCED RELEASE CRITERIA IS OVERLY GENERAL AND UNTESTABLE"
The associated release criteria, [Release_Criteria#basic-graphics-mode-behaviors](8_release_criteria.md#basic-graphics-mode-behaviors), for this test case is overly general and **must** be modified to specific enough to be testable.

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria#basic-graphics-mode-behaviors](8_release_criteria.md#basic-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
This test case will verify that release-blocking installers function as intended using the generic video driver option (“basic graphics mode”) on supported systems and classes of hardware.

{% include 'qa_testcase_supported_systems.md' %}

## Setup
1. Obtain access to supported system and hardware class to be installed.
2. Prepare appropriate media for the selected ISO to be tested.
- Example: [QA:Testcase Media USB dd](Testcase_Media_USB_dd.md)

## How to test
1. Boot the system from the prepared optical, USB media or virtual device attachment.
- Examples: [QA:Testcase Boot Methods Boot ISO](Testcase_Boot_Methods_Boot_Iso.md), [QA:Testcase Boot Methods DVD](Testcase_Boot_Methods_Dvd.md)
2. In the boot menu select the appropriate option to boot the installer.
3. In the installer select the appropriate option to intall in basic graphics mode.
4. Proceed with installation on the test system.<br>**Depending on installer choices this MAY destroy all the data on the test system.**

!!! error "DATA LOSS"
If you choose to complete the installation of the test system any/all data on the system may be lost. Please do not install on a system whose contents you need to keep.

## Expected Results
1. Selection of basic graphics mode in the Anaconda installer is possible.
2. Anaconda installer presents a usable graphical intallation environment.
3. System under test can be installed normally.
4. After reboot system boots into graphical environment.
5. After login user is able to operate the graphical environment.

{% include 'qa_testcase_bottom.md' %}
28 changes: 28 additions & 0 deletions docs/documentation/QA/Testcase_Boot_Methods_Boot_Iso.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: QA:Testcase Boot Methods Boot Iso
author: Trevor Cooper
revision_date: 2022-05-18
rc:
prod: Rocky Linux
ver: 8
level: Final
---

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria#initialization-requirements](9_release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso.

## Setup
1. Prepare your system for booting the boot.iso image. This may involve writing the image to a USB key or burning it to an optical disk. Additionally, attaching the boot.iso to a virtual machine instance as a Virtual Optical Disk or mounting the boot.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required.

## How to test
1. Boot the system from the prepared optical, USB media or virtual device attachment.
2. In the boot menu select the appropriate option to boot the installer.

## Expected Results
1. Graphical boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. If no option is selected, the installer should load after a reasonable timeout.
2. System boots into the Anaconda installer.

{% include 'qa_testcase_bottom.md' %}
28 changes: 28 additions & 0 deletions docs/documentation/QA/Testcase_Boot_Methods_Dvd.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: QA:Testcase Boot Methods DVD
author: Trevor Cooper
revision_date: 2022-05-18
rc:
prod: Rocky Linux
ver: 8
level: Final
---

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria#initialization-requirements](9_release_criteria.md#initialization-requirements) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
This is to verify that the Anaconda installer starts correctly when booting from DVD.iso.

## Setup
1. Prepare your system for booting the DVD.iso image. This may involve writing the image to a USB key or burning it to an optical disk of sufficient capacity. Additionally, attaching the DVD.iso to a virtual machine instance as a Virtual Optical Disk or mounting the DVD.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required.

## How to test
1. Boot the system from the prepared optical, USB media or virtual device attachment.
2. In the boot menu select the appropriate option to boot the installer.

## Expected Results
1. Graphical boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. If no option is selected, the installer should load after a reasonable timeout.
2. System boots into the Anaconda installer.

{% include 'qa_testcase_bottom.md' %}
34 changes: 34 additions & 0 deletions docs/documentation/QA/Testcase_Bootloader_Disk_Selection.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: QA:Testcase Bootloader Disk Selection
author: Al Bowles
revision_date: 2022-07-08
rc:
prod: Rocky Linux
ver:
- 8
- 9
level: Final
---

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria#Bootloader Disk Selection](9_release_criteria.md#bootloader-disk-selection) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
This test case verifies that the user is able to select an alternative disk on which to install the bootloader. It also verifies that, if the user is so inclined, they may choose not to install a bootloader at all.

{% include 'qa_data_loss_warning.md' %}

## Setup
{% include 'qa_setup_boot_to_media.md' %}

## How to test
1. In the Installation Destination spoke, select the disk(s) to install to, then click the "Full disk summary and bootl loader..." button at the bottom of the screen: ![Full disk summary and bootloader...](/assets/images/bootloader.png){ loading=lazy }
1. Click the checkbox next to the disk on which the bootloader is desired
1. Alternatively, uncheck the boot checkbox next to all disks to skip bootloader installation
1. Proceed with installation on the test system.

## Expected Results
1. Installation should complete successfully.
1. Note that if no bootloader is installed, the system may not boot after installation is complete. This is expected.

{% include 'qa_testcase_bottom.md' %}
32 changes: 32 additions & 0 deletions docs/documentation/QA/Testcase_Custom_Boot_Methods_Boot_Iso.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: QA:Testcase Custom Boot Methods Boot Iso
author: Trevor Cooper
revision_date: 2022-06-23
rc:
prod: Rocky Linux
ver: 9
level: Final
---

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria#vnc-graphics-mode-behaviors](9_release_criteria.md#vnc-graphics-mode-behaviors) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
This is to verify that the Anaconda installer starts correctly when booting from the Rocky Linux boot.iso using a custom kernel command line.

## Setup
1. Prepare your system for booting the boot.iso image. This may involve writing the image to a USB key or burning it to an optical disk. Additionally, attaching the boot.iso to a virtual machine instance as a Virtual Optical Disk or mounting the boot.iso to server via baseboard management controller virtual media attach should be possible but is not expressly required.

## How to test
1. Boot the system from the prepared optical, USB media or virtual device attachment.
2. In the boot menu select the appropriate option to boot the installer.
3. Interrupt the normal boot and edit the kernel command line.
4. Add appropriate/required options to the kernel command line and resume booting into the installer.
- Example: For network install from an alternate repository add `--inst.url=http://<server>/<path_to_BaseOS_repo>` and (optionally) `--inst.repo=AppStream,http://<server>/<path_to_AppStream_repo>` to the kernel command line.
- Example: For VNC install in **Direct Mode** add `--inst.vnc` to the kernel command line. For VNC install in **Connect Mode** add `--inst.vnc` and `--inst.vncserver=<host>:<port>` to the kernel command line.

## Expected Results
1. Boot menu is displayed for users to select install options. Navigating the menu and selecting entries must work. Editing the boot command line must be possible. If no option is selected, the installer should load after a reasonable timeout.
2. System boots into the Anaconda installer and any command line options specified are utilized.

{% include 'qa_testcase_bottom.md' %}
200 changes: 200 additions & 0 deletions docs/documentation/QA/Testcase_Debranding.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
---
title: QA:Testcase Debranding
author: Trevor Cooper
revision_date: 2022-05-18
rc:
prod: Rocky Linux
ver: 8
level: Final
---

!!! info "Associated release criterion"
This test case is associated with the [Release_Criteria - Debranding](9_release_criteria.md#debranding) release criterion. If you are doing release validation testing, a failure of this test case may be a breach of that release criterion.

## Description
The Rocky Linux [Release Engineering Team](/team/release_engineering/) builds and maintains tools to manage the debranding of packages received from the upstream vendor. They have published a comprehensive [debranding guide](/team/release_engineering/debranding/) and maintain a [list of packages](https://git.rockylinux.org/rocky/metadata/-/blob/main/patch.yml) requiring debranding patches.

This testcase will verify that all packages available on released media that Rocky Linux Release Engineering has identified as requiring debranding are debranded successfully per their specification.

## Setup
1. Obtain access to an environment with the `dnf`, and `koji` commands and access to [Rocky Linux Gitlab](https://git.rockylinux.org) and [Rocky Linux Koji](https://koji.rockylinux.org)
2. Download the ISO to be tested to test machine.
3. Configure `/etc/koji.conf` to access the [Rocky Linux Koji](https://koji.rockylinux.org).
4. Download a recent copy the [patch.yml](https://git.rockylinux.org/rocky/metadata/-/blob/main/patch.yml) from [Rocky Linux Gitlab](https://git.rockylinux.org).

!!! info "patch.yml"
Packages listed in `patch.yml` are names of source RPMs. Binary RPMs containing content produced by building the patched source RPMs need to be validated. The easiest way to get the list of all possible binary RPMs for a particular package and arch is to ask obtain that information in koji.

## How to test
1. Mount the ISO to be tested locally.
- Example:
```
$ mount -o loop Rocky-8.5-x86_64-dvd1.iso /media
```
2. Determine the path(s) to the `repodata` directory(ies) on the ISO.
- Example:
```
$ find /media -name repodata
```
3. For each package to be validated get the names of the `noarch` and `<arch>` specific packages created from it.
- Example:
```
$ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose <package>
$ koji --quiet latest-build --arch=noarch dist-rocky8-compose <package>
```
4. Use `dnf` to obtain the paths to the binary packages requiring debranding.
- Example:
```
$ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \
--repofrompath Minimal,/media/Minimal --repo Minimal \
<binary_package>
```
5. Copy the `<binary_package>` from the media and examine it's metadata and/or contents to determine if it has obviously been patched.
- Example:
```
$ rpm -q --changelog -p <path_to_binary_package> | head | \
grep "Release Engineering <[email protected]>" -C2 | \
grep -Eq "<pattern_to_find>"
$ rpm2cpio <path_to_binary_package> |
cpio --quiet --extract --to-stdout .<file_to_examine> | \
grep -Eq "<pattern_to_find>"
```
!!! info "NOTE"
Note all debranding patches will patch files directly and leave very obvious traces, some patches don't even add changelog messages to use as an indicator that the package has been patched or debranded. Sometimes the only solution is to extract the binary package and examine the contents directly to find something to test.
6. Unmount the ISO.
- Example:
```
$ umount /media
```
## Expected Results
1. Packages tracked by Release Engineering as requiring debranding and published on installation media are, in fact, debranded per their specification.
<h3>Sample Output</h3>
=== "Success"
```
$ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media
mount: /media: WARNING: device write-protected, mounted read-only.
$ find /media -name repodata
/media/BaseOS/repodata
/media/Minimal/repodata
$ curl -LOR https://git.rockylinux.org/rocky/metadata/-/raw/main/patch.yml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3410 100 3410 0 0 20419 0 --:--:-- --:--:-- --:--:-- 20419
$ yq .debrand.all[] patch.yml | column -x -c 100 -o " "
abrt anaconda anaconda-user-help chrony
cloud-init cockpit crash dhcp
dnf firefox fwupd gcc
gcc-toolset-9-gcc gcc-toolset-10-gcc gcc-toolset-11-gcc gcc-toolset-12-gcc
gnome-settings-daemon grub2 httpd initial-setup
kernel kernel-rt libdnf libreoffice
libreport nginx opa-ff opa-fm
openscap pesign PackageKit python-pip
python3 redhat-rpm-config scap-security-guide shim
shim-unsigned-x64 shim-unsigned-aarch64 sos subscription-manager
systemd thunderbird WALinuxAgent
$ ./yq .debrand.r8[] patch.yml | column -x -c 100 -o " "
dotnet3.0 fwupdate gnome-boxes libguestfs pcs plymouth
python2
NOTE: Only a single package will be shown in this Example.
$ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose sos
$ koji --quiet latest-build --arch=noarch dist-rocky8-compose sos
sos-4.1-9.el8_5.rocky.3.noarch
sos-audit-4.1-9.el8_5.rocky.3.noarch
$ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \
--repofrompath Minimal,/media/Minimal --repo Minimal \
sos sos-audit | grep -E "^file"
file:///media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm
file:///media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm
$ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm | \
head | grep "Release Engineering <[email protected]>" -C2
* Mon Oct 18 2021 Release Engineering <[email protected]> - 4.1-5
- Remove Red Hat branding from sos
$ echo $?
0
$ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm | \
head | grep "Release Engineering <[email protected]>" -C2
* Mon Oct 18 2021 Release Engineering <[email protected]> - 4.1-5
- Remove Red Hat branding from sos
$ echo $?
0
$ umount /media
```
=== "Failure"
```
$ sudo mount -o loop Rocky-8.5-aarch64-minimal.iso /media
mount: /media: WARNING: device write-protected, mounted read-only.
$ find /media -name repodata
/media/BaseOS/repodata
/media/Minimal/repodata
$ curl -LOR https://git.rockylinux.org/rocky/metadata/-/raw/main/patch.yml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3410 100 3410 0 0 20419 0 --:--:-- --:--:-- --:--:-- 20419
$ yq .debrand.all[] patch.yml | column -x -c 100 -o " "
abrt anaconda anaconda-user-help chrony
cloud-init cockpit crash dhcp
dnf firefox fwupd gcc
gcc-toolset-9-gcc gcc-toolset-10-gcc gcc-toolset-11-gcc gcc-toolset-12-gcc
gnome-settings-daemon grub2 httpd initial-setup
kernel kernel-rt libdnf libreoffice
libreport nginx opa-ff opa-fm
openscap pesign PackageKit python-pip
python3 redhat-rpm-config scap-security-guide shim
shim-unsigned-x64 shim-unsigned-aarch64 sos subscription-manager
systemd thunderbird WALinuxAgent
$ ./yq .debrand.r8[] patch.yml | column -x -c 100 -o " "
dotnet3.0 fwupdate gnome-boxes libguestfs pcs plymouth
python2
NOTE: Only a single package will be shown in this Example.
$ koji --quiet latest-build --arch=x86_64 dist-rocky8-compose sos
$ koji --quiet latest-build --arch=noarch dist-rocky8-compose sos
sos-4.1-9.el8_5.rocky.3.noarch
sos-audit-4.1-9.el8_5.rocky.3.noarch
$ dnf download --urls --repofrompath BaseOS,/media/BaseOS --repo BaseOS \
--repofrompath Minimal,/media/Minimal --repo Minimal \
sos sos-audit | grep -E "^file"
file:///media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm
file:///media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm
$ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-4.1-5.el8.noarch.rpm | \
head | grep "Release Engineering <[email protected]>" -C2
$ echo $?
1
$ rpm -q --changelog -p /media/BaseOS/Packages/s/sos-audit-4.1-5.el8.noarch.rpm | \
head | grep "Release Engineering <[email protected]>" -C2
$ echo $?
1
$ umount /media
```
{% include 'qa_testcase_bottom.md' %}
Loading

0 comments on commit b1cbdac

Please sign in to comment.