Skip to content

Pull Request Verification Workflow

Jump to bottom
Daniel Diblik edited this page Aug 15, 2023 · 9 revisions

Workflow

  1. Open the PR.
  2. Work on the PR (commits, rebases, reviews).
  3. Determine if the PR needs to undergo verification through integration tests.
  4. Apply the respective label.
  5. Await acknowledgment or review of the decision by the QE team.
  6. Merge the PR.

Basic concepts

  • The verification of pull requests (triggering integration/verification tests) is semi-automated through the addition of labels.
    Every push to the PR still (re-)triggers the build job and subsequently the test jobs, however none of the test jobs will be submitted to the Testing Farm by Packit unless the respective label (or comment) is added. Test jobs will stay in Queued state until then.

IMPORTANT:
Re-running the test suite does not need to be retriggered by the labels; it can be manually initiated by commenting.
/packit test --labels tier0 for the tier0 suite
/packit test --labels tier1 for the tier1 suite
/packit test for the whole test suite
/packit retest-failed to re-run only failed checks
  • The presence of at least one label is enforced by the enforce_verification_labels GitHub action.
    Without at least one of the enforced labels, the PR cannot be merged.

NOTE:
Details about the labels, the comments they represent, and their association with the enforce labels action are provided in the table in the Labels section below.

  • The tier0 suite is acknowledged as essential, therefore needs to be run in order to sanity check and verify the pull request.
    In some occasions, the tier1 suite can be also recognized as essential to prevent regressions.
    Even if the tier1 suite does not get triggered during the pull request's lifetime, it is still triggered automatically after the pull request gets merged to the main branch.

  • Generally, chore PRs do not need to undergo verification by the test suite, and this decision does not need to be acknowledged by the QE representative.
    Guidelines on which type of PR should be subjected to test suite verification are covered in the Test Strategy (TBD) document.

Labels

The following table describes all labels recognized by the automation in place, along with their meanings, the corresponding comments added by the bot account to the pull request, and the indication of their mandatory status.

Our Humble Automation Service bot (has-bot) handles the comment addition to ease our lives.

Label Description Comment added to the pull request Enforced
tests-skip This PR does not require integration tests to be run. This PR does not require integration tests to be run. ✓
tests-run-tier0 PR ready to run the essential test suit. /packit test --labels tier0 ✓
tests-run-tier1 Run (optional) tier1 test jobs. /packit test --labels tier1 ✗
tests-run-all Run the full test suite. /packit test ✓

NOTE:
Each comment applied by the automation tags [at]conversions-qe group to notify the representatives of the required action.

Clone this wiki locally