Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable running the test-operator role multiple times #2412

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

lpiwowar
Copy link
Contributor

@lpiwowar lpiwowar commented Sep 30, 2024

It was not possible to run the test-operator role multiple times without cleaning up all resources created by the first role run, such as the logs pod created for the logs collection.

This patch makes it possible to run the test operator role multiple times without cleaning up the resources (e.g., after the update in an update job). It is possible because:

  • test-operator-logs pod now has unique name for each job run
  • The variable cifmw_test_operator_*_name is now exposed. By specifying a unique name, we can guarantee no name collisions between the test-operator related resources (logs pods, instances of CRs)

If you want to run test-operator role twice within a single job then make sure that for the second run you specify a value for the cifmw_test_operator_*_name. For example:

cifmw_test_operator_tempest_name: "post-update-tempest-tests"

Copy link

Thanks for the PR! ❤️
I'm marking it as a draft, once your happy with it merging and the PR is passing CI, click the "Ready for review" button below.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/de081b8b9823412e9114978d4382adbb

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 28m 42s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 16m 25s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 13m 42s
✔️ cifmw-multinode-tempest SUCCESS in 1h 36m 24s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 10s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 26s
✔️ build-push-container-cifmw-client SUCCESS in 31m 24s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 15s

Copy link
Collaborator

@lewisdenny lewisdenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @lpiwowar

Would it be better to use an randomly generated ID for each run regardless? This way a user would not need to manually specify a name for the second third run etc.

If the current implementation is best I have a suggestion to add a new section to the README for how to run the role multiple times, just with the details you mentioned in the commit message would be fine.

roles/test_operator/README.md Outdated Show resolved Hide resolved
@lewisdenny lewisdenny self-assigned this Oct 2, 2024
@lpiwowar
Copy link
Contributor Author

lpiwowar commented Oct 2, 2024

Hi @lewisdenny 👋, thank you for the review!

Would it be better to use an randomly generated ID for each run regardless?

Adding the random suffix won't eliminate the need to specify a different name for each execution of the test-operator role. The name as specified in the README is used in Metadata.Name. This value impacts a bunch of resources that get created during the test execution. Adding the random suffix will make the logs and collected artifacts slightly more difficult to read. Also, I think adding the random suffix would go against the principle of idempotency. LMKWYT.

If the current implementation is best I have a suggestion to add a new section to the README

Ok, I'll update the README 👍

It was not possible to run the test-operator role multiple times
without cleaning up all resources created by the first run, such
as the logs pod created for the logs collection.

This patch makes it possible to run the test operator role
multiple times without cleaning up the resources (e.g., after the
update in an update job). It is possible because:

- test-operator-logs pod now has unique name for each job run
- The variable cifmw_test_operator_*_name is now exposed. By
  specifying a unique name, we can guarantee no name collisions
  between the test-operator related resources (logs pods, instances
  of CRs)

If you want to run test-operator role twice within a single job
then make sure that for the second run you specify a value for the
cifmw_test_operator_*_name. For example:

  cifmw_test_operator_tempest_name: "post-update-tempest-tests"
@lpiwowar lpiwowar force-pushed the bugfix/multiple-test-operator-runs branch from c33ee46 to 962fb5a Compare October 2, 2024 08:23
Copy link
Contributor

openshift-ci bot commented Oct 2, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from lewisdenny. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/80c65bcf1cf94b56aebc96c856f4837e

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 02m 49s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 17m 59s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 39s
cifmw-multinode-tempest FAILURE in 1h 25m 34s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 26s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 25s
✔️ build-push-container-cifmw-client SUCCESS in 31m 14s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 21s

@lewisdenny
Copy link
Collaborator

Hi @lewisdenny 👋, thank you for the review!

Would it be better to use an randomly generated ID for each run regardless?

Adding the random suffix won't eliminate the need to specify a different name for each execution of the test-operator role. The name as specified in the README is used in Metadata.Name. This value impacts a bunch of resources that get created during the test execution. Adding the random suffix will make the logs and collected artifacts slightly more difficult to read. Also, I think adding the random suffix would go against the principle of idempotency. LMKWYT.

If the current implementation is best I have a suggestion to add a new section to the README

Ok, I'll update the README 👍

Thank you for the detailed explanation mate. That along with the update README make it clear to me +1 from me.

Copy link
Collaborator

@lewisdenny lewisdenny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Oct 2, 2024
@ciecierski ciecierski requested a review from tosky October 2, 2024 12:36
@pablintino
Copy link
Collaborator

recheck

@tosky
Copy link
Contributor

tosky commented Oct 2, 2024

Does this allow us to run test-operator twice or multiple times?

Copy link
Contributor

@ciecierski ciecierski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/03edffcc7e6940cc90469725209d24e8

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 49m 04s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 19m 57s
cifmw-crc-podified-edpm-baremetal RETRY_LIMIT in 20m 27s
✔️ cifmw-multinode-tempest SUCCESS in 1h 37m 48s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 14s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 21s
✔️ build-push-container-cifmw-client SUCCESS in 30m 59s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 14s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants