Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pod tests to logging job #149

Merged
merged 11 commits into from
Oct 15, 2024
Merged

Conversation

elfiesmelfie
Copy link
Collaborator

@elfiesmelfie elfiesmelfie commented Sep 11, 2024

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/e5f0a1ca93354e8bb492982c7bf1956f

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 50m 25s
✔️ functional-tests-on-osp18 SUCCESS in 1h 37m 02s
functional-logging-tests-osp18 FAILURE in 1h 07m 03s

Copy link

Merge Failed.

This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.
Warning:
Error merging github.com/infrawatch/feature-verification-tests for 149,d71212d7dac4a410690a5411ad13ad6e42e12488

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/b565e21c63d545bb83e8a246d14c1dd5

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 50m 12s
✔️ functional-tests-on-osp18 SUCCESS in 1h 37m 36s
functional-logging-tests-osp18 FAILURE in 1h 03m 07s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/2a39a3f214ba43369d393cf073e9826a

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 13m 08s
✔️ functional-tests-on-osp18 SUCCESS in 1h 46m 33s
functional-logging-tests-osp18 FAILURE in 1h 04m 09s

@@ -0,0 +1,8 @@
---
- name: "Verify pod - {{ common_pod_test_id }}"
Copy link
Collaborator Author

@elfiesmelfie elfiesmelfie Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test id should be elsewhere. need to check the compute logging playbook [1]

[1] https://github.com/infrawatch/feature-verification-tests/blob/master/roles/telemetry_logging/tasks/journal_tests.yml

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason why the test_id shouldn't be here is because the include_tasks task will pass as long as the task file exists.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/07574782ba7c4e7fb7cd65507b5dbcb5

✔️ feature-verification-tests-noop SUCCESS in 5s
✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 29m 01s
✔️ functional-tests-on-osp18 SUCCESS in 1h 38m 40s
functional-logging-tests-osp18 FAILURE in 1h 05m 07s

- podinstance.stdout_lines | length == 0
changed_when: false

- name: Check terminated pod {{ common_pod_test_id }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ayefimov-1 What is the purpose of this second task? It looks like it is repeating the previous test
It might also be mis-named?
Is this just supposed to be getting more information on the pod by name?

If it is there for more information, then I think we can make it optional to run (only run of the pod exists, and don't let this failures effect the test run) since it is informational and not a test

Copy link
Contributor

@ayefimov-1 ayefimov-1 Sep 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal of this test is to validate the pods existence and status.

Line 12 is misnamed and should be called "Check Pod Status {{ common_pod_test_id }}". If this test fails then either there is a bug or that code changed. I was validating the logging tests and few weeks ago and saw that some tests were failing due to code changes.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first tasks is filtering on status.
If the second take is trying to check status, then the first task should not filter on the status.

To leave the first tasks as-is renders the second unnecessary.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5eafe951099446acb1c0ca3da555d4bf

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 54m 09s
✔️ functional-tests-on-osp18 SUCCESS in 1h 40m 17s
functional-logging-tests-osp18 FAILURE in 1h 02m 41s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/da74084f1ffa4ed8a14f40b982a292e9

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 36s
✔️ functional-tests-on-osp18 SUCCESS in 1h 37m 26s
functional-logging-tests-osp18 FAILURE in 1h 03m 51s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5e6d0af9220b4c7595101996672cb90e

✔️ feature-verification-tests-noop SUCCESS in 7s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 56m 18s
✔️ functional-tests-on-osp18 SUCCESS in 1h 39m 25s
functional-logging-tests-osp18 FAILURE in 1h 04m 51s

# vars:
# - <vars from above>

- name: "Verify logging pods are running in openshift-logging"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are passing except for logging-loki-gateway, which has two instances, so the second task fails because there are \n in the pod name.

ansible.builtin.import_role:
name: common

- name: "Verify logging pods are running in minio-dev"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is passing

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/8b5275c352cb4c06b0fd827cccd149b2

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 18m 14s
functional-tests-on-osp18 FAILURE in 1h 06m 23s
functional-logging-tests-osp18 FAILURE in 1h 06m 20s

@elfiesmelfie
Copy link
Collaborator Author

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/8b5275c352cb4c06b0fd827cccd149b2

✔️ feature-verification-tests-noop SUCCESS in 4s ✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 18m 14s ❌ functional-tests-on-osp18 FAILURE in 1h 06m 23s ❌ functional-logging-tests-osp18 FAILURE in 1h 06m 20s

It looks like the functional tests job failed for different reasons to the logging job.
The logging tests passed, but the test operator run was unsuccessful.
We can skip the test-operator run (for now) since we don't currently configure any relevant tests to run anyway

@elfiesmelfie
Copy link
Collaborator Author

I'll sync this to master and disable the test-operator run.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5a7bb2480ce549ca844a50cf18fe8d11

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 45s
functional-tests-on-osp18 FAILURE in 1h 22m 16s
✔️ functional-logging-tests-osp18 SUCCESS in 1h 04m 55s

@elfiesmelfie elfiesmelfie marked this pull request as ready for review October 2, 2024 10:35
@elfiesmelfie
Copy link
Collaborator Author

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/5a7bb2480ce549ca844a50cf18fe8d11

✔️ feature-verification-tests-noop SUCCESS in 4s ✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 45s ❌ functional-tests-on-osp18 FAILURE in 1h 22m 16s ✔️ functional-logging-tests-osp18 SUCCESS in 1h 04m 55s

The failing job is unrelated to the change and needs to be debugged separately.

@elfiesmelfie
Copy link
Collaborator Author

recheck

Copy link
Contributor

@ayefimov-1 ayefimov-1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im not sure why the comment would be true.

"The reason why the test_id shouldn't be here is because the include_tasks task will pass as long as the task file exists."

@elfiesmelfie
Copy link
Collaborator Author

elfiesmelfie commented Oct 8, 2024

Im not sure why the comment would be true.

"The reason why the test_id shouldn't be here is because the include_tasks task will pass as long as the task file exists."

The test ID will also appear in all subsequent task included by the include_task task, I think. It's been a while since I checked in.
To elaborate on the comment: the include_tasks task will pass (i.e. status = passed) as long as the file exists and failed tasks within the included task file does not lead to the include_task task failing, so marking the include_task task as a test will result in the test not being correctly reported.

ci/vars-logging-test.yml Outdated Show resolved Hide resolved
ci/ansible.cfg Outdated Show resolved Hide resolved
- podinstance.stdout_lines | length != 1
changed_when: false

- name: Check pod {{ common_pod_test_id }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the custom_logger is merged, we can go back in and rename these tasks.

The new task name format will be:

name: |
  <prefix> <task_name>
  <test-id>

ci/vars-logging-test.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@vyzigold vyzigold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't comment on the ansible structure as I'm an ansible beginner, but from my little logging knowledge the tests look OK.

roles/common/README.md Show resolved Hide resolved
@@ -0,0 +1,121 @@
---
- name: "Verify logging projects, endpoints, credentials, nodes, pods, services, manifests and subscriptions"
Copy link
Contributor

@mgirgisf mgirgisf Oct 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have this block without any tasks , Is that expected ? can you add please some comments.

ci/logging_tests_controller.yml Show resolved Hide resolved
@mgirgisf mgirgisf force-pushed the efoley_add_pod_test_to_logging branch from 2f45b42 to a6a9667 Compare October 14, 2024 10:31
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/1a432e6fbc47457eadd95fc30970aee3

✔️ feature-verification-tests-noop SUCCESS in 4s
✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 06m 39s
functional-tests-on-osp18 FAILURE in 1h 31m 15s
functional-logging-tests-osp18 FAILURE in 1h 04m 04s
✔️ functional-graphing-tests-osp18 SUCCESS in 1h 04m 27s

@mgirgisf mgirgisf force-pushed the efoley_add_pod_test_to_logging branch from a6a9667 to b5bf96d Compare October 15, 2024 06:53
@mgirgisf mgirgisf self-requested a review October 15, 2024 10:09
@mgirgisf mgirgisf force-pushed the efoley_add_pod_test_to_logging branch from f4b58db to d3f2447 Compare October 15, 2024 11:01
@mgirgisf mgirgisf merged commit 9ecc35c into master Oct 15, 2024
8 checks passed
@mgirgisf mgirgisf deleted the efoley_add_pod_test_to_logging branch October 15, 2024 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants