Add workflow to trigger OCP tests #12542
Conversation
|
Start a new ephemeral environment with changes proposed in this pull request: Fedora Environment Oracle Linux 8 Environment |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
6 times, most recently
from
91760f6
to
f677259
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-moderate-rev-4 /test 4.17-e2e-aws-ocp4-pci-dss-node |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
3 times, most recently
from
5aa9f46
to
2a785c1
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-pci-dss-3-2 Note: if a test is not started it can be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles |
|
@github-actions[bot]: The specified target(s) for
Use
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
3 times, most recently
from
695fc80
to
4934c26
Compare
Use CTF to identify changes in OCP rules, identify the profiles selecting these rules, and start Prow tests.
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
4934c26
to
84b6e16
Compare
Not all profiles are configured in Prow. This uses a hardcoded list of profiles to filter out the "untestable" ones.
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
84b6e16
to
b7c47b0
Compare
|
🤖 Trigger prow tests based on changed rules /test 4.17-e2e-aws-ocp4-high-node Note: if a test is not started it could be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles
|
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
b7c47b0
to
0cf1bbf
Compare
|
I have dropped the artificial change: b7c47b0. |
| if: ${{ steps.ctf.outputs.CTF_OUTPUT_SIZE != '0' && (contains(steps.product.outputs.prop, 'ocp4') || contains(steps.product.outputs.prop, 'rhcos4')) }} | ||
| id: profiles_to_test | ||
| run: | | ||
| OCP_VERSIONS=(4.17 4.16) |
There was a problem hiding this comment.
We'll need to manage this in addition to any OCP version changes in openshift/releases?
There was a problem hiding this comment.
Yes.
Another approach is to trigger the version-less tests, like e2e-aws-ocp4-high-node.
Sidenote: currently the version-less tests are running on 4.16, not the latest 4.17, 🙈
I considered running on two OCP versions, just to have more default coverage.
But it can be wasteful. I'm also okay with running only the version-less tests.
There was a problem hiding this comment.
@rhmdnd Let's stick with the version-less test jobs for now
|
|
||
| # Let's ilter out profiles for which we don't have a CI job configured | ||
| # Here is an example of how to quicly update this variable in the future | ||
| # TESTED_PROFILES=$(grep -r PROFILE= ./ComplianceAsCode-content-master__4.16.yaml | sort -u | sed 's/.*export PROFILE=\(.*\)/\1/') |
There was a problem hiding this comment.
This must be run from the openshift/releases repository, right?
There was a problem hiding this comment.
I don't have a good work around for this.
We could clone the repo and parse the files, or issue a /test comment and parse the comment from the ci bot.
I added it because we don't have tests for all profile ids.
You can see in previous comments that pci-dss-3-2 was chosen for tests, but we don't have CI wired up for that.
There was a problem hiding this comment.
I have changed that to actually curl the job config file and grep it.
So unless the job format or the test command changes, it should keep it self in sync with the tested profiles.
The version-less jobs are expected to be running on latest OCP.
Let's directly parse the source of CI Job configuration to know what profiles we can trigger jobs for. Also, move it out of the rule iteration loop. We just need to define it once.
|
🤖 Trigger prow tests based on changed rules /test e2e-aws-ocp4-high-node Note: if a test is not started it could be that a CI Job is not configure for that particular profile or product. Click here to see all the relevant profiles
|
|
@yuumasato: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
yuumasato
force-pushed
the
add_workflow_to_trigger_ocp_tests
branch
from
7b46c13
to
2999443
Compare
|
I'm dropping 7b46c13a9e32631649c0376873f74ddb169ec951since the workflow ran successfully. |
|
Code Climate has analyzed commit 2999443 and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (50% is the threshold). This pull request will bring the total coverage in the repository to 60.9% (0.0% change). View more on Code Climate. |
|
/packit retest-failed |
Description:
OCP4orRHCOS4profile that selects it to be tested.Rationale:
Review Hints: