core/services/ocr2/plugins/ccip: parallelize tests

[jmank88]

No description provided.

[github-actions]

Flakeguard Summary

Ran new or updated tests between develop and 1799e2a (plugins-ccip-test-parallel).

View Flaky Detector Details | Compare Changes

Found Flaky Tests ❌

Name	Pass Ratio	Panicked?	Timed Out?	Race?	Runs	Successes	Failures	Skips	Package	Package Panicked?	Avg Duration	Code Owners
TestIntegration_CCIP	33.33%	false	false	false	3	1	2	0	github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip	false	3m36.77s	@smartcontractkit/ccip
TestIntegration_CCIP/with_dynamic_price_getter_allowOutOfOrderExecution_false	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip	false	1m39.973333333s	@smartcontractkit/ccip
TestIntegration_CCIP/with_dynamic_price_getter_allowOutOfOrderExecution_false/config_change_or_price_registry_update_while_requests_are_inflight	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip	false	21.486666666s	@smartcontractkit/ccip
TestIntegration_CCIP/with_pipeline_allowOutOfOrderExecution_true	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip	false	1m56.796666666s	@smartcontractkit/ccip
TestIntegration_CCIP/with_pipeline_allowOutOfOrderExecution_true/config_change_or_price_registry_update_while_requests_are_inflight	66.67%	false	false	false	3	2	1	0	github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip	false	22.21s	@smartcontractkit/ccip

Artifacts

For detailed logs of the failed tests, please refer to the artifact failed-test-results-with-logs.json.

[github-actions]

AER Report: CI Core

aer_workflow , commit , Detect Changes , Scheduled Run Frequency , Clean Go Tidy & Generate , Flakeguard Root Project / Get Tests To Run , lint , Core Tests (go_core_tests) , Core Tests (go_core_tests_integration) , Flakeguard Deployment Project , Core Tests (go_core_ccip_deployment_tests) , Core Tests (go_core_race_tests) , Core Tests (go_core_fuzz) , Flakeguard Root Project / Run Tests (github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip, ubuntu-latest) , Flakey Test Detection , SonarQube Scan , Flakeguard Root Project / Report

1. Flaky tests below pass ratio threshold:Run tests with flakeguard

Source of Error:

Run tests with flakeguard	2024-12-18T15:41:22.4892968Z All test results saved to test-result.json
Run tests with flakeguard	2024-12-18T15:41:22.4894145Z Found 5 flaky tests below the pass ratio threshold of 1.00:
Run tests with flakeguard	2024-12-18T15:41:22.4905353Z | Name | Pass Ratio | Panicked? | Timed Out? | Race? | Runs | Successes | Failures | Skips | Package | Package Panicked? | Avg Duration | Code Owners |
Run tests with flakeguard	2024-12-18T15:41:22.4912950Z | TestIntegration_CCIP | 33.33% | false | false | false | 3 | 1 | 2 | 0 | github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip | false | 3m36.77s | Unknown |
Run tests with flakeguard	2024-12-18T15:41:22.4916400Z | TestIntegration_CCIP/with_dynamic_price_getter_allowOutOfOrderExecution_false | 66.67% | false | false | false | 3 | 2 | 1 | 0 | github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip | false | 1m39.973333333s | Unknown |
Run tests with flakeguard	2024-12-18T15:41:22.4920292Z | TestIntegration_CCIP/with_dynamic_price_getter_allowOutOfOrderExecution_false/config_change_or_price_registry_update_while_requests_are_inflight | 66.67% | false | false | false | 3 | 2 | 1 | 0 | github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip | false | 21.486666666s | Unknown |
Run tests with flakeguard	2024-12-18T15:41:22.4924610Z | TestIntegration_CCIP/with_pipeline_allowOutOfOrderExecution_true | 66.67% | false | false | false | 3 | 2 | 1 | 0 | github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip | false | 1m56.796666666s | Unknown |
Run tests with flakeguard	2024-12-18T15:41:22.4928575Z | TestIntegration_CCIP/with_pipeline_allowOutOfOrderExecution_true/config_change_or_price_registry_update_while_requests_are_inflight | 66.67% | false | false | false | 3 | 2 | 1 | 0 | github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/ccip | false | 22.21s | Unknown |
Run tests with flakeguard	2024-12-18T15:41:22.4968985Z ##[error]Process completed with exit code 1.

Why: The error occurred because there were 5 flaky tests that did not meet the required pass ratio threshold of 1.00. These tests had inconsistent results across multiple runs, leading to a failure in the overall test suite.

Suggested fix: Investigate the flaky tests to identify the root cause of their instability. This may involve reviewing the test code for race conditions, dependencies on external systems, or other factors that could cause intermittent failures. Stabilize the tests to ensure consistent pass rates.