Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-1981] use chainlink cli in the keystone smoke test #16146

Merged
merged 87 commits into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
460f04c
WIP, almost working... last step fails, when forwarder calls DF cache
Tofel Jan 14, 2025
5d7ffe2
working version
Tofel Jan 15, 2025
ef63e87
use newer PoR workflow
Tofel Jan 15, 2025
1c13aff
now its broken again
Tofel Jan 16, 2025
4eb030f
Merge 1c13aff4908b2ea514c8fde549d0cb83b9d66c1e into f9cc5147878cc5f45…
Tofel Jan 16, 2025
85b3d62
Update gethwrappers
app-token-issuer-infra-releng[bot] Jan 16, 2025
a62699d
go back to previous config, where all steps were correcty configured
Tofel Jan 16, 2025
025aa34
use older config
Tofel Jan 16, 2025
4ea8c76
fix port
Tofel Jan 16, 2025
2979ad8
use debug consumer contract
Tofel Jan 16, 2025
68c6dfc
WORKING VERSION
Tofel Jan 17, 2025
8909ec5
remove mock streams capability
Tofel Jan 20, 2025
c77aa78
passes with workflow registry deployed with c/d
Tofel Jan 20, 2025
0615056
passes with keystone consumer deployed with c/d
Tofel Jan 20, 2025
cdd5670
passes with keystone forwarder deployed with c/d
Tofel Jan 20, 2025
91886b3
remove some deps
Tofel Jan 20, 2025
b22e391
allow to use chainlink-cli
Tofel Jan 21, 2025
8bea5a0
restore old KeystoneFeedsConsumer.sol contract
Tofel Jan 22, 2025
5266e9f
try to run the test in CI
Tofel Jan 23, 2025
f5026ea
Merge remote-tracking branch 'origin/develop' into tt-1847-ocr3-keyst…
Tofel Jan 23, 2025
93df120
fix integration workflow
Tofel Jan 23, 2025
b9cb7e6
move the function
Tofel Jan 23, 2025
ec738a0
use existing env vars to get the image, use e2e tests workflow versio…
Tofel Jan 23, 2025
0192c51
add missing PRIVATE_KEY env var
Tofel Jan 23, 2025
4a01e35
pass optional roles for GATI to be able to read from repositories
Tofel Jan 23, 2025
a119784
add some comments, update e2e workflow commit
Tofel Jan 23, 2025
d3148c2
add comments
Tofel Jan 23, 2025
78c57f2
fix integration tests workflow file
Tofel Jan 24, 2025
cb9b9c0
newer workflow version
Tofel Jan 24, 2025
296a087
update token name and workflow version
Tofel Jan 24, 2025
92759d2
do not use chainlink CLI in the CI for now (we need a token that can …
Tofel Jan 24, 2025
1188327
fix env var name in CI
Tofel Jan 24, 2025
3838159
fix capability reference
Tofel Jan 24, 2025
91b3e9d
clean up the workflow test
Tofel Jan 24, 2025
cd020ef
move capability test to smoke subfolder
Tofel Jan 24, 2025
e613120
Merge branch 'develop' into tt-1847-ocr3-keystone-por-test-broken
Tofel Jan 24, 2025
fbbd200
restore old versions of feeds consumer wrapper
Tofel Jan 24, 2025
a81413f
rename GITHUB_TOKEN env var to GITHUB_API_TOKEN
Tofel Jan 24, 2025
f8196e2
try one more approach to resolving placeholder env var
Tofel Jan 24, 2025
50d48dd
adjust env file to new location
Tofel Jan 24, 2025
5e320a0
revert testing-related changes to integration tests workflow
Tofel Jan 27, 2025
fcc3055
Merge branch 'develop' into tt-1847-ocr3-keystone-por-test-broken
Tofel Jan 27, 2025
1f7751e
fix lints
Tofel Jan 27, 2025
05c05a9
use function from chainlink-common to generate workflow id, define ne…
Tofel Jan 27, 2025
7022094
add JD
Tofel Jan 27, 2025
8802fa6
use C/D to setup all contracts
Tofel Jan 28, 2025
2326dae
add extra debug information
Tofel Jan 28, 2025
7346b1a
fix lint
Tofel Jan 28, 2025
4d47db5
poll config every 500ms
Tofel Jan 28, 2025
713e3f0
change polling interval to 1s
Tofel Jan 28, 2025
eb037b8
fix config polling intervnal
Tofel Jan 28, 2025
90d68dd
use only chainlink/deployments when interacting with contract
Tofel Jan 28, 2025
9620bdf
JD is failing to create jobs due to a bug and authorization issue
Tofel Jan 28, 2025
59f0c01
do not use JD to setup jobs
Tofel Jan 28, 2025
e15d431
Merge branch 'develop' into tt-1969-debug-capabilities-test
Tofel Jan 29, 2025
9c614f5
CR changes
Tofel Jan 29, 2025
cf0c44b
Merge branch 'tt-1969-debug-capabilities-test' into tt-1847-keystone-…
Tofel Jan 29, 2025
f09a623
adjust to latest changes, move some configs to TOML
Tofel Jan 29, 2025
453e541
CR changes
Tofel Jan 29, 2025
ee61b7b
move comments around
Tofel Jan 29, 2025
679e9fd
add test modification guideline and better debugging
Tofel Jan 29, 2025
c37966c
fix config validations
Tofel Jan 29, 2025
afe7341
Merge branch 'tt-1969-debug-capabilities-test' into tt-1847-keystone-…
Tofel Jan 29, 2025
bad7fa6
fix lints
Tofel Jan 29, 2025
6943e35
use fixed JD version in the CI
Tofel Jan 29, 2025
24072c3
remove workflow registration artifact file
Tofel Jan 29, 2025
318576c
restore older environment file
Tofel Jan 29, 2025
177f5b4
Merge remote-tracking branch 'origin/develop' into tt-1847-keystone-s…
Tofel Jan 30, 2025
57b963f
get p2p ide from node labels, more comments, use contractset
Tofel Jan 30, 2025
a8ecfb4
fix lints and remove superfluous methods
Tofel Jan 30, 2025
f5bb5df
dynamic transmission schedule
Tofel Jan 30, 2025
ee5386f
try out chainlink-cli in the CI
Tofel Jan 30, 2025
09c4b92
use configurable workflow_id also when registering the workflow
Tofel Jan 30, 2025
b26ff97
support configurable capabilities/chainlink-cli versions
Tofel Jan 30, 2025
0bbdad5
remove unused struct
Tofel Jan 30, 2025
3917aae
Merge branch 'tt-1847-keystone-smoke-use-chainlink-deployment' into t…
Tofel Jan 30, 2025
f1fa8eb
fix TOML
Tofel Jan 30, 2025
ca2fe48
make chainlink-cli os/arch aware and extract it to the current folder
Tofel Jan 30, 2025
60c8db1
run chainlink-cli from current directory
Tofel Jan 30, 2025
540f565
use absolute path when running chainlink-cli
Tofel Jan 30, 2025
31bddfd
Merge branch 'develop' into tt-1981-use-chainlink-cli
Tofel Jan 30, 2025
4c676ca
fix if condition
Tofel Jan 30, 2025
6198c67
add ocr3 config accidentally deleted when resolving merge conflicts
Tofel Jan 31, 2025
d2c68bd
Merge branch 'develop' into tt-1981-use-chainlink-cli
Tofel Jan 31, 2025
ac5a7f1
add info about JD image to guidelines
Tofel Jan 31, 2025
07167b8
Merge branch 'develop' into tt-1981-use-chainlink-cli
Tofel Feb 3, 2025
d5f662d
code review changes + renaming of chainlink-cli to CRE CLI + configur…
Tofel Feb 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions integration-tests/smoke/capabilities/.gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# CTFv2 cached environment, no point in committing this
*-cache.toml
# compiled cron capability
# compiled cron capability, it should be downloaded by the test
amd64_cron
# workflow registration artifact
abcdefgasd.yaml
# workflow registration artifact, it's created anew every time the workflow is registered
abcdefgasd.yaml
# chainlink cli binary, it should be downloaded by the test
cre_v*
10 changes: 7 additions & 3 deletions integration-tests/smoke/capabilities/environment-ci.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,14 @@
# without 0x prefix!
feed_id = "018bfe8840700040000000000000000000000000000000000000000000000000"

use_chainlink_cli = false
use_existing = true
use_cre_cli = true
should_compile_new_workflow = false

[workflow_config.existing]
[workflow_config.dependencies]
capabilities_version = "v1.0.0-alpha"
cre_cli_version = "v1.0.2"

[workflow_config.compiled_config]
binary_url = "https://gist.githubusercontent.com/Tofel/8a39af5b68c213d2200446c175b5c99e/raw/cb7b2a56b37e333fe0bdce07b79538c4ce332f5f/binary.wasm.br"
config_url = "https://gist.githubusercontent.com/Tofel/19c80e6297914a79449f916e5e65dfdd/raw/1344c259ef7e970dbabaa1e9e885845b8eba5da9/config.json3674692696"

Expand Down
14 changes: 8 additions & 6 deletions integration-tests/smoke/capabilities/environment.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,18 @@
# without 0x prefix!
feed_id = "018bfe8840700040000000000000000000000000000000000000000000000000"

use_chainlink_cli = true
use_existing = true
use_cre_cli = true
should_compile_new_workflow = true
workflow_folder_location = "path-to-folder-with-main.go-of-your-workflow"

[workflow_config.existing]
[workflow_config.dependencies]
capabilities_version = "v1.0.0-alpha"
cre_cli_version = "v1.0.2"

[workflow_config.compiled_config]
binary_url = "https://gist.githubusercontent.com/Tofel/8a39af5b68c213d2200446c175b5c99e/raw/cb7b2a56b37e333fe0bdce07b79538c4ce332f5f/binary.wasm.br"
config_url = "https://gist.githubusercontent.com/Tofel/19c80e6297914a79449f916e5e65dfdd/raw/1344c259ef7e970dbabaa1e9e885845b8eba5da9/config.json3674692696"

[workflow_config.chainlink_cli]
folder_location = "path-to-folder-with-main.go-of-your-workflow"

[nodeset]
nodes = 5
override_mode = "each"
Expand Down
33 changes: 18 additions & 15 deletions integration-tests/smoke/capabilities/guidelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,16 @@ The test requires several environment variables. Below is a launch configuration
}
```

- **`GITHUB_READ_TOKEN`**: Required for downloading the `cron` capability binary and `chainlink-cli` (if enabled). Requires `content:read` permission for `smartcontractkit/capabilities` and `smartcontractkit/dev-platform` repositories. Use a fine-grained personal access token (PAT) tied to the **organization’s GitHub account**.
- **`GITHUB_READ_TOKEN`**: Required for downloading the `cron` capability binary and CRE CLI (if enabled). Requires `content:read` permission for `smartcontractkit/capabilities` and `smartcontractkit/dev-platform` repositories. Use a fine-grained personal access token (PAT) tied to the **organization’s GitHub account**.
- **`GIST_WRITE_TOKEN`**: Required only for compiling and uploading a new workflow. It needs `gist:read:write` permissions and should be a fine-grained PAT **tied to your personal GitHub account**.

Test also expects you to have the Job Distributor image available locally. By default, `environment.toml` expects image tagged as `jd-test-1:latest`. The easiest way to get it, is to clone the Job Distributor repository and build it locally with:
```bash
docker build -t jd-test-1 -f e2e/Dockerfile.e2e
```

Alternatively, if you have access to the Docker image repository where it's stored you can modify `environment.toml` with the name of the image stored there.

---

## Adding a New Capability
Expand Down Expand Up @@ -123,16 +130,14 @@ For the test to compile and upload the binary, modify your TOML configuration:

```toml
[workflow_config]
use_chainlink_cli = true
use_existing = false

[workflow_config.chainlink_cli]
folder_location = "path-to-folder-with-main.go-of-your-workflow"
use_cre_cli = true
should_compile_new_workflow = true
workflow_folder_location = "path-to-folder-with-main.go-of-your-workflow"
```

### Workflow Configuration

If your workflow requires configuration, modify the test to create and pass the configuration data to `chainlink-cli`:
If your workflow requires configuration, modify the test to create and pass the configuration data to CRE CLI:

```go
configFile, err := os.CreateTemp("", "config.json")
Expand All @@ -156,10 +161,10 @@ If you compiled and uploaded the binary yourself, set the following in your conf

```toml
[workflow_config]
use_chainlink_cli = true
use_existing = true
use_cre_cli = true
should_compile_new_workflow = false

[workflow_config.existing]
[workflow_config.compiled_config]
binary_url = "<binary-url>"
config_url = "<config-url>"
```
Expand All @@ -176,11 +181,9 @@ If the deployer private key or deployment sequence changes, run the test in **up

```toml
[workflow_config]
use_chainlink_cli = true
use_existing = false

[workflow_config.chainlink_cli]
folder_location = "path-to-folder-with-main.go-of-your-workflow"
use_cre_cli = true
should_compile_new_workflow = true
workflow_folder_location = "path-to-folder-with-main.go-of-your-workflow"
```

---
Expand Down
Loading
Loading