Skip to content

Commit

Permalink
Tekton updates (#53) (#54)
Browse files Browse the repository at this point in the history
* Tekton updates (#53)

* Move endpoint to pipeline run

* Doc update

* Switch to generateName

* Fix branch name

* Switch to generateName

* OpenShift doc change

* CP4i fixup

* Fix permissions

Signed-off-by: Trevor Dolby <[email protected]>

* Fix buildah image

---------

Signed-off-by: Trevor Dolby <[email protected]>

* Tekton updates (#55)

* Move endpoint to pipeline run

* Doc update

* Switch to generateName

* Fix branch name

* Switch to generateName

* OpenShift doc change

* CP4i fixup

* Fix permissions

Signed-off-by: Trevor Dolby <[email protected]>

* Fix buildah image

* 12.0.11 fixup

---------

Signed-off-by: Trevor Dolby <[email protected]>

---------

Signed-off-by: Trevor Dolby <[email protected]>
  • Loading branch information
trevor-dolby-at-ibm-com authored Apr 11, 2024
1 parent 409122e commit 91ef94c
Show file tree
Hide file tree
Showing 33 changed files with 125 additions and 92 deletions.
2 changes: 1 addition & 1 deletion .devcontainer/ace-toolkit-xvnc/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ace-demo-pipeline-toolkit-xvnc-devcontainer",
"image": "tdolby/experimental:ace-devcontainer-xvnc-12.0.10.0",
"image": "tdolby/experimental:ace-devcontainer-xvnc-12.0.11.0",
"containerEnv": {
"LICENSE": "accept"
},
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ace-demo-pipeline-devcontainer",
"image": "tdolby/experimental:ace-minimal-devcontainer-12.0.10.0",
"image": "tdolby/experimental:ace-minimal-devcontainer-12.0.11.0",
"containerEnv": {
"LICENSE": "accept"
},
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/quay.io-ace-toolkit-xvnc/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ace-demo-pipeline-quay.io-toolkit-xvnc-devcontainer",
"image": "quay.io/trevor_dolby/ace-devcontainer-xvnc:12.0.10.0",
"image": "quay.io/trevor_dolby/ace-devcontainer-xvnc:12.0.11.0",
"containerEnv": {
"LICENSE": "accept"
},
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/quay.io/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ace-demo-pipeline-devcontainer-quay-io",
"image": "quay.io/trevor_dolby/ace-minimal-devcontainer:12.0.10.0",
"image": "quay.io/trevor_dolby/ace-minimal-devcontainer:12.0.11.0",
"containerEnv": {
"LICENSE": "accept"
},
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docker-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
build:
runs-on: ubuntu-latest
container:
image: tdolby/experimental:ace-minimal-build-12.0.10.0-alpine
image: tdolby/experimental:ace-minimal-build-12.0.11.0-alpine
volumes:
- /var/run/docker.sock:/var/run/docker.sock
options: -u 0
Expand Down Expand Up @@ -37,7 +37,7 @@ jobs:
- name: Build the image
run: |
cd /tmp/build
docker build --file Dockerfile --build-arg BASE_IMAGE=tdolby/experimental:ace-minimal-12.0.10.0-alpine -t tdolby/experimental:tea-github-action-latest .
docker build --file Dockerfile --build-arg BASE_IMAGE=tdolby/experimental:ace-minimal-12.0.11.0-alpine -t tdolby/experimental:tea-github-action-latest .
- name: Push the image
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/main-cp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
# Set the cp.icr.io ace container as runtime.
# Running as root (-u 0) allows the installation of the docker binaries.
container:
image: cp.icr.io/cp/appc/ace:12.0.10.0-r1
image: cp.icr.io/cp/appc/ace:12.0.11.0-r1
options: -u 0
credentials:
username: ${{ secrets.CP_USERNAME }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
# The type of runner that the job will run on
runs-on: ubuntu-latest
container:
image: tdolby/experimental:ace-minimal-12.0.10.0-alpine
image: tdolby/experimental:ace-minimal-12.0.11.0-alpine
options: -u 0
env:
LICENSE: accept
Expand Down
2 changes: 1 addition & 1 deletion TeaRESTApplication/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ curl -X POST --data '{"name": "Assam", "strength": 5}' http://173.193.79.84:3153
curl http://173.193.79.84:31531/tea/index/1
```

Can be run locally as well as in the cloud, and for full details this repo should be pulled into an ACE 12.0.10.0 toolkit (via the egit plugin) and examined there.
Can be run locally as well as in the cloud, and for full details this repo should be pulled into an ACE 12.0.11.0 toolkit (via the egit plugin) and examined there.

## Implementation

Expand Down
2 changes: 1 addition & 1 deletion demo-infrastructure/README-jenkins.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ so that the component tests can run successfully. This is required regardless of
deploy target (integration node or ACEaaS).

For Windows, the ACE_COMMAND environment variable may need to be changed to match a locally-installed
version of ACE (currently set to 12.0.10). Container support is not required.
version of ACE (currently set to 12.0.11). Container support is not required.

For Linux, the pipeline will use containers for the actual build steps, and this requires either
the `ace` container image from cp.icr.io or the `ace-minimal-build` container image to be created
Expand Down
2 changes: 1 addition & 1 deletion demo-infrastructure/docker/ace-minimal-build/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Copyright (c) 2022 Open Technologies for Integration
# Licensed under the MIT license (see LICENSE for details)
ARG BASE_IMAGE=ace-minimal:12.0.10.0-alpine
ARG BASE_IMAGE=ace-minimal:12.0.11.0-alpine
FROM $BASE_IMAGE

#
Expand Down
6 changes: 3 additions & 3 deletions demo-infrastructure/docker/ace-minimal-build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

Used by the pipeline in this repo to run the ACE commands within a CI or other pipeline build.

Built on top of ace-minimal:12.0.10.0-alpine (in a registry of your choice and built from
Built on top of ace-minimal:12.0.11.0-alpine (in a registry of your choice and built from
https://github.com/ot4i/ace-docker/tree/master/experimental/ace-minimal)
but will be pushed to the same registry via the Tekton pipelines in tekton/minimal-image-build
in this repo if using Tekton.

For Jenkins build purposes, this image is needed locally and should be built with
```
docker build -t ace-minimal-build:12.0.10.0-alpine .
docker build -t ace-minimal-build:12.0.11.0-alpine .
```
after building ace-minimal:12.0.10.0-alpine from the ace-docker repo (linked above).
after building ace-minimal:12.0.11.0-alpine from the ace-docker repo (linked above).
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
pipeline {
agent { docker { image 'ace-jenkins:12.0.10.0-windows' } }
agent { docker { image 'ace-jenkins:12.0.11.0-windows' } }
parameters {
/* These values would be better moved to a configuration file and provided by */
/* the Config File Provider plugin (or equivalent), but this is good enough */
Expand Down
2 changes: 1 addition & 1 deletion demo-infrastructure/windows-containers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The repo must be cloned locally, possibly configured with a download URL (see
[ace-basic](https://github.com/ot4i/ace-docker/tree/main/experimental/windows/ace-basic)),
and then the following command should be run in the experimental/windows/ace-basic directory
```
docker build --build-arg FROMIMAGE=jenkins/agent:windowsservercore-ltsc2019 -t ace-jenkins:12.0.10.0-windows .
docker build --build-arg FROMIMAGE=jenkins/agent:windowsservercore-ltsc2019 -t ace-jenkins:12.0.11.0-windows .
```
to create the ace-jenkins image used by this pipeline.

Expand Down
6 changes: 3 additions & 3 deletions tekton/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
# Licensed under the MIT license (see LICENSE for details)

#FROM tdolby/experimental:ace-minimal-11.0.0.11-alpine
ARG BASE_IMAGE=ace-minimal:12.0.10.0-alpine
ARG BASE_IMAGE=ace-minimal:12.0.11.0-alpine
FROM $BASE_IMAGE

# Used for tekton and Maven containers
#
# To run locally, build into an ace-server directory somewhere (such as /tmp/maven-output/ace-server), copy
# this Dockerfile into the parent directory, and then (assuming 12.0.10) run
# this Dockerfile into the parent directory, and then (assuming 12.0.11) run
#
# docker build --build-arg BASE_IMAGE=tdolby/experimental:ace-minimal-12.0.10.0-alpine -t tea-tekton-test .
# docker build --build-arg BASE_IMAGE=tdolby/experimental:ace-minimal-12.0.11.0-alpine -t tea-tekton-test .
#
# from the parent directory (such as /tmp/maven-output)
#
Expand Down
34 changes: 24 additions & 10 deletions tekton/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ kubectl create secret generic jdbc-secret --from-literal=USERID='blah' --from-li
```
with the obvious replacements.

## Tekton dashboard
### Tekton dashboard

The Tekton dashboard (for non-OpenShift users) can be installed as follows:
```
Expand All @@ -95,6 +95,11 @@ dashboard HTTP port can be made available locally as follows:
kubectl --namespace tekton-pipelines port-forward --address 0.0.0.0 svc/tekton-dashboard 9097:9097
```

### Pipeline creation

At this point, the instructions split into two: for deployment to containers, see the following section.
For ACE-as-a-Service, see [ACE-as-a-Service target](#ace-as-a-service-target) below.

## Container deploy target

![Pipeline overview](/demo-infrastructure/images/tekton-pipeline.png)
Expand All @@ -113,12 +118,19 @@ task is only run if `knativeDeploy` is set to `true` when the pipeline is run).

Once that has been accomplished, the simplest way to run the pipeline is
```
kubectl apply -f tekton/ace-pipeline-run.yaml
tkn pipelinerun logs ace-pipeline-run-1 -f
kubectl create -f tekton/ace-pipeline-run.yaml
tkn pipelinerun logs -L -f
```

and this should build the projects, run the unit tests, create a docker image, and then create a deployment that runs the application.

Note that previous versions of the instructions suggested running
```
kubectl apply -f tekton/ace-pipeline-run.yaml
tkn pipelinerun logs ace-pipeline-run-1 -f
```
using a fixed name for the pipeline run, but using a generated name allows build history to be preserved.

### How to know if the container deploy pipeline has succeeded

The end result should be a running container with the tea application deployed, listening for requests on /tea/index at the
Expand Down Expand Up @@ -162,8 +174,8 @@ registry authentication for some reason when using single-node OpenShift with a
After that, the pipeline run YAML should be changed to point to the OpenShift registry, and the
pipeline run as normal:
```
kubectl apply -f tekton/ace-pipeline-run.yaml
tkn pipelinerun logs ace-pipeline-run-1 -f
kubectl create -f tekton/ace-pipeline-run.yaml
tkn pipelinerun logs -L -f
```
The OpenShift Pipeline operator provides a web interface for the pipeline runs also, which may be
an easier way to view progress.
Expand Down Expand Up @@ -199,7 +211,8 @@ appropriate credentials:
kubectl create secret docker-registry ibm-entitlement-key --docker-username=cp --docker-password=myEntitlementKey --docker-server=cp.icr.io
```
Ensure that the ace-tekton-service-account includes the `ibm-entitlement-key` secret for both secrets
and imagePullSecrets. For those without an IBM Entitlement Key, the `ace-minimal` image will also work.
and imagePullSecrets. For those without an IBM Entitlement Key, the `ace-minimal` image will also work, and
the service account will not need to be changed.

Setting up the pipeline requires Tekton to be installed (which may already have happend via OpenShift operators, in which case
skip the first line), tasks to be created, and the pipeline itself to be configured:
Expand All @@ -215,18 +228,19 @@ should be created using values acquired using the ACEaaS console. See
[https://www.ibm.com/docs/en/app-connect/saas?topic=overview-accessing-api](https://www.ibm.com/docs/en/app-connect/saas?topic=overview-accessing-api)
for details on how to find or create the correct credentials, and then set the following
```
kubectl create secret generic aceaas-credentials --from-literal=appConEndpoint=MYENDPOINT --from-literal=appConInstanceID=MYINSTANECID --from-literal=appConClientID=HEXNUMBERSTRING --from-literal=appConApiKey=BASE64APIKEY --from-literal=appConClientSecret=HEXNUMBERCLIENTSECRET
kubectl create secret generic aceaas-credentials --from-literal=appConInstanceID=MYINSTANCEID --from-literal=appConClientID=HEXNUMBERSTRING --from-literal=appConApiKey=BASE64APIKEY --from-literal=appConClientSecret=HEXNUMBERCLIENTSECRET
```
The pipeline should create the required configurations based on the JDBC credentials
and other values if the createConfiguration parameter is set to `true`; this should only be used
for the first pipeline run or after any change to the credentials (see the "ACEaaS API rate
limits" section of [README-aceaas-pipelines.md](/demo-infrastructure/README-aceaas-pipelines.md)
for more information).

Once that has been accomplished, the simplest way to run the pipeline is
Once the required edits to `aceaas-pipeline-run.yaml` have been made (including setting the
ACEaaS API endpoint, if not using the US East region), the simplest way to run the pipeline is
```
kubectl apply -f tekton/aceaas/aceaas-pipeline-run.yaml
tkn pipelinerun logs aceaas-pipeline-run-1 -f
kubectl create -f tekton/aceaas/aceaas-pipeline-run.yaml
tkn pipelinerun logs -L -f
```

and this should build the projects, run the tests, and then deploy to ACEaaS.
Expand Down
8 changes: 2 additions & 6 deletions tekton/ace-pipeline-run.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
# Can use generated names with kubectl create; follow logs with "tkn pipeline logs ace-pipeline -f" to
# avoid needing a fixed name.
#generateName: ace-pipeline-run-

generateName: ace-pipeline-run-
# Fixed name allows "tkn pr delete ace-pipeline-run-1 -f ; kubectl apply -f tekton/ace-pipeline-run.yaml ; tkn pr logs ace-pipeline-run-1 -f"
# which has a slightly nicer log format.
name: ace-pipeline-run-1
#name: ace-pipeline-run-1
spec:
serviceAccountName: ace-tekton-service-account
pipelineRef:
Expand Down
4 changes: 3 additions & 1 deletion tekton/aceaas/40-ibmint-aceaas-deploy-task.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ spec:
type: string
- name: buildImage
type: string
- name: appConEndpoint
type: string
- name: deployPrefix
type: string
results:
Expand Down Expand Up @@ -128,7 +130,7 @@ spec:
echo Deploying BAR file
echo ========================================================================
export appConEndpoint=$(cat /run/secrets/aceaas/appConEndpoint)
export appConEndpoint=$(params.appConEndpoint)
export appConInstanceID=$(cat /run/secrets/aceaas/appConInstanceID)
export appConClientID=$(cat /run/secrets/aceaas/appConClientID)
export appConApiKey=$(cat /run/secrets/aceaas/appConApiKey)
Expand Down
6 changes: 4 additions & 2 deletions tekton/aceaas/41-ibmint-aceaas-config-task.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ spec:
type: string
- name: buildImage
type: string
- name: appConEndpoint
type: string
- name: deployPrefix
type: string
- name: barURL
Expand Down Expand Up @@ -46,7 +48,7 @@ spec:
unset LD_LIBRARY_PATH
set -e # Fail on error
export appConEndpoint=$(cat /run/secrets/aceaas/appConEndpoint)
export appConEndpoint=$(params.appConEndpoint)
export appConInstanceID=$(cat /run/secrets/aceaas/appConInstanceID)
export appConClientID=$(cat /run/secrets/aceaas/appConClientID)
export appConApiKey=$(cat /run/secrets/aceaas/appConApiKey)
Expand Down Expand Up @@ -154,7 +156,7 @@ spec:
unset LD_LIBRARY_PATH
set -e # Fail on error
export appConEndpoint=$(cat /run/secrets/aceaas/appConEndpoint)
export appConEndpoint=$(params.appConEndpoint)
export appConInstanceID=$(cat /run/secrets/aceaas/appConInstanceID)
export appConClientID=$(cat /run/secrets/aceaas/appConClientID)
export appConApiKey=$(cat /run/secrets/aceaas/appConApiKey)
Expand Down
11 changes: 5 additions & 6 deletions tekton/aceaas/aceaas-pipeline-run.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
# Can use generated names with kubectl create; follow logs with "tkn pipeline logs aceaas-pipeline -f" to
# avoid needing a fixed name.
#generateName: aceaas-pipeline-run-

generateName: aceaas-pipeline-run-
# Fixed name allows "tkn pr delete aceaas-pipeline-run-1 -f ; kubectl apply -f tekton/aceaas/aceaas-pipeline-run.yaml ; tkn pr logs aceaas-pipeline-run-1 -f"
# which has a slightly nicer log format.
name: aceaas-pipeline-run-1
#name: aceaas-pipeline-run-1
spec:
serviceAccountName: ace-tekton-service-account
pipelineRef:
Expand All @@ -26,3 +22,6 @@ spec:
value: "tdolby"
- name: createConfiguration
value: "true"
- name: appConEndpoint
# US East default
value: "api.p-vir-c1.appconnect.automation.ibm.com"
7 changes: 7 additions & 0 deletions tekton/aceaas/aceaas-pipeline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ spec:
type: string
- name: deployPrefix
type: string
- name: appConEndpoint
type: string
default: "api.p-vir-c1.appconnect.automation.ibm.com"
- name: createConfiguration
type: string
default: "false"
Expand All @@ -30,6 +33,8 @@ spec:
value: $(params.buildImage)
- name: deployPrefix
value: $(params.deployPrefix)
- name: appConEndpoint
value: $(params.appConEndpoint)
- name: create-config-and-runtime
taskRef:
name: aceaas-create-config-and-runtime
Expand All @@ -40,6 +45,8 @@ spec:
value: $(params.revision)
- name: buildImage
value: $(params.buildImage)
- name: appConEndpoint
value: $(params.appConEndpoint)
- name: deployPrefix
value: $(params.deployPrefix)
- name: barURL
Expand Down
10 changes: 5 additions & 5 deletions tekton/minimal-image-build/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ and proceed through the pages until the main download page with a link:

The link is likely to be of the form
```
https://iwm.dhe.ibm.com/sdfdl/v2/regs2/mbford/Xa.2/Xb.WJL1cUPI9gANEhP8GuPD_qX1rj6x5R4yTUM7s_C2ue8/Xc.12.0.10.0-ACE-LINUX64-DEVELOPER.tar.gz/Xd./Xf.LpR.D1vk/Xg.12164875/Xi.swg-wmbfd/XY.regsrvs/XZ.pPVETUejcqPsVfDVKbdNu6IRpo4TkyKu/12.0.10.0-ACE-LINUX64-DEVELOPER.tar.gz
https://iwm.dhe.ibm.com/sdfdl/v2/regs2/mbford/Xa.2/Xb.WJL1cUPI9gANEhP8GuPD_qX1rj6x5R4yTUM7s_C2ue8/Xc.12.0.11.0-ACE-LINUX64-DEVELOPER.tar.gz/Xd./Xf.LpR.D1vk/Xg.12164875/Xi.swg-wmbfd/XY.regsrvs/XZ.pPVETUejcqPsVfDVKbdNu6IRpo4TkyKu/12.0.11.0-ACE-LINUX64-DEVELOPER.tar.gz
```
Copy that link into the aceDownloadUrl parameter, adjusting the version numbers in the other files as needed.

Expand All @@ -84,8 +84,8 @@ image and push it to the registry:
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
kubectl apply -f tekton/minimal-image-build/01-ace-minimal-image-build-and-push-task.yaml
kubectl apply -f tekton/minimal-image-build/ace-minimal-image-pipeline.yaml
kubectl apply -f tekton/minimal-image-build/ace-minimal-image-pipeline-run.yaml
tkn pipelinerun logs ace-minimal-image-pipeline-run-1 -f
kubectl create -f tekton/minimal-image-build/ace-minimal-image-pipeline-run.yaml
tkn pipelinerun logs -L -f
```

The ace-minimal-build-image-pipeline builds not only the ace-minimal-build image but also
Expand All @@ -95,8 +95,8 @@ kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/
kubectl apply -f tekton/minimal-image-build/01-ace-minimal-image-build-and-push-task.yaml
kubectl apply -f tekton/minimal-image-build/02-ace-minimal-build-image-build-and-push-task.yaml
kubectl apply -f tekton/minimal-image-build/ace-minimal-build-image-pipeline.yaml
kubectl apply -f tekton/minimal-image-build/ace-minimal-build-image-pipeline-run.yaml
tkn pipelinerun logs ace-minimal-build-image-pipeline-run-1 -f
kubectl create -f tekton/minimal-image-build/ace-minimal-build-image-pipeline-run.yaml
tkn pipelinerun logs -L -f
```

## OpenShift
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: ace-minimal-build-image-pipeline-run-1
generateName: ace-minimal-build-image-pipeline-run-
# Fixed name allows "tkn pr delete ace-minimal-build-image-pipeline-run-1 -f ; kubectl apply -f tekton/ace-minimal-build-image-pipeline-run.yaml ; tkn pr logs ace-minimal-build-image-pipeline-run-1 -f"
#name: ace-minimal-build-image-pipeline-run-1
spec:
serviceAccountName: ace-tekton-service-account
# Use this instead if building in a CP4i environment
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: ace-minimal-image-pipeline-run-1
generateName: ace-minimal-image-pipeline-run-
# Fixed name allows "tkn pr delete ace-minimal-image-pipeline-run-1 -f ; kubectl apply -f tekton/ace-minimal-image-pipeline-run.yaml ; tkn pr logs ace-minimal-image-pipeline-run-1 -f"
#name: ace-minimal-image-pipeline-run-1
spec:
serviceAccountName: ace-tekton-service-account
# Use this instead if building in a CP4i environment
Expand Down
Loading

0 comments on commit 91ef94c

Please sign in to comment.