From e82ab70a02db01982aaa516abaf7c218d9f383c0 Mon Sep 17 00:00:00 2001 From: Victoria Martinez de la Cruz Date: Thu, 11 Jul 2024 14:47:56 +0200 Subject: [PATCH] Create index image from pre-built bundles (#597) * Create index image from pre-built bundles Deploy from bundles can now be used along with deploy from index enabled. This way, an index image can be created with specific pre built bundles and deploy STF with those. * Fix set_fact syntax for SGO/SGO bundle info dicts * Add ImageStreams for STO and SGO * Fix minor typo in STO/SGO bundle info dict * Set correct STO and SGO bundles path We need to use the internal registry address to build the index image that will serve as base for the catalog * Drop build number from SGO and STO bundles We only need the tag when creating the index image Keeping this value will generate errors * Get STO and SGO tags from the bundle path * Fix typo on SGO and STO images path * Update README with bundle + index instructions Add more details on how to deploy from index with pre built bundles * Use stf_channel on the deploy_stf step Define this value to stable-1.5 when deploying from bundles and index, unstable (the default) if bundles have been built locally. Also, honor the sgo and sto bundles tags. Ideally, we would update the sgo and sto image path to be only the path (without the tag) and define the tag separately. In this case, it is responsibility from the user to make sure that the tag in the image path and the tag provided in sto and sgo image path match when deploying from externally built bundles and index. * Update missing reference to SGO bundles channel * Update some logic paths to simplify the execution * Honor the stf_channel variable Nightly bundles use the "unstable" channel while RH catalog bundles use "stable-1.5" channel We should use nightly bundles "unstable" by default and pass "stable-1.5" when deploying from RH catalog * Get opm for getting bundles information We need opm to get the bundle version when deploying from index with pre-built bundles Get opm following the same logic we use for operator-sdk * Get the operator bundle version for STO and SGO Inspect the provided bundles with opm render and obtain the operator bundle version for each of them. * Set a default version for opm Let's stick to latest-4.14 for now, with the option of updating to a different one if needed * Fix typo when executing opm render * Revert "Fix typo when executing opm render" This reverts commit 1050227747d741b2d9f1c11885bd2eb91fb5ee73. * Revert "Set a default version for opm" This reverts commit d6a51df8b2b5883e19a29c00abd1b0d6286c53af. * Revert "Get the operator bundle version for STO and SGO" This reverts commit c1d9db36574f3ddc2fa46a5cd1a7eff379d9b24f. * Revert "Get opm for getting bundles information" This reverts commit 642df50f8171b5709638c407cbc552df49068f27. * Get STO and SGO operator bundle version using oc oc image info command provides operator bundle version information. Use this to get the proper version numbers when building the index image * Get STO and SGO bundle versions from info Parse the oc image info output to get the STO and SGO bundle versions * Add extra logic to avoid variables overwrite sto_bundle_info and sgo_bundle_info is overriding even though the step is skipped Add an extra logic check to avoid variables override --- build/stf-run-ci/README.md | 30 ++++++++++-- build/stf-run-ci/tasks/create_catalog.yml | 56 ++++++++++++++++++++++- build/stf-run-ci/tasks/main.yml | 29 ++---------- 3 files changed, 86 insertions(+), 29 deletions(-) diff --git a/build/stf-run-ci/README.md b/build/stf-run-ci/README.md index b9b1afbc0..ecc4fd38e 100644 --- a/build/stf-run-ci/README.md +++ b/build/stf-run-ci/README.md @@ -20,7 +20,7 @@ choose to override: | `__deploy_stf` | {true,false} | true | Whether to deploy an instance of STF | | `__local_build_enabled` | {true,false} | true | Whether to deploy STF from local built artifacts. Also see `working_branch`, `sg_branch`, `sgo_branch` | | `__deploy_from_bundles_enabled` | {true,false} | false | Whether to deploy STF from OLM bundles (TODO: compat with `__local_build_enabled`) | -| `__deploy_from_index_enabled` | {true,false} | false | Whether to deploy STF from locally built bundles and index image. | +| `__deploy_from_index_enabled` | {true,false} | false | Whether to deploy STF from locally built bundles/OLM bundles and index image. | | `__disconnected_deploy` | {true,false} | false | Whether to deploy on a disconnected cluster | | `__service_telemetry_bundle_image_path` | | `quay.io/infrawatch-operators/service-telemetry-operator-bundle:nightly-head` | Image path to Service Telemetry Operator bundle | | `__smart_gateway_bundle_image_path` | | `quay.io/infrawatch-operators/smart-gateway-operator-bundle:nightly-head` | Image path to Smart Gateway Operator bundle | @@ -83,9 +83,10 @@ choose to override: You can deploy Service Telemetry Framework using this role in a few configuration methods: -* local build artifacts from Git repository cloned locally -* local build artifacts, local bundle artifacts, and Subscription via OLM using locally built index image -* standard deployment using Subscription and OLM +* local build artifacts from Git repository cloned locally (local build) +* local build artifacts, local bundle artifacts, and Subscription via OLM using locally built index image (local build + deploy from index) +* externally build bundle artifacts and Subscription via OLM using locally built index image (deploy from bundles + deploy from index) +* standard deployment using Subscription and OLM (deploy from bundles) * supporting components but no instance of Service Telemetry Operator ## Basic deployment @@ -134,6 +135,27 @@ You can perform a deployment using OLM and a Subscription from locally built art ansible-playbook -e __local_build_enabled=true -e __deploy_from_index_enabled=true run-ci.yaml ``` +## Deployment with pre-build bundles and index + +Instead of relying on the operator-sdk to deploy from selected bundles using the "operator-sdk run bundle" utility, +you can perform a deployment using OLM and a Subscription to a locally created index image like this: + +```sh +ansible-playbook -e __local_build_enabled=false -e __deploy_from_bundles_enabled=true \ + -e __deploy_from_index_enabled=true \ + -e __service_telemetry_bundle_image_path=//stf-service-telemetry-operator-bundle: \ + -e __smart_gateway_bundle_image_path=//stf-smart-gateway-operator-bundle: \ + -e pull_secret_registry= \ + -e pull_secret_user= \ + -e pull_secret_pass= + run-ci.yaml +``` + +Since you will fetch the selected images from a bundle registry, it is required that you have all the required +access credentials for the desired registry correctly configured. Check the "Deployment with pre-build bundles" +docs above to get more information about this. + + # License Apache v2.0 diff --git a/build/stf-run-ci/tasks/create_catalog.yml b/build/stf-run-ci/tasks/create_catalog.yml index feed3b56f..828f0c905 100644 --- a/build/stf-run-ci/tasks/create_catalog.yml +++ b/build/stf-run-ci/tasks/create_catalog.yml @@ -7,11 +7,65 @@ # Updating to use stdout_lines[-1] so that any additional info that gets added to generate_bundles (e.g. for debug) doesn't break this task # Adding from_json so that the JSON output is parsed into a dictionary -- name: Create info variables from bundle generation output +- name: Create info variables from bundle generation output (local build) + when: __local_build_enabled | bool and not __deploy_from_bundles_enabled | bool ansible.builtin.set_fact: sto_bundle_info: "{{ generate_bundle_sto.stdout_lines[-1] | from_json }}" sgo_bundle_info: "{{ generate_bundle_sgo.stdout_lines[-1] | from_json }}" +- name: Create info variables from provided pre-built bundles (deploy from bundles) + when: __deploy_from_bundles_enabled | bool and not __local_build_enabled | bool + block: + - name: Get STO operator bundle info + ansible.builtin.command: oc image info {{ __service_telemetry_bundle_image_path }} + register: sto_prebuilt_image_info + + - name: Get SGO operator bundle info + ansible.builtin.command: oc image info {{ __smart_gateway_bundle_image_path }} + register: sgo_prebuilt_image_info + + - name: Get STO and SGO bundle versions + ansible.builtin.set_fact: + sto_prebuilt_bundle_version: "{{ sto_prebuilt_image_info.stdout_lines[-1] | split('=') | last }}" + sgo_prebuilt_bundle_version: "{{ sgo_prebuilt_image_info.stdout_lines[-1] | split('=') | last }}" + + - name: Set info variables from provided pre-built bundles + ansible.builtin.set_fact: + sto_bundle_info: + 'bundle_default_channel': "{{ stf_channel }}" + 'bundle_channels': "{{ stf_channel }}" + 'operator_bundle_version': "{{ sto_prebuilt_bundle_version }}" + sgo_bundle_info: + 'bundle_default_channel': "{{ stf_channel }}" + 'bundle_channels': "{{ stf_channel }}" + 'operator_bundle_version': "{{ sgo_prebuilt_bundle_version }}" + +- name: Show STO and SGO bundle info that will used in the index image + ansible.builtin.debug: + msg: + - "{{ sto_bundle_info }}" + - "{{ sgo_bundle_info }}" + +- name: Create ImageStream for STO and SGO (deploying from bundles) + when: __deploy_from_bundles_enabled | bool + block: + - name: Set correct STO and SGO bundle paths when deploying from index with pre-built bundles + ansible.builtin.set_fact: + sto_bundle_image_path: "{{ __internal_registry_path }}/{{ namespace }}/service-telemetry-operator-bundle:{{ sto_bundle_image_tag }}" + sgo_bundle_image_path: "{{ __internal_registry_path }}/{{ namespace }}/smart-gateway-operator-bundle:{{ sgo_bundle_image_tag }}" + + - name: Create ImageStream for STO + ansible.builtin.command: + cmd: | + oc import-image -n {{ namespace }} service-telemetry-operator-bundle:{{ sto_bundle_image_tag }} --from={{ __service_telemetry_bundle_image_path }} --confirm --insecure + register: sto_is + + - name: Create ImageStream for SGO + ansible.builtin.command: + cmd: | + oc import-image -n {{ namespace }} smart-gateway-operator-bundle:{{ sgo_bundle_image_tag }} --from={{ __smart_gateway_bundle_image_path }} --confirm --insecure + register: sgo_is + - name: Get the builder-dockercfg Secret name ansible.builtin.command: oc get secret -n {{ namespace }} --field-selector='type==kubernetes.io/dockercfg' -ojsonpath='{.items[?(@.metadata.annotations.kubernetes\.io/service-account\.name=="builder")].metadata.name}' register: secret_builder_dockercfg_name diff --git a/build/stf-run-ci/tasks/main.yml b/build/stf-run-ci/tasks/main.yml index 73508fedc..df29982ab 100644 --- a/build/stf-run-ci/tasks/main.yml +++ b/build/stf-run-ci/tasks/main.yml @@ -28,27 +28,6 @@ - __local_build_enabled | bool - __deploy_from_bundles_enabled | bool -- name: Fail when deploying from index image and local build disabled - ansible.builtin.fail: - msg: __deploy_from_index_enabled must also have __local_build_enabled - when: - - __deploy_from_index_enabled | bool - - not __local_build_enabled | bool - -- name: Fail when deploying from index images and deployment from bundles also requested (mutually exclusive methods) - ansible.builtin.fail: - msg: __deploy_from_index_enabled can not be used with __deploy_from_bundles_enabled - when: - - __deploy_from_index_enabled | bool - - __deploy_from_bundles_enabled | bool - -- name: Fail when disconnected deploy and other deployment options also requested - ansible.builtin.fail: - msg: __disconnected_deploy cannot be used if __deploy_from_bundles_enabled, __deploy_from_index_enabled or __local_build_enabled - when: - - __disconnected_deploy | bool - - __deploy_from_bundles_enabled | bool or __deploy_from_index_enabled | bool or __local_build_enabled | bool - - name: Get the list of nodes kubernetes.core.k8s_info: kind: Node @@ -159,7 +138,7 @@ pod-security.kubernetes.io/audit: restricted pod-security.kubernetes.io/warn: restricted -- when: __deploy_from_index_enabled | bool +- when: __deploy_from_index_enabled | bool and __local_build_enabled | bool tags: - create_bundles block: @@ -180,11 +159,13 @@ tags: - build +- when: __deploy_from_index_enabled | bool + block: - name: Create file-based catalog ansible.builtin.include_tasks: create_catalog.yml # -- deploy -- when: not __local_build_enabled | bool +- when: not __local_build_enabled | bool and not __deploy_from_index_enabled | bool block: - name: Setup Service Telemetry Framework from supplied bundle URLs ansible.builtin.include_tasks: setup_stf_from_bundles.yml @@ -206,7 +187,7 @@ name: service-telemetry-operator namespace: "{{ namespace }}" spec: - channel: unstable + channel: "{{ stf_channel }}" installPlanApproval: Automatic name: service-telemetry-operator source: service-telemetry-framework-operators