OSDOCS-4112: Multi-arch support for Operator projects

_topic_maps/_topic_map.yml

@@ -1839,6 +1839,8 @@ Topics:
     File: osdk-monitoring-prometheus
   - Name: Configuring leader election
     File: osdk-leader-election
+  - Name: Configuring support for multiple platforms
+    File: osdk-multi-arch-support
   - Name: Object pruning utility
     File: osdk-pruning-utility
   - Name: Migrating package manifest projects to bundle format

modules/osdk-multi-arch-building-images.adoc

@@ -0,0 +1,105 @@
+// Module included in the following assemblies:
+//
+// * operators/operator_sdk/osdk-multi-arch-support.adoc
+
+:_content-type: PROCEDURE
+[id="osdk-multi-arch-building-images_{context}"]
+= Building a manifest list of the platforms your Operator supports
+
+You can use the `make docker-buildx` command to build a manifest list of the platforms supported by your Operator and operands. A manifest list references specific image manifests for one or more architectures. An image manifest specifies the platforms that an image supports.
+
+For more information, see link:https://specs.opencontainers.org/image-spec/image-index[OpenContainers Image Index Spec] or link:https://docs.docker.com/registry/spec/manifest-v2-2/#manifest-list[Image Manifest v2, Schema 2].
+
+[IMPORTANT]
+====
+If your Operator project deploys an application or other workload resources, the following procedure assumes the application's multi-platform images are built during the application release process.
+====
+
+.Prerequisites
+
+* An Operator project built using the Operator SDK version {osdk_ver} or later
+* Docker installed
+
+.Procedure
+
+. Inspect the image manifests of your Operator and operands to find which platforms your Operator project can support. Run the following command to inspect an image manifest:
++
+[source,terminal]
+----
+$ docker manifest inspect <image_manifest> <1>
+----
+<1> Specifies an image manifest, such as `redhat/ubi9:latest`.
++
+The platforms that your Operator and operands mutually support determine the platform compatibility of your Operator project.
++
+.Example output
+[source,json]
+----
+{
+    "manifests": [
+        {
+            "digest": "sha256:c0669ef34cdc14332c0f1ab0c2c01acb91d96014b172f1a76f3a39e63d1f0bda",
+            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
+            "platform": {
+                "architecture": "amd64",
+                "os": "linux"
+            },
+            "size": 528
+        },
+...
+        {
+            "digest": "sha256:30e6d35703c578ee703230b9dc87ada2ba958c1928615ac8a674fcbbcbb0f281",
+            "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
+            "platform": {
+                "architecture": "arm64",
+                "os": "linux",
+                "variant": "v8"
+            },
+            "size": 528
+        },
+...
+----
+
+. If the previous command does not output platform information, then the specified base image might be a single image instead of an image manifest. You can find which architectures an image supports by running the following command:
++
+[source,terminal]
+----
+$ docker inspect <image>
+----
+
+. For Go-based Operator projects, the Operator SDK explicitly references the `amd64` architecture in your project's Dockerfile. Make the following change
+to your Dockerfile to set an environment variable to the value specified by the platform flag:
++
+.Example Dockerfile
+[source,docker]
+----
+FROM golang:1.19 as builder
+ARG TARGETOS
+ARG TARGETARCH
+...
+RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager main.go <1>
+----
+<1> Change the `GOARCH` field from `amd64` to `$TARGETARCH`.
+
+. Your Operator project's makefile defines the `PLATFORMS` environment variable. If your Operator's images do not support all of the platforms set by default, edit the variable to specify the supported platforms. The following example defines the supported platforms as `linux/arm64` and `linux/amd64`:
++
+.Example makefile
+[source,make]
+----
+# ...
+PLATFORMS ?= linux/arm64,linux/amd64 <1>
+.PHONY: docker-buildx
+# ...
+----
++
+<1> The following `PLATFORMS` values are set by default: `linux/arm64`, `linux/amd64`, `linux/s390x`, and `linux/ppc64le`.
++
+When you run the `make docker buildx` command to generate a manifest list, the Operator SDK creates an image manifest for each of the platforms specified by the `PLATFORMS` variable.
+
+. Run the following command from your Operator project directory to build your manager image. Running the command builds a manager image with multi-platform support and pushes the manifest list to your registry.
++
+[source,terminal]
+----
+$ make docker-buildx \
+  IMG=<image_registry>/<organization_name>/<repository_name>:<version_or_sha>
+----

modules/osdk-multi-arch-node-affinity.adoc

@@ -0,0 +1,13 @@
+// Module included in the following assemblies:
+//
+// * operators/operator_sdk/osdk-multi-arch-support.adoc
+
+:_content-type: CONCEPT
+[id="osdk-multi-arch-node-affinity_{context}"]
+= About node affinity rules for multi-architecture compute machines and Operator workloads
+
+You must set node affinity rules to ensure your Operator workloads can run on multi-architecture compute machines. Node affinity is a set of rules used by the scheduler to define a pod's placement. Setting node affinity rules ensures your Operator's workloads are scheduled to compute machines with compatible architectures.
+
+If your Operator performs better on particular architectures, you can set preferred node affinity rules to schedule pods to machines with the specified architectures.
+
+For more information, see "About clusters with multi-architecture compute machines" and "Controlling pod placement on nodes using node affinity rules".

modules/osdk-multi-arch-node-preference.adoc

@@ -0,0 +1,63 @@
+// Module included in the following assemblies:
+//
+// * operators/operator_sdk/osdk-multi-arch-support.adoc
+
+:_content-type: PROCEDURE
+[id="osdk-multi-arch-node-preference_{context}"]
+= Using preferred node affinity rules to configure support for multi-architecture compute machines for Operator projects
+
+If your Operator performs better on particular architectures, you can configure preferred node affinity rules to schedule pods to nodes to the specified architectures.
+
+.Prerequisites
+
+* An Operator project created or maintained with Operator SDK {osdk_ver} or later.
+* A manifest list defining the platforms your Operator supports.
+* Required node affinity rules are set for your Operator project.
+
+.Procedure
+
+. Search your Operator project for Kubernetes manifests that define pod spec and pod template spec objects.
++
+.Example Kubernetes manifest
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: s1
+spec:
+  containers:
+    - name: <container_name>
+      image: docker.io/<org>/<image_name>
+----
+
+. Set your Operator's preferred node affinity rules in the Kubernetes manifests that define pod spec and pod template spec objects, similar to the following example:
++
+.Example Kubernetes manifest
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: s1
+spec:
+  containers:
+    - name: <container_name>
+      image: docker.io/<org>/<image_name>
+  affinity:
+      nodeAffinity:
+        preferredDuringSchedulingIgnoredDuringExecution: <1>
+          - preference:
+            matchExpressions: <2>
+              - key: kubernetes.io/arch <3>
+                operator: In <4>
+                values:
+                - amd64
+                - arm64
+            weight: 90 <5>
+----
+<1> Defines a preferred rule.
+<2> If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node only if all `matchExpressions` are satisfied.
+<3> Specifies the architectures defined in the manifest list.
+<4> Specifies an `operator`. The Operator can be `In`, `NotIn`,  `Exists`, or `DoesNotExist`. For example, use the value of `In` to require the label to be in the node.
+<5> Specifies a weight for the node, valid values are `1`-`100`. The node with highest weight is preferred.

modules/osdk-multi-arch-node-reqs.adoc

@@ -0,0 +1,130 @@
+// Module included in the following assemblies:
+//
+// * operators/operator_sdk/osdk-multi-arch-support.adoc
+
+:_content-type: PROCEDURE
+[id="osdk-multi-arch-node-reqs_{context}"]
+= Using required node affinity rules to support multi-architecture compute machines for Operator projects
+
+If you want your Operator to support multi-architecture compute machines, you must define your Operator's required node affinity rules.
+
+.Prerequisites
+
+* An Operator project created or maintained with Operator SDK {osdk_ver} or later.
+* A manifest list defining the platforms your Operator supports.
+
+.Procedure
+
+. Search your Operator project for Kubernetes manifests that define pod spec and pod template spec objects.
++
+[IMPORTANT]
+====
+Because object type names are not declared in YAML files, look for the mandatory `containers` field in your Kubernetes manifests. The `containers` field is required when specifying both pod spec and pod template spec objects.
+
+You must set node affinity rules in all Kubernetes manifests that define a pod spec or pod template spec, including objects such as `Pod`, `Deployment`, `DaemonSet`, and `StatefulSet`.
+====
++
+.Example Kubernetes manifest
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: s1
+spec:
+  containers:
+    - name: <container_name>
+      image: docker.io/<org>/<image_name>
+----
+
+. Set the required node affinity rules in the Kubernetes manifests that define pod spec and pod template spec objects, similar to the following example:
++
+.Example Kubernetes manifest
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: s1
+spec:
+  containers:
+    - name: <container_name>
+      image: docker.io/<org>/<image_name>
+  affinity:
+    nodeAffinity:
+      requiredDuringSchedulingIgnoredDuringExecution: <1>
+        nodeSelectorTerms: <2>
+        - matchExpressions: <3>
+          - key: kubernetes.io/arch <4>
+            operator: In
+            values:
+            - amd64
+            - arm64
+            - ppc64le
+            - s390x
+          - key: kubernetes.io/os <5>
+            operator: In
+            values:
+                - linux
+----
+<1> Defines a required rule.
+<2> If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node if one of the `nodeSelectorTerms` is satisfied.
+<3> If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node only if all `matchExpressions` are satisfied.
+<4> Specifies the architectures defined in the manifest list.
+<5> Specifies the operating systems defined in the manifest list.
+
+. Go-based Operator projects that use dynamically created workloads might embed pod spec and pod template spec objects in the Operator's logic.
++
+If your project embeds pod spec or pod template spec objects in the Operator's logic, edit your Operator's logic similar to the following example. The following example shows how to update a `PodSpec` object by using the Go API:
++
+[source,go]
+----
+Template: corev1.PodTemplateSpec{
+    ...
+    Spec: corev1.PodSpec{
+        Affinity: &corev1.Affinity{
+            NodeAffinity: &corev1.NodeAffinity{
+                RequiredDuringSchedulingIgnoredDuringExecution: &corev1.NodeSelector{
+                    NodeSelectorTerms: []corev1.NodeSelectorTerm{
+                        {
+                            MatchExpressions: []corev1.NodeSelectorRequirement{
+                                {
+                                    Key:      "kubernetes.io/arch",
+                                    Operator: "In",
+                                    Values:   []string{"amd64","arm64","ppc64le","s390x"},
+                                },
+                                {
+                                    Key:      "kubernetes.io/os",
+                                    Operator: "In",
+                                    Values:   []string{"linux"},
+                                },
+                            },
+                        },
+                    },
+                },
+            },
+        },
+        SecurityContext: &corev1.PodSecurityContext{
+            ...
+        },
+        Containers: []corev1.Container{{
+            ...
+        }},
+    },
+----
++
+where:
+
+`RequiredDuringSchedulingIgnoredDuringExecution`:: Defines a required rule.
+`NodeSelectorTerms`:: If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node if one of the `nodeSelectorTerms` is satisfied.
+`MatchExpressions`:: If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node only if all `matchExpressions` are satisfied.
+`kubernetes.io/arch`:: Specifies the architectures defined in the manifest list.
+`kubernetes.io/os`:: Specifies the operating systems defined in the manifest list.
+
+[WARNING]
+====
+If you do not set node affinity rules and a container is scheduled to a compute machine with an incompatible architecture, the pod fails and triggers one of the following events:
+
+`CrashLoopBackOff`:: Occurs when an image manifest's entry point fails to run and an `exec format error` message is printed in the logs.
+`ImagePullBackOff`:: Occurs when a manifest list does not include a manifest for the architecture where a pod is scheduled or the node affinity terms are set to the wrong values.
+====

modules/osdk-multi-arch-validate.adoc

@@ -0,0 +1,43 @@
+// Module included in the following assemblies:
+//
+// * operators/operator_sdk/osdk-multi-arch-support.adoc
+
+:_content-type: PROCEDURE
+[id="osdk-multi-arch-validate_{context}"]
+= Validating your Operator's multi-platform readiness
+
+You can validate your Operator's multi-platform readiness by running the `bundle validate` command. The command verifies that your Operator project meets the following conditions:
+
+* Your Operator's manager image supports the platforms labeled in the cluster service version (CSV) file.
+* Your Operator's CSV has labels for the supported platforms for Operator Lifecycle Manager (OLM) and OperatorHub.
+
+.Procedure
+
+* Run the following command to validate your Operator project for multiple architecture readiness:
++
+[source,terminal]
+----
+$ operator-sdk bundle validate ./bundle \
+  --select-optional name=multiarch
+----
++
+.Example validation message
+[source,text]
+----
+INFO[0020] All validation tests have completed successfully
+----
++
+.Example error message for missing CSV labels in the manager image
+[source,text]
+----
+ERRO[0016] Error: Value test-operator.v0.0.1: not all images specified are providing the support described via the CSV labels. Note that (SO.architecture): (linux.ppc64le) was not found for the image(s) [quay.io/example-org/test-operator:v1alpha1]
+ERRO[0016] Error: Value test-operator.v0.0.1: not all images specified are providing the support described via the CSV labels. Note that (SO.architecture): (linux.s390x) was not found for the image(s) [quay.io/example-org/test-operator:v1alpha1]
+ERRO[0016] Error: Value test-operator.v0.0.1: not all images specified are providing the support described via the CSV labels. Note that (SO.architecture): (linux.amd64) was not found for the image(s) [quay.io/example-org/test-operator:v1alpha1]
+ERRO[0016] Error: Value test-operator.v0.0.1: not all images specified are providing the support described via the CSV labels. Note that (SO.architecture): (linux.arm64) was not found for the image(s) [quay.io/example-org/test-operator:v1alpha1]
+----
++
+.Example error message for missing OperatorHub flags
+[source,text]
+----
+WARN[0014] Warning: Value test-operator.v0.0.1: check if the CSV is missing the label (operatorframework.io/arch.<value>) for the Arch(s): ["amd64" "arm64" "ppc64le" "s390x"]. Be aware that your Operator manager image ["quay.io/example-org/test-operator:v1alpha1"] provides this support. Thus, it is very likely that you want to provide it and if you support more than amd64 architectures, you MUST,use the required labels for all which are supported.Otherwise, your solution cannot be listed on the cluster for these architectures
+----

operators/operator_sdk/osdk-bundle-validate.adoc

@@ -17,3 +17,12 @@ include::modules/osdk-bundle-validate-tests.adoc[leveloffset=+1]
 * xref:../../operators/understanding/olm-packaging-format.adoc#olm-bundle-format_olm-packaging-format[Bundle format]
 
 include::modules/osdk-bundle-validate-run.adoc[leveloffset=+1]
+
+ifndef::openshift-rosa,openshift-dedicated[]
+include::modules/osdk-multi-arch-validate.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+.Additional resources
+
+* xref:../../operators/operator_sdk/osdk-multi-arch-support.adoc#osdk-multi-platform-support[Configuring Operator projects for multi-platform support]
+endif::[]