Add support for building local kind Node image #6399

hjiajing · 2024-06-05T02:15:29Z

When triggering Manually upstream conformance test on Linux, if the input kubernetes version has no corresponding kind node image, the job will build a kind Node image with the Kubernetes source code, and create Kind Cluster with local Node image.

luolanzone · 2024-06-05T05:59:47Z

.github/workflows/conformance.yml

          curl -Lo ./kind https://github.com/kubernetes-sigs/kind/releases/download/${KIND_VERSION}/kind-$(uname)-amd64
          chmod +x ./kind
          sudo mv kind /usr/local/bin
+      - name: Prepare local Machine for Conformance test
+        run: |
+          if [ "${{ inputs.k8s-version }}" != "latest" ]; then


We should support both branch and tags like 'v1.30.0-rc.0', 'v1.31.0-alpha.0' from K8s repo. The purpose to build local Kind image is to run comformance test against the latest one or different release candidates.

Got it, replaced this if condition with a regex.

luolanzone · 2024-06-05T06:17:11Z

ci/kind/kind-setup.sh

+# When k8s-version is "latest", build node image with the latest k8s version
+if [[ "$K8S_VERSION" == "latest" ]]; then
+    echo "=== Building kind node image with the latest k8s version ==="
+    kind_node_image="kindest/node:latest"


Suggested change

# When k8s-version is "latest", build node image with the latest k8s version

if [[ "$K8S_VERSION" == "latest" ]]; then

echo "=== Building kind node image with the latest k8s version ==="

kind_node_image="kindest/node:latest"

# When k8s-version is "latest", build node image with the latest K8s version

if [[ "$K8S_VERSION" == "latest" ]]; then

echo "=== Building Kind node image with the latest K8s version ==="

kind_node_image="kindest/node:latest"

ci/run-k8s-e2e-tests.sh

.github/workflows/conformance.yml

rajnkamr · 2024-06-07T07:56:33Z

.github/workflows/conformance.yml

+          conformance_image_options=()
+          # If the K8s version is 'latest', we need to find the latest conformance image.
+          if [ ${{ inputs.k8s-version == 'latest' }} ]; then
+            conformance_image_options+=(--conformance-image "registry.k8s.io/conformance:$(skopeo list-tags docker://registry.k8s.io/conformance | jq -r '.Tags[]' | sort -V | tail -n 1)")


Suggested change

conformance_image_options+=(--conformance-image "registry.k8s.io/conformance:$(skopeo list-tags docker://registry.k8s.io/conformance | jq -r '.Tags[]' | sort -V | tail -n 1)")

latest_tag=$(skopeo list-tags docker://registry.k8s.io/conformance | jq -r '.Tags[]' | sort -V | tail -n 1)

# Add the conformance image option

conformance_image_options+=(--conformance-image "registry.k8s.io/conformance:$latest_tag")

Changed, thanks for then comment.

luolanzone · 2024-06-11T06:41:17Z

ci/kind/kind-setup.sh

@@ -665,6 +666,14 @@ if version_lt "$kind_version" "0.12.0" && [[ "$KUBE_PROXY_MODE" == "none" ]]; th
    exit 1
 fi

+# When k8s-version is not vx.y.z, we need to build the K8s node image in local.
+if [[ ! $K8S_VERSION =~ ^(v)?[0-9]+\.[0-9]+\.[0-9]+$ ]]; then


Does it mean we can only support local K8s node image building when the K8s version like v1.30.0-alpha, v1.31.0-rc1?
How do we support a future version like v1.31.0 when the Kind haven't supported it yet in its release?
For example, Kind 0.23 only supports following images by default:
https://github.com/kubernetes-sigs/kind/releases/tag/v0.23.0

Images pre-built for this release: v1.30.0: kindest/node:v1.30.0@sha256:047357ac0cfea04663786a612ba1eaba9702bef25227a794b52890dd8bcd692e v1.29.4: kindest/node:v1.29.4@sha256:3abb816a5b1061fb15c6e9e60856ec40d56b7b52bcea5f5f1350bc6e2320b6f8 v1.28.9: kindest/node:v1.28.9@sha256:dca54bc6a6079dd34699d53d7d4ffa2e853e46a20cd12d619a09207e35300bd0 v1.27.13: kindest/node:v1.27.13@sha256:17439fa5b32290e3ead39ead1250dca1d822d94a10d26f1981756cd51b24b9d8 v1.26.15: kindest/node:v1.26.15@sha256:84333e26cae1d70361bb7339efb568df1871419f2019c80f9a12b7e2d485fe19 v1.25.16: kindest/node:v1.25.16@sha256:5da57dfc290ac3599e775e63b8b6c49c0c85d3fec771cd7d55b45fae14b38d3b

Yes, we cannot guarantee Kind will release the latest K8s Node, so I used docker pull command to check whether the Node image exists or not. If not, then we will try to build image in local.

luolanzone

LGTM overall, two nits

.github/workflows/conformance.yml

ci/kind/kind-setup.sh

luolanzone · 2024-06-13T03:33:02Z

@hjiajing please update the PR details info.

tnqn · 2024-07-05T08:43:21Z

ci/kind/kind-setup.sh

+    git clone https://github.com/kubernetes/kubernetes.git ${THIS_DIR}/kubernetes
+    tag=${K8S_VERSION}
+    if [[ $K8S_VERSION == "latest" ]]; then
+        tag="master"
+    fi
+    git -C ${THIS_DIR}/kubernetes checkout $tag


Suggested change

git clone https://github.com/kubernetes/kubernetes.git ${THIS_DIR}/kubernetes

tag=${K8S_VERSION}

if [[ $K8S_VERSION == "latest" ]]; then

tag="master"

fi

git -C ${THIS_DIR}/kubernetes checkout $tag

tag=${K8S_VERSION}

if [[ $K8S_VERSION == "latest" ]]; then

tag="master"

fi

git clone --branch $tag --depth 1 https://github.com/kubernetes/kubernetes.git ${THIS_DIR}/kubernetes

# time git clone https://github.com/kubernetes/kubernetes.git real 1m22.043s user 4m29.718s sys 0m39.540s # time git clone --branch v1.30.0 --depth 1 https://github.com/kubernetes/kubernetes.git kubernetes-depth real 0m11.055s user 0m5.463s sys 0m1.935s # du -sh * 1.4G kubernetes 387M kubernetes-depth

i tried use --branch --depth 1, but I found that when kind tries to build images from master branch, it cannot found workspace information as follows:

hjiajing@ubuntu-dev (master) [130] % ./hack/print-workspace-status.sh ~/kubernetes STABLE_BUILD_GIT_COMMIT 3125877de0aefbb642833beaa3a4c189d833e797 STABLE_BUILD_SCM_STATUS clean STABLE_BUILD_SCM_REVISION STABLE_BUILD_MAJOR_VERSION STABLE_BUILD_MINOR_VERSION STABLE_DOCKER_TAG STABLE_DOCKER_REGISTRY registry.k8s.io STABLE_DOCKER_PUSH_REGISTRY staging-k8s.gcr.io gitCommit 3125877de0aefbb642833beaa3a4c189d833e797 gitTreeState clean gitVersion gitMajor gitMinor buildDate 2024-07-09T02:48:57Z

But since we decide not to support master branch, we can use --branch --depth now.

ci/kind/kind-setup.sh

.github/workflows/conformance.yml

jianjuns

A nit

jianjuns · 2024-07-05T20:06:45Z

ci/kind/kind-setup.sh

@@ -665,6 +666,21 @@ if version_lt "$kind_version" "0.12.0" && [[ "$KUBE_PROXY_MODE" == "none" ]]; th
    exit 1
 fi

+# Build Kind node image with the specified K8s version if it doesn't exist in local and remote registry neither


Remove "neither", or change to "in either the local or remote registry".

According to Quan's comment, this part is moved to conformance.yml, this comment is removed.

tnqn

LGTM

.github/workflows/conformance.yml

tnqn · 2024-07-09T09:55:07Z

The PR description needs update

When trigger Manually upstream conformance test on Linux, if the input kubernetes version has no corresponding kind Node image, the job will build kind Node image with the Kubernetes source code. Signed-off-by: Jiajing Hu <[email protected]>

hjiajing · 2024-07-10T15:11:10Z

The PR description needs update

Updated, thanks for the reminder.

tnqn · 2024-07-11T03:06:21Z

/skip-all

tnqn · 2024-07-11T03:07:16Z

/skip-all

hjiajing requested a review from luolanzone June 5, 2024 02:15

luolanzone reviewed Jun 5, 2024

View reviewed changes

.github/workflows/conformance.yml Outdated Show resolved Hide resolved

hjiajing force-pushed the build-k8s branch from aa315ab to 8649906 Compare June 7, 2024 07:01

rajnkamr reviewed Jun 7, 2024

View reviewed changes

luolanzone reviewed Jun 11, 2024

View reviewed changes

hjiajing force-pushed the build-k8s branch from 8649906 to d19423e Compare June 12, 2024 05:14

luolanzone reviewed Jun 13, 2024

View reviewed changes

.github/workflows/conformance.yml Outdated Show resolved Hide resolved

.github/workflows/conformance.yml Outdated Show resolved Hide resolved

ci/kind/kind-setup.sh Outdated Show resolved Hide resolved

hjiajing force-pushed the build-k8s branch from d19423e to 84570a1 Compare June 20, 2024 15:28

luolanzone requested a review from tnqn June 26, 2024 06:43

luolanzone added this to the Antrea v2.1 release milestone Jul 4, 2024

tnqn reviewed Jul 5, 2024

View reviewed changes

jianjuns reviewed Jul 5, 2024

View reviewed changes

hjiajing force-pushed the build-k8s branch from 84570a1 to c074787 Compare July 9, 2024 07:50

tnqn previously approved these changes Jul 9, 2024

View reviewed changes

tnqn reviewed Jul 9, 2024

View reviewed changes

.github/workflows/conformance.yml Outdated Show resolved Hide resolved

Support building local kind Node image

44f6c3d

When trigger Manually upstream conformance test on Linux, if the input kubernetes version has no corresponding kind Node image, the job will build kind Node image with the Kubernetes source code. Signed-off-by: Jiajing Hu <[email protected]>

hjiajing dismissed tnqn’s stale review via 44f6c3d July 10, 2024 15:09

hjiajing force-pushed the build-k8s branch from c074787 to 44f6c3d Compare July 10, 2024 15:09

tnqn approved these changes Jul 10, 2024

View reviewed changes

tnqn merged commit 42a0aaa into antrea-io:main Jul 11, 2024
54 of 58 checks passed

luolanzone mentioned this pull request Jul 12, 2024

Add local built K8s images support in conformance test on Kind workflow #6248

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for building local kind Node image #6399

Add support for building local kind Node image #6399

hjiajing commented Jun 5, 2024 •

edited

Loading

luolanzone Jun 5, 2024

hjiajing Jun 7, 2024

luolanzone Jun 5, 2024

hjiajing Jun 7, 2024

rajnkamr Jun 7, 2024

hjiajing Jun 12, 2024

luolanzone Jun 11, 2024 •

edited

Loading

hjiajing Jun 12, 2024

luolanzone left a comment

luolanzone commented Jun 13, 2024

tnqn Jul 5, 2024

hjiajing Jul 9, 2024

jianjuns left a comment

jianjuns Jul 5, 2024

hjiajing Jul 9, 2024

tnqn left a comment

tnqn commented Jul 9, 2024

hjiajing commented Jul 10, 2024

tnqn commented Jul 11, 2024

tnqn commented Jul 11, 2024

-            conformance_image_options+=(--conformance-image "registry.k8s.io/conformance:$(skopeo list-tags docker://registry.k8s.io/conformance | jq -r '.Tags[]' | sort -V | tail -n 1)")
+            latest_tag=$(skopeo list-tags docker://registry.k8s.io/conformance | jq -r '.Tags[]' | sort -V | tail -n 1)
+  # Add the conformance image option
+  conformance_image_options+=(--conformance-image "registry.k8s.io/conformance:$latest_tag")

Add support for building local kind Node image #6399

Add support for building local kind Node image #6399

Conversation

hjiajing commented Jun 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luolanzone Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luolanzone left a comment

Choose a reason for hiding this comment

luolanzone commented Jun 13, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jianjuns left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tnqn left a comment

Choose a reason for hiding this comment

tnqn commented Jul 9, 2024

hjiajing commented Jul 10, 2024

tnqn commented Jul 11, 2024

tnqn commented Jul 11, 2024

hjiajing commented Jun 5, 2024 •

edited

Loading

luolanzone Jun 11, 2024 •

edited

Loading