Adapt k8s to v1.19.3. (#1760)

Fixes #1651 Signed-off-by: Tao He <[email protected]> Signed-off-by: Ye Cao <[email protected]> Co-authored-by: Tao He <[email protected]>
v6d-io · Jul 19, 2024 · 1e02fce · 1e02fce
1 parent ef61fd8
commit 1e02fce
Show file tree

Hide file tree

Showing 33 changed files with 1,805 additions and 1,065 deletions.
diff --git a/docs/notes/cloud-native/deploy-kubernetes.rst b/docs/notes/cloud-native/deploy-kubernetes.rst
@@ -3,7 +3,9 @@
 Deploy on Kubernetes
 ====================
 
-Vineyard is managed by the :ref:`vineyard-operator` on Kubernetes.
+Vineyard is managed by the :ref:`vineyard-operator` on Kubernetes. If you want to use
+the Vineyard Scheduler, please make sure the kubernetes cluster is within a proper version
+range (v1.19 - v1.24).
 
 Quick start
 -----------

diff --git a/docs/notes/cloud-native/vineyard-operator.rst b/docs/notes/cloud-native/vineyard-operator.rst
@@ -461,6 +461,12 @@ on Vineyard by placing them as close as possible to their input data, thereby re
 migration costs. The Vineyard scheduler plugin is developed based on the `Kubernetes Scheduling
 Framework`_, and its overall scheduling strategy can be summarized as follows:
 
+.. note::
+
+    Vineyard Scheduler is compatible with Kubernetes versions 1.19 to 1.24. Ensure your Kubernetes 
+    cluster is within this version range for proper functionality.
+
+
 - All Vineyard workloads can only be deployed on nodes where the Vineyard daemon server is
   present.
 - If a workload does not depend on any other workload, it will be scheduled using a

diff --git a/docs/notes/developers/faq.rst b/docs/notes/developers/faq.rst
@@ -92,17 +92,27 @@ concerns, please feel free to `open an issue`_ or `post it to discussions`_.
 
 9. *What's the minimal Kubernetes version requirement for vineyard operator?*
 
-  At present, we only test the vineyard operator based on Kubernetes 1.24.0. 
-  So we highly recommend using Kubernetes 1.24.0 or above.
+  At present, the vineyard scheduler (a component of vineyard operator) is compatible with
+  Kubernetes versions 1.19 to 1.24. Ensure your Kubernetes cluster is within this version
+  range for proper functionality.
 
-10. *Why the vineyard operator can't be deployed on Kubernetes?*
+10. *Can the vineyard operator be deployed in the latest kubernetes cluster?*
+
+  The answer is yes.
+
+  Actually, the k8s compatibility of vineyard operator is bound by the vineyard scheduler.
+  You can refer the `KubeSchedulerConfiguration doc`_ to bump up the 
+  `Vineyard Scheduler KubeSchedulerConfiguration`_ to V1 and then bump up the k8s version
+  in the `go.mod`_.
+
+11. *Why the vineyard operator can't be deployed on Kubernetes?*
 
   If you use the helm to deploy the vineyard operator, you may find the vineyard operator
   can't be deployed successfully after a long time. In this case, you should check whether
   the command contains the flag `--wait`. If so, you should remove the flag `--wait` and
   try to install the operator again.
 
-11. *How to connect to the vineyard cluster deployed by the vineyard operator?*
+12. *How to connect to the vineyard cluster deployed by the vineyard operator?*
 
   There are two ways to connect to the vineyard cluster deployed by the vineyard operator:
 
@@ -112,13 +122,17 @@ concerns, please feel free to `open an issue`_ or `post it to discussions`_.
   - `Through RPC`. Connect to the vineyard cluster through the RPC service exposed by the 
     vineyard operator. You could refer to the `guide`_ for more details.
 
-12. *Is there a way to install the vineyard cluster on Kubernetes quickly?*
+13. *Is there a way to install the vineyard cluster on Kubernetes quickly?*
 
   To reduce the complexity of the installation, we provide a `command line tool`_
   to install the vineyard cluster on Kubernetes quickly.
 
+
 .. _open an issue: https://github.com/v6d-io/v6d/issues/new
 .. _post it to discussions: https://github.com/v6d-io/v6d/discussions/new
 .. _guide: ../../tutorials/kubernetes/using-vineyard-operator.rst
 .. _command line tool: ../../notes/cloud-native/vineyardctl.md
-.. _vineyard client: ../../notes/references/python-api.html#vineyard.Client
+.. _vineyard client: ../../notes/references/python-api.html#vineyard.Client
+.. _KubeSchedulerConfiguration doc: https://kubernetes.io/docs/reference/scheduling/config/
+.. _Vineyard Scheduler KubeSchedulerConfiguration: https://github.com/v6d-io/v6d/blob/main/k8s/config/scheduler/config.yaml
+.. _go.mod: https://github.com/v6d-io/v6d/blob/main/k8s/go.mod
diff --git a/docs/tutorials/data-processing/accelerate-data-sharing-in-kedro.rst b/docs/tutorials/data-processing/accelerate-data-sharing-in-kedro.rst
@@ -18,12 +18,12 @@ Prepare the Kubernetes cluster
 
 To deploy Kedro pipelines on Kubernetes, you must have a kubernetes cluster.
 
-.. tip::
+.. note::
 
-    If you already have a K8s cluster, just skip this section and continue
-    on deploying.
+    Vineyard Scheduler is compatible with Kubernetes versions 1.19 to 1.24. Ensure your Kubernetes 
+    cluster is within this version range for proper functionality.
 
-We recommend `kind v0.20.0 <https://kind.sigs.k8s.io/>`_ to create a multi-node
+We recommend `kind v0.13.0 <https://kind.sigs.k8s.io/>`_ to create a multi-node
 Kubernetes cluster on your local machine as follows:
 
 .. code:: bash
@@ -33,13 +33,13 @@ Kubernetes cluster on your local machine as follows:
     apiVersion: kind.x-k8s.io/v1alpha4
     nodes:
     - role: control-plane
-      image: kindest/node:v1.25.11
+      image: kindest/node:v1.19.3
     - role: worker
-      image: kindest/node:v1.25.11
+      image: kindest/node:v1.19.3
     - role: worker
-      image: kindest/node:v1.25.11
+      image: kindest/node:v1.19.3
     - role: worker
-      image: kindest/node:v1.25.11
+      image: kindest/node:v1.19.3
     EOF
 
 

diff --git a/docs/tutorials/kubernetes/data-sharing-with-vineyard-on-kubernetes.rst b/docs/tutorials/kubernetes/data-sharing-with-vineyard-on-kubernetes.rst
@@ -27,7 +27,7 @@ to share the vineyard data via the command `vineyardctl schedule workload`.
 Prerequisites
 -------------
 
-- A kubernetes cluster with version >= 1.25.10.
+- A kubernetes cluster >= v1.19.
 - Install the latest vineyardctl command line tool refer to `vineyardctl installation`_.
 
 Data sharing between different containers

diff --git a/...ials/kubernetes/efficient-data-sharing-in-kubeflow-with-vineyard-csi-driver.rst b/...ials/kubernetes/efficient-data-sharing-in-kubeflow-with-vineyard-csi-driver.rst
@@ -10,7 +10,7 @@ Next, we will show you how to use the Vineyard CSI Driver to speed up a kubeflow
 Prerequisites
 =============
 
-- A kubernetes cluster with version >= 1.25.10. If you don't have one by hand, you can refer to the guide `Initialize Kubernetes Cluster`_ to create one.
+- A kubernetes cluster >= 1.19. If you don't have one by hand, you can refer to the guide `Initialize Kubernetes Cluster`_ to create one.
 - Install the `Vineyardctl`_ by following the official guide.
 - Install the argo workflow cli >= 3.4.8.
 - Install the kfp package <= 1.8.0 for kubeflow **v1** or >= 2.0.1 for kubeflow **v2**.

diff --git a/go/vineyard/go.mod b/go/vineyard/go.mod
@@ -4,34 +4,55 @@ go 1.19
 
 require (
 	github.com/apache/arrow/go/v11 v11.0.0
-	github.com/go-logr/logr v1.2.3
-	github.com/go-logr/zapr v1.2.3
+	github.com/go-logr/logr v1.3.0
 	github.com/goccy/go-json v0.10.0
 	github.com/pkg/errors v0.9.1
-	github.com/stretchr/testify v1.8.0
-	go.uber.org/zap v1.24.0
+	github.com/stretchr/testify v1.8.4
+	go.uber.org/zap v1.26.0
 	golang.org/x/exp v0.0.0-20220827204233-334a2380cb91
+	sigs.k8s.io/controller-runtime v0.17.2
 )
 
 require (
 	github.com/andybalholm/brotli v1.0.4 // indirect
 	github.com/apache/thrift v0.16.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/go-logr/zapr v1.3.0 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/snappy v0.0.4 // indirect
 	github.com/google/flatbuffers v2.0.8+incompatible // indirect
+	github.com/google/gofuzz v1.2.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/klauspost/asmfmt v1.3.2 // indirect
 	github.com/klauspost/compress v1.16.6 // indirect
 	github.com/klauspost/cpuid/v2 v2.2.5 // indirect
 	github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
 	github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/pierrec/lz4/v4 v4.1.15 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
 	github.com/zeebo/xxh3 v1.0.2 // indirect
-	go.uber.org/atomic v1.10.0 // indirect
-	go.uber.org/multierr v1.9.0 // indirect
-	golang.org/x/mod v0.6.0 // indirect
-	golang.org/x/sys v0.9.0 // indirect
-	golang.org/x/tools v0.2.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	golang.org/x/mod v0.14.0 // indirect
+	golang.org/x/net v0.19.0 // indirect
+	golang.org/x/sys v0.16.0 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	golang.org/x/tools v0.16.1 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
+	k8s.io/apimachinery v0.29.0 // indirect
+	k8s.io/klog/v2 v2.110.1 // indirect
+	k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
+)
+
+replace (
+	github.com/go-logr/logr => github.com/go-logr/logr v0.3.0
+	github.com/go-logr/zapr => github.com/go-logr/zapr v0.2.0
+	k8s.io/klog/v2 => k8s.io/klog/v2 v2.2.0
+	sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.7.2
 )