some more examples and docs ; autoload k3p.yaml if found in cwd

tinyzimmer · Dec 14, 2020 · 0ac43b3 · 0ac43b3
1 parent fe014ca
commit 0ac43b3
Show file tree

Hide file tree

Showing 18 changed files with 394 additions and 28 deletions.
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # k3p
 
-A `k3s` packager and installer, primarily intended for airgapped deployments
+A `k3s` packager and installer, originally and primarily intended for air-gapped deployments, but could see use cases elsewhere.
 
 For documentation on `k3p` usage, see the [command docs here](doc/k3p.md).
 
@@ -12,7 +12,9 @@ For documentation on `k3p` usage, see the [command docs here](doc/k3p.md).
 
 ## Quickstart
 
-Will publish releases via actions in the future. For now, on a system with `git` and `go` installed.
+First download a binary for your system from the [releases](https://github.com/tinyzimmer/k3p/releases) page.
+
+Or, to build from source, on a system with, `make`, `git`, and `go` installed.
 
 ```bash
 git clone https://github.com/tinyzimmer/k3p
@@ -97,4 +99,5 @@ whoami-5dc4dd9cdf-qvvnz   1/1     Running   0          32s
 ```
 
 For further information on adding worker nodes and/or setting up HA, you can view the command documentation, 
-however more complete documentation will come in the future in the form of [examples](examples/).
+however more complete documentation will come in the future in the form of [examples](examples/) and other docs.
+There are already a few simple examples that you can use to get a general understanding of the workflow.
diff --git a/doc/k3p_build.md b/doc/k3p_build.md
@@ -11,7 +11,7 @@ k3p build [flags]
 ```
   -a, --arch string             The architecture to package the distribution for. Only (amd64, arm, and arm64 are supported) (default "amd64")
   -C, --channel string          The release channel to retrieve the version of k3s from (default "stable")
-  -c, --config string           An optional config file providing variables to be used at installation
+  -c, --config string           An optional file providing variables and other configurations to be used at installation, if a k3p.yaml in the current directory exists it will be used automatically
   -E, --eula string             A file containing an End User License Agreement to display to the user upon installing the package
   -e, --exclude strings         Directories to exclude when reading the manifest directory
       --exclude-images          Don't include container images with the final archive

diff --git a/examples/docker/README.md b/examples/docker/README.md
@@ -0,0 +1,120 @@
+# Playing with Docker
+
+This directory goes into more detail on smoke testing packages with docker. 
+If you have used [`k3d`](https://github.com/rancher/k3d) in the past most of this will be familiar to you.
+
+To start off, build the package in this directory (for the purpose of these examples we'll exclude images from the archive):
+
+```bash
+# Build the package and give it a unique name
+$ k3p build --exclude-images --name=k3p-docker
+
+2020/12/14 10:09:59  [INFO]     Building package "k3p-docker"
+2020/12/14 10:09:59  [INFO]     Detecting latest k3s version for channel stable
+2020/12/14 10:10:00  [INFO]     Latest k3s version is v1.19.4+k3s1
+2020/12/14 10:10:00  [INFO]     Packaging distribution for version "v1.19.4+k3s1" using "amd64" architecture
+2020/12/14 10:10:00  [INFO]     Downloading core k3s components
+2020/12/14 10:10:00  [INFO]     Fetching checksums...
+2020/12/14 10:10:00  [INFO]     Fetching k3s install script...
+2020/12/14 10:10:00  [INFO]     Fetching k3s binary...
+2020/12/14 10:10:00  [INFO]     Skipping bundling k3s airgap images with the package
+2020/12/14 10:10:00  [INFO]     Validating checksums...
+2020/12/14 10:10:00  [INFO]     Searching "/home/tinyzimmer/devel/k3p/examples/docker" for kubernetes manifests to include in the archive
+2020/12/14 10:10:00  [INFO]     Detected kubernetes manifest: "/home/tinyzimmer/devel/k3p/examples/docker/whoami.yaml"
+2020/12/14 10:10:00  [INFO]     Skipping bundling container images with the package
+2020/12/14 10:10:00  [INFO]     Writing package metadata
+2020/12/14 10:10:00  [INFO]     Archiving version "latest" of "k3p-docker" to "/home/tinyzimmer/devel/k3p/examples/docker/package.tar"
+```
+
+To install this package to a simple single node cluster running in docker you can do the following:
+
+```bash
+# --write-kubeconfig is optional and will extract the kubeconfig once the server is up
+# otherwise instructions are printed for fetching it directly from the container
+$ k3p install package.tar --docker --write-kubeconfig kubeconfig.yaml
+2020/12/14 10:11:13  [INFO]     Loading the archive
+2020/12/14 10:11:13  [INFO]     Creating docker network k3p-docker
+2020/12/14 10:11:13  [INFO]     Creating docker volume k3p-docker-server-0
+2020/12/14 10:11:14  [INFO]     Copying the archive to the rancher installation directory
+2020/12/14 10:11:14  [INFO]     Installing binaries to /usr/local/bin
+2020/12/14 10:11:14  [INFO]     Installing scripts to /usr/local/bin/k3p-scripts
+2020/12/14 10:11:14  [INFO]     Installing manifests to /var/lib/rancher/k3s/server/manifests
+2020/12/14 10:11:14  [INFO]     Running k3s installation script
+2020/12/14 10:11:14  [INFO]     Starting k3s docker node k3p-docker-server-0
+2020/12/14 10:11:15  [INFO]     Starting k3s docker node k3p-docker-serverlb
+2020/12/14 10:11:15  [INFO]     Waiting for server to write the admin kubeconfig
+2020/12/14 10:11:17  [INFO]     Writing the kubeconfig to "kubeconfig.yaml"
+2020/12/14 10:11:17  [INFO]     The cluster has been installed
+2020/12/14 10:11:17  [INFO]     You can view the cluster by running `kubectl --kubeconfig kubeconfig.yaml cluster-info`
+
+$ kubectl --kubeconfig kubeconfig.yaml cluster-info
+Kubernetes master is running at https://127.0.0.1:6443
+CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
+Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy
+
+To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
+
+$ kubectl --kubeconfig kubeconfig.yaml get pod
+NAME                      READY   STATUS    RESTARTS   AGE
+whoami-5db874f58d-dcx48   1/1     Running   0          54s
+
+# To remove the cluster when you are done
+$ k3p uninstall --name=k3p-docker   # The --name flag supports tab completion
+2020/12/14 10:13:53  [INFO]     Removing docker cluster k3p-docker
+2020/12/14 10:13:53  [INFO]     Removing docker container and volumes for k3p-docker-serverlb
+2020/12/14 10:13:53  [INFO]     Removing docker container and volumes for k3p-docker-server-0
+2020/12/14 10:13:54  [INFO]     Removing docker network k3p-docker
+```
+
+You can specify server/agent count and configurations also (this is the same as for a regular install)
+
+```bash
+$ k3p install package.tar --docker --write-kubeconfig kubeconfig.yaml \
+    --servers 3 --agents 3 \             # Specify number of server and agent nodes
+    --k3s-server-arg="--disable=traefik" # can be specified multiple times, there is also an agent equivalent
+
+# ...
+# ...
+
+$ kubectl --kubeconfig kubeconfig.yaml get node
+NAME                  STATUS   ROLES         AGE   VERSION
+k3p-docker-agent-0    Ready    worker        47s   v1.19.4+k3s1
+k3p-docker-agent-1    Ready    worker        50s   v1.19.4+k3s1
+k3p-docker-agent-2    Ready    worker        49s   v1.19.4+k3s1
+k3p-docker-server-0   Ready    etcd,master   55s   v1.19.4+k3s1
+k3p-docker-server-1   Ready    etcd,master   24s   v1.19.4+k3s1
+k3p-docker-server-2   Ready    etcd,master   44s   v1.19.4+k3s1
+
+$ kubectl --kubeconfig kubeconfig.yaml get pod -A
+NAMESPACE     NAME                                     READY   STATUS    RESTARTS   AGE
+default       whoami-5db874f58d-xtgrl                  1/1     Running   0          2m42s
+kube-system   coredns-66c464876b-sr4fw                 1/1     Running   0          2m42s
+kube-system   local-path-provisioner-7ff9579c6-5wpnq   1/1     Running   0          2m42s
+kube-system   metrics-server-7b4f8b595-rmwl5           1/1     Running   0          2m42s
+```
+
+Forwarding ports to specific nodes in the cluster works the same as `k3d`
+
+```bash
+$ k3p install package.tar --docker \
+    --publish 8080:80@loadbalancer \  # Forward 8080 on the local machine to 80 on the LoadBalancer
+    --publish 8081:80@server[0]       # Forward 8081 on the local machine to 80 on the first server instance
+
+2020/12/14 10:29:10  [INFO]     Loading the archive
+2020/12/14 10:29:10  [INFO]     Creating docker network k3p-docker
+2020/12/14 10:29:10  [INFO]     Creating docker volume k3p-docker-server-0
+2020/12/14 10:29:10  [INFO]     Copying the archive to the rancher installation directory
+2020/12/14 10:29:10  [INFO]     Installing binaries to /usr/local/bin
+2020/12/14 10:29:10  [INFO]     Installing scripts to /usr/local/bin/k3p-scripts
+2020/12/14 10:29:10  [INFO]     Installing manifests to /var/lib/rancher/k3s/server/manifests
+2020/12/14 10:29:11  [INFO]     Running k3s installation script
+2020/12/14 10:29:11  [INFO]     Starting k3s docker node k3p-docker-server-0
+2020/12/14 10:29:11  [INFO]     Starting k3s docker node k3p-docker-serverlb
+2020/12/14 10:29:12  [INFO]     The cluster has been installed
+2020/12/14 10:29:12  [INFO]     You can retrieve the kubeconfig by running `docker cp k3p-docker-server-0:/etc/rancher/k3s/k3s.yaml ./kubeconfig.yaml`
+
+$ docker ps
+CONTAINER ID   IMAGE                      COMMAND                  CREATED         STATUS         PORTS                                          NAMES
+b90d5ac9107a   rancher/k3d-proxy:latest   "/bin/sh -c nginx-pr…"   7 seconds ago   Up 5 seconds   0.0.0.0:6443->6443/tcp, 0.0.0.0:8080->80/tcp   k3p-docker-serverlb
+f9d226f0a17b   rancher/k3s:v1.19.4-k3s1   "/bin/k3s server --t…"   7 seconds ago   Up 6 seconds   0.0.0.0:8081->80/tcp                           k3p-docker-server-0
+```
diff --git a/examples/docker/whoami.yaml b/examples/docker/whoami.yaml
@@ -0,0 +1,57 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: whoami
+  namespace: default
+  labels:
+    app: whoami
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: whoami
+  template:
+    metadata:
+      labels:
+        app: whoami
+      annotations:
+    spec:
+      containers:
+        - name: whoami
+          image: "traefik/whoami:latest"
+          imagePullPolicy: IfNotPresent
+          ports:
+            - name: http
+              containerPort: 80
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: whoami
+  namespace: default
+  labels:
+    app: whoami
+spec:
+  type: ClusterIP
+  ports:
+  - name: http
+    port: 80
+    targetPort: 80
+  selector:
+    app: whoami
+---
+kind: Ingress
+apiVersion: extensions/v1beta1
+metadata:
+  name: whoami
+  namespace: default
+spec:
+  rules:
+    - host: localhost
+      http:
+        paths:
+          - path: /
+            backend:
+              serviceName: whoami
+              servicePort: 80
diff --git a/examples/ha/README.md b/examples/ha/README.md
@@ -0,0 +1,95 @@
+## HA Deployments
+
+In terms of the contents of the packag we again use a simple `whoami` example, except with this time specifying pod anti-affinity to ensure pods are not co-located on the same node.
+K3p can be used to add new nodes to the cluster either locally via the `install` command, or remotely via the `node add` command.
+
+### Creating the Initial Node
+
+With the experimental k3s embedded etcd HA, one node has to be started with the `--cluster-init` flag, and then additional control-plane instances can be added through joining the initial node.
+
+With the package in this directory already built, SSH in to your first host and run `k3p install` with the `--init-ha` flag. (This can also be done remotely with the `--host` flag).
+
+```bash
+[core@coreos1 ~]$ sudo k3p install package.tar --init-ha
+
+2020/12/14 12:52:47  [INFO]     Loading the archive
+2020/12/14 12:52:48  [INFO]     Copying the archive to the rancher installation directory
+2020/12/14 12:52:49  [INFO]     Generating a node token for additional control-plane instances
+2020/12/14 12:52:49  [INFO]     Installing binaries to /usr/local/bin
+2020/12/14 12:52:49  [INFO]     Installing scripts to /usr/local/bin/k3p-scripts
+2020/12/14 12:52:49  [INFO]     Installing images to /var/lib/rancher/k3s/agent/images
+2020/12/14 12:52:50  [INFO]     Installing manifests to /var/lib/rancher/k3s/server/manifests
+2020/12/14 12:52:50  [INFO]     Running k3s installation script
+2020/12/14 12:52:50  [K3S]      [INFO]  Skipping k3s download and verify
+2020/12/14 12:52:50  [K3S]      [INFO]  Skipping installation of SELinux RPM
+2020/12/14 12:52:50  [K3S]      [INFO]  Creating /usr/local/bin/kubectl symlink to k3s
+2020/12/14 12:52:50  [K3S]      [INFO]  Creating /usr/local/bin/crictl symlink to k3s
+2020/12/14 12:52:50  [K3S]      [INFO]  Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
+2020/12/14 12:52:50  [K3S]      [INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
+2020/12/14 12:52:50  [K3S]      [INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
+2020/12/14 12:52:50  [K3S]      [INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
+2020/12/14 12:52:50  [K3S]      [INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
+2020/12/14 12:52:50  [K3S]      [INFO]  systemd: Enabling k3s unit
+2020/12/14 12:52:50  [K3S]      Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
+2020/12/14 12:52:51  [K3S]      [INFO]  systemd: Starting k3s
+2020/12/14 12:52:59  [INFO]     The cluster has been installed
+2020/12/14 12:52:59  [INFO]     You can view the cluster by running `k3s kubectl cluster-info`
+
+# A token was generated for joining new control-plane instances during the install.
+# A pre-generated one can also be used. To retrieve the generated one you can run
+[core@coreos1 ~]$ sudo k3p token get server
+fFUiC96GBQ69XgENdvhabseBd53vSUVWuYhrLJKVRX08a3M9RA8qSYypBxLMX0iCEPBnWl6BmZ6WKIw4pAhtbQYhMWveiGI3YbkGMkwJQnTfuTnkBzzMIvsitvBiwqg3
+
+# So far we have a single node, and we are unable to schedule two of our pods
+[core@coreos1 ~]$ sudo k3s kubectl get node
+NAME      STATUS   ROLES         AGE   VERSION
+coreos1   Ready    etcd,master   68s   v1.19.4+k3s1
+
+[core@coreos1 ~]$ sudo k3s kubectl get pod
+NAME                      READY   STATUS    RESTARTS   AGE
+whoami-5f47859667-87l9q   1/1     Running   0          63s
+whoami-5f47859667-l77k6   0/1     Pending   0          63s
+whoami-5f47859667-n8jvh   0/1     Pending   0          63s
+```
+
+To join a second and third instance to the cluster there are two (actually three) ways we can do this. The first way is to install the package again to the other instances, using the `--join` flag to signal joining an existing cluster.
+
+```bash
+[core@coreos2 ~]$ sudo k3p install package.tar \
+    --join https://172.18.64.84:6443 \  # The IP and API port of the first instance
+    --join-role server \                # Join as a server instance (the default option is as an agent and uses a different token)
+    --join-token fFUiC96GBQ69XgENdvhabseBd53vSUVWuYhrLJKVRX08a3M9RA8qSYypBxLMX0iCEPBnWl6BmZ6WKIw4pAhtbQYhMWveiGI3YbkGMkwJQnTfuTnkBzzMIvsitvBiwqg3
+
+# ...
+# ...
+```
+
+You can also use `k3p node add` from the initial node to bring in new instances using SSH. If you have public key authentication setup you can use that, otherwise it will prompt for a password. 
+
+You can also do this from a remote instance with the `--leader` flag assuming it uses the same SSH credentials as the new node you are adding.
+
+```bash
+[core@coreos1 ~]$ sudo k3p node add 172.18.64.91 \  # The remote address of the node
+    --ssh-user core \                               # The user to use for SSH
+    --private-key ~/.ssh/id_rsa \                   # The SSH private key (or omit to be prompted for a password)
+    --node-role server                              # Join as a server
+
+# ...
+# ...
+```
+
+Once that is done you will have a highly available cluster and deployment
+
+```bash
+[core@coreos1 ~]$ sudo k3s kubectl get node
+NAME      STATUS   ROLES         AGE     VERSION
+coreos1   Ready    etcd,master   11m     v1.19.4+k3s1
+coreos2   Ready    etcd,master   6m20s   v1.19.4+k3s1
+coreos3   Ready    etcd,master   2m57s   v1.19.4+k3s1
+
+[core@coreos1 ~]$ sudo k3s kubectl get pod
+NAME                      READY   STATUS    RESTARTS   AGE
+whoami-5f47859667-87l9q   1/1     Running   0          11m
+whoami-5f47859667-l77k6   1/1     Running   0          11m
+whoami-5f47859667-n8jvh   1/1     Running   0          11m
+```
diff --git a/examples/ha/whoami.yaml b/examples/ha/whoami.yaml
@@ -0,0 +1,67 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: whoami
+  namespace: default
+  labels:
+    app: whoami
+spec:
+  replicas: 3
+  selector:
+    matchLabels:
+      app: whoami
+  template:
+    metadata:
+      labels:
+        app: whoami
+      annotations:
+    spec:
+      containers:
+        - name: whoami
+          image: "traefik/whoami:latest"
+          imagePullPolicy: Never
+          ports:
+            - name: http
+              containerPort: 80
+      affinity:
+        podAntiAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+          - labelSelector:
+              matchExpressions:
+              - key: app
+                operator: In
+                values:
+                - whoami
+            topologyKey: "kubernetes.io/hostname"
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: whoami
+  namespace: default
+  labels:
+    app: whoami
+spec:
+  type: ClusterIP
+  ports:
+  - name: http
+    port: 80
+    targetPort: 80
+  selector:
+    app: whoami
+---
+kind: Ingress
+apiVersion: extensions/v1beta1
+metadata:
+  name: whoami
+  namespace: default
+spec:
+  rules:
+    - host: localhost
+      http:
+        paths:
+          - path: /
+            backend:
+              serviceName: whoami
+              servicePort: 80
diff --git a/examples/kvdi/README.md → examples/helm-charts/README.md b/examples/kvdi/README.md → examples/helm-charts/README.md
@@ -18,7 +18,7 @@ Then build a package in this directory using the provided config:
 
 ```sh
 # For the sake of producing a smaller artifact, we'll use the --exclude-images flag
-$ k3p build -c config.yaml  --exclude-images --name kvdi
+$ k3p build --exclude-images --name kvdi
 2020/12/11 20:32:08  [INFO]     Building package "kvdi"
 2020/12/11 20:32:08  [INFO]     Detecting latest k3s version for channel stable
 2020/12/11 20:32:09  [INFO]     Latest k3s version is v1.19.4+k3s1
@@ -35,4 +35,4 @@ $ k3p build -c config.yaml  --exclude-images --name kvdi
 2020/12/11 20:32:09  [INFO]     Skipping bundling container images with the package
 2020/12/11 20:32:09  [INFO]     Writing package metadata
 2020/12/11 20:32:09  [INFO]     Archiving version "latest" of "kvdi" to "/home/tinyzimmer/devel/k3p/examples/kvdi/package.tar"
-```
+```
diff --git a/examples/kvdi/config.yaml → examples/helm-charts/k3p.yaml b/examples/kvdi/config.yaml → examples/helm-charts/k3p.yaml