Author | Last updated |
---|---|
Saul Nachman & Moshe Immerman |
22/07/2021 |
This is part 2 of a series demonstrating the Template Operator's capabilities. This post covers templating out a coordinated deployment, service and ingress using a CRD, as well as making a homemade CRD.
Powering up with Custom Resource Definitions (CRDs)
Using CRDs with the Template Operator is a powerful technique allowing platform engineers to set up complex deployments with minimal configuration.
See the project readme for installation instructions.
To test the deployment in the browser, set up ingress to your cluster – see, for example, the minikube or kind docs.
As a platform engineer, I need it to be easy for development teams to expose applications to the internet, so that they can move quickly and encounter only as much complexity as they must.
Improving team productivity – one demand placed on platform teams – involves providing teams new capabilities while also reducing complexity.
In the example, you will use the TutorialService
spec, which has three fields as well as the standard metadata:
image
(required, string)domain
(required, string)replicas
(optional, int)
Using just these values, you can generate a properly configured deployment, service and ingress.
Note: You can see how this CRD was generated in Making a homemade CRD.
Install the tutorial TutorialService
CRD:
kubectl apply -f https://raw.githubusercontent.com/flanksource/template-operator/v0.4.0/examples/tutorial-crd.yaml
Apply the two TutorialService
instances:
cat <<EOF | kubectl apply -f -
kind: Namespace
apiVersion: v1
metadata:
name: crd-tutorial
---
apiVersion: tutorial.tutorial/v1
kind: TutorialService
metadata:
name: tutorial-application-1
namespace: crd-tutorial
spec:
image: nginx:1.21.1
domain: test
replicas: 2
---
apiVersion: tutorial.tutorial/v1
kind: TutorialService
metadata:
name: tutorial-application-2
namespace: crd-tutorial
spec:
image: nginx:1.21.1
domain: test
# Notice there is no 'replicas' field specified here.
EOF
Apply the template, which will reconcile using the TutorialService
instances above:
cat <<EOF | kubectl apply -f -
apiVersion: templating.flanksource.com/v1
kind: Template
metadata:
name: tutorial-service
spec:
# The "source" field selects for the objects to monitor.
# API docs here: https://pkg.go.dev/github.com/flanksource/template-operator/api/v1#ResourceSelector
source:
# Selects for the apiVersion
apiVersion: tutorial.tutorial/v1
# Selects for the kind – in this case TutorialService instances
kind: TutorialService
resources:
# Adds a deployment
- apiVersion: apps/v1
kind: Deployment
metadata:
# {{.metadata.name}} comes from the source object (".").
# Syntax is based on go text templates with gomplate functions (https://docs.gomplate.ca).
name: "{{.metadata.name}}"
namespace: "{{.metadata.namespace}}"
labels:
app: "{{.metadata.name}}"
spec:
# Notice that when replicas are not specified the default value is set to "2".
replicas: '{{.spec.replicas | default "2"}}'
selector:
matchLabels:
app: "{{.metadata.name}}"
template:
metadata:
labels:
app: "{{.metadata.name}}"
spec:
containers:
- name: "{{.metadata.name}}"
# The container image is consumed here.
image: '{{.spec.image}}'
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
protocol: TCP
# Adds a service
- apiVersion: v1
kind: Service
metadata:
name: "{{.metadata.name}}"
namespace: "{{.metadata.namespace}}"
spec:
selector:
app: "{{.metadata.name}}"
ports:
- port: 8080
targetPort: 80
# Adds an ingress
- apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/tls-acme: true
name: "{{.metadata.name}}"
namespace: "{{.metadata.namespace}}"
labels:
app: "{{.metadata.name}}"
spec:
rules:
# The domain is consumed here
- host: "{{.metadata.name}}.{{.spec.domain}}"
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: "{{.metadata.name}}"
port:
number: 8080
tls:
- hosts:
- '{{.metadata.name}}.{{.spec.domain}}'
secretName: "{{.metadata.name}}-tls"
EOF
Once the Template Operator has reconciled (see Part 1 for how to find the logs), run:
kubectl get ingress -n crd-tutorial
kubectl get services -n crd-tutorial
kubectl get pods -n crd-tutorial
to see, respectively:
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
crd-tutorial tutorial-application-1 <none> tutorial-application-1.test localhost 80, 443 64m
crd-tutorial tutorial-application-2 <none> tutorial-application-2.test localhost 80, 443 64m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
tutorial-application-1 ClusterIP 10.96.74.220 <none> 8080/TCP 65m
tutorial-application-2 ClusterIP 10.96.113.251 <none> 8080/TCP 65m
NAME READY STATUS RESTARTS AGE
tutorial-application-1-55fcbb44d-bhpnk 1/1 Running 0 65m
tutorial-application-1-55fcbb44d-qw9rk 1/1 Running 0 65m
tutorial-application-2-5765d6fcdd-6hz47 1/1 Running 0 65m
tutorial-application-2-5765d6fcdd-7qbx9 1/1 Running 0 65m
Add to your hosts file what is necessary to expose URLs tutorial-application-1.test
and tutorial-application-2.test
, then visit https://tutorial-application-2.test and https://tutorial-application-2.test to see the default nginx index page.
There are a few options for CRD generators, such as OperatorSDK and kubebuilder .
For this example, we've used kubebuilder
, but any CRD generator or a handwritten CRD would work as well.
For kubebuilder
, install kubebuilder and go 1.13 <= version < 1.16 (for downversioning, gvm is a decent management tool).
kubebuilder init --domain tutorial --repo tutorial-operator
kubebuilder create api --group tutorial --version v1 --kind TutorialService
When asked, don't create the controller.
You won't need much of what's scaffolded but will need the api/v1
folder and the (yet to be created) crd/bases
folder.
First you'll need to add fields to the spec.
Open api/v1/tutorialservice_types.go
to see something like the following code.
The example added fields under TutorialServiceSpec
and TutorialServiceStatus
has been removed.
Those in TutorialServiceSpec
are what you'll modify.
package v1
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
// TutorialServiceSpec defines the desired state of TutorialService
type TutorialServiceSpec struct {
Image string `json:"image"`
Domain string `json:"domain"`
Replicas int `json:"replicas,omitempty"`
}
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
// TutorialService is the Schema for the tutorialservices API
type TutorialService struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec TutorialServiceSpec `json:"spec,omitempty"`
}
//+kubebuilder:object:root=true
// TutorialServiceList contains a list of TutorialService
type TutorialServiceList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []TutorialService `json:"items"`
}
func init() {
SchemeBuilder.Register(&TutorialService{}, &TutorialServiceList{})
}
To generate the new CRD, run make manifests
from where you set up the project. Examine your file tree to a file (crd/bases/tutorial.tutorial_tutorialservices.yaml
) containing something like the following snippet.
This is the same CRD used in the example above.
cat <<EOF | kubectl apply -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.4.1
creationTimestamp: null
name: tutorialservices.tutorial.tutorial
spec:
group: tutorial.tutorial
names:
kind: TutorialService
listKind: TutorialServiceList
plural: tutorialservices
singular: tutorialservice
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
description: TutorialService is the Schema for the tutorialservices API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TutorialServiceSpec defines the desired state of TutorialService
properties:
domain:
type: string
image:
type: string
replicas:
type: integer
# Note that the properties missing the 'omitempty' annotation are `required`.
required:
- domain
- image
type: object
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
EOF