Skip to content

Commit

Permalink
Merge pull request #49 from stakater/hibernation-usecase
Browse files Browse the repository at this point in the history
Add Hibernation usecase
  • Loading branch information
LilShah authored Oct 14, 2022
2 parents 061fee9 + 62f5b59 commit 8946acd
Show file tree
Hide file tree
Showing 12 changed files with 146 additions and 23 deletions.
4 changes: 4 additions & 0 deletions docs/.vuepress/configs/sidebar.ts
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,10 @@ export const sidebarEn: SidebarConfig = [
text: "Extending Manager ClusterRole",
link: "/content/sre/tenant-operator/usecases/manager-clusterrole",
},
{
text: "Freeing up Unused Resources",
link: "/content/sre/tenant-operator/usecases/hibernation.md",
},
],
},
"/content/sre/tenant-operator/hibernation.md",
Expand Down
12 changes: 6 additions & 6 deletions docs/content/sre/tenant-operator/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,23 @@ RBAC is one of the most complicated and error-prone parts of Kubernetes. With Te

Tenant-Operator provides several ClusterRoles that are automatically bound to the Tenants Namespaces used for managing access to the Namespaces and the resources they contain. You can also modify the default roles or create new roles to have full control and customize access control for your users and teams.

Tenant-Operator also is able to leverage existing OpenShift groups or external groups synced from 3rd party identity management system for maintaining Tenant membership in your organizations current user management system.
Tenant-Operator is also able to leverage existing OpenShift groups or external groups synced from 3rd party identity management systems, for maintaining Tenant membership in your organization's current user management system.

## Self-Service

With Tenant-Operator, you can empower your users to safely provision namespaces for themselves and their teams (typically mapped to SSO groups). Team-owned namespaces and the resources inside of them count towards the team's quotas rather than the user's individual limits and are automatically shared with all team members according to the access rules you configure in tenant-operator.

Also by leveraging Tenant-Operators templating mechanism, namespaces can be provisioned and automatically pre-populated with any kind of resource or multiple resources such as network policies, docker pull secrets or even Helm charts etc
Also by leveraging Tenant-Operator's templating mechanism, namespaces can be provisioned and automatically pre-populated with any kind of resource or multiple resources such as network policies, docker pull secrets or even Helm charts etc

## HashiCorp Vault Multitenancy

Tenant-Operator is not only providing strong Multi Tenancy for the OpenShift internals but also extends the tenants permission model to HashiCorp Vault where it can create Vault paths and greatly ease the overhead of managing RBAC in Vault.

## ArgoCD Multitenancy
## ArgoCD Multitenancy

Tenant-Operator is not only providing strong Multi Tenancy for the OpenShift internals but also extends the tenants permission model to ArgoCD were it can provision AppProjects and Allowed Repositories for your tenants greatly ease the overhead of managing RBAC in ArgoCD.

## Cost/Resource Optimization
## Cost/Resource Optimization

Tenant-Operator provides a mechanism for defining Resource Quotas at the tenant scope, meaning all namespaces belonging to a particular tenant share the defined quota, which is why you are able to safely enable dev teams to self serve their namespaces whilst being confident that they can only use the resources allocated based on budget and business needs.

Expand All @@ -44,7 +44,7 @@ Common use cases for namespace templates may be:
- Adding development tooling to a namespace
- Deploying pre-populated databases with test data
- Equipping new namespaces with optional credentials such as image pull secrets

## Tenant level Hibernation
Tenant-operator can downscale Deployments and StatefulSets in a tenant's Namespace according to a defined sleep schedule. The Deployments and StatefulSets are brought back to their required replicas according to the provided wake schedule.

Expand All @@ -53,7 +53,7 @@ Tenant-operator can manage Mattermost to create Teams for tenant users. All tena

## Everything as Code/GitOps Ready

Tenant-Operator is designed and built to be 100% OpenShift native and to be configured and managed the same familiar way as native OpenShift resources so is perfect for modern shops that are dedicated to GitOps as it is fully configurable using Custom Resources
Tenant-Operator is designed and built to be 100% OpenShift-native and to be configured and managed the same familiar way as native OpenShift resources so is perfect for modern shops that are dedicated to GitOps as it is fully configurable using Custom Resources.

## Preventing Clusters Sprawl
Share a single cluster with multiple teams, groups of users, or departments by saving operational and management efforts.
Expand Down
4 changes: 2 additions & 2 deletions docs/content/sre/tenant-operator/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ You can uninstall Tenant-Operator by following these steps

* Now the operator has been uninstalled.

* `Optional:` you can also manually remove tenant operators CRDs and it's resources from the cluster.
* `Optional:` you can also manually remove tenant operators CRDs and its resources from the cluster.

## Installing via Subscription

Expand Down Expand Up @@ -203,7 +203,7 @@ subscription.operators.coreos.com "tenant-operator" deleted

* Now the operator has been uninstalled.

* `Optional:` you can also manually remove tenant operators CRDs and it's resources from the cluster.
* `Optional:` you can also manually remove tenant operators CRDs and it's resources from the cluster.

## Installing via Helm

Expand Down
4 changes: 2 additions & 2 deletions docs/content/sre/tenant-operator/integration-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,11 @@ users:

### Privileged Namespaces

`privilegedNamespaces:` Contains the list of `namespaces` ignored by the tenant-operator. The tenant-operator will not manage the `namespaces` in this list. Values in this list are regex patterns. For example, to ignore the `default` namespace, we can specify `^default$`; and to ignore all `namespaces` starting with the `openshift-` prefix, we can specify `^openshift-*`.
`privilegedNamespaces:` Contains the list of `namespaces` ignored by the tenant-operator. The tenant-operator will not manage the `namespaces` in this list. Values in this list are regex patterns. For example, to ignore the `default` namespace, we can specify `^default$`; and to ignore all `namespaces` starting with the `openshift-` prefix, we can specify `^openshift-*`.

### Privileged ServiceAccounts

`privilegedServiceAccounts:` Contains the list of `ServiceAccounts` ignored by the tenant-operator. The tenant-operator will not manage the `ServiceAccounts` in this list. Values in this list are regex patterns. For example, to ignore all `ServiceAccounts` starting with the `system:serviceaccount:openshift-` prefix, we can use `^system:serviceaccount:openshift-*`; and to ignore the `system:serviceaccount:builder` service account we can use `^system:serviceaccount:builder$.`
`privilegedServiceAccounts:` Contains the list of `ServiceAccounts` ignored by the tenant-operator. The tenant-operator will not manage the `ServiceAccounts` in this list. Values in this list are regex patterns. For example, to ignore all `ServiceAccounts` starting with the `system:serviceaccount:openshift-` prefix, we can use `^system:serviceaccount:openshift-*`; and to ignore the `system:serviceaccount:builder` service account we can use `^system:serviceaccount:builder$.`

### Namespace Access Policy

Expand Down
2 changes: 1 addition & 1 deletion docs/content/sre/tenant-operator/tenant-roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ fig 4. Shows viewers role in a tenant using Tenant Operator

Viewer role will only have view access on there `Project`.

#### Access Permissions
#### Access Permissions

* ServiceAccount access in `Project`
* get
Expand Down
2 changes: 1 addition & 1 deletion docs/content/sre/tenant-operator/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ If a user is added to tenant resource, and the user does not exist in RHSSO, the

Description/steps after which this situation occurs:

1. Add a new user to Tenant CR
1. Add a new user to Tenant CR
2. Attempt to log in to Vault with the added user
3. Vault denies that the user exists, and signs the user up via RHSSO. User is now created on RHSSO (You may check for the user on RHSSO).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ resources:
"jenkins.io/credentials-description": credentials from Kubernetes
```
Once the template has been created, Bill has to edit the `Tenant` to add unique label to namespaces in which the secret has to be deployed.
Once the template has been created, Bill has to edit the `Tenant` to add unique label to namespaces in which the secret has to be deployed.
For this, he can use the support for [common](./tenant.html#distributing-common-labels-and-annotations-to-tenant-namespaces-via-tenant-custom-resource) and [specific](./tenant.html#distributing-specific-labels-and-annotations-to-tenant-namespaces-via-tenant-custom-resource) labels across namespaces.

Bill has to specify a label on namespaces in which he needs the secret. He can add it to all namespaces inside a tenant or some specific namespaces depending on the use case.
Expand All @@ -54,14 +54,14 @@ spec:
- build
- prod
# use this if you want to add label to some specific namespaces
# use this if you want to add label to some specific namespaces
specificMetadata:
- namespaces:
- test-namespace
labels:
distribute-image-pull-secret: true
# use this if you want to add label to all namespaces under your tenant
# use this if you want to add label to all namespaces under your tenant
commonMetadata:
labels:
distribute-image-pull-secret: true
Expand Down
116 changes: 116 additions & 0 deletions docs/content/sre/tenant-operator/usecases/hibernation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# Freeing up unused resources at night time with tenant hibernation

Bill is a cluster admin who wants to free unused cluster resources in an effort to reduce costs during the night when the cluster isn't being used.

First, Bill creates a tenant with the `hibernation` schedules mentioned in the spec, or adds the hibernation field to existing tenant:

```yaml
apiVersion: tenantoperator.stakater.com/v1beta1
kind: Tenant
metadata:
name: sigma
spec:
hibernation:
sleepSchedule: 0 20 * * 1-5
wakeSchedule: 0 8 * * 1-5
owners:
users:
- user
editors:
users:
- user1
quota: medium
namespaces:
withoutTenantPrefix:
- build
- stage
- dev
```
The schedules above will make all namespaces that belong to the tenant put the deployments and statefulSets within those namespaces to sleep, by reducing their pod count to 0 at 8 PM every weekday. At 8 AM on a weekday, the namespaces will then wake up by restoring their applications' previous pod count.
Bill can verify this behaviour by checking the newly created ResourceSupervisor resource at running time:
```bash
oc get ResourceSupervisor -A
NAME AGE
sigma 5m
```

The ResourceSupervisor will look like this at running time:

```yaml
apiVersion: tenantoperator.stakater.com/v1beta1
kind: ResourceSupervisor
metadata:
finalizers:
- tenantoperator.stakater.com/resourcesupervisor
name: example
spec:
argocd:
appProjects: []
namespace: ''
hibernation:
sleepSchedule: 0 20 * * 1-5
wakeSchedule: 0 8 * * 1-5
status:
currentStatus: running
nextReconcileTime: '2022-10-12T20:00:00Z'
```
The ResourceSupervisor will look like this at sleeping time:
```yaml
apiVersion: tenantoperator.stakater.com/v1beta1
kind: ResourceSupervisor
metadata:
finalizers:
- tenantoperator.stakater.com/resourcesupervisor
name: example
spec:
argocd:
appProjects: []
namespace: ''
hibernation:
sleepSchedule: 0 20 * * 1-5
wakeSchedule: 0 8 * * 1-5
status:
currentStatus: sleeping
nextReconcileTime: '2022-10-13T08:00:00Z'
sleepingApplications:
- Namespace: build
kind: Deployment
name: example
replicas: 3
- Namespace: stage
kind: Deployment
name: example
replicas: 3
```
Bill wants to prevent the `build` namespace from going to sleep, so they can add the `hibernation.stakater.com/exclude: 'true'` annotation to it. The ResourceSupervisor will now look like this after reconciling:

```yaml
apiVersion: tenantoperator.stakater.com/v1beta1
kind: ResourceSupervisor
metadata:
finalizers:
- tenantoperator.stakater.com/resourcesupervisor
name: example
spec:
argocd:
appProjects: []
namespace: ''
hibernation:
sleepSchedule: 0 20 * * 1-5
wakeSchedule: 0 8 * * 1-5
status:
currentStatus: sleeping
nextReconcileTime: '2022-10-13T08:00:00Z'
sleepingApplications:
- Namespace: stage
kind: Deployment
name: example
replicas: 3
```
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,6 @@ managerRoleExtendedRules:
- update
- watch
```
## What’s next
See how Bill can [hibernate unused namespaces at night](./hibernation.html)
2 changes: 1 addition & 1 deletion docs/content/sre/tenant-operator/usecases/namespace.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Additionally, to remove namespaces from a tenant, simply remove the tenant label

## Remove Namespaces from your Cluster via GitOps.

GitOps is a quick and efficient way to automate the management of your K8S resources.
GitOps is a quick and efficient way to automate the management of your K8s resources.

To remove namespaces from your cluster via GitOps;

Expand Down
12 changes: 6 additions & 6 deletions docs/content/sre/tenant-operator/usecases/quota.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Using Tenant-Operator, the cluster-admin can set and enforce resource quotas and
Bill is a cluster admin who will first creates a resource quota where he sets the maximum resource limits that Anna's tenant will have.
Here `limitrange` is an optional field, cluster admin can skip it if not needed.

The annotation `quota.tenantoperator.stakater.com/is-default: "true"` sets the quota as default quota.
The annotation `quota.tenantoperator.stakater.com/is-default: "true"` sets the quota as default quota.

```yaml
kubectl create -f - << EOF
Expand All @@ -30,11 +30,11 @@ spec:
limits:
- type: "Pod"
max:
cpu: "2"
memory: "1Gi"
cpu: "2"
memory: "1Gi"
min:
cpu: "200m"
memory: "100Mi"
cpu: "200m"
memory: "100Mi"
EOF
```

Expand All @@ -46,7 +46,7 @@ NAME STATE AGE
small Active 3m
```

Bill then proceeds to create a tenant for Anna, while also linking the newly created `quota`.
Bill then proceeds to create a tenant for Anna, while also linking the newly created `quota`.

```yaml
kubectl create -f - << EOF
Expand Down
2 changes: 1 addition & 1 deletion docs/content/sre/tenant-operator/usecases/template.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,4 @@ Parameters can be used with both `manifests` and `helm charts`

## What’s next

See how Anna can deploy a [template in a namespace](./deploying-templates.html)
See how Anna can deploy a [template in a namespace](./deploying-templates.html)

0 comments on commit 8946acd

Please sign in to comment.