Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: restructured local engine (hostpath, lvm, and zfs) #404

Merged
merged 11 commits into from
Apr 5, 2024
Merged
12 changes: 12 additions & 0 deletions docs/i18n/en/docusaurus-plugin-content-docs/current.json
Original file line number Diff line number Diff line change
Expand Up @@ -174,5 +174,17 @@
"sidebar.docs.category.Migration using Velero": {
"message": "Migration using Velero",
"description": "The label for category Migration using Velero in sidebar docs"
},
"sidebar.docs.category.Local PV Hostpath": {
datacore-gthomas marked this conversation as resolved.
Show resolved Hide resolved
"message": "Local PV Hostpath",
"description": "The label for category Local PV Hostpath in sidebar docs"
},
"sidebar.docs.category.Local PV LVM": {
"message": "Local PV LVM",
"description": "The label for category Local PV LVM in sidebar docs"
},
"sidebar.docs.category.Local PV ZFS": {
"message": "Local PV ZFS",
"description": "The label for category Local PV ZFS in sidebar docs"
}
}
62 changes: 24 additions & 38 deletions docs/main/concepts/data-engines/data-engines.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ title: Overview of OpenEBS Data Engines
keywords:
- Data Engines
- OpenEBS Data Engines
- Local Storage
- Local Engines
- Replicated Storage
- Replicated Engine
description: OpenEBS Data Engine is the core component that acts as an end-point for serving the IO to the applications. OpenEBS Data engines are akin to Storage controllers or sometimes also know for implementing the software defined storage capabilities.
---

Expand Down Expand Up @@ -44,7 +48,7 @@ The following table summarizes the recommendation for small to medium instances,
| Ephemeral Node or Storage | Non-Ephemeral | Non-Ephemeral | Ephemeral |
| Size of Cluster | Single Node | Multiple Nodes | Multiple Nodes |
| Storage Deployment Type | Hyperconverged | Hyperconverged | Disaggregated |
| Recommended Data Engines | Local Engine | Local Engine and Replicated Engine | Replicated Engine |
| Recommended Data Engines | Local Storage | Local Storage and Replicated Storage | Replicated Storage |

The following table summarizes the recommendation for small to medium instances with fast SSDs capable of higher IOPS and Throughput, typically connected using NVMe:

Expand All @@ -53,7 +57,7 @@ The following table summarizes the recommendation for small to medium instances
| Ephemeral Node or Storage | Non-Ephemeral | Non-Ephemeral | Ephemeral |
| Size of Cluster | Single Node | Multiple Nodes | Multiple Nodes |
| Storage Deployment Type | Hyperconverged | Hyperconverged | Disaggregated |
| Recommended Data Engines | Local Engine | Local Engine and Replicated Engine | Replicated Engine |
| Recommended Data Engines | Local Storage | Local Storage and Replicated Storage | Replicated Storage |


## Stateful Workload Capabilities
Expand Down Expand Up @@ -83,7 +87,7 @@ Each stateful application comes with a certain capabilities and depends on the [
| Workload Type | Distributed | Stand-alone | Distributed and/or Stand-alone |
| ----------------------------| :--------------: | :---------------------: | :---------------------------: |
| Required Capabilities | Performance | Availability | Performance and Availability |
| Recommended Data Engines | Local Engine | Replicated Engine | Replicated Engine |
| Recommended Data Engines | Local Storage | Replicated Storage | Replicated Storage |


## Data Engine Capabilities
Expand All @@ -96,15 +100,15 @@ All OpenEBS Data Engines support:

OpenEBS Data Engines can be classified into two categories.

### Local Engines
### Local Storage

OpenEBS Local Engines can create Persistent Volumes (PVs) out of local disks or hostpaths or using the volume managers on the Kubernetes worker nodes. Local Engines are well suited for cloud native applications that have the availability, scalability features built into them. Local Engines are also well suited for stateful workloads that are short lived like Machine Learning (ML) jobs or edge cases where there is a single node Kubernetes cluster.
OpenEBS Local Storage or Local Engines can create Persistent Volumes (PVs) out of local disks or hostpaths or use the volume managers on the Kubernetes worker nodes. Local Storage are well suited for cloud native applications that have the availability, scalability features built into them. Local Storage are also well suited for stateful workloads that are short lived like Machine Learning (ML) jobs or edge cases where there is a single node Kubernetes cluster.

:::note
Local volumes are only available from the the node on which the persistent volume is created. If that node fails, the application pod will not be re-scheduled to another node.
Local Storage are only available from the the node on which the persistent volume is created. If that node fails, the application pod will not be re-scheduled to another node.
:::

The below table identifies few differences among the different OpenEBS Local Engines.
The below table identifies few differences among the different OpenEBS Local Storage.

| Feature | Hostpath | Rawfile | Device | ZFS | LVM |
| -------------------------------------------- | :---: | :------: | :--------: | :------: | :------: |
Expand All @@ -120,12 +124,12 @@ The below table identifies few differences among the different OpenEBS Local Eng
| Works on OS Mounted storage | Yes | Yes | No | No | No |


### Replicated Engines
### Replicated Storage

Replicated Engines (aka Replicated Volumes) are those that can synchronously replicate the data to multiple nodes. These engines provide protection against node failures, by allowing the volume to be accessible from one of the other nodes where the data was replicated to. The replication can also be setup across availability zones helping applications move across availability zones. Replicated Volumes are also capable of enterprise storage features like snapshots, clone, volume expansion, and so forth.
Replicated Storage or Replicated Engine (f.k.a Mayastor) are those that can synchronously replicate the data to multiple nodes. These engines provide protection against node failures, by allowing the volume to be accessible from one of the other nodes where the data was replicated to. The replication can also be setup across availability zones helping applications move across availability zones. Replicated Volumes are also capable of enterprise storage features like snapshots, clone, volume expansion, and so forth.

:::tip
Depending on the type of storage attached to your Kubernetes worker nodes and application performance requirements, you can select from [Local Engine](local-engine.md) or[Replicated Engine](replicated-engine.md).
Depending on the type of storage attached to your Kubernetes worker nodes and application performance requirements, you can select from [Local Storage](local-engine.md) or[Replicated Storage](replicated-engine.md).
:::

:::note
Expand All @@ -136,48 +140,30 @@ An important aspect of the OpenEBS Data Layer is that each volume replica is a f
- When Volume Snapshots is taken, the snapshot is taken on all its healthy volume replicas.
:::

Below table identifies few differences among the different OpenEBS Replicated engines.

| Feature | Jiva | cStor | Mayastor |
| -------------------------------------------- | :---: | :------: | :--------: |
| Full Backup and Restore using Velero | Yes | Yes | Yes |
| Synchronous replication | Yes | Yes | Yes |
| Protect against node failures (replace node) | Yes | Yes | Yes |
| Use with ephemeral storage on nodes | Yes | Yes | Yes |
| Thin Provisioning | Yes | Yes | Yes |
| Disk pool or aggregate support | Yes | Yes | Planned |
| Disk resiliency (RAID support ) | Yes | Yes | Planned |
| On demand capacity expansion | No | Yes | Planned |
| Snapshots | No | Yes | Planned |
| Clones | No | Yes | Planned |
| Incremental Backups | No | Yes | Planned |
| Suitable for high capacity (>50GB) workloads | No | Yes | Yes |
| Near disk performance | No | No | Yes |

### Ideal Conditions for selecting OpenEBS Local Engine
### Use-cases for OpenEBS Local Storage

- When applications are managing replication and availability themselves and there is no need of replication at storage layer. In most such situations, the applications are deployed as `statefulset`.
- Local Engine is recommended when dedicated local disks are not available for a given application or dedicated storage is not needed for a given application.
- Local Storage is recommended when dedicated local disks are not available for a given application or dedicated storage is not needed for a given application.
- When near disk performance is a need along with features like snapshots, volume expansion, pooling of storage from multiple storage devices.

### Ideal Conditions for selecting OpenEBS Replicated Engine
### Use-cases for OpenEBS Replicated Storage

- When you need high performance storage using NVMe SSDs and the cluster is capable of NVMeoF.
- When you need replication or availability features to protected against node failures.
- Replicated Engine is designed for the next gen compute and storage technology and is under active development.
- Replicated Storage is designed for the next gen compute and storage technology and is under active development.

As Replicated Engine is under active development, you can also influence the features that are being built by joining [OpenEBS community on Kubernetes Slack](https://kubernetes.slack.com). If you are already signed up, head to our discussions at [#openebs](https://kubernetes.slack.com/messages/openebs/) channel.
For latest updates and discussions, community users are invited to join [OpenEBS community on Kubernetes Slack](https://kubernetes.slack.com). If you are already signed up, head to our discussions at [#openebs](https://kubernetes.slack.com/messages/openebs/) channel.

### Summary

A short summary is provided below.
datacore-gthomas marked this conversation as resolved.
Show resolved Hide resolved

- Local Engine is preferred if your application is in production and does not need storage level replication.
- Replicated Engine is preferred if your application needs low latency and near disk throughput, requires storage level replication and your nodes have high CPU, RAM, and NVMe capabilities.
- Local Storage is preferred if your application is in production and does not need storage level replication.
- Replicated Storage is preferred if your application needs low latency and near disk throughput, requires storage level replication and your nodes have high CPU, RAM, and NVMe capabilities.


## See Also

- [User Guides](../../user-guides/)
- [Local Engine User Guide](../../user-guides/local-engine-user-guide/)
- [Replicated Engine User Guide](../../user-guides/replicated-engine-user-guide/)
[User Guides](../../user-guides/)
[Local Storage User Guide](../../user-guides/local-engine-user-guide/)
[Replicated Storage User Guide](../../user-guides/replicated-engine-user-guide/)
34 changes: 17 additions & 17 deletions docs/main/concepts/data-engines/local-engine.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
id: localengine
title: OpenEBS Local Engine
title: OpenEBS Local Storage
keywords:
- Local Engine
- OpenEBS Local Engine
description: This document provides you with a brief explanation of OpenEBS Local Engine, quick start guides, and when to use OpenEBS Local Engine, and when not to use OpenEBS Local Engine, and its limitations.
- Local Storage
- OpenEBS Local Storage
description: This document provides you with a brief explanation of OpenEBS Local Storage, quick start guides, and when to use OpenEBS Local Storage, and when not to use OpenEBS Local Storage, and its limitations.
---

## Local Engine Overview
## Local Storage Overview

OpenEBS provides Dynamic PV provisioners for [Kubernetes Local Volumes](https://kubernetes.io/docs/concepts/storage/volumes/#local). A Local Engine (aka Local Volume) implies that storage is available only from a single node. A local volume represents a mounted local storage device such as a disk, partition, or directory.
OpenEBS provides Dynamic PV provisioners for [Kubernetes Local Volumes](https://kubernetes.io/docs/concepts/storage/volumes/#local). A Local Storage (aka Local Volume) implies that storage is available only from a single node. A local volume represents a mounted local storage device such as a disk, partition, or directory.

As the local volume is accessible only from a single node, local volumes are subject to the availability of the underlying node and are not suitable for all applications. If a node becomes unhealthy, then the local volume will also become inaccessible and a Pod using it will not be able to run. Applications using local volumes must be able to tolerate this reduced availability, as well as potential data loss, depending on the durability characteristics of the underlying disk.

Expand All @@ -35,12 +35,12 @@ OpenEBS helps users to take local volumes into production by providing features

OpenEBS provides Local Volume that can be used to provide locally mounted storage to Kubernetes Stateful workloads. Refer to the [Quickstart Guide](../../quickstart-guide/) for more information.

## When to use OpenEBS Local Engine?
## When to use OpenEBS Local Storage?

- High performance is needed by those applications which manage their own replication, data protection, and other features such as snapshots and clones.
- When local disks need to be managed dynamically and monitored for impending notice of them going bad.

## When not to use OpenEBS Local Engine?
## When not to use OpenEBS Local Storage?

- When applications expect replication from storage.
- When the volume size may need to be changed dynamically but the underlying disk is not resizable.
Expand All @@ -49,7 +49,7 @@ OpenEBS provides Local Volume that can be used to provide locally mounted storag

OpenEBS Local Volumes can be backed up and restored along with the application using [Velero](https://velero.io).

Velero uses [Restic](https://github.com/restic/restic) for backing up and restoring Kubernetes local volumes. Velero can be configured to save the backups either in the cloud or on-premise with any S3 compatible storage like MinIO. When user initiates the backup, Velero via the Restic, will copy the entire data from the Local Engine to the remote location. Later, when the user wants to restore the application, Velero injects an init container into the application that will download and populate the data into the volume from the backed up location. For more details on how Velero Restic works, refer to the [Velero Restic integration](https://velero.io/docs/v1.3.2/restic/) documentation.
Velero uses [Restic](https://github.com/restic/restic) for backing up and restoring Kubernetes local volumes. Velero can be configured to save the backups either in the cloud or on-premise with any S3 compatible storage like MinIO. When user initiates the backup, Velero via the Restic, will copy the entire data from the Local Storage to the remote location. Later, when the user wants to restore the application, Velero injects an init container into the application that will download and populate the data into the volume from the backed up location. For more details on how Velero Restic works, refer to the [Velero Restic integration](https://velero.io/docs/v1.3.2/restic/) documentation.

While the preferred way for Backup and Restore for cloud native applications using Local Volumes is to use the application specific backup solution, you can use the Velero based Backup and Restore in the following cases:
- Application does not natively provide a Backup and Restore solution
Expand All @@ -75,7 +75,7 @@ A quick summary of the steps to restore include:

1. Install and Setup Velero, with the same provider where backups were saved.

2. Local Engines are created with node affinity. As the node names will change when a new cluster is created, create the required PVC(s) prior to proceeding with restore.
2. Local Storage Volumes are created with node affinity. As the node names will change when a new cluster is created, create the required PVC(s) prior to proceeding with restore.
```
kubectl apply -f https://openebs.github.io/charts/examples/local-hostpath/local-hostpath-pvc.yaml
```
Expand All @@ -85,17 +85,17 @@ A quick summary of the steps to restore include:
velero restore create rbb-01 --from-backup bbb-01 -l app=test-velero-backup
```

## Limitations (or Roadmap items) of OpenEBS Local Engine
## Limitations (or Roadmap items) of OpenEBS Local Storage
datacore-gthomas marked this conversation as resolved.
Show resolved Hide resolved

- Size of the Local Engine cannot be increased dynamically.
- Disk quotas are not enforced by Local Engine. An underlying device or hostpath can have more data than requested by a PVC or storage class. Enforcing the capacity is a roadmap feature.
- Size of the Local Storage cannot be increased dynamically.
- Disk quotas are not enforced by Local Storage. An underlying device or hostpath can have more data than requested by a PVC or storage class. Enforcing the capacity is a roadmap feature.
- Enforce capacity and PVC resource quotas on the local disks or host paths.
- SMART statistics of the managed disks is also a potential feature in the roadmap.

## See Also

- [OpenEBS Architecture](../architecture.md)
- [Local Engine Prerequisites](../../user-guides/local-engine-user-guide/prerequisites.mdx)
- [Installation](../../quickstart-guide/installation.md)
- [Local Engine User Guide](../../user-guides/local-engine-user-guide/)
[OpenEBS Architecture](../architecture.md)
[Local Storage Prerequisites](../../user-guides/local-engine-user-guide/prerequisites.mdx)
[Installation](../../quickstart-guide/installation.md)
[Local Storage User Guide](../../user-guides/local-engine-user-guide/)

Loading
Loading