Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add datastore backup and upgrade notes #452

Merged
merged 3 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/operate/data-backup.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
sidebar_position: 8
description: "Strategies for backing up and restoring the Restate data store"
---

import Admonition from '@theme/Admonition';

# Data Backup

<Admonition type="note">
Future versions of Restate will support distributed deployment with spanning multiple machnes enhancing the availability you can achieve with your Restate cluster. This document only covers single-node Restate deployments.
</Admonition>

The Restate server persists both metadata (such as the details of deployed services, in-flight invocations) and data (e.g., virtual object and workflow state keys) in its data store, which is located in its base directory (by default, the `restate-data` path relative to the startup working directory). Restate is configured to perform write-ahead logging with fsync enabled to ensure that effects are fully persisted before being acknowledged to participating services.

Backing up the full contents of the Restate base directory will ensure that you can recover this state in the event of a server failure. We recommend placing the data directory on fast block storage that supports atomic snapshots, such as Amazon EBS volume snapshots. Alternatively, you can stop the restate-server process, archive the base directory contents, and then restart the process. This ensures that the backup contains an atomic view of the persisted state.

In addition to the data store, you should also make sure you have a back up of the effective Restate server configuration. Be aware that this may be spread across command line arguments, environment variables, and the server configuration file.

## Restoring Backups

<Admonition type={"caution"} title={"Avoid multiple instances of Restate"}>
Restate cannot guarantee that it is the only instance of the given node. You must ensure that only one instance of any given Restate node is running when restoring the data store from a backup. Running multiple instances could lead to a "split-brain" scenario where different servers process invocations for the same set of services, causing state divergence.
</Admonition>

To restore from backup, ensure the following:

* Use a Restate server release that is compatible with the version that produced the data store snapshot. See the [Upgrading](upgrading) section.
* Use an equivalent [Restate server configuration](/operate/configuration/server). In particular, ensure that the `cluster-name` and `node-name` attributes match those of the previous Restate server operating on this data store.
* Exclusive access to a data store restored from the most recent atomic snapshot of the previous Restate installation.
32 changes: 32 additions & 0 deletions docs/operate/upgrading.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
sidebar_position: 7
description: "Restate installation software version upgrades, compatibility policy, rollback strategy"
---

import Admonition from '@theme/Admonition';

# Upgrading

Restate follows [Semantic Versioning](https://semver.org/). The server persists compatibility markers which enable it to detect incompatible data versions. However, you should be careful to follow supported version migration paths and perform [data backups](data-backup) when performing software upgardes.

## Version compatibility

<Admonition type="note">
Consult the release notes for the specific details of any new version when planning upgrades.
</Admonition>

Upgrading to the latest patch version should always be possible and is recommended to benefit from the latest bug fixes and enhancements.

Incremental minor version upgrades will retain functional compatibility with the immediate prior version. That is, for any minor version update, you will be able to upgrade from `x.y` to `x.(y+1)` while retaining all persisted data and metadata. You must not skip minor version upgrades, i.e. go directly from `x.y` to `x.(y+2)`, as it may bypass necessary data store migrations required for preserving forward compatibility.

If you encounter any issues with a new version, you can downgrade a Restate installation to the latest patch level of the previous minor version. For example, you can safely rollback the Restate server version from `x.(y).0` to `x.(y-1).z` if you encounter compatibility issues. However, this rollback is only supported if you have not used any new opt-in features exclusive to the newer version. You cannot downgrade more than one minor version behind the most recent version used with the data store using this approach.

## Service compatibility

Registered Restate services must use an SDK compatible with the service protocol version(s) of the running Restate server. Note that Restate SDKs follow independent versioning from the server. You can find the latest SDK compatibility matrix in the respective SDK version documentation.

* [Restate Java SDK](https://github.com/restatedev/sdk-java#versions)
* [Restate TypeScript SDK](https://github.com/restatedev/sdk-typescript#versions)
* [Restate Go SDK](https://github.com/restatedev/sdk-go#versions)
* [Restate Python SDK](https://github.com/restatedev/sdk-python#versions)
* [Restate Rust SDK](https://github.com/restatedev/sdk-rust#versions)
Loading