Add docs for using Litestream with LiteFS (#1730)

litefs/backup.html.markerb

@@ -47,4 +47,84 @@ aws s3 cp /path/to/backup.gz s3://mybucket/backup-`date +%d%H`.gz
 This backup can be run on any of your nodes as even the replicas should have
 minimal lag behind the primary. If you run this on multiple nodes (such as all
 your candidates), make sure they are backing up to different storage locations
-so they do not overwrite one another.
+so they do not overwrite one another.
+
+
+## Continuous backup via Litestream
+
+[Litestream](https://litestream.io/) is a tool for continuously streaming
+incremental backups to S3-compatible object storage. You can use it on top of
+LiteFS to provide point-in-time recovery in case your cluster fails for any reason.
+
+<div class="warning">
+This approach only works if you are running a static lease on your cluster.
+</div>
+
+
+### Initialize your bucket
+
+First, you'll need to create a bucket on S3-compatible storage. In this example,
+we'll use [Tigris](https://www.tigrisdata.com/) since it integrates nicely with
+Fly.io.
+
+```sh
+fly storage create
+```
+
+This command will return several configuration variables and you'll need
+`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `BUCKET_NAME` later in this setup.
+
+### Set up your configuration file
+
+Once your bucket is created, you'll need to create a
+[Litestream configuration file](https://litestream.io/reference/config/). In
+this example, change `my.db` and `mybucket` to your respective database file and
+bucket names:
+
+```yml
+dbs:
+  - path: /litefs/my.db
+    meta-path: /var/lib/litefs/my.db-litestream
+    replicas:
+      - type: s3
+        bucket: "mybucket"
+        path: "my.db"
+        endpoint: "fly.storage.tigris.dev"
+        force-path-style: true
+```
+
+If you save this file to `/etc/litestream.yml` then Litestream will pick it up
+automatically for any Litestream commands you use.
+
+The `meta-path` should be a path on a persistent volume. In this example, we're
+assuming that `/var/lib/litefs` has been mounted from a volume.
+
+### Running Litestream
+
+Litestream should be run after LiteFS has been initialized so we can specify it
+in the LiteFS `exec`. Litestream can run its own `exec` so you should specify 
+your application there.
+
+```yml
+# litefs.yml
+
+exec: "litestream replicate -exec myapp"
+```
+
+Litestream should only be run on the primary node. It requires write access to
+the database so it will not run on read-only replicas.
+
+### Verifying your setup
+
+Once you have your application running, you should be able to run Litestream's
+[restore](https://litestream.io/reference/restore/) command to fetch the current
+version of the database:
+
+```sh
+litestream restore -o ~/myrestored.db /litefs/my.db
+```
+
+If you need to recover from a loss of the database, you'll need to restore your
+SQLite database to a file and then use [litefs import](/docs/litefs/import) to 
+import it into your cluster. Litestream cannot restore directly into your LiteFS
+mount.

litefs/cloud-backups.html.markerb

@@ -16,45 +16,39 @@ your state.
 
 In this doc, we'll show you how to add LiteFS Cloud to an existing LiteFS
 cluster. If you're not using LiteFS yet, take a look at the
-[getting started guide][], which walks you through setting LiteFS up for your
-app, including LiteFS Cloud.
+[getting started guide](/docs/litefs/getting-started-fly), which walks you
+through setting LiteFS up for your app, including LiteFS Cloud.
 
 If you'd rather implement your own backups, take a look at the
-[Backing up your LiteFS cluster][] docs.
-
-[getting started guide]: /docs/litefs/getting-started-fly
-[Backing up your LiteFS cluster]: /docs/litefs/backup
+[Backing up your LiteFS cluster](/docs/litefs/backup) docs.
 
 ## Creating a LiteFS Cloud cluster
 
 If you want to back up your database using LiteFS Cloud, you'll need to create
 a LiteFS Cloud cluster. You can do this in the Fly.io dashboard. First,
-[sign up for an account][Sign up] if you haven't already. After you're signed in,
+[sign up for an account](/docs/hands-on/sign-up-sign-in/) if you haven't already. After you're signed in,
 navigate to the LiteFS Cloud section on the left navbar, and then use the Create
 button to create a new cluster.
-[Here's a handy link to the LiteFS Cloud dashboard][LiteFS Cloud Dashboard] if you prefer!
+[Here's a handy link to the LiteFS Cloud dashboard](https://fly.io/dashboard/personal/litefs) if you prefer!
 
 When you create your LiteFS Cloud cluster, you'll be asked to save an auth token.
 Keep this handy - you'll use it below.
 
-[Sign up]: /docs/hands-on/sign-up-sign-in/
-[LiteFS Cloud Dashboard]: https://fly.io/dashboard/personal/litefs
 
 ## Configuring LiteFS to use LiteFS Cloud
 
 ### Update LiteFS
 
 Before configuring LiteFS Cloud, you must update your LiteFS to a version that supports
 LiteFS Cloud. Update to `v0.5.0` or greater. You can do this by
-[installing the latest version][LiteFS Installation].
+[installing the latest version](/docs/litefs/getting-started-fly/#installing-litefs).
 
-[LiteFS Installation]: /docs/litefs/getting-started-fly/#installing-litefs
 
 ### LiteFS Cloud Configuration
 
 When you created your LiteFS Cloud cluster, you got a LiteFS Cloud API authentication
 token. (If you lost it, you can create a new one in
-[the LiteFS Cloud dashboard][LiteFS Cloud Dashboard].)
+[the LiteFS Cloud dashboard](https://fly.io/dashboard/personal/litefs).)
 
 This token should be made available to LiteFS via the `LITEFS_CLOUD_TOKEN` environment
 variable. If you're running on Fly.io, you can add a secret for it with:
@@ -82,4 +76,3 @@ systemctl restart litefs
 
 Remember that this needs to be run for every LiteFS node.
 
-[LiteFS Cloud Dashboard]: https://fly.io/dashboard/personal/litefs

litefs/cloud-restore.html.markerb

@@ -23,7 +23,7 @@ did the first (erroneous) restore!
 ## Restoring your LiteFS database
 
 To restore your LiteFS database to a snapshot, navigate to
-the [Fly.io LiteFS dashboard][LiteFS Dashboard], find
+the [Fly.io LiteFS dashboard](https://fly.io/dashboard/personal/litefs), find
 the LiteFS Cloud cluster and database you would like to restore,
 and click the "view backups" link. Then you can select a date and time
 to restore from. Since restore is a destructive function (you are likely
@@ -37,22 +37,20 @@ after you make a write to your database. For production applications (where
 writes are frequent), this is unlikely to be noticeable, but if you're
 testing it out, you may need to manually make a write to see changes!
 
-[LiteFS Dashboard]: https://fly.io/dashboard/personal/litefs
 
 ## Limitations
 
 ### Snapshot age
 
 LiteFS Cloud stores "snapshots" as a series of
-[LTX files][]. For cost and performance reasons,
+[LTX files](https://github.com/superfly/litefs/blob/main/docs/ARCHITECTURE.md#lite-transaction-files-ltx). For cost and performance reasons,
 LiteFS Cloud compacts these files into files containing transactions
 from five minute intervals. This means that when you select a
 timestamp for restoring, LiteFS Cloud takes the most recent file
 available from before that timestamp, which means that you may miss
 some transactions that happened within the five minute period before
 the timestamp you selected.
 
-[LTX files]: https://github.com/superfly/litefs/blob/main/docs/ARCHITECTURE.md#lite-transaction-files-ltx
 
 ### Snapshot restore requires database write