Add readthedocs documentation (wal-g#953)

citusdata · May 7, 2021 · 28ea3b1 · 28ea3b1
1 parent 9fc2672
commit 28ea3b1
Show file tree

Hide file tree

Showing 11 changed files with 235 additions and 200 deletions.
diff --git a/.readthedocs.yaml b/.readthedocs.yaml
@@ -0,0 +1,11 @@
+# Required
+version: 2
+
+mkdocs:
+  configuration: mkdocs.yml
+
+# Optionally set the version of Python and requirements required to build your docs
+python:
+   version: 3.7
+   install:
+   - requirements: docs/requirements.txt
diff --git a/FoundationDB.md → docs/FoundationDB.md b/FoundationDB.md → docs/FoundationDB.md
@@ -1,4 +1,4 @@
-## WAL-G for FoundationDB
+# WAL-G for FoundationDB
 
 **Work in progress**
 
@@ -13,15 +13,15 @@ Optional:
 
 - To build with libsodium, just set `USE_LIBSODIUM` environment variable.
 - To build with lzo decompressor, just set `USE_LZO` environment variable.
-```
+```plaintext
 go get github.com/wal-g/wal-g
 cd $GOPATH/src/github.com/wal-g/wal-g
 make install
 make deps
 make fdb_build
 ```
 Users can also install WAL-G by using `make install`. Specifying the GOBIN environment variable before installing allows the user to specify the installation location. On default, `make install` puts the compiled binary in `go/bin`.
-```
+```plaintext
 export GOBIN=/usr/local/bin
 cd $GOPATH/src/github.com/wal-g/wal-g
 make install
@@ -32,11 +32,11 @@ make fdb_install
 Usage
 -----
 
-* ``backup-fetch``
+### ``backup-fetch``
 
 Command for sending backup from storage to stream in order to restore it in the database.
 
-```
+```bash
 wal-g backup-fetch example_backup
 ```
 
@@ -45,15 +45,15 @@ Variable _WALG_STREAM_RESTORE_COMMAND_ is required for use backup-fetch
 
 WAL-G can also fetch the latest backup using:
 
-```
+```bash
 wal-g backup-fetch LATEST
 ```
 
-* ``backup-push``
+### ``backup-push``
 
 Command for compressing, encrypting and sending backup from stream to storage.
 
-```
+```bash
 wal-g backup-push
 ```
 

diff --git a/MongoDB.md → docs/MongoDB.md b/MongoDB.md → docs/MongoDB.md
@@ -1,4 +1,4 @@
-## WAL-G for MongoDB
+# WAL-G for MongoDB
 
 **Interface of MongoDB features is currently unstable**
 
@@ -13,15 +13,15 @@ Optional:
 
 - To build with libsodium, just set `USE_LIBSODIUM` environment variable.
 - To build with lzo decompressor, just set `USE_LZO` environment variable.
-```
+```plaintext
 go get github.com/wal-g/wal-g
 cd $GOPATH/src/github.com/wal-g/wal-g
 make install
 make deps
 make mongo_build
 ```
 Users can also install WAL-G by using `make install`. Specifying the GOBIN environment variable before installing allows the user to specify the installation location. On default, `make install` puts the compiled binary in `go/bin`.
-```
+```plaintext
 export GOBIN=/usr/local/bin
 cd $GOPATH/src/github.com/wal-g/wal-g
 make install
@@ -32,41 +32,41 @@ make mongo_install
 Configuration
 -------------
 
-#### `WALG_STREAM_CREATE_COMMAND`
+* `WALG_STREAM_CREATE_COMMAND`
 
 Command to create MongoDB backup, should return backup as single stream to STDOUT. Required for backup procedure.
 
-#### `WALG_STREAM_RESTORE_COMMAND`
+* `WALG_STREAM_RESTORE_COMMAND`
 
 Command to unpack MongoDB backup, should take backup (created by `WALG_STREAM_CREATE_COMMAND`) 
 to STDIN and push it to MongoDB instance. Required for restore procedure.
 
-#### `MONGODB_URI`
+* `MONGODB_URI`
 
 URI used to connect to a MongoDB instance. Required for backup and oplog archiving procedure.
 
-#### `OPLOG_ARCHIVE_AFTER_SIZE`
+* `OPLOG_ARCHIVE_AFTER_SIZE`
 
 Oplog archive batch in bytes which triggers upload to storage.
 
-#### `OPLOG_ARCHIVE_TIMEOUT_INTERVAL`
+* `OPLOG_ARCHIVE_TIMEOUT_INTERVAL`
 
 Time interval (passed since previous upload) to trigger upload to storage.
 
 Format: [golang duration string](https://golang.org/pkg/time/#ParseDuration).
 
-#### `MONGODB_LAST_WRITE_UPDATE_INTERVAL`
+* `MONGODB_LAST_WRITE_UPDATE_INTERVAL`
 
 Interval to update the latest majority optime. wal-g archives only majority committed operations.
 Format: [golang duration string](https://golang.org/pkg/time/#ParseDuration).
 
 
-#### `OPLOG_PUSH_WAIT_FOR_BECOME_PRIMARY`
+* `OPLOG_PUSH_WAIT_FOR_BECOME_PRIMARY`
 
 Wait for primary and start archiving or exit immediately. 
 Archiving works only on primary, but it's useful to run wal-g on all replicaset nodes with `OPLOG_PUSH_WAIT_FOR_BECOME_PRIMARY: true` to handle replica set elections. Then new primary will catch up archiving after elections.
 
-#### `OPLOG_PITR_DISCOVERY_INTERVAL`
+* `OPLOG_PITR_DISCOVERY_INTERVAL`
 
 Defines the longest possible point-in-time recovery period.
 It's lasts from starting timestamp of the oldest backup (within `OPLOG_PITR_DISCOVERY_INTERVAL`) until now. 
@@ -75,20 +75,20 @@ Setting is used by oplog archives [purging](#oplog-purge).
 Format: [golang duration string](https://golang.org/pkg/time/#ParseDuration).
 
 
-#### `OPLOG_PUSH_STATS_ENABLED`
+* `OPLOG_PUSH_STATS_ENABLED`
 
 Enables statistics collecting of oplog archiving procedure.
 
-#### `OPLOG_PUSH_STATS_UPDATE_INTERVAL`
+* `OPLOG_PUSH_STATS_UPDATE_INTERVAL`
 
 Interval to update oplog archiving statistics. Disabled if reset to 0.
 
-#### `OPLOG_PUSH_STATS_LOGGING_INTERVAL`
+* `OPLOG_PUSH_STATS_LOGGING_INTERVAL`
 
 Interval to log oplog archiving statistics. Disabled if reset to 0.
 Format: [golang duration string](https://golang.org/pkg/time/#ParseDuration).
 
-#### `OPLOG_PUSH_STATS_EXPOSE_HTTP`
+* `OPLOG_PUSH_STATS_EXPOSE_HTTP`
 
 Exposes http-handler with oplog archiving statistics: `stats/oplog_push`.
 HTTP-server listens `HTTP_LISTEN` port (default: 8090).
@@ -99,41 +99,41 @@ Usage
 
 WAL-G mongodb extension currently supports these commands:
 
-#### ``backup-push``
+### ``backup-push``
 
 Creates new backup and send it to storage.
 
 Runs `WALG_STREAM_CREATE_COMMAND` to create backup.
 
-```
+```bash
 wal-g backup-push
 ```
 
-#### `backup-list`
+### `backup-list`
 
 Lists currently available backups in storage.
 
-```
+```bash
 wal-g backup-list
 ```
 
-#### `backup-fetch`
+### `backup-fetch`
 
 Fetches backup from storage and restores passes data to `WALG_STREAM_RESTORE_COMMAND` to restore backup.
 
 User should specify the name of the backup to fetch.
 
-```
+```bash
 wal-g backup-fetch example_backup
 ```
 
-#### `backup-show`
+### `backup-show`
 
 Fetches backup metadata from storage to STDOUT.
 
 User should specify the name of the backup to show.
 
-```
+```bash
 # wal-g backup-show stream_20201027T224823Z
 {
        "BackupName": "stream_20201027T224823Z",
@@ -165,23 +165,23 @@ User should specify the name of the backup to show.
        "StartLocalTime": "2020-10-28T01:48:23.121314+03:00"
 }
 ```
-#### `backup-delete`
+### `backup-delete`
 
 Deletes backup from storage.
 
 User should specify the name of the backup to delete.
 
 Dry-run
-```
+```bash
 wal-g backup-delete example_backup
 ```
 
 Perform delete
-```
+```bash
 wal-g backup-delete example_backup --confirm
 ```
 
-#### `oplog-push`
+### `oplog-push`
 
 Fetches oplog from mongodb instance (`MONGODB_URI`) and uploads to storage.
 
@@ -191,31 +191,31 @@ wal-g forces upload when,
 
 Archiving collects writes if optime is readable by majority reads. Optime is updated every `MONGODB_LAST_WRITE_UPDATE_INTERVAL`.
 
-```
+```bash
 wal-g oplog-push
 ```
 
 Note: archiving works only on primary, but you can run it on any replicaset node using config option `OPLOG_PUSH_WAIT_FOR_BECOME_PRIMARY: true`.
 
-#### `oplog-replay`
+### `oplog-replay`
 
 Fetches oplog archives from storage and applies to mongodb instance (`MONGODB_URI`)
 
 User should specify SINCE and UNTIL boundaries (format: `timestamp.inc`, eg `1593554809.32`). Both of them should exist in storage.
 SINCE is included and UNTIL is NOT.
 
-```
+```bash
 wal-g oplog-replay 1593554109.1 1593559109.1
 ```
 
-##### Common constraints:
+### Common constraints:
 
 - SINCE: operation timestamp before full backup started.
 - UNTIL: operation timestamp after backup finished.
 
 Use `MongoMeta.Before.LastMajTS` and `MongoMeta.After.LastMajTS` fields from backup [metadata](#backup-show).
 
-#### `oplog-fetch`
+### `oplog-fetch`
 
 Fetches oplog archives from storage and passes to STDOUT.
 
@@ -224,23 +224,23 @@ SINCE is included and UNTIL is NOT.
 
 Supported formats to output: `json`, `bson`, `bson-raw`
 
-```
+```bash
 wal-g oplog-fetch 1593554109.1 1593559109.1 --format json
 ```
 
-#### `oplog-purge`
+### `oplog-purge`
 
 Purges outdated oplog archives from storage. Clean-up will retain:
 - oplog archives in [PITR interval](#oplog_pitr_discovery_interval)
 - oplog archives within backup creation period
 
 Dry-run
-```
+```bash
 wal-g oplog-purge
 ```
 
 Perform clean-up
-```
+```bash
 wal-g oplog-purge --confirm
 ```
 
@@ -250,7 +250,7 @@ Typical configurations
 ### Full backup/restore only
 
 Here's typical wal-g configuration for that case:
-```
+```bash
 MONGODB_URI:                 'mongodb://user:password@localhost:27018/?authSource=admin&connect=direct&socketTimeoutMS=60000&connectTimeoutMS=10000&tls=true'
 WALG_STREAM_CREATE_COMMAND:  'mongodump --archive --oplog --uri="mongodb://user:password@localhost:27018/?authSource=admin&connectTimeoutMS=10000&tls=true"'
 WALG_STREAM_RESTORE_COMMAND: 'mongorestore --archive --oplogReplay --uri="mongodb://user:password@localhost:27018/?authSource=admin&connectTimeoutMS=10000&tls=true"'
@@ -261,7 +261,7 @@ You may also add `--drop` option to restore command. This option drops the colle
 ### Continuous archiving and point-in-time recovery 
 
 Here's typical wal-g configuration for that case:
-```
+```bash
 # Used to fetch oplog documents
 MONGODB_URI:                 'mongodb://user:password@localhost:27018/?authSource=admin&connect=direct&socketTimeoutMS=60000&connectTimeoutMS=10000&tls=true'
 
@@ -291,18 +291,18 @@ WALG_STREAM_RESTORE_COMMAND: 'mongorestore --archive --uri="mongodb://user:passw
 ### How to restore backup to point in time
 
 Suppose you want to restore your cluster to `2020-10-28T12:11:10+03:00`.
-```
+```bash
 # wal-g backup-list -v
 name                    finish_local_time         ts_before     ts_after      data_size   permanent
 stream_20201025T222118Z 2020-10-26T01:50:17+03:00 1603664478.21 1603666217.6  18521875261 false
 stream_20201026T220833Z 2020-10-27T01:37:42+03:00 1603750113.39 1603751861.6  18552792676 false
 stream_20201027T224823Z 2020-10-28T02:08:55+03:00 1603838903.4  1603840135.34 18952713762 false
 ```
 Pick the closest backup and restore it (don't forget about [constraints](#common-constraints)):
-```
+```bash
 # wal-g backup-fetch stream_20201027T224823Z
 ```
 Replay oplog from backup `ts_before` to `2020-10-28T12:11:10+03:00`:
-```
+```bash
 # wal-g oplog-replay 1603838903.4 1603876270.1
-```
+```