Skip to content

Commit

Permalink
Update 04.restack.md (#482)
Browse files Browse the repository at this point in the history
* Update 04.restack.md

new version with the three clouds

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

* Update content/docs/09.administrator-guide/02.deployment/04.restack.md

Co-authored-by: Anna Geller <[email protected]>

---------

Co-authored-by: Anna Geller <[email protected]>
  • Loading branch information
atapiawastaken and anna-geller authored Sep 6, 2023
1 parent a6e3aed commit 72e7e8c
Showing 1 changed file with 217 additions and 37 deletions.
254 changes: 217 additions & 37 deletions content/docs/09.administrator-guide/02.deployment/04.restack.md
Original file line number Diff line number Diff line change
@@ -1,66 +1,209 @@
---
title: Deployment on AWS with Restack
title: Deploy to AWS, GCP or Azure with Restack
---

[Restack](https://restack.io) lets you deploy Kestra on your own AWS infrastructure on Kubernetes.
[Restack](https://restack.io) provides the tools to deploy open-source products to your cloud without requiring DevOps resources.


## Getting Started

To deploy Kestra with Restack:

- [Sign up for a Restack account](#sign-up-for-a-restack-account).
- [Add AWS credentials with AdministratorAccess](#add-aws-credentials-with-administratoraccess).
- [One-click cluster creation with Restack](#one-click-cluster-creation-with-restack).
- [Deploy Kestra on Restack](#deploy-kestra-on-restack).
- [Start using Kestra](#start-using-kestra).
- [Deploy multiple instances of Kestra](#deploy-multiple-instances-of-kestra).
- [Sign up for a Restack account](#sign-up-for-a-restack-account)
- [Connect your AWS account](#connect-your-aws-account)
- [Connect your GCP account](#connect-your-gcp-account)
- [Connect your Azure account](#connect-your-azure-account)
- [One-click cluster creation with Restack](#one-click-cluster-creation-with-restack)
- [Deploy Kestra on Restack](#deploy-kestra-on-restack)
- [Start using Kestra](#start-using-kestra)
- [Deploy your own Kestra image](#deploy-your-kestra-image)
- [Manage secrets and environment variables](#manage-secrets-and-environment-variables).


## Sign up for a Restack account

To Sign up for a Restack account, visit [www.restack.io/signup](https://www.restack.io/signup). You can sign up with your corporate email address or your GitHub profile. You do not need a credit card to sign up.
To create a Restack account, visit [www.restack.io/signup](https://www.restack.io/signup). You can sign up for free (_no credit card required_) using your company's email address or your GitHub account.

![restack-signup](/docs/administrator-guide/deployment/restack/restack-sign-up.png)

If you already have an account, login to Restack at [www.restack.io/login](https://www.restack.io/login).

## Add AWS credentials with AdministratorAccess
## Connect your AWS account

Log into Restack Console and navigate to Settings > Cloud > Connect Cloud:

![](https://cdn.sanity.io/images/ev3amoz3/production/4c3c9385fd468fd49fcade0b405f089e14303bba-4320x2280.png)

Then, create an IAM Role by following the following steps:

1. Log into your AWS Management Console as a user with administrator privileges and go to the IAM section.

2. Click the Roles tab and click on Create Role in the IAM sidebar.

![](https://cdn.sanity.io/images/ev3amoz3/production/7d1ba16ba5610e8947ca20d8746ee8424a9d4621-2000x1062.png)

3. In the "Select type of trusted entity" section, select the "Custom trust policy" option

![](https://cdn.sanity.io/images/ev3amoz3/production/666cb5015b3c5a29e6eef6216bdd97ef67297dc8-1770x1110.png)

4. Copy and paste the following JSON policy into the custom policy field (_you also can copy the same policy from the Restack UI_):

```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::223746943638:user/connect",
"arn:aws:iam::223746943638:role/connect"
]
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "restack"
}
}
}
]
}
```

5. Click the "Next" button and under the "Add permissions" section, add the `AdministratorAccess` policy.

![](https://cdn.sanity.io/images/ev3amoz3/production/2685ee99b153f46b38f99607407dfba72c5c6522-2000x1160.png)

6. Click the "Next" button and type `restack` in the "Role name" field. Then, review the IAM role and click on the "Create role" button.

![](https://cdn.sanity.io/images/ev3amoz3/production/fb14345d290077195b986cb3931c0d39689dca6f-2000x1255.png)

7. From the IAM > Roles overview, navigate to the role you've just created and copy its ARN.

![](https://cdn.sanity.io/images/ev3amoz3/production/e3750c4fb52873a781e328e81c70237d1968d5da-2560x1440.png)

8. Paste the Role ARN to Restack's user interface and validate the setup by clicking on the 'Connect cloud' button. This will validate if Restack can successfully assume the role in your AWS account.

![](https://cdn.sanity.io/images/ev3amoz3/production/ef820e88fd8cd5bd64ca24b79f22bc96cf9fd255-2560x1440.png)

If the AWS cloud account has been successfully added and Restack can connect to it, you can proceed to create a cluster and provision your Kestra instance.


**How can restack assume the IAM role created in your account?**

- The `sts:AssumeRole` is an AWS Security Token Service (STS) API operation that allows an IAM user or role in one account to assume a role in another AWS account. This allows the user or role to access resources in the second account that they would not normally have access to.

- The `sts:AssumeRole` operation returns a set of temporary security credentials that the user or role can use to access resources in the second account. These credentials include an access key, a secret access key, and a session token, which can be used to make AWS API requests.


---

## Connect your GCP account


You can connect your GCP account to Restack in just a few steps.


First, make sure that the following requirements are satisfied:

1. You have an active Google Cloud account or organization
2. You have an active project that can be used to deploy resources from Restack
3. You have an active billing profile associated with that project


1. Create a new service account in your Google project. This service account will be used to manage permissions to deploy resources with Restack.

![](https://cdn.sanity.io/images/ev3amoz3/production/43f688c517cfca2c2e430d17d0069ed3fb74b2e8-6016x3384.png)



2. Grant this service account access to the project with the `Owner` role

![](https://cdn.sanity.io/images/ev3amoz3/production/e30384b7512b54dd7770562d3ab5c10c05eecf6c-6016x3384.png)

3. Click on "Grant access" and add the `[email protected]` as a new principal. Then, assign the `Service Account Token Creator` role to that user, as shown in the image below


![](https://cdn.sanity.io/images/ev3amoz3/production/4d051eea7ecfd580fd1d5d3f730cebb238e17f3f-6016x3384.png)

4. Finally, click on "Save" and "Done" to finish the service account creation process.

5. Add your GCP project ID and the email of your service account to Restack's user interface in order to add your GCP cloud account and validate the connectivity established via the service account.

To deploy Kestra in your own AWS infrastructure with Restack, you will need to add your credentials as the next step.
![](https://cdn.sanity.io/images/ev3amoz3/production/bfc406eb19cd1c67245fbd8d7c8e84ceafc7e020-1176x1920.png)

Make sure that this account has *AdministratorAccess*. This is how Restack can ensure an end-to-end cluster creation and cluster management process.

1. Navigate to **Clusters** in the left-hand navigation menu.
2. Select the **Credentials** tab.
3. Click **Add credential**.
![add-credentials](/docs/administrator-guide/deployment/restack/restack-add-credentials.png)
4. Give a suitable title to your credentials for managing them later.
5. Enter your [**AWS Access Key ID** and **AWS Secret Access key**.](https://docs.aws.amazon.com/accounts/latest/reference/root-user-access-key.html)
6. Click **Add credential**.
![add-aws-credentials](/docs/administrator-guide/deployment/restack/restack-add-AWS-creds.png)
---

## Connect your Azure account

You can connect your Microsoft Azure account to Restack in just a few steps.

First, make sure that the following requirements are satisfied:
1. You have an active Azure account
2. You have an active Azure subscription
3. You have an active Azure user with the "Owner" role.


Follow these steps to create a new Azure App Registration in order to authenticate your Azure account with Restack and deploy Azure resources from Restack.

1. Create a new App Registration in your Azure Active Directory

![](https://cdn.sanity.io/images/ev3amoz3/production/6b355dd2401ab94754258f4fe12ff51c52c6f0bc-3020x1652.png)

![](https://cdn.sanity.io/images/ev3amoz3/production/dc0fa6c05a65e60d9ff7c323b25b5ea116e4e7e1-3024x1650.png)

2. Copy the Application (client) ID and paste it into the Restack UI setup as the Client ID.

![](https://cdn.sanity.io/images/ev3amoz3/production/392b15df09de170e1bb7632880f42ff093852853-2564x1730.png)

3. Copy the Directory (tenant) ID and paste it in Restack UI as the Tenant ID.

![](https://cdn.sanity.io/images/ev3amoz3/production/cb0e76db65825a3d097b764735e6de29d68195ff-2590x1728.png?w=450)

4. For your newly created App Registration, add an API Permission called "user_impersonation".

![](https://cdn.sanity.io/images/ev3amoz3/production/d3c38be874bce1bfe63c60a1bd6a2da75b77cce5-3024x1648.png)

5. For your App Registration, go to "Certificates & secrets" and create a new client secret. Copy the value of the secret and paste it in Restack UI as the Client Secret.

![](https://cdn.sanity.io/images/ev3amoz3/production/2cc097fd4e153bab779de038f153a83bb2a38c59-3024x1648.png)

6. Go to your "Subscriptions", and choose the subscription you want to use for Restack. In the "Access Control (IAM)" section, access "Role assignments" and add a new role assignment. Choose the "Owner" role (Privileged administrator roles) & Azure Kubernetes Service RBAC Cluster Admin (Job function roles) and click "Next".

![](https://cdn.sanity.io/images/ev3amoz3/production/6451cf86e77b7fb886d63578734858fe00c8f149-3024x1650.png)
![](https://cdn.sanity.io/images/ev3amoz3/production/1121d4525b7659406f38ddcab0e745ff1d84cb1f-3024x1650.png)

7. Choose "User, group or service principal" in "Assign access to" and click on "Select members". In the opened search bar, type in "restack". The "restack" App Registration you've created should pop up. Select it and click on "Next". Lastly, click on "Review + assign". Now in the "Role assignments" section, you should see the "restack" app.

![](https://cdn.sanity.io/images/ev3amoz3/production/3a8e74393fc5b3ee3b0598ef69475b9e4d738891-3024x1650.png)

8. Go to "Subscriptions", copy the Subscription ID, and paste it into the Restack UI.

![](https://cdn.sanity.io/images/ev3amoz3/production/dfaefa1f878e41dce37339de365cb6e32c88ea9d-2564x1726.png)

9. Click on "Connect cloud".

>[How to get your AWS Access key ID and AWS Secret Access Key](https://docs.aws.amazon.com/accounts/latest/reference/root-user-access-key.html)
If the cloud account has been successfully added and Restack can connect to it, you can proceed to create a cluster and provision your Kestra instance.

---

## One-click cluster creation with Restack

::alert{type="info"}
Running your application on a Kubernetes cluster lets you deploy, scale and monitor the application reliably.
::

Once you have added your credentials:
1. Navigate to the **Clusters** tab on the same page and click on **Create cluster**.
1. Navigate to the **Clusters** tab and click on **Create cluster**.
![create-cluster](/docs/administrator-guide/deployment/restack/restack-create-cluster.png)
2. Give a suitable name to your cluster.
2. Give a name to your cluster.
3. Select the region you want to deploy the cluster in.
4. Select the AWS credentials you added in the previous step.
![cluster-details](/docs/administrator-guide/deployment/restack/restack-cluster-details.png)

The cluster creation process will start automatically. Once the cluster is ready, you will get an email on the email id connected with your account.
The cluster creation process will start automatically. Once the cluster is ready, you will receive an email with the confirmation.

Creating a cluster is a one-time process. From here you can add other open source tools or multiple instances of Kestra in the same cluster.

![creating-cluster](/docs/administrator-guide/deployment/restack/restack-creating-cluster.png)
![cluster-created](/docs/administrator-guide/deployment/restack/restack-cluster-created.png)

Any application you deploy in your cluster will be accessible via a free **restack domain**.
Contact the Restack team via chat to set a custom domain for your Kestra instances.

## Deploy Kestra on Restack

Expand All @@ -74,10 +217,47 @@ Contact the Restack team via chat to set a custom domain for your Kestra instanc

Kestra will be deployed on your cluster and you can access it using the link under the *URL* tab.
![lightdash-deployed](/docs/administrator-guide/deployment/restack/kestra-deployed.png)
You can also check the workloads and volumes that are deployed within Kestra.
![access-lightdash](/docs/administrator-guide/deployment/restack/kestra-access.png)

## Deploy multiple instances of Kestra

Restack makes it easier to deploy multiple instances of Kestra on the same or multiple clusters.
<br/>So you can test the latest version before upgrading or have a dedicated instance for development and for production.
---

## Deploy your Kestra image


Activate **Restack CI/CD previews** under Stack > Settings to deploy a custom Kestra image by connecting your GitHub repository.

With CI/CD preview activated, every time you make a pull request to your GitHub repo, Restack takes every file and deploys them to your cloud.

For every pull request, Restack provides you with a URL with a preview environment, allowing you to review the pull request. Once you merge your pull request, Restack flushes the preview environments and builds the deployed code from the repository into your production environment.

The following [Kestra Restack repository](https://github.com/restackio/starter-kestra) can help you get started.


---

## Manage secrets and environment variables


You can edit Kestra secrets and environment variables directly in the Restack UI.

On the Restack UI, click on `Stacks`, then on your application, and then on `Settings`. From there, you will be able to add, modify, or delete your secrets and environment variables.

When adding a new environment variable, consider the following:

First, add an environment key and value in lowercase with no hyphens or underscores in the Restack UI. Here is an example:

```
Key: openai
Value: my openai key
```
Then, in the Kestra Flow, you can use globals to get the value. Following the previous example, you can use the following syntax:

```
{{ globals.openai }}
```

For more information about global environment variables, check the following [documentation page](https://kestra.io/docs/developer-guide/variables#environment-variables).


![](https://cdn.sanity.io/images/ev3amoz3/production/5b7ea6b1dc491e5029b7a68bc6ab2ed1679266df-649x777.png)

1 comment on commit 72e7e8c

@vercel
Copy link

@vercel vercel bot commented on 72e7e8c Sep 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

kestra-io – ./

kestra-io-git-main-kestra.vercel.app
kestra-io.vercel.app
kestra-io-kestra.vercel.app

Please sign in to comment.