From b65f5840d0421271d818a370a529045ab7e3b807 Mon Sep 17 00:00:00 2001 From: peterschmidt85 Date: Thu, 24 Aug 2023 11:40:21 +0200 Subject: [PATCH] - [Docs] Added the `Clouds` guide - [Docs] Lots of minor edits --- docs/assets/stylesheets/extra.css | 14 +-- docs/docs/guides/clouds.md | 81 ++++++++++++++ docs/docs/guides/dev-environments.md | 64 +++-------- docs/docs/guides/services.md | 66 +++--------- docs/docs/guides/tasks.md | 138 ++++++++---------------- docs/docs/index.md | 77 ++++--------- docs/docs/projects.md | 69 ------------ docs/docs/reference/cli/run.md | 155 ++++++++++++++------------- docs/docs/reference/profiles.yml.md | 81 +++++++++----- mkdocs.yml | 2 +- 10 files changed, 314 insertions(+), 433 deletions(-) create mode 100644 docs/docs/guides/clouds.md delete mode 100644 docs/docs/projects.md diff --git a/docs/assets/stylesheets/extra.css b/docs/assets/stylesheets/extra.css index 9551c8672..b35f1debc 100644 --- a/docs/assets/stylesheets/extra.css +++ b/docs/assets/stylesheets/extra.css @@ -344,7 +344,7 @@ } .md-typeset :not(pre) > a code { - color: #8752d7; + color: #ce00ff; } .md-typeset pre > code { @@ -483,9 +483,9 @@ code .md-code__nav:hover .md-code__button { } [data-md-color-scheme=slate][data-md-color-primary=black], [data-md-color-primary=white] { - --md-primary-fg-color: #6e14ed; - --md-accent-fg-color: #6e14ed; - --md-typeset-a-color: #6e14ed; + --md-primary-fg-color: #0048ff; + --md-accent-fg-color: #ce00ff; + --md-typeset-a-color: #ce00ff; --md-code-hl-function-color: #e3b4fb; --md-code-hl-keyword-color: #e37cff; --md-code-hl-string-color: #b4f9c6; @@ -858,14 +858,16 @@ html .md-footer-meta.md-typeset a:is(:focus,:hover) { color: var(--md-typeset-a-color); } -.md-typeset .grid.cards>:-webkit-any(ul,ol)>li a, .md-typeset .grid>.card a { +.md-typeset .grid.cards > :-webkit-any(ul,ol) > li a, +.md-typeset .grid.cards > :-webkit-any(ul,ol) > li span, +.md-typeset .grid > .card a { color: var(--md-default-fg-color); text-decoration: none; display: block; margin: 0; padding: 1rem 1.4rem; border-radius: 6px; - border: rgba(0,0,0,0.6) 1px solid; + border: rgba(0,0,0,0.6) 0.5px solid; } .md-typeset .grid.cards>ol>li:focus-within,.md-typeset .grid.cards>ol>li:hover,.md-typeset .grid.cards>ul>li:focus-within,.md-typeset .grid.cards>ul>li:hover,.md-typeset .grid>.card:focus-within,.md-typeset .grid>.card:hover { diff --git a/docs/docs/guides/clouds.md b/docs/docs/guides/clouds.md new file mode 100644 index 000000000..104a08ae6 --- /dev/null +++ b/docs/docs/guides/clouds.md @@ -0,0 +1,81 @@ +# Clouds + +With `dstack`, you can run LLM workloads using compute resources from multiple cloud GPU providers. +All you need to do is sign up with these providers, and then pass +their credentials to `dstack`. + +### Why multiple clouds? + +
+- **GPU availability** + Because of high demand, it's easier to obtain GPU from various cloud providers through a single interface. +- **GPU price** + Leverage smaller cloud services (e.g., Lambda Cloud) and spot instances across larger providers. +- **No vendor lock-in** + An open-source and cloud-agnostic interface enables easy switching between cloud providers. +
+ +## Creating cloud accounts + +First, you have to create accounts with each cloud provider. + +Currently, `dstack` supports [AWS](https://portal.aws.amazon.com/billing/signup), +[GCP](https://console.cloud.google.com/freetrial), +[Azure](https://azure.microsoft.com/en-us/free), and [Lambda](https://cloud.lambdalabs.com/sign-up). +To request support for more providers, please submit or upvote +relevant issues in [our tracker](https://github.com/dstackai/dstack/issues). + +??? info "Applying for cloud credits" + Startups can apply for extra credits, usually by reaching out directly to the provider in the case of smaller providers, + or through a partner program (such as [NVIDIA Inception](https://www.nvidia.com/en-us/startups/)) for larger providers. + +??? info "Requesting GPU quotas" + + Larger providers require you to request GPU quotas, essentially obtaining permission from their support + team, prior to utilizing GPUs with your account. If planning to use GPU through credits, approval for the request might + take extra time. + + Quotas need to be requested separately for each family of GPUs and for each region where GPU usage is intended. + + To use spot instances with certain cloud providers (e.g. AWS), you should request quotes + for such instances separately. + +## Configuring clouds with dstack + +First, you need to start the `dstack` server, log in to the UI, open the project settings, and add a backend for each +cloud. + +![](../../assets/images/dstack-hub-view-project-empty.png){ width=800 } + +Configuring backends involves providing cloud credentials, and specifying storage. + +
+- [**AWS** + Learn how to set up an Amazon Web Services backend. + ](../../reference/backends/aws/) +- [**GCP** + Learn how to set up a Google Cloud backend. + ](../../reference/backends/gcp/) +- [**Azure** + Learn how to set up an Microsoft Azure backend. + ](../../reference/backends/azure/) +- [**Lambda** + Learn how to set up a Lambda Cloud backend. + ](../../reference/backends/lambda/) + +
+ +## Requesting resources + +You can request resources using the [`--gpu`](../reference/cli/run.md#GPU) +and [`--memory`](../reference/cli/run.md#MEMORY) arguments with `dstack run`, +or through [`resources`](../reference/profiles.yml.md#RESOURCES) with `.dstack/profiles.yml`. + +Both the [`dstack run`](../reference/cli/run.md) command and [`.dstack/profiles.yml`](../reference/profiles.yml.md) +support various other options, including requesting spot instances, defining the maximum run duration or price, and +more. + +!!! info "Automatic instance discovery" + Remember, you can't specify an instance type by name. Instead, ask for resources by GPU name or memory amount. + `dstack` will automatically select the suitable instance type from a cloud provider and region with the best + price and availability. \ No newline at end of file diff --git a/docs/docs/guides/dev-environments.md b/docs/docs/guides/dev-environments.md index 34bf98990..f02e6df0c 100644 --- a/docs/docs/guides/dev-environments.md +++ b/docs/docs/guides/dev-environments.md @@ -89,7 +89,7 @@ You can install packages using `pip` and `conda` executables from `setup`. If there is no pre-built image, the `dstack run` command will build it and upload it to the storage. If the pre-built image is already available, the `dstack run` command will reuse it. -The `.dstack.yml` has many other properties. To view them all, refer to the [Reference](../reference/dstack.yml/dev-environment.md). +For more details on the file syntax, refer to [`.dstack.yml`](../reference/dstack.yml/dev-environment.md). ## Run the configuration @@ -122,58 +122,18 @@ To open the dev environment via a desktop IDE, click the URL in the output. By default, VS Code comes with pre-installed Python and Jupyter extensions. -### Configure resources, price, etc +### Requesting resources -For every run, you can specify hardware resources like memory and GPU, along with various run policies (e.g., maximum -hourly price, use of spot instances, etc.). +You can request resources using the [`--gpu`](../reference/cli/run.md#GPU) +and [`--memory`](../reference/cli/run.md#MEMORY) arguments with `dstack run`, +or through [`resources`](../reference/profiles.yml.md#RESOURCES) with `.dstack/profiles.yml`. -| Example | Description | -|-----------------------------|--------------------------------------------| -| `dstack run . --gpu A10` | Use an instance with `NVIDIA A10` GPU | -| `dstack run . --gpu A100:8` | Use an instance with 8 `NVIDIA A100` GPUs | -| `dstack run . --gpu 24GB` | Use an instance with a GPU that has `24GB` | +Both the [`dstack run`](../reference/cli/run.md) command and [`.dstack/profiles.yml`](../reference/profiles.yml.md) +support various other options, including requesting spot instances, defining the maximum run duration or price, and +more. -The `dstack run` command has many options. To view them, refer to the [Reference](../reference/cli/run.md). +!!! info "Automatic instance discovery" + `dstack` will automatically select the suitable instance type from a cloud provider and region with the best + price and availability. -??? info "Profiles" - ### Configure profiles (optional) - - Instead of configuring resources, price, and policies through `dstack run`, you can use profiles. To set up a profile, - create the `.dstack/profiles.yml` file in the root folder of the project. - -
- - ```yaml - profiles: - - name: large - - resources: - memory: 24GB # (Optional) The minimum amount of RAM memory - gpu: - memory: 48GB # (Optional) The minimum amount of GPU memory - - max_price: 1.5 # (Optional) The maximim price per instance, in dollards. - - max_duration: 1d # (Optional) The maximum duration of the run. - - spot_policy: auto # (Optional) The spot policy. Supports `spot`, `on-demand, and `auto`. - - backends: [azure, lambda] # (Optional) Use only listed backends - - default: true # (Optional) - ``` - -
- - #### Spot instances - - If `spot_policy` is set to `auto`, `dstack` gives priority to spot instances. If unavailable, it uses on-demand instances. - To reduce costs, set `spot_policy` to `spot`. Keep in mind that spot instances are much cheaper but may be interrupted. - Your code should handle interruptions and resume from saved checkpoints. - - #### Default profile - - By default, the `dstack run` command uses the default profile. You - can override it by passing the `--profile` argument to the `dstack run` command. - - For more details on the syntax of the `profiles.yml` file, refer to the [Reference](../reference/profiles.yml.md). \ No newline at end of file +For more details on the run command, refer to [`dstack run`](reference/cli/run.md). \ No newline at end of file diff --git a/docs/docs/guides/services.md b/docs/docs/guides/services.md index aacef371b..83d3d7356 100644 --- a/docs/docs/guides/services.md +++ b/docs/docs/guides/services.md @@ -32,7 +32,7 @@ Before running a service, you need to configure a gateway address to run the ser ??? info "Gateway" - ### Configure a gateway address + ### Configure a gateway First, you have to create a gateway in one of the clouds of your choice. @@ -148,7 +148,7 @@ You can install packages using `pip` and `conda` executables from `commands`. If there is no pre-built image, the `dstack run` command will build it and upload it to the storage. If the pre-built image is already available, the `dstack run` command will reuse it. -The `.dstack.yml` has many other properties. To view them all, refer to the [Reference](../reference/dstack.yml/service.md). +For more details on the file syntax, refer to [`.dstack.yml`](../reference/dstack.yml/service.md). ## Run the configuration @@ -175,58 +175,18 @@ Serving HTTP on https://myservice.mydomain.com ... This command deploys the service, and forwards the traffic to the gateway address. -### Configure resources, price, etc +### Requesting resources -For every run, you can specify hardware resources like memory and GPU, along with various run policies (e.g., maximum -hourly price, use of spot instances, etc.). +You can request resources using the [`--gpu`](../reference/cli/run.md#GPU) +and [`--memory`](../reference/cli/run.md#MEMORY) arguments with `dstack run`, +or through [`resources`](../reference/profiles.yml.md#RESOURCES) with `.dstack/profiles.yml`. -| Example | Description | -|-----------------------------|--------------------------------------------| -| `dstack run . --gpu A10` | Use an instance with `NVIDIA A10` GPU | -| `dstack run . --gpu A100:8` | Use an instance with 8 `NVIDIA A100` GPUs | -| `dstack run . --gpu 24GB` | Use an instance with a GPU that has `24GB` | +Both the [`dstack run`](../reference/cli/run.md) command and [`.dstack/profiles.yml`](../reference/profiles.yml.md) +support various other options, including requesting spot instances, defining the maximum run duration or price, and +more. -The `dstack run` command has many options. To view them, refer to the [Reference](../reference/cli/run.md). +!!! info "Automatic instance discovery" + `dstack` will automatically select the suitable instance type from a cloud provider and region with the best + price and availability. -??? info "Profiles" - ### Configure profiles (optional) - - Instead of configuring resources, price, and policies through `dstack run`, you can use profiles. To set up a profile, - create the `.dstack/profiles.yml` file in the root folder of the project. - -
- - ```yaml - profiles: - - name: large - - resources: - memory: 24GB # (Optional) The minimum amount of RAM memory - gpu: - memory: 48GB # (Optional) The minimum amount of GPU memory - - max_price: 1.5 # (Optional) The maximim price per instance, in dollards. - - max_duration: 1d # (Optional) The maximum duration of the run. - - spot_policy: auto # (Optional) The spot policy. Supports `spot`, `on-demand, and `auto`. - - backends: [azure, lambda] # (Optional) Use only listed backends - - default: true # (Optional) - ``` - -
- - #### Spot instances - - If `spot_policy` is set to `auto`, `dstack` gives priority to spot instances. If unavailable, it uses on-demand instances. - To reduce costs, set `spot_policy` to `spot`. Keep in mind that spot instances are much cheaper but may be interrupted. - Your code should handle interruptions and resume from saved checkpoints. - - #### Default profile - - By default, the `dstack run` command uses the default profile. You - can override it by passing the `--profile` argument to the `dstack run` command. - - For more details on the syntax of the `profiles.yml` file, refer to the [Reference](../reference/profiles.yml.md). \ No newline at end of file +For more details on the run command, refer to [`dstack run`](reference/cli/run.md). diff --git a/docs/docs/guides/tasks.md b/docs/docs/guides/tasks.md index 291c28e3a..61492af8c 100644 --- a/docs/docs/guides/tasks.md +++ b/docs/docs/guides/tasks.md @@ -46,6 +46,37 @@ commands: +??? info "Port mapping" + If you've configured ports, the CLI forwards them to your local machine, using the same port numbers. + Yet, you can choose to override the local ports if needed. + + The following command will make the application available on `http://127.0.0.1:3000`. + +
+ + ```shell + $ dstack run . -f serve.dstack.yml --port 3000:7860 + ``` + +
+ + Alternatively, you can hardcode the port mapping directly into the configuration (not recommended): + +
+ + ```yaml + type: task + + ports: + - 3000:7860 + + commands: + - pip install -r requirements.txt + - gradio app.py + ``` + +
+ ### Configure the environment By default, `dstack` uses its own Docker images to run tasks, which are pre-configured with Python, Conda, and essential CUDA drivers. @@ -103,7 +134,7 @@ You can install packages using `pip` and `conda` executables from `commands`. If there is no pre-built image, the `dstack run` command will build it and upload it to the storage. If the pre-built image is already available, the `dstack run` command will reuse it. -The `.dstack.yml` has many other properties. To view them all, refer to the [Reference](../reference/dstack.yml/task.md). +For more details on the file syntax, refer to [`.dstack.yml`](../reference/dstack.yml/task.md). ## Run the task @@ -133,37 +164,6 @@ Epoch 2: 100% 1719/1719 [00:18<00:00, 92.32it/s, loss=0.0981, acc=0.969] The `dstack run` command provisions cloud resources, pre-installs the environment, code, runs the task, and establishes an SSH tunnel for secure access. -??? info "Port mapping" - If you've configured ports, the CLI forwards them to your local machine, using the same port numbers. - Yet, you can choose to override the local ports if needed. - - The following command will make the application available on `http://127.0.0.1:3000`. - -
- - ```shell - $ dstack run . -f serve.dstack.yml --port 3000:7860 - ``` - -
- - Alternatively, you can hardcode the port mapping directly into the configuration (not recommended): - -
- - ```yaml - type: task - - ports: - - 3000:7860 - - commands: - - pip install -r requirements.txt - - gradio app.py - ``` - -
- ### Parametrize tasks If you want, it's possible to parametrize tasks with user arguments. Here's an example: @@ -191,69 +191,19 @@ $ dstack run . -f args.dstack.yml --train_batch_size=1 --num_train_epochs=100 The `dstack run` command will pass `--train_batch_size=1` and `--num_train_epochs=100` as arguments to `train.py`. -### Configure resources, price, etc - -For every run, you can specify hardware resources like memory and GPU, along with various run policies (e.g., maximum -hourly price, use of spot instances, etc.). +### Requesting resources -| Example | Description | -|-----------------------------|--------------------------------------------| -| `dstack run . --gpu A10` | Use an instance with `NVIDIA A10` GPU | -| `dstack run . --gpu A100:8` | Use an instance with 8 `NVIDIA A100` GPUs | -| `dstack run . --gpu 24GB` | Use an instance with a GPU that has `24GB` | +You can request resources using the [`--gpu`](../reference/cli/run.md#GPU) +and [`--memory`](../reference/cli/run.md#MEMORY) arguments with `dstack run`, +or through [`resources`](../reference/profiles.yml.md#RESOURCES) with `.dstack/profiles.yml`. -The `dstack run` command has many options. To view them, refer to the [Reference](../reference/cli/run.md). +Both the [`dstack run`](../reference/cli/run.md) command and [`.dstack/profiles.yml`](../reference/profiles.yml.md) +support various other options, including requesting spot instances, defining the maximum run duration or price, and +more. -??? info "Profiles" - ### Configure profiles (optional) - - Instead of configuring resources, price, and policies through `dstack run`, you can use profiles. To set up a profile, - create the `.dstack/profiles.yml` file in the root folder of the project. - -
- - ```yaml - profiles: - - name: large - - resources: - memory: 24GB # (Optional) The minimum amount of RAM memory - gpu: - memory: 48GB # (Optional) The minimum amount of GPU memory - - retry_policy: # (Optional) - limit: 30min - - max_price: 1.5 # (Optional) The maximim price per instance, in dollards. - - max_duration: 1d # (Optional) The maximum duration of the run. - - spot_policy: auto # (Optional) The spot policy. Supports `spot`, `on-demand, and `auto`. - - backends: [azure, lambda] # (Optional) Use only listed backends - - default: true # (Optional) - ``` - -
- - #### Spot instances - - If `spot_policy` is set to `auto`, `dstack` gives priority to spot instances. If unavailable, it uses on-demand instances. - To reduce costs, set `spot_policy` to `spot`. Keep in mind that spot instances are much cheaper but may be interrupted. - Your code should handle interruptions and resume from saved checkpoints. - - #### Retry policy - - If `dstack` can't find capacity, an error displays. To enable continuous capacity search, use `retry_policy` with a - `limit`. For example, setting it to `30min` makes `dstack` search for capacity for 30 minutes. - - #### Default profile - - By default, the `dstack run` command uses the default profile. You - can override it by passing the `--profile` argument to the `dstack run` command. - - For more details on the syntax of the `profiles.yml` file, refer to the [Reference](../reference/profiles.yml.md). +!!! info "Automatic instance discovery" + `dstack` will automatically select the suitable instance type from a cloud provider and region with the best + price and availability. ??? info "Reload mode (experimental)" @@ -269,4 +219,6 @@ The `dstack run` command has many options. To view them, refer to the [Reference This feature allows you to run an app in the cloud while continuing to edit the source code locally and have the app - reload changes on the fly. \ No newline at end of file + reload changes on the fly. + +For more details on the run command, refer to [`dstack run`](reference/cli/run.md). \ No newline at end of file diff --git a/docs/docs/index.md b/docs/docs/index.md index 62fc578d5..e2d571917 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -18,14 +18,12 @@ The server is available at http://127.0.0.1:3000?token=b934d226-e24a-4eab-eb92b3 -## Configure backends +## Configure clouds Upon startup, the server sets up the default project called `main`. +Prior to using `dstack`, make sure to [configure clouds](guides/clouds.md#configuring-backends). -![](../assets/images/dstack-hub-view-project-empty.png){ width=800 } - -Prior to using `dstack`, open the `main` project's settings, and add cloud backends -(e.g., [AWS](reference/backends/aws.md), [GCP](reference/backends/gcp.md), [Azure](reference/backends/azure.md), [Lambda](reference/backends/lambda.md), etc.). +[//]: # (![](../assets/images/dstack-hub-view-project-empty.png){ width=800 }) [//]: # (Once cloud backends are configured, `dstack` will be able to provision cloud resources across configured clouds, ensuring) [//]: # (the best price and higher availability.) @@ -125,7 +123,7 @@ the [gateway](guides/services.md#configure-a-gateway-address). [//]: # ( ) [//]: # ( Each folder may have one default configuration file named `.dstack.yml`.) -For more details on the syntax of configuration file, refer to the [`.dstack.yml` Reference](../docs/reference/dstack.yml/index.md). +For more details on the file syntax, refer to [`.dstack.yml`](../docs/reference/dstack.yml/index.md). ## Run the configuration @@ -179,55 +177,18 @@ Launching in *reload mode* on: http://127.0.0.1:7860 (Press CTRL+C to quit) [//]: # (!!! info "Port forwarding") [//]: # ( By default, `dstack` forwards the ports used by dev environments and tasks to your local machine for convenient access.) -For more details on how the `dstack run` command works, refer to the [CLI Reference](reference/cli/run.md). - -[//]: # (TODO: Not sure if port forwarding is worth mentioning) -[//]: # (TODO: Mention resource configuration and/or profiles) -[//]: # (TODO: Move .gitignore to the `dstack run`'s reference page) - -[//]: # (## Profiles) -[//]: # () -[//]: # (If you have [configured](projects.md) a project that runs dev environments and tasks in the cloud, you can define multiple) -[//]: # (profiles. Each profile can configure the project to use and the resources required for the run.) -[//]: # () -[//]: # (To define profiles, create the `profiles.yml` file in the `.dstack` folder within your project directory. Here's an example:) -[//]: # () -[//]: # (
) -[//]: # () -[//]: # (```yaml) -[//]: # (profiles:) -[//]: # ( - name: gcp-t4) -[//]: # ( project: gcp) -[//]: # ( ) -[//]: # ( resources:) -[//]: # ( memory: 24GB) -[//]: # ( gpu:) -[//]: # ( name: T4) -[//]: # ( ) -[//]: # ( spot_policy: auto) -[//]: # ( retry_policy:) -[//]: # ( limit: 30min) -[//]: # ( max_duration: 1d) -[//]: # ( ) -[//]: # ( default: true) -[//]: # (```) -[//]: # () -[//]: # (
) - -[//]: # (!!! info "Spot instances") -[//]: # ( If `spot_policy` is set to `auto`, `dstack` prioritizes spot instances.) -[//]: # ( If these are unavailable, it uses `on-demand` instances. To cut costs, set `spot_policy` to `spot`.) -[//]: # ( ) -[//]: # ( If `dstack` can't find capacity, an error displays. To enable continuous capacity search, use `retry_policy` with a ) -[//]: # ( `limit`. For example, setting it to `30min` makes `dstack` search for capacity for 30 minutes.) -[//]: # () -[//]: # ( Note that spot instances are significantly cheaper but can be interrupted. Your code should ideally ) -[//]: # ( handle interruptions and resume work from saved checkpoints.) -[//]: # () -[//]: # (Now, if you use the `dstack run` command, `dstack` will use the default profile.) -[//]: # () -[//]: # (!!! info "Multiple profiles") -[//]: # ( You can define multiple profiles according to your needs and use any of them with the `dstack run` command by specifying) -[//]: # ( the desired profile using the `--profile` argument.) -[//]: # () -[//]: # (For more details on the syntax of the `profiles.yml` file, refer to the [Reference](reference/profiles.yml.md).) \ No newline at end of file +For more details on the run command, refer to [`dstack run`](reference/cli/run.md). + +### Requesting resources + +You can request resources using the [`--gpu`](reference/cli/run.md#GPU) +and [`--memory`](reference/cli/run.md#MEMORY) arguments with `dstack run`, +or through [`resources`](reference/profiles.yml.md#RESOURCES) with `.dstack/profiles.yml`. + +Both the [`dstack run`](reference/cli/run.md) command and [`.dstack/profiles.yml`](reference/profiles.yml.md) +support various other options, including requesting spot instances, defining the maximum run duration or price, and +more. + +!!! info "Automatic instance discovery" + `dstack` will automatically select the suitable instance type from a cloud provider and region with the best + price and availability. \ No newline at end of file diff --git a/docs/docs/projects.md b/docs/docs/projects.md deleted file mode 100644 index 27228b647..000000000 --- a/docs/docs/projects.md +++ /dev/null @@ -1,69 +0,0 @@ -# Projects - -A project allows you to configure where to run dev environments, tasks, and services, as well as manage users that -access it. - -At startup, `dstack` sets up a default project for local execution. To run dev environments, tasks, and services in your -desired cloud account (AWS, GCP, Azure, etc.), you must create the corresponding project and configure the `dstack` CLI to use it. - -!!! info "NOTE:" - You can configure multiple projects and switch between them using the `dstack` CLI. - -## Creating projects - -To create a new project, log in to the Hub application, open the `Projects` page, and click the `Add` button. - -![](../assets/images/dstack-hub-create-project.png){ width=800 } - -Then, you need to select the type of backend where you want to provision infrastructure and store data, and provide the corresponding backend settings. -For instructions specific to a particular cloud, please refer to the relevant sections below. - -??? info "AWS" - To use AWS, you will require an S3 bucket for storing state and artifacts, as well as credentials to access the - corresponding cloud services. - - [Learn more →](reference/backends/aws.md){ .md-button .md-button--primary } - -??? info "GCP" - To use GCP, you will require a cloud bucket for storing state and artifacts, as well as a - service account to access the corresponding cloud services. - - [Learn more →](reference/backends/gcp.md){ .md-button .md-button--primary } - -??? info "Azure" - To use Azure, you will require a storage account for storing state and artifacts, as well as Azure AD app credentials - to access the corresponding cloud services. - - [Learn more →](reference/backends/azure.md){ .md-button .md-button--primary } - -??? info "Lambda" - To use Lambda, you will need an S3 bucket for storing state and artifacts, along with AWS credentials and a Lambda Cloud - API key. - - [Learn more →](reference/backends/lambda.md){ .md-button .md-button--primary } - -## Configuring the CLI - -Once you have created the project, you will find the `CLI` code snippet in its `Settings`. - -![](../assets/images/dstack-hub-view-project.png){ width=800 } - -Execute this code in the terminal to configure the project with the CLI. - -
- -```shell -$ dstack config --url http://localhost:3000 --project aws --token 34ccc68b-8579-44ff-8923-026619ddb20d -``` - -
- -To use this project with the CLI, you need to pass its name using the `--project` argument in CLI commands (such as -`dstack run`, `dstack init`, etc.) - -!!! info "NOTE:" - If you want to set the project as the default, add the `--default` flag to the `dstack config` command. - -[//]: # (TODO [TASK]: Mention where the configuration is stored (i.e. `~/.dstack/config.yaml`) - -[//]: # (TODO [MEDIUM]: Currently, there is no settings, such as quota management, max duration, etc.) \ No newline at end of file diff --git a/docs/docs/reference/cli/run.md b/docs/docs/reference/cli/run.md index 06b89d957..95560b686 100644 --- a/docs/docs/reference/cli/run.md +++ b/docs/docs/reference/cli/run.md @@ -8,33 +8,23 @@ This command runs a given configuration. ```shell $ dstack run --help -Usage: dstack run [-h] [-f FILE] [-n NAME] [-y] [-d] [--project PROJECT] - [--profile PROFILE] [--reload] WORKING_DIR [ARGS ...] - -Positional Arguments: - WORKING_DIR The working directory of the run - ARGS Run arguments - -Options: - -f, --file FILE The path to the run configuration file. Defaults to - WORKING_DIR/.dstack.yml. - -n, --name NAME The name of the run. - If not specified, a random name is assigned. - -y, --yes Do not ask for plan confirmation - -d, --detach Do not poll logs and run status - --reload Enable auto-reload - --project PROJECT The name of the project - --profile PROFILE The name of the profile +Usage: dstack run WORKING_DIR [-f CONFIGURATION_FILE] [-n RUN_NAME] + [-e ENV ...] + [-p PORT ...] + [--gpu GPU] [--memory MEMORY] [--shm-size SHM_SIZE] + [--spot-auto | --spot | --on-demand] + [--no-retry | --retry-limit RETRY] + [--max-price MAX_PRICE] + [--backend BACKEND_TYPE ...] + [--profile PROFILE_NAME] [--project PROJECT_NAME] + [--reload] + [-d] + [-y] + [ARGS ...] ``` -[//]: # (TODO: Document `--port`, `--env`, `--gpu`, etc.) - -!!! info "Default configuration" - By default, the `dstack run` command looks for the default configuration file named `.dstack.yml` in the given working - directory. If your configuration file is named differently, you can specify a path to it using the `-f` argument. - ??? info ".gitignore" When running dev environments or tasks, `dstack` uses the exact version of code that is present in the folder where you use the `dstack run` command. @@ -43,56 +33,73 @@ Options: make sure to create a `.gitignore` file and include these large files or folders that you don't want to include when running dev environments or tasks. -[//]: # (TODO: Ports aren't part of the `dstack run --help` output) - ## Arguments reference -The following arguments are required: - -- `WORKING_DIR` - (Required) The working directory of the run (e.g. `.`) - -The following arguments are optional: - -- `-f FILE`, `--f FILE` – (Optional) The path to the run configuration file. Defaults to `WORKING_DIR/.dstack.yml`. -- `-n NAME`, `--name NAME` - (Optional) The name of the run. If not specified, a random name is assigned. -- `-y`, `--yes` - (Optional) Do not ask for plan confirmation -- `-d`, `--detach` – (Optional) Run in the detached mode to disable logs and run status polling. By default, the run is in the attached mode, so the logs are printed in real-time. -- `--reload` – (Optional) Enable auto-reload -- `--project PROJECT` – (Optional) The name of the project -- `--profile PROFILE` – (Optional) The name of the profile - -[//]: # (- `-t TAG`, `--tag TAG` – (Optional) A tag name. Warning, if the tag exists, it will be overridden.) -- `--backend [BACKEND ...]` – (Optional) Backend(s) to consider for provisioning -- `--max-offers MAX_OFFERS` – (Optional) The maximum number of best offers shown in run plan -- `-p PORT`, `--port PORT` – (Optional) Requests port or define mapping (`EXTERNAL_PORT:INTERNAL_PORT`) -- `-e ENV`, `--env ENV` – (Optional) Set environment variable (`NAME=value`) -- `--gpu` – (Optional) Request a GPU for the run. Specify any: name, count, memory (`NAME:COUNT:MEMORY` or `NAME` or `COUNT:MEMORY`, etc...) -- `--max-price` – (Optional) The maximum price per hour, in dollars -- `ARGS` – (Optional) Use `ARGS` to pass custom run arguments - -Spot policy (the arguments are mutually exclusive): - -- `--spot-policy` – The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`. -- `--spot` – A shorthand for `--spot-policy spot` -- `--on-demand` – A shorthand for `--spot-policy on-demand` -- `--spot-auto` – A shorthand for `--spot-policy auto` - -Retry policy (the arguments are mutually exclusive): - -- `--no-retry` – Do not retry the run on failure -- `--retry` – Retry the run on failure. Use a default retry limit (1h). -- `--retry-limit` – Retry the run on failure with a custom retry limit, e.g. 4h or 1d - -Build policies: - -[//]: # (- `--use-build` – Use the build if available, otherwise fail) -- `--build` – If the environment is not pre-built yet, pre-build it. If the environment is already pre-built, reuse it. -- `--force-build` – Force pre-building the environment, even if it has been pre-built before. - -[//]: # (- `--build-only` — Just create the build and save it) - -[//]: # (Tags should be dropped) - -[//]: # (!!! info "NOTE:") -[//]: # ( By default, the run is in the attached mode, so you'll see the output in real-time.) - +### General + +- WORKING_DIR – (Required) The working directory of the run. Example: `dstack run .` +- -f CONFIGURATION_FILE, --file CONFIGURATION_FILE – (Optional) The configuration file to run. + If not specified, `dstack` will use the `WORKING_DIR/.dstack.yml` configuration by default. + Example: `dstack run . -f llama-2/serve.dstack.yml` +- -n RUN_NAME, --name RUN_NAME – (Optional) The name of the run. If not specified, `dstack` will pick a random name. + Example: `dstack run . -n my-dev-environment` +- -e ENV, --env ENV – (Optional) + Set environment variables. Example: `dstack run . -e MYVAR1=foo -e MYVAR2=bar` +- -p PORT, --port PORT – (Optional) + Configure port forwarding. Examples: `dstack run . -p 8000` (forwards the `8000` port of the task to the same port on + your local machine) or `dstack run . -p 8001:8000` (forwards the `8000` port of the task to the `8001` port on + your local machine). +- -y, --yes - (Optional) Do not ask a confirmation +- -p PROFILE, --profile PROFILE – (Optional) The name of the profile +- -p PROJECT, --project PROJECT – (Optional) The name of the project +- -d, --detach – (Optional) Run in the detached mode (no logs are shown in the output, and the command doesn't wait until the run is finished) + +### Compute + +- --gpu GPU – (Optional) Request GPU. Examples: + `dstack run . --gpu A10` or `dstack run . --gpu 24B` or `dstack run . --gpu A100:8` +- --memory MEMORY – (Optional) The minimum size of memory. Examples: + `dstack run . --memory 64GB` +- --shm_size SHM_SIZE – (Optional) The size of shared memory. + Required to set if you are using parallel communicating processes. + Example: `dstack run . --shm_size 8GB` +- --max_price MAX_PRICE – (Optional) The maximum price per hour, in dollars. + Example: `dstack run . --max-price 1.1` +- -backend BACKEND – (Optional) + Force using listed backends only. Possible values: `aws`, `azure`, `gcp`, `lambda`. + If not specified, all configured backends are tried. Example: `dstack run . --backend aws --backend gcp` +- --spot – (Optional) Force using spot instances only. + Example: `dstack run . --spot` +- --spot-auto – (Optional) Force using spot instances + if they are available and on-demand instances otherwise. This is the default for tasks and services. + Example: `dstack run . --spot-auto` +- --on-demand – (Optional) Force using on-demand instances. + This is the default for dev environments. + Example: `dstack run . --on-demand` +- --no-retry – (Optional) Do not wait for capacity. + This is the default. + Example: `dstack run . --no-retry` +- --retry-limit RETRY_LIMIT – (Optional) The duration + to wait for capacity. + Example: `dstack run . --retry-limit 3h` or `dstack run . --retry-limit 2d` +- --max-duration MAX_DURATION – (Optional) The maximum duration of a run. + After it elapses, the run is forced to stop. Protects from running idle instances. Defaults to `6h` for dev environments and to `72h` for tasks. + Examples: `dstack run . --max-duration 3h` or `dstack run . --max-duration 2d` or `dstack run . --max-duration off`. + +### Arguments + +- ARGS – (Optional) Pass [custom arguments](../../guides/tasks.md#parametrize-tasks) to the task. + +### Experimental + +- --build – Pre-build the environment (if you're using the `build` property in [`dstack.yml`](../dstack.yml/index.md)) if it's not pre-built yet. + Example: `dstack run . --build` +- --force-build – Force pre-building the environment (if you're using the `build` property in [`dstack.yml`](../dstack.yml), even if it has been pre-built before. + Example: `dstack run . --force-build` +- --reload – (Optional) Enable auto-reloading of your local changes into the running task. + Maybe be used together with Streamlit, Fast API, Gradio for auto-reloading local changes. + Example: `dstack run . --reload` +- --max-offers MAX_OFFERS – (Optional) Set the maximum number of offers shown + before the confirmation. + +[//]: # (TODO: Add a link to reference/dstack.yml#build) diff --git a/docs/docs/reference/profiles.yml.md b/docs/docs/reference/profiles.yml.md index f757dd805..df90843a6 100644 --- a/docs/docs/reference/profiles.yml.md +++ b/docs/docs/reference/profiles.yml.md @@ -1,29 +1,56 @@ # profiles.yml -Profiles configure the project to use and the resources required for the run. - -Profiles are defined in the `.dstack/profiles.yml` file within your project directory. - -Below is a full reference of all available properties. - -- `profiles` - (Required) The root property (of an `array` type) - - `name` - (Required) The name of the profile - - `backends` - (Optional) A list of backends that will be tried for provisioning. Supported backends are `local`, `aws`, `azure`, `gcp`, and `lambda`. If not specified, all configured backends are tried. - - `resources` - (Optional) The minimum required resources - - `memory` - (Optional) The minimum size of RAM memory (e.g., `"16GB"`). - - `gpu` - (Optional) The minimum number of GPUs, their model name and memory - - `name` - (Optional) The name of the GPU model (e.g., `"K80"`, `"V100"`, `"A100"`, etc) - - `count` - (Optional) The minimum number of GPUs. Defaults to `1`. - - `memory` - (Optional) The minimum size of GPU memory (e.g., `"16GB"`) - - `shm_size` - (Optional) The size of shared memory (e.g., `"8GB"`). If you are using parallel communicating - processes (e.g., dataloaders in PyTorch), you may need to configure this. - - `spot_policy` - (Optional) The policy for provisioning spot or on-demand instances: `spot`, `on-demand`, or `auto`. `spot` provisions a spot instance. `on-demand` provisions a on-demand instance. `auto` first tries to provision a spot instance and then tries on-demand if spot is not available. Defaults to `on-demand` for dev environments and to `auto` for tasks. - - `retry_policy` - (Optional) The policy for re-submitting the run. - - `retry` - (Optional) Whether to retry the run on failure or not. Default to `false` - - `limit` - (Optional) The maximum period of retrying the run, e.g., `4h` or `1d`. Defaults to `1h` if `retry` is `true`. - - `max_duration` - (Optional) The maximum duration of a run (e.g., `2h`, `1d`, etc). After it elapses, the run is forced to stop. Protects from running idle instances. Defaults to `6h` for dev environments and to `72h` for tasks. Use `max_duration: off` to disable maximum run duration. - - `max_price` - (Optional) The maximum price per hour, in dollars - -[//]: # (TODO: Add examples) - -[//]: # (TODO: Add more explanations of how it works, incl. how to pass defined profiles to the CLI) \ No newline at end of file +Instead of configuring resources, spot and retry policies, max price, and other parameters +through [`dstack run`](cli/run.md), you can use profiles. +To set up a profile, create the `.dstack/profiles.yml` file in the root folder of the project. + +## Usage example + +
+ +```yaml +profiles: + - name: large + + resources: + memory: 24GB # (Optional) The minimum amount of RAM memory + gpu: + name: A100 # (Optional) The name of the GPU + memory: 40GB # (Optional) The minimum amount of GPU memory + shm_size: 8GB # (Optional) The size of shared memory + + spot_policy: auto # (Optional) The spot policy. Supports `spot`, `on-demand, and `auto`. + + max_price: 1.5 # (Optional) The maximum price per instance per hour + + max_duration: 1d # (Optional) The maximum duration of the run. + + retry: + retry-limit: 3h # (Optional) To wait for capacity + + backends: [azure, lambda] # (Optional) Use only listed backends + + default: true # (Optional) Activate the profile by default +``` + +
+ +### Schema reference + +- profiles - (Required) + - name - (Required) The name of the profile. Can be passed via [`--profile PROFILE`](cli/run.md#PROFILE) to `dstack run` + - resources - (Optional) + - memory - (Optional) The minimum size of memory. Example: `64GB` + - gpu - (Optional) + - name - (Optional) The name of the GPU. Examples: `T4`, `V100`, `A10`, `L4`, `A100`, etc.) + - count - (Optional) The minimum number of GPUs. Defaults to `1`. + - memory - (Optional) The minimum size of GPU memory. Example: `20GB` + - shm_size - (Optional) The size of shared memory. + Required to set if you are using parallel communicating processes. Example: `8GB` + - spot_policy - (Optional) The spot policy. Example: `spot` (spot instances only), `on-demand` (on-demand instances only), or `auto` (spot if available or on-demand otherwise). Defaults to `on-demand` for dev environments and to `auto` for tasks and services. + - retry_policy - (Optional) The policy for re-submitting the run. + - limit - (Optional) The duration to wait for capacity. Example: `3h` or `2d`. + - max_duration - (Optional) The maximum duration of a run. After it elapses, the run is forced to stop. Protects from running idle instances. Defaults to `6h` for dev environments and to `72h` for tasks. Examples: `3h` or `2d` or `off`. + - max_price - (Optional) The maximum price per hour, in dollars. Example: `1.1` or `0.8` + - backends - (Optional) Force using listed backends only. Possible values: `aws`, `azure`, `gcp`, `lambda`. If not specified, all configured backends are tried. + - default - (Optional) If set to `true`, it will be activated by default \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index b6965b091..bb2e85a9e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -142,8 +142,8 @@ nav: - pip: docs/installation/pip.md - Docker: docs/installation/docker.md - HF Spaces: docs/installation/hf-spaces.md -# - Projects: docs/projects.md - Guides: + - Clouds: docs/guides/clouds.md - Dev environments: docs/guides/dev-environments.md - Tasks: docs/guides/tasks.md - Services: docs/guides/services.md