Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update configuration #157

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −11
Open

update configuration #157

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 22 additions & 11 deletions resourcing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,27 @@ icon: "desktop"

## Running Locally

When running locally through Docker, we recommend making at least 4vCPU cores and 10GB of RAM available to Docker
(16GB is preferred). This can be controlled in the **Resources** section of the Docker Desktop settings menu.
When running locally through Docker, we recommend allocating at least 4 vCPU cores and 10GB of RAM to Docker (16GB is preferred). You can adjust these settings in the **Resources** section of the Docker Desktop settings menu.

To prevent Danswer from consuming excessive resources and potentially impacting your system's performance, you can set resource limits on the Docker containers. This is done by applying the `docker-compose.resources.yml` file when starting Danswer. This file allows you to control and limit the resources used by each Danswer component.

To apply resource limits:

1. Use the `docker-compose.resources.yml` file provided in the `deployment/docker_compose` directory. This file contains default resource limits and reservations for each service.

2. When starting Danswer, include the resource file in your Docker Compose command:

```bash
docker-compose -f docker-compose.dev.yml -f docker-compose.resources.yml up
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this match the cmds in the quickstart (e.g. -p danswer-stack + remove the - in docker-compose)?

```

3. You can adjust the resource limits by setting environment variables. For more details on available environment variables and default values, refer to the [`docker-compose.resources.yml`](https://github.com/onyx-dot-app/onyx/blob/main/deployment/docker_compose/docker-compose.resources.yml) file.

## Single Cloud Instance

We generally recommend setting everything up on a single instance
(e.g. an AWS EC2 instance, a Google Compute Engine instance, an Azure VM, etc.) via Docker Compose as it's
the simplest way to get started. For a step-by-step guide on how to do this, checkout our
the simplest way to get started. For a step-by-step guide on how to do this, checkout our
[EC2 deployment guide](https://docs.danswer.dev/production/aws/ec2).

For most use cases a single reasonably sized instance should be more than enough to guarantee excellent
Expand All @@ -24,14 +37,14 @@ If you go with this approach, we recommend:

- **CPU:** >= 4 vCPU cores (we recommend 8 vCPU cores if possible)
- **Memory:** >= 10 GB of RAM (for best performance, 16 is recommended)
- Note: If you are switching embedding models, you will need >= 11 GB RAM as both sets of models need
- Note: If you are switching embedding models, you will need >= 11 GB RAM as both sets of models need
to be loaded in simultaneously
- **Disk:** >= 50 GB + ~2.5x the size of the indexed documents. Disk is generally very cheap, so
we would recommend getting extra disk beyond this recommendation to be safe.
- Note: Vespa does not allow writes when disk usage is >75%, so make sure to always have some headroom here
- Note: old, unused docker images often take up a bunch of space when performing frequent upgrades of Danswer.
- Note: Vespa, used by Danswer for document indexing, requires Haswell (2013) or later CPUs. For older CPUs, use the `vespaengine/vespa-generic-intel-x86_64` image in your `docker-compose.yml`. This generic image is slower but ensures compatibility. For details, see [Vespa CPU Support](https://docs.vespa.ai/en/cpu-support.html).
To clean up these unused images, run: `docker system prune --all`.
To clean up these unused images, run: `docker system prune --all`.

For reference, we have chosen to give each Danswer Cloud customer a `m7g.xlarge` instance by default,
which has 4vCPU cores + 16GB of RAM + 200GB of disk space. We're comfortable using 4vCPU cores in a
Expand All @@ -41,7 +54,7 @@ production deployment.

## Kubernetes / AWS ECS

If you prefer to give each component it's own dedicated resources for more efficient
If you prefer to give each component it's own dedicated resources for more efficient
scaling, we recommend giving each container access to at least the following resources:

`api_server` - 1 CPU, 2Gi Memory
Expand All @@ -52,11 +65,9 @@ scaling, we recommend giving each container access to at least the following res

`postgres` - 500m CPU, 2Gi Memory

`vespa` - >=4 CPU, >= 4Gi Memory. This is the bare minimum, and we would generally recommend
higher than this. The resources required here also scales linearly with the number of documents indexed.
For reference, with 50GB of documents, we would generally recommend at least 10 CPU, 20Gi Memory +
`vespa` - >=4 CPU, >= 4Gi Memory. This is the bare minimum, and we would generally recommend
higher than this. The resources required here also scales linearly with the number of documents indexed.
For reference, with 50GB of documents, we would generally recommend at least 10 CPU, 20Gi Memory +
tuning the `VESPA_SEARCHER_THREADS` environment variable.

`nginx` - 250m CPU, 128Mi Memory