Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(self-managed): refine docker compose page #4816

Merged
merged 6 commits into from
Jan 10, 2025
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ in terms of tenant assignment, Operate - Zeebe connection must be secured. Check

### Troubleshooting multi-tenancy in Operate

If users can view data from the `<default>` tenant only and no data from other tenants (and you have not [configured multi-tenancy using Helm](https://github.com/camunda/camunda-platform-helm/tree/main/charts/camunda-platform-8.6#global-parameters)), multi-tenancy is not enabled in Operate. Refer to the [configuration instructions above](#multi-tenancy).
If users can view data from the `<default>` tenant only and no data from other tenants (and you have not [configured multi-tenancy using Helm](https://artifacthub.io/packages/helm/camunda/camunda-platform#global-parameters)), multi-tenancy is not enabled in Operate. Refer to the [configuration instructions above](#multi-tenancy).

If multi-tenancy is enabled in Operate but disabled in [Identity](/self-managed/identity/what-is-identity.md), users will not have any tenant authorizations in Operate
and will not be able to access the data of any tenants in Operate.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Multi-tenancy must be enabled for each required component. Using the single glob
## Helm charts

When using Helm charts, you can enable multi-tenancy globally with the flag `global.multitenancy.enabled`.
Visit [the Helm chart configuration](https://github.com/camunda/camunda-platform-helm/tree/main/charts/camunda-platform-8.6#global-parameters) for additional details.
Visit [the Helm chart configuration](https://artifacthub.io/packages/helm/camunda/camunda-platform#global-parameters) for additional details.

## Environment variables

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Running a dual-region configuration requires users to detect and manage any regi

- A dual-region Camunda 8 setup installed in two different regions, preferably derived from our [AWS dual-region concept](/self-managed/setup/deploy/amazon/amazon-eks/dual-region.md).
- In that guide, we're showcasing Kubernetes dual-region installation, based on the following tools:
- [Helm (3.x)](https://helm.sh/docs/intro/install/) for installing and upgrading the [Camunda Helm chart](https://github.com/camunda/camunda-platform-helm).
- [Helm (3.x)](https://helm.sh/docs/intro/install/) for installing and upgrading the [Camunda Helm chart](https://artifacthub.io/packages/helm/camunda/camunda-platform).
- [Kubectl (1.30.x)](https://kubernetes.io/docs/tasks/tools/#kubectl) to interact with the Kubernetes cluster.
- (deprecated) [zbctl](/apis-tools/community-clients/cli-client/index.md) to interact with the Zeebe cluster.
- `cURL` or similar to interact with the REST API.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ for container to container communication align with the IP ranges Keycloak consi
[external requests documentation](https://www.keycloak.org/docs/latest/server_admin/#_ssl_modes)
to be external and therefore require SSL.

As the [Camunda Helm Charts](https://github.com/camunda/camunda-platform-helm) currently do
As the [Camunda Helm Charts](hhttps://artifacthub.io/packages/helm/camunda/camunda-platform) currently do
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
not provide support for the distribution of the Keycloak TLS key to the other containers, we recommend viewing the solution available in the
[Identity documentation](/self-managed/identity/troubleshooting/troubleshoot-identity.md#solution-2-identity-making-requests-from-an-external-ip-address).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ This guide requires you to have previously completed or reviewed the steps taken
## Prerequisites

- An [AWS account](https://docs.aws.amazon.com/accounts/latest/reference/accounts-welcome.html) to create resources within AWS.
- [Helm (3.16+)](https://helm.sh/docs/intro/install/) for installing and upgrading the [Camunda Helm chart](https://github.com/camunda/camunda-platform-helm).
- [Helm (3.16+)](https://helm.sh/docs/intro/install/) for installing and upgrading the [Camunda Helm chart](https://artifacthub.io/packages/helm/camunda/camunda-platform).
- [Kubectl (1.30+)](https://kubernetes.io/docs/tasks/tools/#kubectl) to interact with the cluster.
- [Terraform (1.9+)](https://developer.hashicorp.com/terraform/downloads)

Expand Down
252 changes: 240 additions & 12 deletions docs/self-managed/setup/deploy/local/docker-compose.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,254 @@
---
id: docker-compose
title: "Docker Compose"
keywords: ["camunda docker"]
keywords: ["camunda docker", "docker compose"]
---

A Docker Compose configuration to run Zeebe, Operate, Tasklist, Optimize, Identity, and Connectors Bundle is available in the [camunda-platform](https://github.com/camunda/camunda-platform/blob/main/docker-compose.yaml) repository.
Follow the instructions in the [README](https://github.com/camunda/camunda-platform#using-docker-compose).
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
While the [Docker images](/self-managed/setup/deploy/other/docker.md) themselves are supported for production usage, the Docker Compose files are designed to be used by developers to run an environment locally; they are not designed to be used in production. We recommend to use [Kubernetes](/self-managed/setup/install.md) in production.
A Docker Compose configuration to run Camunda Self-Managed components (Zeebe, Operate, Tasklist, Optimize, Identity, and Connectors).

:::danger
While the [Docker images](/self-managed/setup/deploy/other/docker.md) themselves are supported for production usage, the Docker Compose files are designed to be used by developers to run an environment locally, and are not designed to be used in production. We recommend [Kubernetes](/self-managed/setup/install.md) for production use cases.
:::

## Prerequisites

- **Docker Compose:** Version 1.27.0+ (supports the [latest compose specification](https://docs.docker.com/compose/compose-file/)).
- **Docker:** Version 20.10.16+
- **Keycloak:** (Local development only). Add Keycloak to resolve to 127.0.0.1 on your local machine, and set `KEYCLOAK_HOST=keycloak` in the `.env` file for token refresh and logout functionality.

## Run Camunda 8 with Docker Compose

To start a complete instance of Camunda 8 Self-Managed environment locally:

1. Clone the [Camunda Self-Managed repository](https://github.com/camunda/camunda-self-managed), and navigate to the directory of the desired Camunda version (for example, `/version/camunda-8.6`).
2. Run the following command:

```shell
docker compose up -d
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
```

1. Wait for the environment to initialize. This may take several minutes. Monitor the logs, especially the Keycloak container log, to ensure the components have started.

### Configuration options

Running `docker compose up -d` starts all Camunda components, including Identity. The [Camunda Self-Managed repository](https://github.com/camunda/camunda-self-managed) also contains the additional configuration files for lightweight development.

- **docker-compose.yaml:** Contains all Camunda 8 Components for a full stack deployment: Zeebe, Operate, Tasklist, Connectors, Optimize, Identity, Elasticsearch, Keycloak, Web Modeler, and PostgreSQL.
- **docker-compose-core.yaml:** Contains Camunda 8 Orchestration cluster components: Zeebe, Tasklist, Operate, and Connectors.
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
- **docker-compose-web-modeler.yaml:** Contains the Camunda 8 Web Modeler standalone installation. For more information, see the [Web Modeler instructions](#web-modeler).

To start Camunda with an alternate configuration, specify a file using the following command:

```shell
docker compose -f docker-compose-core.yaml up -d
```

### Access components

Running components can be accessed with the username `demo` and password `demo`:

- Operate: [http://localhost:8081](http://localhost:8081)
- Tasklist: [http://localhost:8082](http://localhost:8082)
- Optimize: [http://localhost:8083](http://localhost:8083)
- Identity: [http://localhost:8084](http://localhost:8084)
- Web Modeler [http://localhost:8070](http://localhost:8070)
- Elasticsearch: [http://localhost:9200](http://localhost:9200)

Keycloak is used to manage users, and can be accessed with the user `admin` and password `admin`:

- Keycloak: [http://localhost:18080/auth/](http://localhost:18080/auth/)

The workflow engine Zeebe is available using gRPC:

- Zeebe: `localhost:26500`

### Stop Camunda 8

Run the following command to stop Camunda 8:

```shell
docker compose down -v
jessesimpson36 marked this conversation as resolved.
Show resolved Hide resolved
```

:::caution
This will also delete any data you created.
:::

To tear down the environment and keep any data, run the following command:

```shell
docker compose down
```

Zeebe, Operate, Tasklist, along with Optimize require a separate network from Identity as you'll see in the docker-compose file.
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved

## Modeling and process execution

### Web Modeler

:::info
Non-production installations of Web Modeler are restricted to five collaborators per project. Refer to the [licensing documentation](/docs/reference/licenses.md) for more information.
:::

#### Standalone setup

Web Modeler is included by default when running the `docker-compose.yaml` configuration, but can be run independently using Identity, Keycloak and Postgres as dependencies.

The following command uses the provided `docker-compose-web-modeler.yaml` configuration file to only start Web Modeler and its dependencies:

```shell
docker compose -f docker-compose-web-modeler.yaml up -d
```

To tear down the environment (including all data and volumes), run the following command:

```shell
docker compose -f docker-compose-web-modeler.yaml down -v
```

#### Deploy or execute a process

The local Zeebe instance started when using the provided `docker-compose.yaml` is pre-configured in Web Modeler.

Once you are ready to deploy or execute a process, this instance can be used without needing to enter the cluster endpoint manually. The correct authentication type is also preset based on the [`ZEEBE_AUTHENTICATION_MODE` environment variable](#securing-the-zeebe-api).

<Tabs groupId="web modeler" defaultValue="with" queryString values={
[
{label: 'With Zeebe request authentication', value: 'with' },
{label: 'Without authentication', value: 'without' },
]}>

<TabItem value="with">

If you enabled authentication for gRPC requests on Zeebe, use the following client credentials when deploying and executing processes:

- Client ID: `zeebe`
- Client secret: `zecret`

:::note
The correct OAuth token URL and audience are preset internally.
:::

</TabItem>

<TabItem value='without'>

No additional input is required.

</TabItem>
</Tabs>

#### Emails

The provided configuration includes [Mailpit](https://github.com/axllent/mailpit) as a test SMTP server. It captures all emails sent by Web Modeler, but does not forward them to the actual recipients.

You can access emails in Mailpit's Web UI at [http://localhost:8075](http://localhost:8075).

### Desktop Modeler

:::info
Desktop Modeler is [open source and free to use](https://github.com/camunda/camunda-modeler).
:::

This Docker Compose configuration serves two purposes:
[Download the Desktop Modeler](https://camunda.com/download/modeler/) and start modeling BPMN, DMN and Camunda Forms on your local machine.

### Use the CLI

<Tabs groupId="cli" defaultValue="with" queryString values={
[
{label: 'With Zeebe request authentication', value: 'with' },
{label: 'Without authentication', value: 'without' },
]}>

<TabItem value="with">

If you enabled authentication for GRPC requests on Zeebe, provide the following client credentials when deploying and executing processes:

- Authentication: `OAuth`
- URL: `http://localhost:26500`
- Client ID: `zeebe`
- Client secret: `zecret`
- OAuth URL: `http://localhost:18080/auth/realms/camunda-platform/protocol/openid-connect/token`
- Audience: `zeebe-api`

</TabItem>

<TabItem value="without">

Once you are ready to deploy or execute processes, use the following settings to deploy to the local Zeebe instance:

- Authentication: `None`
- URL: `http://localhost:26500`

</TabItem>
</Tabs>

## Secure the Zeebe API

1. It can be used to start up a development environment locally.
2. It documents how the various components need to be wired together.
By default, the Zeebe GRPC API is publicly accessible without requiring any client credentials for development purposes.

To enable authentication of GRPC requests in Zeebe, setting the environment variable `ZEEBE_AUTHENTICATION_MODE` to `identity`, for example:

```shell
ZEEBE_AUTHENTICATION_MODE=identity docker compose up -d
```

The default value can also be modified directly in the included `.env` file.

## Connectors

Both the full and lightweight Docker Compose files contain a configuration for [Out-of-the-box Connectors](/components/connectors/out-of-the-box-connectors/available-connectors-overview.md). Refer to the [Connector installation guide](/self-managed/connectors-deployment/install-and-start.md) for details on how to provide the related Connector templates for modeling.

### Connector secrets

Secrets can be added into the Connector runtime using the included `connector-secrets.txt` file. Add secrets in the format `NAME=VALUE`
per line. The secrets will then be available in the Connector runtime in the format `secrets.NAME`.

### Custom connectors

To add custom Connectors, create a new Docker Image bundling them as described in the [Connectors repository](https://github.com/camunda/connectors).

Alternatively, you can mount new Connector JARs as volumes into the `/opt/app` folder by adding this to the Docker Compose file. Keep in mind that the Connector JARs must include all necessary dependencies inside the JAR.

## Kibana
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've formatted/edited these instructions, but the Kibana profile doesn't run for me in the provided 8.6 or alpha/8.7 compose file. Not sure if this is an error in my setup or in the profile. Can you verify Kibana works and that these instructions are correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried it with Camunda 8.6 Docker Compose, and it works as expected.
What issue/error do you have?


A [Kibana](https://www.elastic.co/kibana/) profile is available in the provided configuration files to support inspection and exploration of the Camunda 8 data in Elasticsearch.

Enable the profile by adding `--profile kibana` to your Docker Compose command:

```shell
docker compose --profile kibana up -d
```

This profile will start Kibana in addition to the default components. Kibana can be used to explore the records exported by Zeebe into Elasticsearch, or to discover the data in Elasticsearch used by the other components (for example, Operate).

Navigate to the Kibana web application and explore the data without login credentials:

- Kibana: [http://localhost:5601](http://localhost:5601)

:::note
We recommend to use [Helm + KIND](/self-managed/setup/deploy/local/local-kubernetes-cluster.md) instead of Docker Compose for local environments, as the Helm configurations are battle-tested and much closer to production systems.

You need to configure the index patterns in Kibana before you can explore the data.

- Go to `Management > Stack Management > Kibana > Index Patterns`.
- Create a new index pattern. For example, `zeebe-record-*` matches the exported records.
- If you don't see any indexes then make sure to export some data first (e.g. deploy a process). The indexes of the records are created when the first record of this type is exported.
- Go to `Analytics > Discover` and select the index pattern.

:::

## Web Modeler
## Troubleshooting

### Running on arm64 based hardware

When using arm64-based hardware (for example, an M1 or M2 Mac), the Keycloak container might not start, as Bitnami only
provides amd64-based images for versions < 22. You can build and tag an arm-based image locally using the following command:

```shell
$ DOCKER_BUILDKIT=0 docker build -t bitnami/keycloak:19.0.3 "https://github.com/camunda/camunda-platform.git#8.2.15:.keycloak/"
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved
```

An additional Docker Compose configuration to run Web Modeler is also available in the
[camunda-platform](https://github.com/camunda/camunda-platform/blob/main/docker-compose-web-modeler.yaml) repository. Follow the instructions in the [README](https://github.com/camunda/camunda-platform#web-modeler-self-managed) to utilize this configuration.
After building and tagging the image, you can start the environment.
2 changes: 1 addition & 1 deletion docs/self-managed/setup/guides/ingress-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Camunda 8 Helm chart doesn't manage or deploy Ingress controllers, it only deplo
## Preparation

- An Ingress controller should be deployed in advance. The examples below use the [ingress-nginx controller](https://github.com/kubernetes/ingress-nginx), but any Ingress controller could be used by setting `ingress.className`.
- TLS configuration is not handled in the examples because it varies between different workflows. It could be configured directly using `ingress.tls` options or via an external tool like [Cert-Manager](https://github.com/cert-manager/cert-manager) using `ingress.annotations`. For more details, check available [configuration options](https://github.com/camunda/camunda-platform-helm/tree/main/charts/camunda-platform-8.6#configuration).
- TLS configuration is not handled in the examples because it varies between different workflows. It could be configured directly using `ingress.tls` options or via an external tool like [Cert-Manager](https://github.com/cert-manager/cert-manager) using `ingress.annotations`. For more details, check available [configuration options](https://artifacthub.io/packages/helm/camunda/camunda-platform#configuration).

## Configuration

Expand Down
2 changes: 1 addition & 1 deletion docs/self-managed/setup/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,6 @@ For details on supported environments (e.g. Java or Elasticsearch versions), see
- [**Docker**](./deploy/other/docker.md): Component [Docker images](https://hub.docker.com/u/camunda) are available for use in production on Linux systems. Windows or macOS are only supported for development environments.
- [**Manual**](./deploy/local/manual.md): The Java applications can run on a local or virtual machine if it provides a supported Java Virtual Machine (JVM). This allows you to run Camunda on virtual machines or bare metal and offers a significant amount of flexibility. However, you will need to configure the details for the components to interact correctly yourself. We consider this a last resort. Note that Windows/Mac is **not** supported for production usage of Zeebe.

A **Docker Compose** configuration file is also provided for local development, and is **not** optimized for production usage. You can find setup instructions in the [camunda-platform](https://github.com/camunda/camunda-platform) repository.
A **Docker Compose** configuration file is also provided for local development, and is **not** optimized for production usage. You can find setup instructions in the [Camunda Self-Managed repository](https://github.com/camunda/camunda-self-managed#docker-compose).
aabouzaid marked this conversation as resolved.
Show resolved Hide resolved

For more details on deployment, see [sizing your environment](../../components/best-practices/architecture/sizing-your-environment.md#camunda-8-self-managed).
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ in terms of tenant assignment, Operate - Zeebe connection must be secured. Check

### Troubleshooting multi-tenancy in Operate

If users can view data from the `<default>` tenant only and no data from other tenants (and you have not [configured multi-tenancy using Helm](https://github.com/camunda/camunda-platform-helm/blob/main/charts/camunda-platform-8.3/README.md#global-parameters)), multi-tenancy is not enabled in Operate. Refer to the [configuration instructions above](#multi-tenancy).
If users can view data from the `<default>` tenant only and no data from other tenants (and you have not [configured multi-tenancy using Helm](https://helm.camunda.io/camunda-platform/version-matrix/camunda-8.3/)), multi-tenancy is not enabled in Operate. Refer to the [configuration instructions above](#multi-tenancy).

If multi-tenancy is enabled in Operate but disabled in [Identity](/self-managed/identity/what-is-identity.md), users will not have any tenant authorizations in Operate
and will not be able to access the data of any tenants in Operate.
Expand Down
Loading
Loading