-
Notifications
You must be signed in to change notification settings - Fork 87
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(docs): keep getting started high level (#410)
point to detailed troubleshooting guide only if needed
- Loading branch information
1 parent
afba2e2
commit b3d7939
Showing
8 changed files
with
230 additions
and
35 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
--- | ||
title: 📝 Architecture | ||
title: 🏗️ Architecture | ||
--- | ||
|
||
## The Kestra internal components | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
--- | ||
title: 📝 Api Guide | ||
title: 💻 API Guide | ||
rightBar: false | ||
editLink: false | ||
--- | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,204 @@ | ||
--- | ||
title: 🔧 Troubleshooting | ||
--- | ||
|
||
Something doesn't work as expected? Check out this page to help you troubleshoot. | ||
|
||
|
||
## Support us on social medias | ||
|
||
- Star Kestra on [GitHub](https://github.com/kestra-io/kestra). | ||
- Follow us on [Twitter](https://twitter.com/kestra_io) and [LinkedIn](https://www.linkedin.com/company/kestra). | ||
- Get in touch with us on [Slack](https://kestra.io/slack). | ||
|
||
|
||
### Networking in Docker Compose | ||
|
||
The docker-compose file provided as part of the [Getting Started](01.getting-started.md) guide doesn't configure networking for the Kestra containers. This means that you won't be able to access any services exposed via `localhost` on your local machine (e.g., another Docker container with a mapped port). Your machine and Docker container use a different network. To use a locally exposed service from Kestra container, you can use the `host.docker.internal` hostname or `172.17.0.1`. The `host.docker.internal` address allows you to reach your host machine's services from Kestra's container. | ||
|
||
Alternatively, you can leverage Docker network. By default, your Kestra container will be placed in a `default` network. You can add your custom services to the `docker-compose.yml` file provided by Kestra and use the services' alias (keys from `services`) to reach them. Even better would be if you create a new network e.g. network `kestra_net` and add your services to it. Then you can add this network to the `networks` section of the `kestra` service. With this, you will have access via `localhost` to all your exposed ports. The example below shows how you can add `iceberg-rest`, `minio` and `mc` (i.e. Minio client) to your Kestra Docker Compose file. | ||
|
||
```yaml | ||
version: "3" | ||
|
||
volumes: | ||
postgres-data: | ||
driver: local | ||
kestra-data: | ||
driver: local | ||
|
||
networks: | ||
kestra_net: | ||
|
||
services: | ||
postgres: | ||
image: postgres | ||
volumes: | ||
- postgres-data:/var/lib/postgresql/data | ||
environment: | ||
POSTGRES_DB: kestra | ||
POSTGRES_USER: kestra | ||
POSTGRES_PASSWORD: k3str4 | ||
healthcheck: | ||
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"] | ||
interval: 30s | ||
timeout: 10s | ||
retries: 10 | ||
networks: | ||
kestra_net: | ||
|
||
iceberg-rest: | ||
image: tabulario/iceberg-rest | ||
ports: | ||
- 8181:8181 | ||
environment: | ||
- AWS_ACCESS_KEY_ID=admin | ||
- AWS_SECRET_ACCESS_KEY=password | ||
- AWS_REGION=us-east-1 | ||
- CATALOG_WAREHOUSE=s3://warehouse/ | ||
- CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO | ||
- CATALOG_S3_ENDPOINT=http://minio:9000 | ||
networks: | ||
kestra_net: | ||
|
||
minio: | ||
image: minio/minio | ||
container_name: minio | ||
environment: | ||
- MINIO_ROOT_USER=admin | ||
- MINIO_ROOT_PASSWORD=password | ||
- MINIO_DOMAIN=minio | ||
networks: | ||
kestra_net: | ||
aliases: | ||
- warehouse.minio | ||
ports: | ||
- 9001:9001 | ||
- 9000:9000 | ||
command: ["server", "/data", "--console-address", ":9001"] | ||
|
||
mc: | ||
depends_on: | ||
- minio | ||
image: minio/mc | ||
container_name: mc | ||
networks: | ||
kestra_net: | ||
environment: | ||
- AWS_ACCESS_KEY_ID=admin | ||
- AWS_SECRET_ACCESS_KEY=password | ||
- AWS_REGION=us-east-1 | ||
entrypoint: > | ||
/bin/sh -c " | ||
until (/usr/bin/mc config host add minio http://minio:9000 admin password) do echo '...waiting...' && sleep 1; done; | ||
/usr/bin/mc rm -r --force minio/warehouse; | ||
/usr/bin/mc mb minio/warehouse; | ||
/usr/bin/mc policy set public minio/warehouse; | ||
tail -f /dev/null | ||
" | ||
kestra: | ||
image: kestra/kestra:latest-full | ||
pull_policy: always | ||
entrypoint: /bin/bash | ||
# Note that this is meant for development only. Refer to the documentation for production deployments of Kestra which runs without a root user. | ||
user: "root" | ||
command: | ||
- -c | ||
- /app/kestra server standalone --worker-thread=128 | ||
volumes: | ||
- kestra-data:/app/storage | ||
- /var/run/docker.sock:/var/run/docker.sock | ||
- /tmp/kestra-wd:/tmp/kestra-wd | ||
environment: | ||
KESTRA_CONFIGURATION: | | ||
datasources: | ||
postgres: | ||
url: jdbc:postgresql://postgres:5432/kestra | ||
driverClassName: org.postgresql.Driver | ||
username: kestra | ||
password: k3str4 | ||
kestra: | ||
server: | ||
basic-auth: | ||
enabled: false | ||
username: admin | ||
password: kestra | ||
repository: | ||
type: postgres | ||
storage: | ||
type: local | ||
local: | ||
base-path: "/app/storage" | ||
queue: | ||
type: postgres | ||
tasks: | ||
tmp-dir: | ||
path: /tmp/kestra-wd/tmp | ||
url: http://localhost:8080/ | ||
ports: | ||
- "8080:8080" | ||
- "8081:8081" | ||
depends_on: | ||
postgres: | ||
condition: service_started | ||
networks: | ||
kestra_net: | ||
``` | ||
Finally, you can also use the `host` network mode for the `kestra` service. This will make your container use your host network and you will be able to reach all your exposed ports. This means you have to change the `services.kestra.environment.KESTRA_CONFIGURATION.datasources.postgres.url` to `jdbc:postgresql://localhost:5432/kestra`. This is the easiest way but it can be a security risk. See the example below using `network_mode`. | ||
|
||
```yaml | ||
version: "3" | ||
volumes: | ||
kestra-data: | ||
driver: local | ||
services: | ||
kestra: | ||
image: kestra/kestra:latest-full | ||
pull_policy: always | ||
entrypoint: /bin/bash | ||
network_mode: host | ||
environment: | ||
JAVA_OPTS: "--add-opens java.base/java.nio=ALL-UNNAMED" | ||
NODE_OPTIONS: "--max-old-space-size=4096" | ||
KESTRA_CONFIGURATION: | | ||
datasources: | ||
postgres: | ||
url: jdbc:postgresql://localhost:5432/kestra | ||
driverClassName: org.postgresql.Driver | ||
username: kestra | ||
password: k3str4 | ||
kestra: | ||
server: | ||
basic-auth: | ||
enabled: false | ||
username: admin | ||
password: kestra | ||
anonymous-usage-report: | ||
enabled: true | ||
repository: | ||
type: postgres | ||
storage: | ||
type: local | ||
local: | ||
base-path: "/app/storage" | ||
queue: | ||
type: postgres | ||
tasks: | ||
tmp-dir: | ||
path: /tmp/kestra-wd/tmp | ||
scripts: | ||
docker: | ||
volume-enabled: true | ||
defaults: # just one example to show global taskDefaults | ||
- type: io.kestra.plugin.airbyte.connections.Sync | ||
url: http://host.docker.internal:8000/ | ||
username: airbyte | ||
password: password | ||
url: http://localhost:8080/ | ||
variables: | ||
env-vars-prefix: "" # to avoid requiring KESTRA_ prefix on env vars | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
b3d7939
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Successfully deployed to the following URLs:
kestra-io – ./
kestra-io.vercel.app
kestra-io-git-main-kestra.vercel.app
kestra-io-kestra.vercel.app