Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dont require local data api dependency for development #154

Merged
merged 6 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# GFW Tile Service
Raster and vector tile API for datasets in the [GFW Data API](https://github.com/wri/gfw-data-api) and Titiler dynamic tiling for raster assets in publicly accessible cloud storage.

## Developing
### Option 1: Developing against the Data API postgres database in one of the cloud environments (dev, staging or production):
* Make sure you have ssh access to the bastion host of the aws account (contact a Data API Engineering team member to get help with this).

* Open ssh tunnel connection to the database you'd like to connect to. For example, for the `staging` environment:

```ssh -i ~/.ssh/id_rsa -N -L 5432:application-autoscaling-698c9c01-db99-4430-a97a-6baaae853dc6.cljrqsduwhdo.us-east-1.rds.amazonaws.com:5432 ec2-user@gfw-staging```

* Set the environment variables for the **read only** credentials of the above database. The environment variables are `GFW_DB_NAME`, `GFW_DB_USER_RO` and `GFW_DB_PASSWORD_RO`. These are also listed in the `docker-compose.dev.yml` file.

* In `docker-compose.dev.yml`, set `DATA_LAKE_BUCKET` to the desired environment's bucket name. By default, the `staging` environment bucket (`gfw-data-lake-staging`) will be used.

* In `docker-compose.dev.yml`, set `AWS_DEFAULT_PROFILE` to your aws profile in `~/.aws` that will grant your dev instance access to the aws resources including the data lake bucket above in the aws account of the interest (contact a Data API Engineering team member to get an account).

* Run the start up script from the root directory:
```./scripts/develop```

### Option 2: Developing against a local instance of Data API database

* Start dev instance of Data API locally using the instructions [here](https://github.com/wri/gfw-data-api?tab=readme-ov-file#run-locally-with-docker)

* Run the start up script from the root directory with the option to point to the local Data API:
```./scripts/develop --local_data_api```
33 changes: 33 additions & 0 deletions docker-compose-local-data-api.dev.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
version: "3.5"

services:
app:
build:
context: .
args:
- ENV=dev
container_name: tile-cache_dev
restart: on-failure
volumes:
- $HOME/.aws:/root/.aws:ro
- ./app:/app/app
networks:
- gfw-data-api
environment:
- DB_HOST_RO=database_12
- DATABASE_RO=geostore
- DB_USER_RO=gfw_readonly
- DB_PASSWORD_RO=readonly # pragma: allowlist secret
- DB_PORT_RO=5432
- AWS_DEFAULT_PROFILE=gfw-dev
- LOG_LEVEL=debug
- RASTER_TILER_LAMBDA_NAME=test
- ENV=dev
- PLANET_API_KEY
ports:
- 8088:80
entrypoint: wait_for_postgres.sh /start-reload.sh

networks:
gfw-data-api:
name: gfw-data-api_dev_default
12 changes: 7 additions & 5 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,18 @@ services:
networks:
- gfw-data-api
environment:
- DB_HOST_RO=database_12
- DATABASE_RO=geostore
- DB_USER_RO=gfw_readonly
- DB_PASSWORD_RO=readonly # pragma: allowlist secret
- DB_HOST_RO=host.docker.internal
- DATABASE_RO=${GFW_DB_NAME}
- DB_USER_RO=${GFW_DB_USER_RO}
- DB_PASSWORD_RO=${GFW_DB_PASSWORD_RO}
- DB_PORT_RO=5432
- AWS_DEFAULT_PROFILE=gfw-dev
- AWS_DEFAULT_PROFILE=gfw-staging
- DATA_LAKE_BUCKET=gfw-data-lake-staging
- LOG_LEVEL=debug
- RASTER_TILER_LAMBDA_NAME=test
- ENV=dev
- PLANET_API_KEY
- AWS_REQUEST_PAYER=requester
ports:
- 8088:80
entrypoint: wait_for_postgres.sh /start-reload.sh
Expand Down
11 changes: 8 additions & 3 deletions scripts/develop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ set -e
# Default values
POSITIONAL=()
BUILD=true
COMPOSE_FILE="docker-compose.dev.yml"

# extracting cmd line arguments
while [[ $# -gt 0 ]]
do
Expand All @@ -15,6 +17,10 @@ do
BUILD=false
shift # past argument
;;
--local_data_api)
COMPOSE_FILE="docker-compose-local-data-api.dev.yml"
shift # past argument
;;
*) # unknown option
POSITIONAL+=("$1") # save it in an array for later
shift # past argument
Expand All @@ -24,8 +30,7 @@ done
set -- "${POSITIONAL[@]}" # restore positional parameters

if [ "${BUILD}" = true ]; then
docker compose -f docker-compose.dev.yml up --abort-on-container-exit --remove-orphans --build
docker compose -f "${COMPOSE_FILE}" up --abort-on-container-exit --remove-orphans --build
else
docker compose -f docker-compose.dev.yml up --abort-on-container-exit --remove-orphans
docker compose -f "${COMPOSE_FILE}" up --abort-on-container-exit --remove-orphans
fi

Loading