Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make dumpus API easier to self-host #48

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,5 @@ CELERY_HOSTNAME=""

DISWHO_JWT_SECRET=""
DISWHO_BASE_URL=""

DISCORD_SECRET=""
90 changes: 90 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
services:

broker:
container_name: dumpus-broker
image: rabbitmq:latest
ports:
- "5672:5672"

healthcheck:
test: [ "CMD", "rabbitmq-diagnostics", "--quiet", "ping" ]
timeout: 10s
retries: 5
start_period: 10s

environment:
RABBITMQ_DEFAULT_USER: dumpus
RABBITMQ_DEFAULT_PASS: dumpus

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, it would be better if you used environment variables instead of hard coding usernames and passwords 😅

environment:
  RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
  RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}


restart: unless-stopped

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider on-failure in case of an error


db:
container_name: dumpus-db
image: postgres:latest
ports:
- "6641:5432"
Comment on lines +20 to +24

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a volume here would be useful for database persistence

db:
  volumes:
    - postgres_data:/var/lib/postgresql/data
volumes:
  postgres_data:


healthcheck:
test: [ "CMD-SHELL", "PGUSER=dumpus", "pg_isready" ]
interval: 1s
timeout: 10s
retries: 5

environment:
POSTGRES_USER: dumpus

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing as the first comment, use environment variables

POSTGRES_PASSWORD: dumpus
POSTGRES_DB: dumpus

restart: unless-stopped

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider on-failure in case of an error


worker:
container_name: dumpus-worker
build:
dockerfile: Dockerfile.worker

healthcheck:
test: [ "CMD-SHELL", "celery", "-A", "tasks", "inspect", "ping", "-d", "regular_process@$HOSTNAME" ]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test: ["CMD-SHELL", "celery -A tasks inspect ping -d regular_process@$$HOSTNAME"]

This is a better way to implement the health check because the command isn't split into multiple arguments but rather a single shell command.

interval: 2s
timeout: 10s
retries: 5

depends_on:
broker:
condition: service_healthy
restart: true
db:
condition: service_healthy
restart: true

environment:
POSTGRES_URL: "postgresql+psycopg2://dumpus:dumpus@db:5432/dumpus"
RABBITMQ_URL: "amqp://dumpus:dumpus@broker:5672/"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to update the connection strings


api:
container_name: dumpus-api
build:
dockerfile: Dockerfile.api

ports:
- "5000:5000"
depends_on:
worker:
condition: service_healthy
restart: true

environment:
POSTGRES_URL: "postgresql+psycopg2://dumpus:dumpus@db:5432/dumpus"
RABBITMQ_URL: "amqp://dumpus:dumpus@broker:5672/"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure to update the connection strings


# flower:
# container_name: flower
# build:
# dockerfile: Dockerfile.flower

# ports:
# - "5566:5566"

# depends_on:
# worker:
# condition: service_healthy
# restart: true
12 changes: 9 additions & 3 deletions src/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,13 +99,19 @@ def generate_diswho_jwt():

def fetch_diswho_user(user_id):
diswho_base_url = os.getenv('DISWHO_BASE_URL')
diswho_jwt = generate_diswho_jwt()
if diswho_base_url:
base_url = diswho_base_url + "/user"
diswho_jwt = generate_diswho_jwt()
auth = f"Bearer {diswho_jwt}"
else:
base_url = os.getenv('DISCORD_BASE_URL', 'https://discord.com/api/v8/users/')
auth = f"Bot {os.getenv('DISCORD_SECRET')}"

headers = {
'authorization': f'Bearer {diswho_jwt}'
'authorization': auth
}

r = requests.get(f'{diswho_base_url}/user/{user_id}', headers=headers)
r = requests.get(f'{base_url}/{user_id}', headers=headers)
if r.status_code == 200:
return r.json()
else:
Expand Down