Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(filemanager): docker caching and current state default #601

Merged
merged 5 commits into from
Oct 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ services:
depends_on:
- db
healthcheck:
test: "curl http://localhost:8000/api/v1/s3_objects/count"
test: "curl http://localhost:8000/api/v1/s3/count"
start_period: 30s
interval: 10s
timeout: 2s
Expand Down
18 changes: 9 additions & 9 deletions lib/workload/stateless/stacks/filemanager/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,33 +4,33 @@

FROM public.ecr.aws/docker/library/rust:1.81 AS chef

ARG DATABASE_URL
ENV DATABASE_URL $DATABASE_URL
ENV RUSTC_WRAPPER="/usr/local/cargo/bin/sccache"

WORKDIR /app

RUN apt -y update && apt -y install curl && \
curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash
# rustfmt is used for code gen.
RUN cargo install cargo-chef && rustup component add rustfmt
RUN cargo binstall -y sccache && cargo binstall -y cargo-chef && rustup component add rustfmt

FROM chef AS planner

COPY . .
RUN cargo chef prepare
RUN --mount=type=cache,target=/root/.cache/sccache cargo chef prepare

FROM chef AS builder

COPY --from=planner /app/recipe.json recipe.json

# Cargo chef caches compilation.
RUN cargo chef cook
# Cargo chef and sccache caches compilation.
RUN --mount=type=cache,target=/root/.cache/sccache cargo chef cook --bin filemanager-api-server && sccache --show-stats

COPY . .
RUN cargo build --bin filemanager-api-server
RUN --mount=type=cache,target=/root/.cache/sccache cargo build --bin filemanager-api-server && sccache --show-stats

FROM public.ecr.aws/docker/library/debian:bookworm-slim AS runtime
RUN env

# curl is used for healthcheck.
RUN apt -y update && apt -y install curl

COPY --from=builder /app/target/debug/filemanager-api-server /usr/local/bin
ENTRYPOINT ["/usr/local/bin/filemanager-api-server"]
6 changes: 1 addition & 5 deletions lib/workload/stateless/stacks/filemanager/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,6 @@ This project is split up into multiple crates in a workspace. For development, d

## Rust code development

The filemanager uses docker to run a local postgres database to track objects, and sqlx, which connects to the database
at compile time to ensure that queries are valid. Compilation will emit errors if a query cannot successfully be run
on postgres database.

Makefile is used to simplify development. To get started run:

```sh
Expand Down Expand Up @@ -51,7 +47,7 @@ Unit tests can be run with:
make test
```

Which runs `cargo test`.
Which runs `cargo test`. This will also launch a local postgres database for testing.

To lint the code and format it, run:

Expand Down
21 changes: 17 additions & 4 deletions lib/workload/stateless/stacks/filemanager/compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,25 @@ services:
build: database
restart: always
environment:
- POSTGRES_DB=filemanager
- POSTGRES_DB=${POSTGRES_DB:-filemanager}
- POSTGRES_USER=filemanager
- POSTGRES_PASSWORD=filemanager
- PGPORT=4321
ports:
- "${FILEMANAGER_DATABASE_HOST}:${FILEMANAGER_DATABASE_PORT}:4321"
healthcheck:
test: ['CMD-SHELL', 'pg_isready -d orcabus -U orcabus']
interval: 10s
timeout: 60s
retries: 5
start_period: 90s
api:
build:
context: .
environment:
# Container database address for running server inside a docker container.
- DATABASE_URL=postgresql://filemanager:filemanager@postgres:4321/filemanager
- RUST_LOG=debug
- DATABASE_URL=${DATABASE_URL:-postgresql://filemanager:filemanager@postgres:4321/filemanager}
- RUST_LOG=trace
- FILEMANAGER_API_CORS_ALLOW_ORIGINS=${FILEMANAGER_API_CORS_ALLOW_ORIGINS:-http://localhost:3000}
- FILEMANAGER_API_CORS_ALLOW_HEADERS=${FILEMANAGER_API_CORS_ALLOW_HEADERS:-accept,authorization,content-type,user-agent,x-csrftoken,x-requested-with,x-amz-security-token,x-amz-date,content-disposition}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-access_key_id}
Expand All @@ -25,4 +31,11 @@ services:
ports:
- '${API_PORT}:8000'
restart: always

depends_on:
- postgres
healthcheck:
test: "curl http://localhost:8000/api/v1/s3/count"
start_period: 30s
interval: 10s
timeout: 2s
retries: 5
Loading