Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DM-48304: Add cli Docker container #6

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 21 additions & 3 deletions .github/workflows/docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,29 @@ jobs:
fetch-depth: 0

- uses: lsst-sqre/build-and-push-to-ghcr@v1
id: build
id: base
with:
image: "lsst/ppdb-base"
dockerfile: "docker/Dockerfile.base"
github_token: ${{ secrets.GITHUB_TOKEN }}
push: true

- uses: lsst-sqre/build-and-push-to-ghcr@v1
id: cli
with:
image: "lsst/ppdb-cli"
dockerfile: "docker/Dockerfile.cli"
github_token: ${{ secrets.GITHUB_TOKEN }}
push: true
build-args: |
BASE_IMAGE=ghcr.io/lsst/ppdb-base:${{ steps.base.outputs.tag }}

- uses: lsst-sqre/build-and-push-to-ghcr@v1
id: replication
with:
image: "lsst/ppdb-replication"
dockerfile: "docker/Dockerfile.replication"
github_token: ${{ secrets.GITHUB_TOKEN }}
push: true

- run: echo Pushed ghcr.io/lsst/ppdb-replication:${{ steps.build.outputs.tag }}
build-args: |
BASE_IMAGE=ghcr.io/lsst/ppdb-base:${{ steps.base.outputs.tag }}
39 changes: 39 additions & 0 deletions docker/Dockerfile.base
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
FROM python:3.11.6-slim-bookworm

ENV DEBIAN_FRONTEND=noninteractive

# Update and install OS dependencies
RUN apt-get -y update && \
apt-get -y upgrade && \
apt-get -y install --no-install-recommends git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Install required python build dependencies
RUN pip install --upgrade --no-cache-dir pip setuptools wheel uv

# Install optional dependencies
RUN uv pip install --no-cache-dir --system psycopg2-binary

# Create the build directory
WORKDIR /build
COPY . /build

# Install requirements
RUN uv pip install --no-cache-dir --system -r requirements.txt

# Install the package
RUN uv pip install --no-cache-dir --system --no-deps .

# Remove the build directory
RUN rm -rf /build

# Setup the application scripts
WORKDIR /app

# Install sdm_schemas
# Change this using: -e SDM_SCHEMAS_REF=branch_or_tag_name
ENV SDM_SCHEMAS_REF=main
COPY ./docker/scripts/download-sdm-schemas.sh .
RUN ./download-sdm-schemas.sh && rm download-sdm-schemas.sh
ENV SDM_SCHEMAS_DIR=/app/sdm_schemas
4 changes: 4 additions & 0 deletions docker/Dockerfile.cli
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
ARG BASE_IMAGE=ghcr.io/lsst/ppdb-base:main
FROM ${BASE_IMAGE}

ENTRYPOINT ["ppdb-cli"]
38 changes: 4 additions & 34 deletions docker/Dockerfile.replication
Original file line number Diff line number Diff line change
@@ -1,44 +1,14 @@
FROM python:3.11.6-slim-bookworm
ARG BASE_IMAGE=ghcr.io/lsst/ppdb-base:main
FROM ${BASE_IMAGE}

ENV DEBIAN_FRONTEND=noninteractive
# Install optional requirements
RUN uv pip install --no-cache-dir --system cassandra-driver

# Update and install OS dependencies
RUN apt-get -y update && \
apt-get -y upgrade && \
apt-get -y install --no-install-recommends git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

# Install required python build dependencies
RUN pip install --upgrade --no-cache-dir pip setuptools wheel uv

# Create the build directory
WORKDIR /build
COPY . /build

# Install requirements
RUN uv pip install --no-cache-dir --system cassandra-driver psycopg2-binary
RUN uv pip install --no-cache-dir --system -r requirements.txt

# Install the package
RUN uv pip install --no-cache-dir --system --no-deps .

# Setup the application scripts
WORKDIR /app

# Install sdm_schemas
# Change this using: -e SDM_SCHEMAS_REF=branch_or_tag_name
ENV SDM_SCHEMAS_REF=main
COPY ./docker/scripts/download-sdm-schemas.sh .
RUN ./download-sdm-schemas.sh && rm download-sdm-schemas.sh
ENV SDM_SCHEMAS_DIR=/app/sdm_schemas

# Copy the entrypoint script
COPY docker/scripts/entrypoint-replication.sh .
RUN chmod +x /app/entrypoint-replication.sh

# Remove the build directory
RUN rm -rf /build

# Run the wrapper script for the ppdb-replication command
CMD ["/app/entrypoint-replication.sh"]
19 changes: 19 additions & 0 deletions docker/scripts/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/usr/bin/env bash

##############################################################################
# Build the PPDB Docker images - run this script from the root of the project.
#
# This should only be used for local development. The production images are
# built and pushed to the container registry from a GitHub Action.
##############################################################################

set -e

# Build the base image
docker build -t lsst/ppdb-base:local -f docker/Dockerfile.base .

# Build the CLI image
docker build --build-arg BASE_IMAGE=lsst/ppdb-base:local -t lsst/ppdb-cli:local -f docker/Dockerfile.cli .

# Build the replication image
docker build --build-arg BASE_IMAGE=lsst/ppdb-base:local -t lsst/ppdb-replication:local -f docker/Dockerfile.replication .
21 changes: 9 additions & 12 deletions docker/scripts/entrypoint-replication.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,16 @@ set -euo pipefail
command -v ppdb-replication >/dev/null 2>&1 || { echo "ppdb-replication command not found"; exit 1; }
echo "Found ppdb-replication command"

# Function to check if an environment variable is set
check_env_var() {
local var_name="$1"
local var_value="${!var_name}"
if [ -z "${var_value:-}" ]; then
echo "$var_name is a required environment variable"
exit 1
fi
}

# Check if the required environment variables are set
check_env_var "PPDB_REPLICATION_APDB_CONFIG"
check_env_var "PPDB_REPLICATION_PPDB_CONFIG"
if [ -z "${PPDB_REPLICATION_APDB_CONFIG:-}" ]; then
echo "ERROR: PPDB_REPLICATION_APDB_CONFIG is a required environment variable"
exit 1
fi

if [ -z "${PPDB_REPLICATION_PPDB_CONFIG:-}" ]; then
echo "ERROR: PPDB_REPLICATION_PPDB_CONFIG is a required environment variable"
exit 1
fi

# Build the command from the environment variables
_CMD="ppdb-replication"
Expand Down
Loading