Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add github action to build and publish docker image #41

Merged
merged 1 commit into from
Dec 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
target/
.git/config
53 changes: 53 additions & 0 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Copyright 2023 The SeamDB Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: docker
on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
docker:
runs-on: ubuntu-latest
env:
dockerhub_org: ${{ vars.DOCKERHUB_ORG || vars.DOCKERHUB_USERNAME || 'anonymous' }}
steps:
- name: Login to Docker Hub
if: github.event_name == 'push' && vars.DOCKERHUB_USERNAME != ''
uses: docker/login-action@v3
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Generate tag for version
id: generate-tag
if: github.event_name == 'push' && github.ref_type == 'tag'
run: echo "VERSION_TAG=,${{ env.dockerhub_org }}/seamdb:$GITHUB_REF_NAME" >> $GITHUB_OUTPUT
- name: More platforms on push
id: more-platforms
if: github.event_name == 'push'
run: echo "MORE_PLATFORMS=,linux/arm64,darwin/amd64,darwin/arm64" >> $GITHUB_OUTPUT
- name: Build and push
uses: docker/build-push-action@v6
with:
platforms: linux/amd64${{ steps.more-platforms.outputs.MORE_PLATFORMS }}
push: ${{ github.event_name == 'push' && vars.DOCKERHUB_USERNAME != '' }}
tags: ${{ env.dockerhub_org }}/seamdb:latest${{ steps.generate-tag.outputs.VERSION_TAG }}
85 changes: 85 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Copyright 2023 The SeamDB Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

FROM --platform=$BUILDPLATFORM rust:1-bookworm AS build

ARG TARGETOS
ARG TARGETARCH

RUN apt update && apt install -y cmake protobuf-compiler clang wget unzip

ENV ETCD_VERSION=v3.5.17

RUN if [ "$TARGETOS" = "linux" ]; then \
wget https://github.com/etcd-io/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-$TARGETOS-$TARGETARCH.tar.gz && \
mkdir /opt/etcd && \
tar xvf etcd-$ETCD_VERSION-$TARGETOS-$TARGETARCH.tar.gz --strip-components 1 --directory=/opt/etcd; \
elif [ "$TARGETOS" = "darwin" ]; then \
wget https://github.com/etcd-io/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-$TARGETOS-$TARGETARCH.zip && \
unzip etcd-$ETCD_VERSION-$TARGETOS-$TARGETARCH.zip && \
mv etcd-$ETCD_VERSION-$TARGETOS-$TARGETARCH /opt/etcd; \
else \
echo "unsupported platform $BUILDPLATFORM"; \
fi

RUN wget https://dlcdn.apache.org/kafka/3.9.0/kafka_2.13-3.9.0.tgz && \
mkdir /opt/kafka && \
tar xvf kafka_2.13-3.9.0.tgz --strip-components 1 --directory=/opt/kafka

COPY . seamdb

RUN make -C seamdb release


# Final image here
FROM --platform=$BUILDPLATFORM debian:bookworm

RUN apt update && apt install -y openjdk-17-jdk wget lsof

COPY --from=build /opt/etcd /opt/etcd
COPY --from=build /opt/kafka /opt/kafka
COPY --from=build /seamdb/target/release/seamdbd /usr/bin/

ENV PATH=/opt/kafka/bin:/opt/etcd:$PATH

# https://kafka.apache.org/quickstart
RUN kafka-storage.sh format --standalone -t `kafka-storage.sh random-uuid` -c /opt/kafka/config/kraft/reconfig-server.properties

ENV RUST_BACKTRACE=full RUST_LOG=seamdb=trace

RUN rm -rf entrypoint.sh && \
echo "kafka-server-start.sh /opt/kafka/config/kraft/reconfig-server.properties >> /var/log/kafka.log 2>&1 &" >> entrypoint.sh && \
echo "" >> entrypoint.sh && \
echo "etcd >> /var/log/etcd.log 2>&1 &" >> entrypoint.sh && \
echo "" >> entrypoint.sh && \
echo "until lsof -i :2379 > /dev/null" >> entrypoint.sh && \
echo "do" >> entrypoint.sh && \
echo " echo 'waiting etcd up'" >> entrypoint.sh && \
echo " sleep 1" >> entrypoint.sh && \
echo "done" >> entrypoint.sh && \
echo "echo 'etcd is ready'" >> entrypoint.sh && \
echo "" >> entrypoint.sh && \
echo "until lsof -i :9092 > /dev/null" >> entrypoint.sh && \
echo "do" >> entrypoint.sh && \
echo " echo 'waiting kafka up'" >> entrypoint.sh && \
echo " sleep 1" >> entrypoint.sh && \
echo "done" >> entrypoint.sh && \
echo "echo 'kafka is ready'" >> entrypoint.sh && \
echo "" >> entrypoint.sh && \
echo "seamdbd --cluster.uri etcd://127.0.0.1:2379/seamdb1 --log.uri kafka://127.0.0.1:9092" >> entrypoint.sh && \
chmod u+x entrypoint.sh

EXPOSE 5432

CMD ./entrypoint.sh

Check warning on line 85 in Dockerfile

View workflow job for this annotation

GitHub Actions / docker

JSON arguments recommended for ENTRYPOINT/CMD to prevent unintended behavior related to OS signals

JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals More info: https://docs.docker.com/go/dockerfile/rule/json-args-recommended/
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
# SeamDB
SeamDB aims to be a computation cluster to seam existing services to provide database interfaces. Personnally, I treat it as LevelDB in distributed fashion.

## Give it a try

```
docker run --rm -p 5432:5432 -d kezhuw/seamdb

docker run -it --rm jbergknoff/postgresql-client postgresql://host.docker.internal/db1

db1=> CREATE DATABASE db1;
db1=> CREATE TABLE table1 (id serial PRIMARY KEY, count bigint NOT NULL, price real NOT NULL, description text);
db1=> INSERT INTO table1 (count, price, description) VALUES (4, 15.6, NULL), (3, 7.8, 'NNNNNN'), (8, 3.4, 'a'), (8, 2.9, 'b');
db1=> SELECT id, count, price description FROM table1 ORDER BY count DESC, id ASC;
db1=> SELECT sum(count) AS count, max(price) AS max_price, min(price) AS min_price, sum(count*price) AS sales_amount from table1 ORDER BY max(price) DESC;
```

## Seaming services
* Etcd or alikes as bootstrap cluster meta. It serves `CURRENT` as in LevelDB.
* Kafka or alikes as manifest log and data log. It serves `MANIFEST` and `LOG` as in LevelDB.
Expand Down
Loading