Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: Test Models Against the Full Docker Stack #3183

Closed
Closed
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 97 additions & 0 deletions .github/workflows/docker-stack-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Docker-Stack-Test

on:
# every Monday at 4:30 AM

schedule:
- cron: '30 4 * * 1'


env:
R_LIBS_USER: /usr/local/lib/R/site-library
LC_ALL: en_US.UTF-8
NCPUS: 2
PGHOST: postgres
CI: true

jobs:


# ----------------------------------------------------------------------
# DOCKER STACK TESTS
# ----------------------------------------------------------------------
Stack-Test:

runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}

strategy:
fail-fast: false


services:
postgres:
image: mdillon/postgis:9.5
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5

container:
image: pecan/depends
Comment on lines +38 to +39
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm rusty on the details here -- Does running this test inside the depends container mean these deps are used directly, or do we wind up pulling the whole container again to be run inside itself? Would it work to not specify a container at all let the Docker stack run directly on the host?


steps:
# checkout source code
- name: work around https://github.com/actions/checkout/issues/766
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
- uses: actions/checkout@v3
with:
set-safe-directory: false

- name: Set up Docker Compose
run: |
sudo apt-get update
sudo apt-get -y install docker-compose
- name: Build and run containers
run: |
docker system prune --all --force
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is a prune needed here? Won't it be running in a fresh VM (presumably with ~empty docker cache) every time?

cp docker/env.example .env
echo "COMPOSE_PROJECT_NAME=pecan" >> .env
echo "PECAN_VERSION=develop" >> .env
echo "UID=$(id -u)" >> .env
echo "GID=$(id -g)" >> .env
docker-compose up -d postgres
docker run --rm --network pecan_pecan pecan/db

docker-compose run bety user guestuser guestuser "Guest User" [email protected] 4 4

docker-compose run bety user carya illinois "Carya Demo User" [email protected] 1 1
docker run --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop
docker run --rm --network pecan_pecan --volume pecan_pecan:/data pecan/data:develop chown -R "$(id -u).$(id -g)" /data

docker run --user="$(id -u)" --rm --network pecan_pecan --volume pecan_pecan:/data --env FQDN=docker pecan/data:develop

docker-compose up -d


- name: Wait for services to start
run: |

docker-compose ps
sleep 20 # Adjust the duration as needed
docker-compose ps
sleep 20
docker-compose ps
sleep 20
Comment on lines +78 to +83
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a loop over an actual status check; maybe something like while [[ $(docker compose ps --format '{{.State}}'|sort|uniq) != "running" ]]; do sleep 10; done



- name: Display running containers
run: |
docker-compose ps
docker network ls
docker network inspect bridge
Comment on lines +86 to +90
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How useful is this step for unattended CI run? Will these outputs be helpful for debugging or will they just be filling the log up?


# run tests against the full docker stack
- name: Run Integration Test
run: |
chmod +x ./tests/Docker-Stack-Test.sh
./tests/Docker-Stack-Test.sh

1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,7 @@ services:
api:
image: pecan/api:${PECAN_VERSION:-latest}
user: "${UID:-1001}:${GID:-1001}"
restart: unless-stopped
Copy link
Collaborator Author

@allgandalf allgandalf Jun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be useful to automatically restart the api container if it ever fails when we bring up the whole docker stack

networks:
- pecan
environment:
Expand Down
47 changes: 47 additions & 0 deletions tests/Docker-Stack-Test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
curl -v -L -X POST -H "Host: pecan.localhost" \
-F 'hostname=docker' \
-F 'modelid=1000000014' \
-F 'sitegroupid=1' \
-F 'siteid=756' \
-F 'sitename=Duke Forest - loblolly pine (US-Dk3)' \
-F 'pft[]=temperate.coniferous' \
-F 'start=2004/01/01' \
-F 'end=2004/12/31' \
-F 'input_met=CRUNCEP' \
-F 'email=' \
-F 'notes=' \
'http://172.17.0.1/pecan/04-runpecan.php'

sleep 200
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, probably want to loop over a status check and sleep rather than assume a set duration


curl -v -L -H "Host: pecan.localhost" \
'http://172.17.0.1/pecan/dataset.php?workflowid=99000000001&type=file&name=out/99000000001/sipnet.out'

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a step here verifying that sipnet.out is free from errors / contains the expected output values?


# -----------------------------------------------
# cURL Request Template
#-----------------------------------------------

# Be sure to use the -H header and also use the bridge IP address of the Docker host (172.17.0.1) and not localhost


# curl -v -L -X POST -H "Host: pecan.localhost" \
# -F 'hostname=' \
# -F 'modelid=' \
# -F 'sitegroupid=' \
# -F 'siteid=' \
# -F 'sitename=' \
# -F 'pft[]=' \
# -F 'start=' \
# -F 'end=' \
# -F 'input_met=' \
# -F 'email=' \
# -F 'notes=' \
# 'http://172.17.0.1/pecan/04-runpecan.php'

# Adjust the sleep time according to the model you are running

# Append the workflowid and ID after `out/` by 1 for every consecutive run

# curl -v -L -H "Host: pecan.localhost" \
# 'http://172.17.0.1/pecan/dataset.php?workflowid=(ID)&type=file&name=out/(ID)/sipnet.out'
Comment on lines +21 to +47
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These lines seem redundant with the ones above

Suggested change
# -----------------------------------------------
# cURL Request Template
#-----------------------------------------------
# Be sure to use the -H header and also use the bridge IP address of the Docker host (172.17.0.1) and not localhost
# curl -v -L -X POST -H "Host: pecan.localhost" \
# -F 'hostname=' \
# -F 'modelid=' \
# -F 'sitegroupid=' \
# -F 'siteid=' \
# -F 'sitename=' \
# -F 'pft[]=' \
# -F 'start=' \
# -F 'end=' \
# -F 'input_met=' \
# -F 'email=' \
# -F 'notes=' \
# 'http://172.17.0.1/pecan/04-runpecan.php'
# Adjust the sleep time according to the model you are running
# Append the workflowid and ID after `out/` by 1 for every consecutive run
# curl -v -L -H "Host: pecan.localhost" \
# 'http://172.17.0.1/pecan/dataset.php?workflowid=(ID)&type=file&name=out/(ID)/sipnet.out'