Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating data export job and matching new schema #74

Merged
merged 4 commits into from
Mar 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/export-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,9 @@ jobs:
working-directory: data-serving/scripts/export-data
env:
REGISTRY: ${{ steps.login-ecr.outputs.registry }}
REPO_COUNTRY: gdh-tcp-country-exporter
REPO_FULL: gdh-tcp-full-exporter
IMAGE_TAG: ${{ github.sha }}
run: |
docker build -t $REGISTRY/$REPO_COUNTRY:$IMAGE_TAG -t $REGISTRY/$REPO_COUNTRY .
docker push $REGISTRY/$REPO_COUNTRY:$IMAGE_TAG
docker push $REGISTRY/$REPO_COUNTRY:latest
Comment on lines -32 to -38
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just comment it out. We still might need the country exporter in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just add it back if we need it. It's in the commit history and the other repo.

docker build -f ./Dockerfile_full_export -t $REGISTRY/$REPO_FULL:$IMAGE_TAG -t $REGISTRY/$REPO_FULL .
docker push $REGISTRY/$REPO_FULL:$IMAGE_TAG
docker push $REGISTRY/$REPO_FULL:latest
42 changes: 0 additions & 42 deletions .github/workflows/export-tests.yml

This file was deleted.

11 changes: 0 additions & 11 deletions data-serving/scripts/export-data/Dockerfile

This file was deleted.

10 changes: 3 additions & 7 deletions data-serving/scripts/export-data/Dockerfile_full_export
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
FROM alpine:3.16
RUN apk update
RUN apk add bash aws-cli mongodb-tools python3 curl
RUN apk add bash aws-cli mongodb-tools curl

COPY common.sh .
COPY full_export.sh .
COPY data_dictionary.txt .
COPY citation.txt .
COPY country_export.sh .
COPY logger.py .
COPY transform.py .
COPY query.json .
COPY fields.txt .

ENTRYPOINT ["./full_export.sh"]
30 changes: 0 additions & 30 deletions data-serving/scripts/export-data/citation.txt

This file was deleted.

11 changes: 0 additions & 11 deletions data-serving/scripts/export-data/common.sh

This file was deleted.

26 changes: 0 additions & 26 deletions data-serving/scripts/export-data/country_export.sh

This file was deleted.

Loading