Making changes to the metadata service to use the new base image #99

EliseCastle23 · 2023-11-22T23:14:28Z

Improvements

Update to use new Amazon Linux base image and use the same structure as our other python services.
Utilizing "gen3" user instead of "root" for more secure containers
Moving to Poetry to manage our virtual environments
Multi-stage Docker builds for smaller images
Move to Gunicorn

github-actions · 2023-11-22T23:14:54Z

The style in this PR agrees with black. ✔️

This formatting comment was generated automatically by a script in uc-cdis/wool.

…or metadata to work as expected

github-actions · 2024-09-04T19:54:48Z

Please find the ci env pod logs here

github-actions · 2024-09-04T23:35:57Z

Please find the ci env pod logs here

github-actions · 2024-09-05T16:02:39Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#23d18b}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-09-06T16:59:46Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#23d18b}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-16T21:07:30Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-17T16:21:00Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#f14c4c}{\tt{error}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{4}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-17T17:51:11Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#f14c4c}{\tt{error}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{4}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-17T19:49:42Z

Please find the ci env pod logs here

github-actions · 2024-10-17T20:46:45Z

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-17T21:47:34Z

Please find the ci env pod logs here

github-actions · 2024-10-17T22:42:58Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#f14c4c}{\tt{error}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{4}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-18T20:42:43Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#f14c4c}{\tt{error}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{4}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-10-21T21:06:30Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#23d18b}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

Avantol13 · 2024-10-23T18:08:27Z

pyproject.toml

@@ -9,7 +9,7 @@ license = "Apache-2.0"
 python = "^3.9"
 fastapi = "^0.65.2"
 uvicorn = {extras = ["standard"], version="^0.18.2"}
-gunicorn = ">=20.0.4"
+gunicorn = "^22.0.0"


I'd like to keep all our deps >= until we hit a version that doesn't work, then we create a ticket, pin, and try to fix soon

We're not consistent with this, but in newer services, we've tried to keep every >=

github-actions · 2024-11-04T17:43:47Z

Please find the ci env pod logs here

github-actions · 2024-11-07T18:22:48Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-11-07T20:12:22Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#23d18b}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2024-11-18T18:30:35Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_discoverypage.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_aggregate\_mds.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_metadata\_ingestion.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#23d18b}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

…cdis#99) Improvements Update to use new Amazon Linux base image and use the same structure as our other python services. Utilizing "gen3" user instead of "root" for more secure containers Moving to Poetry to manage our virtual environments Multi-stage Docker builds for smaller images Move to Gunicorn

Making changes to the metadata service to use the new base image

26324ef

github-actions bot added the @requires-metadata label Nov 22, 2023

EliseCastle23 marked this pull request as draft November 22, 2023 23:15

EliseCastle23 and others added 2 commits November 27, 2023 09:52

needed to add the additional logger gunicorn configuration in order f…

73650e0

…or metadata to work as expected

Merge branch 'master' into feat/GPE-1115

f699ec4

EliseCastle23 added 3 commits September 4, 2024 14:49

adding nginx to metadata-service

6521d58

created new lock file

5e1047c

changing location of virtual environment

bac5968

EliseCastle23 added 3 commits September 5, 2024 08:25

pushing up change to activate the proper virtual environment

1a791da

adding a few more changes

d5b6655

/venv to /env

3b654a5

changing command

bdc0b6b

EliseCastle23 marked this pull request as ready for review October 16, 2024 17:33

EliseCastle23 added 4 commits October 16, 2024 13:48

Merge branch 'master' into feat/GPE-1115

f46c532

updating poetry lock

9f44740

updating gunicorn

1109c62

removing asgi and replacing with wsgi

540cd54

adding asgi stuff back

1db8ab9

EliseCastle23 added jenkins-brain skip-k8s-cleanup labels Oct 18, 2024

Avantol13 reviewed Oct 23, 2024

View reviewed changes

EliseCastle23 added 4 commits November 1, 2024 11:29

adding python nginx as base image

ab30f7f

using poetry as virtual environment

6cc4757

removing command

70d5894

Merge branch 'master' into feat/GPE-1115

adf3081

EliseCastle23 added 2 commits November 7, 2024 08:44

Merge branch 'master' into feat/GPE-1115

feccd6b

updating lock file

83749bd

EliseCastle23 removed jenkins-brain skip-k8s-cleanup labels Nov 7, 2024

m0nhawk approved these changes Nov 18, 2024

View reviewed changes

Merge branch 'master' into feat/GPE-1115

abb32d3

jawadqur approved these changes Nov 18, 2024

View reviewed changes

EliseCastle23 merged commit 732ca79 into master Nov 18, 2024
8 checks passed

EliseCastle23 deleted the feat/GPE-1115 branch November 18, 2024 20:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Making changes to the metadata service to use the new base image #99

Making changes to the metadata service to use the new base image #99

EliseCastle23 commented Nov 22, 2023 •

edited

Loading

github-actions bot commented Nov 22, 2023

github-actions bot commented Sep 4, 2024

github-actions bot commented Sep 4, 2024

github-actions bot commented Sep 5, 2024

github-actions bot commented Sep 6, 2024

github-actions bot commented Oct 16, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 18, 2024

github-actions bot commented Oct 21, 2024

Avantol13 Oct 23, 2024

github-actions bot commented Nov 4, 2024

github-actions bot commented Nov 7, 2024

github-actions bot commented Nov 7, 2024

github-actions bot commented Nov 18, 2024

Making changes to the metadata service to use the new base image #99

Making changes to the metadata service to use the new base image #99

Conversation

EliseCastle23 commented Nov 22, 2023 • edited Loading

Improvements

github-actions bot commented Nov 22, 2023

github-actions bot commented Sep 4, 2024

github-actions bot commented Sep 4, 2024

github-actions bot commented Sep 5, 2024

github-actions bot commented Sep 6, 2024

github-actions bot commented Oct 16, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 17, 2024

github-actions bot commented Oct 18, 2024

github-actions bot commented Oct 21, 2024

Avantol13 Oct 23, 2024

Choose a reason for hiding this comment

github-actions bot commented Nov 4, 2024

github-actions bot commented Nov 7, 2024

github-actions bot commented Nov 7, 2024

github-actions bot commented Nov 18, 2024

EliseCastle23 commented Nov 22, 2023 •

edited

Loading