Merge branch 'develop' of https://github.com/metacell/cloud-harness i…

…nto feature/CH-95

.dockerignore

@@ -19,4 +19,5 @@ __pycache__
 /application-templates
 /deployment-configuration
 /cloud-harness
-.openapi-generator
+.openapi-generator
+docker-compose.yaml

.github/pull_request_template.md

@@ -1,24 +1,29 @@
-Closes #
+Closes [CH-X](https://metacell.atlassian.net/browse/CH-X)
 
-Implemented solution: ... 
+# Implemented solution
 
-How to test this PR: ...
+... 
 
-### Sanity checks:
+# How to test this PR
+
+...
+
+# Sanity checks:
 - [ ] The pull request is explicitly linked to the relevant issue(s)
 - [ ] The issue is well described: clearly states the problem and the general proposed solution(s)
-- [ ] From the issue and the current PR it is explicitly stated how to test the current change
+- [ ] In this PR it is explicitly stated how to test the current change
 - [ ] The labels in the issue set the scope and the type of issue (bug, feature, etc.)
+- [ ] The relevant components are indicated in the issue (if any)
 - [ ] All the automated test checks are passing
-- [ ] All the linked issues are included in one milestone
-- [ ] All the linked issues are in the Review/QA column of the [board](https://app.zenhub.com/workspaces/cloud-harness-5fdb203b7e195b0015a273d7/board)
+- [ ] All the linked issues are included in one Sprint
+- [ ] All the linked issues are in the Review state
 - [ ] All the linked issues are assigned
 
-### Breaking changes (select one):
+# Breaking changes (select one):
 - [ ] The present changes do not change the preexisting api in any way
-- [ ] This PR and the issue are tagged as a `breaking-change`
+- [ ] This PR and the issue are tagged as a `breaking-change` and the migration procedure is well described [above](#implemented-solution)
 
-### Possible deployment updates issues (select one):
+# Possible deployment updates issues (select one):
 - [ ] There is no reason why deployments based on CloudHarness may break after the current update
 - [ ] This PR and the issue are tagged as `alert:deployment`
 

.github/workflows/lint-check.yml

@@ -0,0 +1,23 @@
+name: Lint Check
+
+on:
+  push
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Set up Python
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.x'
+
+    - name: Install autopep8
+      run: pip install autopep8
+
+    - name: Run lint-check.sh
+      run: bash lint-check.sh

.gitignore

@@ -7,6 +7,8 @@ node_modules
 .coverage
 *.DS_Store
 deployment/helm
+deployment/compose
+docker-compose.yaml
 *.egg-info
 *.idea
 /build
@@ -20,3 +22,4 @@ __pycache__
 .env
 tools/deployment-cli-tools/tests/resources/migration
 tools/deployment-cli-tools/tests/resources/migration.bak
+/.venv

README.md

@@ -2,12 +2,12 @@
 <img src="https://github.com/MetaCell/cloud-harness/blob/develop/cloudharness.png?raw=true" alt="drawing" width="200"/>
 </p>
 
-CloudHarness is a base infrastructure facilitator for microservice based applications deployed on Kubernetes.
+CloudHarness is a base infrastructure facilitator for microservice based applications deployed primarily on Kubernetes.
 Can scaffold and maintain your cloud solution on top of Cloudharness without writing
 Kubernetes templates, with in place common utilities and applications already configured for you.
 
 What building your cloud solution with CloudHarness gives to you:
-- Common framework and utilities to develop and deploy micro-service application 
+- Common framework and utilities to develop and deploy micro-service application
   - Helm chart automatic generation
     - deployments
     - services
@@ -17,6 +17,12 @@ What building your cloud solution with CloudHarness gives to you:
     - access gatekeepers configuration
     - secrets
     - templated config maps from files
+  - Docker compose configuration generation
+    - services
+    - traefik configuration
+    - databases (postgreql)
+    - access gatekeepers configuration
+    - secrets and configmaps
   * Automatic build and push of images
   * REST-API scaffolding building based on OpenApi
   * Continuous deployment script generation
@@ -46,17 +52,18 @@ In particular, these questions may rise:
  - How to manage databases without being locked to a specific vendor solution?
  - How to perform database backups?
  - How to manage secret data?
- - What about having a precounfigured account management application? 
- - Sooner rather than later I'll need an orchestration queue. Why not have that just ready to use? 
+ - What about having a precounfigured account management application?
+ - Sooner rather than later I'll need an orchestration queue. Why not have that just ready to use?
 
 # Command line tools
 
 CloudHarness provides the following command line tools to help application scaffolding and deployment.
 
-* `harness-deployment` - generate the helm chart to deploy on Kubernetes. 
+* `harness-deployment` - generate the helm chart to deploy on Kubernetes.
 * `harness-application` - create a new CloudHarness REST application.
 * `harness-generate` - generates server and client code for all CloudHarness REST applications.
 * `harness-test` - run end to end tests
+
 # Get started
 
 ## Prerequisites
@@ -67,15 +74,15 @@ Cloudharness can be used on all major operative systems.
 - Linux: supported and tested
 - MacOS: supported and tested
 - Windows/WSL2: supported and tested
-- Windows native: mostly working, unsupported  
+- Windows native: mostly working, unsupported
 
 ### Python
 Python 3.9 must be installed.
 
 It is recommended to setup a virtual environment.
-With conda: 
+With conda:
 ```bash
-conda create --name ch python=3.9
+conda create --name ch python=3.12
 conda activate ch
 ```
 
@@ -94,6 +101,10 @@ conda activate ch
 
 [Skaffold](https://skaffold.dev/docs/install/) is the way to go to build and debug your application in your local development environment.
 
+### Docker compose
+
+[Docker Compose](https://docs.docker.com/compose/) is required if the docker compose system is the target (instead of Kubernetes).
+
 ### Node environment
 
 A node environment with npm is required for developing web applications and to run end to end tests.
@@ -108,7 +119,6 @@ A JRE is needed to run the code generators based on openapi-generator.
 
 For more info, see [here](https://openapi-generator.tech/docs/installation).
 
-
 ## CloudHarness command line tools
 To use the cli tools, install requirements first:
 
@@ -127,24 +137,25 @@ To (re)generate the code for your applications, run `harness-generate` from the
 The script will look for all openapi applications, and regenerate the Flask server code and documentation.
 Note: the script will eventually override any manually modified file. To avoid that, define a file openapi-generator-ignore.
 
-# Extend CloudHarness to build your solution
-CloudHarness is born to be extended. In order to extend CloudHarness you just need to mirror the folder structure:
-* **applications**: place here your custom applications, or override default ones
-* **deployment-configuration**: override the helm chart default values and templates
-* **infrastructure**: define base images to use in your application
+# Extend CloudHarness to build your project
+
+CloudHarness is born to be extended. 
+
+The quickest way to start is to install Cloud Harness, copy the *blueprint* folder and build from that with the cli tools, such as
+`harness-application`, `harness-generate`, `harness-deployment`.
 
-or simply copy the *blueprint* folder.
+See the [developers documentation](docs/dev.md#start-your-project) for more information.
 
 # Build and deploy
 
 The script `harness-deployment` scans your applications and configurations to create the build and deploy artifacts.
 Created artifacts include:
- - Helm chart
+ - Helm chart (or docker compose configuration file)
  - Skaffold build and run configuration
  - Visual Studio Code debug and run configuration
  - Codefresh pipeline yaml specification (optional)
 
-With your solution folder structure looking like
+With your project folder structure looking like
 
 ```
 applications
@@ -153,7 +164,7 @@ infrastructure
 cloud-harness
 ```
 
-run 
+run
 
 ```
 harness-deployment cloud-harness . [PARAMS]

application-templates/base/test/api/test_st.py

@@ -3,7 +3,7 @@
 import schemathesis as st
 from schemathesis.checks import response_schema_conformance, not_a_server_error
 
-from cloudharness_test import apitest_init # include to perform default authorization
+from cloudharness_test import apitest_init  # include to perform default authorization
 
 app_url = os.environ.get("APP_URL", "http://samples.ch.local/api")
 
@@ -15,4 +15,3 @@ def test_ping(case):
     response = case.call()
     pprint(response.__dict__)
     assert response.status_code == 200, "this api errors on purpose"
-

application-templates/base/test/e2e/landing.spec.ts

@@ -29,7 +29,7 @@ describe("End to end test", () => {
       return el.textContent;
     });
 
-    expect(await page.title()).toEqual("Samples");
-    expect(title).toEqual("Sample React application is working!");
+    expect(await page.title()).not.toBeNull();
+    expect(title).not.toBeNull();
   });
 });

application-templates/django-app/api/templates/main.jinja2

@@ -49,13 +49,17 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
-from cloudharness.middleware import set_authentication_token
+from cloudharness.middleware import set_authentication_token, get_authentication_token
 @app.middleware("http")
 async def add_process_time_header(request: Request, call_next):
     # retrieve the bearer token from the header
     # and save it for use in the AuthClient
-    authorization = request.headers.get('Authorization')
+    authorization = request.headers.get('Authorization') or request.cookies.get('kc-access')
+
     if authorization:
+        if 'Bearer ' in authorization:
+            authorization = authorization.split('Bearer ')[1]
+
         set_authentication_token(authorization)
 
     return await call_next(request)
@@ -67,16 +71,17 @@ if os.environ.get('KUBERNETES_SERVICE_HOST', None):
     # start the kafka event listener when running in/for k8s
     import cloudharness_django.services.events
 
-# enable the Bearer Authentication
-security = HTTPBearer()
-
-async def has_access(credentials: HTTPBasicCredentials = Depends(security)):
+async def has_access():
     """
     Function that is used to validate the token in the case that it requires it
     """
     if not os.environ.get('KUBERNETES_SERVICE_HOST', None):
         return {}
-    token = credentials.credentials
+
+    token = get_authentication_token()
+
+    if not token:
+        raise HTTPException(status_code=401)
 
     try:
         payload = get_auth_service().get_auth_client().decode_token(token)

application-templates/django-app/api/test_st.py

@@ -3,7 +3,7 @@
 import schemathesis as st
 from schemathesis.checks import response_schema_conformance, not_a_server_error
 
-from cloudharness_test import apitest_init # include to perform default authorization
+from cloudharness_test import apitest_init  # include to perform default authorization
 
 app_url = os.environ.get("APP_URL", "http://samples.ch.local/api")
 
@@ -20,8 +20,9 @@ def test_ping(case):
     pprint(response.__dict__)
     assert response.status_code == 200, "this api errors on purpose"
 
+
 def test_state_machine():
     schema.as_state_machine().run()
 # APIWorkflow = schema.as_state_machine()
 # APIWorkflow.run()
-# TestAPI = APIWorkflow.TestCase
+# TestAPI = APIWorkflow.TestCase

application-templates/django-app/backend/__APP_NAME__/asgi.py

@@ -16,9 +16,9 @@
 application = get_asgi_application()
 
 # init the auth service
-from cloudharness_django.services import init_services
+from cloudharness_django.services import init_services  # noqa E402
 
 init_services()
 
 # start the kafka event listener
-import cloudharness_django.services.events
+import cloudharness_django.services.events  # noqa E402

application-templates/django-app/backend/__APP_NAME__/settings.py

@@ -120,13 +120,13 @@
 # ***********************************************************************
 # * __APP_NAME__ settings
 # ***********************************************************************
-from cloudharness.applications import get_configuration
-from cloudharness.utils.config import ALLVALUES_PATH, CloudharnessConfig
+from cloudharness.applications import get_configuration  # noqa E402
+from cloudharness.utils.config import ALLVALUES_PATH, CloudharnessConfig  # noqa E402
 
 # ***********************************************************************
 # * import base CloudHarness Django settings
 # ***********************************************************************
-from cloudharness_django.settings import *
+from cloudharness_django.settings import *  # noqa E402
 
 # add the local apps
 INSTALLED_APPS += [

application-templates/django-app/backend/__APP_NAME__/urls.py

@@ -24,7 +24,7 @@
 
 urlpatterns = [path("admin/", admin.site.urls)]
 urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) + \
-               static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
+    static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
 urlpatterns += [re_path(r"^(?P<path>.*)$", index, name="index")]
 
 admin.site.site_header = "__APP_NAME__ Admin"

application-templates/django-app/backend/__APP_NAME__/wsgi.py

@@ -16,9 +16,9 @@
 application = get_wsgi_application()
 
 # init the auth service
-from cloudharness_django.services import init_services
+from cloudharness_django.services import init_services  # noqa E402
 
 init_services()
 
 # start the kafka event listener
-import cloudharness_django.services.events
+import cloudharness_django.services.events  # noqa E402

application-templates/django-app/backend/api/controllers/__init__.py

@@ -1 +1 @@
-import api.controllers.test as test_controller
+import api.controllers.test as test_controller

application-templates/flask-server/backend/__APP_NAME__/__main__.py

@@ -7,4 +7,3 @@
 
 if __name__ == '__main__':
     main()
-

application-templates/flask-server/backend/requirements.txt

@@ -1,5 +1,7 @@
 connexion[swagger-ui]==2.14.2
-swagger-ui-bundle >= 0.0.2
+Flask == 2.2.5
+swagger-ui-bundle==0.0.9
 python_dateutil >= 2.6.0
 setuptools >= 21.0.0
-Flask<3.0.0
+
+

application-templates/flask-server/backend/setup.py

@@ -14,11 +14,12 @@
 # http://pypi.python.org/pypi/setuptools
 
 REQUIRES = [
-    "connexion>=2.0.2",
-    "swagger-ui-bundle>=0.0.2",
+    "connexion[swagger-ui]<3.0.0",
+    "Flask>=2.2.5",
     "python_dateutil>=2.6.0",
     "pyjwt>=2.6.0",
-    "cloudharness"
+    "swagger-ui-bundle>=0.0.2",
+    "cloudharness",
 ]
 
 setup(
@@ -38,4 +39,3 @@
     __APP_NAME__
     """
 )
-