Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Added Mongo Storage and cache #31

Merged
merged 28 commits into from
Jul 25, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
14e8565
Use datetime and uuid4 lib instead of placeholders
salvatorelaiso Jul 21, 2023
8916cdf
Refactor curve param in init function
salvatorelaiso Jul 21, 2023
2977c0f
chore: linting
salvatorelaiso Jul 21, 2023
4e98f24
Set client_id from config metadata for issuer and subject
salvatorelaiso Jul 21, 2023
60f9870
Add default_sign_alg in config
salvatorelaiso Jul 21, 2023
2bb0240
Set htu param in JWT from config
salvatorelaiso Jul 21, 2023
a531dc0
Move signing algorithm into Helper init
salvatorelaiso Jul 21, 2023
1c89900
Merge remote-tracking branch 'upstream/dev' into dev
salvatorelaiso Jul 21, 2023
1545a03
Merge branch 'main' into dev
salvatorelaiso Jul 21, 2023
787e5dc
Removed unecessary directory
pderose Jul 21, 2023
58bc95f
Merge branch 'dev' of https://github.com/italia/eudi-wallet-it-python…
pderose Jul 21, 2023
d0ada24
Merge branch 'dev' into dev
PascalDR Jul 21, 2023
029ba28
Update tests
salvatorelaiso Jul 21, 2023
21cd8dd
Merge branch 'dev' of https://github.com/PascalDR/eudi-wallet-it-pyth…
pderose Jul 21, 2023
bd58830
Removed unecessary directory
pderose Jul 21, 2023
45b7112
Initial support to MongoDB storge
pderose Jul 21, 2023
a3e7a06
Added cache support
pderose Jul 24, 2023
8028cb9
Updated file names
pderose Jul 24, 2023
a76a321
Added Mongodb instalation in pipeline
pderose Jul 24, 2023
f26f2c6
Merge branch 'PascalDR-feature/storage' into dev
pderose Jul 24, 2023
c35ae82
Remove unnecessary file
salvatorelaiso Jul 24, 2023
5f898d5
Merge remote-tracking branch 'upstream/dev' into dev
salvatorelaiso Jul 24, 2023
a4a39d0
Commented phpmyadmin section of docker compose
pderose Jul 24, 2023
5fba712
Added requirement
pderose Jul 24, 2023
2dfbce3
Added dependency in CI/CD
pderose Jul 24, 2023
bbcd929
Added mongodb settings in backend configuration
pderose Jul 25, 2023
f1a68d6
Moved dependency from required to extra
pderose Jul 25, 2023
42d403d
Added dependency to CI/CD
pderose Jul 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .DS_Store
Binary file not shown.
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
MYSQL_DATABASE=wordpress
MYSQL_USER=wp_user
MYSQL_PASSWORD=wp_password
MYSQL_ROOT_PASSWORD=root_password
11 changes: 11 additions & 0 deletions .github/workflows/python-app.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ jobs:
python -m pip install -U setuptools
python -m pip install -e .
python -m pip install satosa
- name: Install MongoDB
run: |
sudo apt-get install -y gnupg wget
sudo wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
sudo echo "deb http://repo.mongodb.org/apt/debian buster/mongodb-org/4.4 main" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
sudo apt-get update
sudo apt-get install -y mongodb-org
- name: Start MongoDB
run: |
sudo systemctl start mongod

- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
Expand Down
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -160,4 +160,8 @@ cython_debug/
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/

env
*.db

wordpress/*
!wordpress/wp-content/
wordpress/wp-content/plugins/onelogin-saml-sso/
58 changes: 58 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
version: '3.1'

services:
database:
mem_limit: 2048m
image: mariadb:10.6.4-focal
restart: unless-stopped
ports:
- 3306:3306
env_file: .env
environment:
MYSQL_DATABASE: '${MYSQL_DATABASE}'
MYSQL_USER: '${MYSQL_USER}'
MYSQL_PASSWORD: '${MYSQL_PASSWORD}'
MYSQL_ROOT_PASSWORD: '${MYSQL_ROOT_PASSWORD}'
volumes:
- db-data:/var/lib/mysql
networks:
- wordpress-network

phpmyadmin:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please comment this to leave it at the will of the user having phpadmin or not

depends_on:
- database
image: arm64v8/phpmyadmin:5.2.1-apache
restart: unless-stopped
ports:
- 8081:80
env_file: .env
environment:
PMA_HOST: database
MYSQL_ROOT_PASSWORD: '${MYSQL_ROOT_PASSWORD}'
networks:
- wordpress-network

wordpress:
depends_on:
- database
image: wordpress:5.4.0-apache
restart: unless-stopped
ports:
- 8080:80
env_file: .env
environment:
WORDPRESS_DB_HOST: database:3306
WORDPRESS_DB_NAME: '${MYSQL_DATABASE}'
WORDPRESS_DB_USER: '${MYSQL_USER}'
WORDPRESS_DB_PASSWORD: '${MYSQL_PASSWORD}'
volumes:
- ./wordpress/:/var/www/html/
networks:
- wordpress-network

volumes:
db-data:

networks:
wordpress-network:
driver: bridge
4 changes: 4 additions & 0 deletions example/satosa/pyeudiw_backend.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ config:
- RS256
- ES256

default_sign_alg: RS256

#Various informations of the client
client_id: <base>/<name>
client_name: Name of an example organization
Expand Down Expand Up @@ -71,6 +73,8 @@ config:
use: sig
x5c:
- '...'
# time in ms
token_exp_delta: 600000
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this collides with the new configuration, please use it


#This section contains the details for presentation request
presentation_definitions:
Expand Down
42 changes: 22 additions & 20 deletions pyeudiw/satosa/backend.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import uuid
from datetime import datetime, timedelta
import json
import logging
Expand Down Expand Up @@ -32,18 +33,23 @@ def __init__(self, auth_callback_func, internal_attributes, config, base_url, na
:type auth_callback_func:
(satosa.context.Context, satosa.internal.InternalData) -> satosa.response.Response
:type internal_attributes: dict[string, dict[str, str | list[str]]]
:type config: dict[str, dict[str, str] | list[str]]
:type config: dict[str, dict[str, str] | list[str] | str]
:type base_url: str
:type name: str
"""

super().__init__(auth_callback_func, internal_attributes, base_url, name)

self.client_id = config['wallet_relying_party']['client_id']

self.default_sign_alg = config['default_sign_alg']

self.absolute_redirect_url = config['wallet_relying_party']['redirect_uris'][0]
self.absolute_request_url = config['wallet_relying_party']['request_uris'][0]

self.qrcode_settings = config['qrcode_settings']
self.qr_settings = config['qr_code_settings']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has changed in dev branch, please align this accordingly

self.token_exp_delta = config['jwks']['token_exp_delta']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same comment as the previous


self.config = config

logger.debug(f"Loaded configuration:\n{json.dumps(config)}")
Expand Down Expand Up @@ -84,10 +90,10 @@ def entity_configuration_endpoint(self, context, *args):
jwk = JWK()

data = {
"exp": int((datetime.now() + timedelta(minutes=6)).timestamp()),
"exp": int((datetime.now() + timedelta(milliseconds=self.token_exp_delta)).timestamp()),
"iat": int(datetime.now().timestamp()),
"iss": "https://rp.example.it",
"sub": "https://rp.example.it",
"iss": self.client_id,
"sub": self.client_id,
"jwks": {
"keys": [jwk.export_public()]
},
Expand All @@ -96,14 +102,14 @@ def entity_configuration_endpoint(self, context, *args):
}
}

jwshelper = JWSHelper(jwk)
jwshelper = JWSHelper(jwk, alg=self.default_sign_alg)

return Response(
jwshelper.sign(
plain_dict=data,
protected={
"alg": "RS256",
"kid": "2HnoFS3YnC9tjiCaivhWLVUJ3AxwGGz_98uRFaqMEEs",
"alg": self.default_sign_alg,
"kid": jwk.jwk["kid"],
"typ": "entity-statement+jwt"
}
),
Expand All @@ -112,7 +118,7 @@ def entity_configuration_endpoint(self, context, *args):

def pre_request_endpoint(self, context, *args):
payload = {'client_id': self.client_id,
'request_uri': self.complete_request_url}
'request_uri': self.absolute_redirect_url}
query = urlencode(payload, quote_via=quote_plus)
response = base64.b64encode(
bytes(f'eudiw://authorize?{query}', 'UTF-8'))
Expand All @@ -126,16 +132,14 @@ def pre_request_endpoint(self, context, *args):
def redirect_endpoint(self, context, *args):
jwk = JWK()

helper = JWSHelper(jwk)
helper = JWSHelper(jwk, alg=self.default_sign_alg)
jwt = helper.sign({
"jti": "f47c96a1-f928-4768-aa30-ef32dc78aa69",
"jti": str(uuid.uuid4()),
"htm": "GET",
"htu": "https://verifier.example.org/request_uri",
"htu": self.absolute_request_url,
"iat": int(datetime.now().timestamp()),
"ath": "fUHyO2r2Z3DZ53EsNrWBb0xWXoaNy59IiKCAqksmQEo"
},
"RS256",
)
})

response = {"request": jwt}

Expand All @@ -148,7 +152,7 @@ def redirect_endpoint(self, context, *args):
def request_endpoint(self, context, *args):
jwk = JWK()

helper = JWSHelper(jwk)
helper = JWSHelper(jwk, alg=self.default_sign_alg)
jwt = helper.sign({
"state": "3be39b69-6ac1-41aa-921b-3e6c07ddcb03",
"vp_token": "eyJhbGciOiJFUzI1NiIs...PT0iXX0",
Expand All @@ -168,9 +172,7 @@ def request_endpoint(self, context, *args):
}
]
}
},
"RS256",
)
})

response = {"response": jwt}

Expand Down Expand Up @@ -223,4 +225,4 @@ def authn_response(self, context, binding):
:param context: The current context
:param binding: The saml binding type
:return: response
"""
"""
Empty file added pyeudiw/storage/__init__.py
Empty file.
8 changes: 8 additions & 0 deletions pyeudiw/storage/base_cache.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from typing import Callable

class BaseCache():
def try_retrieve(self, object_name: str, on_not_found: Callable[[], str]) -> dict:
raise NotImplementedError()

def overwrite(self, object_name: str, value_gen_fn: Callable[[], str]) -> dict:
raise NotImplementedError()
9 changes: 9 additions & 0 deletions pyeudiw/storage/base_storage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
class BaseStorage(object):
def init_session(self, dpop_proof: dict, attestation: dict):
NotImplementedError()

def update_request_object(self, document_id: str, request_object: dict):
NotImplementedError()

def update_response_object(self, nonce: str, state: str, response_object: dict):
NotImplementedError()
64 changes: 64 additions & 0 deletions pyeudiw/storage/mongo_cache.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import pymongo
from datetime import datetime
from typing import Callable

from .base_cache import BaseCache

class MongoCache(BaseCache):
def __init__(self, storage_conf: dict, url: str, connection_params: dict = None) -> None:
super().__init__()

self.storage_conf = storage_conf
self.url = url
self.connection_params = connection_params

self.client = None
self.db = None

def _connect(self):
if not self.client or not self.client.server_info():
self.client = pymongo.MongoClient(self.url, **self.connection_params)
self.db = getattr(self.client, self.storage_conf["db_name"])
PascalDR marked this conversation as resolved.
Show resolved Hide resolved
self.collection = getattr(self.db, "cache_storage")

def try_retrieve(self, object_name: str, on_not_found: Callable[[], str]) -> dict:
self._connect()

query = {"object_name": object_name}

cache_object = self.collection.find_one(query)

if cache_object is None:
creation_date = datetime.timestamp(datetime.now())
cache_object = {
"object_name": object_name,
"data": on_not_found(),
"creation_date": creation_date
}

self.collection.insert_one(cache_object)

return cache_object

def overwrite(self, object_name: str, value_gen_fn: Callable[[], str]) -> dict:
self._connect()

new_data = value_gen_fn()
updated_date = datetime.timestamp(datetime.now())

cache_object = {
"object_name": object_name,
"data": new_data,
"creation_date": updated_date
}

query = {"object_name": object_name}

self.collection.update_one(query, {
"$set": {
"data": new_data,
"creation_date": updated_date
}
})

return cache_object
Loading