Feature/impl factory pattern repos

.env.example

@@ -0,0 +1,13 @@
+DB_ENV=production
+DATABASE_HOST=
+DATABASE_USER=
+DATABASE_PASSWORD=
+DATABASE_NAME=
+TESTING_DATABASE_HOST=localhost
+TESTING_DATABASE_USER=postgres
+TESTING_DATABASE_PASSWORD=password
+TESTING_DATABASE_NAME=ftf_test_db
+
+POLITRACK_API_URL=
+POLITRACK_USERNAME=
+POLITRACK_SECRET_PASSWORD=

.github/workflows/ci.yml

@@ -37,7 +37,8 @@ jobs:
           POLITRACK_USERNAME: ${{ secrets.POLITRACK_USERNAME}}
           POLITRACK_SECRET_PASSWORD: ${{ secrets.POLITRACK_SECRET_PASSWORD}}
           REDIS_HOST: ${{ secrets.REDIS_HOST}}
-        run: poetry run pytest
+        # Exempt tests/e2e because tests/api uses production database to test whereas tests/e2e uses localhost == avoid conflicts
+        run: poetry run pytest -m "not e2e"
 
   cd:
     runs-on: ubuntu-latest
@@ -56,8 +57,8 @@ jobs:
         with:
           aws_access_key: ${{ secrets.AWS_ACCESS_KEY_ID }}
           aws_secret_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
-          application_name: Face-The-Facts Backend
-          environment_name: ${{ github.ref == 'refs/heads/main' && 'Face-The-Facts-Backend-Main' || 'Face-The-FactsBackend-Development' }}
+          application_name: ${{ github.ref == 'refs/heads/main' && 'backend-production' || 'backend-development' }}
+          environment_name: ${{ github.ref == 'refs/heads/main' && 'Backend-production-env' || 'Backend-development-env' }}
           version_label: ${{ github.sha }}
           use_existing_version_if_available: true
           region: eu-central-1

README-feature/existing-features.md

@@ -0,0 +1,44 @@
+# Here is What I have done.
+
+- Added Pylint.
+- Generated a UML.
+- Analysed the UML.
+- Proposed potential improvement.
+
+# Description
+
+In this branch, I automatically generated a UML by [Pyreverse](https://pylint.readthedocs.io/en/latest/pyreverse.html) underling Pylint to analyse existing structures.
+
+# UML
+
+Since the generated UML is huge (1.2MB), I stored it in Google Drive.
+
+Here is the [link](https://drive.google.com/file/d/1X8WbZdDZLS34gVqFH47N0_pWty6tRQ66/view).
+
+# Analysis
+
+This analysis provides an overview of the structure and design patterns used in the "Face The Facts" project. The project primarily employs the FastAPI framework for its web functionality and SQLAlchemy for database management. Additionally, the project adheres to a functional programming paradigm, except in cases where FastAPI schemas and database models require a different approach.
+
+## FastAPI Schemas
+
+The project leverages FastAPI for web-related functionality, particularly in defining schemas. These schemas inherit from either the FTFBaseModel class or the BaseModel class from the Pydantic library. This choice depends on the specific requirements of the schema and its usage within the project.
+
+## Database Classes
+
+Database management in the project relies on SQLAlchemy. All database classes inherit from the Base class provided by SQLAlchemy. This inheritance ensures consistency in database modeling and allows for efficient data manipulation and querying.
+
+# Potential Improvement
+
+## Repository patterns
+
+In the current implementation of the project's crud.py module, database operations are handled directly within the functions. While this approach can work effectively for smaller projects, a more structured and scalable approach could be to adopt the Repository Pattern.
+
+Here is benefits:
+
+1. Abstraction: It provides an abstraction layer between the application code and the database, making it easier to switch databases or data sources if needed.
+
+2. Testability: With data access logic decoupled from business logic, unit testing becomes more straightforward, as you can mock the repository for testing purposes.
+
+3. Maintainability: Changes to the data access layer can be isolated, reducing the impact on the rest of the codebase.
+
+4. Flexibility: Different implementations of repositories can be used for different data sources (e.g., databases, APIs).

README-feature/impl-factory_pattern_repos.md

@@ -0,0 +1,54 @@
+# Implementation of the Factory Pattern for repositories
+
+## Here is what I have done
+
+- Implemented the factory pattern for repositories (party_donation, party).
+- Implemented e2e testing routes (**/v1/partydonations**, **/plugin/partydonations**, **/plugin/parties**).
+- Refactored existing functions to fetch data for the routes with the repository factories.
+
+## Description
+
+I implemented the factory_pattern to handle multiple repositories. As long as unit tests, I wrote e2e tests for existing party donation-related routes. After checking the tests, I refactored the existing functions to fetch data (e.g., get_all_party_donations, get_parties) with repository factories.
+
+(Before refactoring: partydonations)
+
+```bash
+e30a345 feat: Test plugin party_donation endpoint
+95bf7c8 feat: Test pary_donation_v1 endpoint by postgres
+a5b23ea feat: Set up env for testing
+```
+
+(After refactoring: partydonations)
+
+```bash
+6f246d4 feat: Replace with repo
+```
+
+(Before refactoring: parties)
+
+```bash
+0f0c8da feat: Restructure e2e testing
+5450de6 feat: Test parties endpoint
+```
+
+(After refactoring: parties)
+
+```bash
+ba30cd7 feat: Replace with repo
+```
+
+## Potential Improvement
+
+The current implementation for the updating tables(src/cron_jobs/utils/append_db.py) is highly coupled. For example, the `append_party` function combines multiple responsibilities, such as retrieving missing data from third-party and database updates, which violates SRP.
+
+For testing and implementing an event-driven pattern, we should decouple the services.
+
+1. Data Fetching Service:
+
+- Fetch missing party data from the third-party API.
+- Publish messages containing the retrieved data.
+
+2. Database Update Service:
+
+- Subscribe to the messages published by the data-fetching service.
+- Process and update the database based on the received messages.

README-feature/impl-repository_pattern.md

@@ -0,0 +1,18 @@
+# Implementation of the Repository Pattern for Party Donation
+
+## Here is what I have done
+
+- Implemented the repository pattern for party donation.
+- Conducted tests for the repository pattern.
+
+## Description
+
+I implemented the repository pattern for party donation table and test it.
+
+## Missing
+
+Refactoring party donation related routes (FastAPI routes) with the repository pattern. Plus, implementation of the repository pattern for other tables (e.g., Party, PartyDonationOrganization and etc).
+
+## Potential Improvement
+
+In the Architecture Patterns, the repository pattern (SQLAlchemyRepository) was implemented for just one table. But in reality, SQLAlchemyRepository must handle multiple tables. To manage multiple tables, abstract factory pattern (e.g., RepositoryFactory class) should be implemented.

README-feature/tests-existing-code.md

@@ -0,0 +1,18 @@
+# Party Donation Related Tables Testing
+
+## Here is what I have done
+
+- Conducted unit tests for Party, PartyDonation, and PartyDonationOrganization tables using SQLite.
+- Conducted unit tests for get_all_party_donations used by **/partydonations**.
+
+## Description
+
+In preparation for the implementation of an event-driven pipeline for updating party donation information, I conducted tests on party donation related tables. This ensures the data integrity of Party, PartyDonation, and PartyDonationOrganization tables and lays the foundation for future updates.
+
+## Missing
+
+Testing for populating tables (inserting and updating the database by scraped data) is currently missing. We plan to complete this functionality in the near future.
+
+## Potential Improvement
+
+To enhance the testability of our functions, we can consider decoupling functions and addressing dependency injection. This will make our code more modular and easier to test.