Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #6

Merged
merged 26 commits into from
Mar 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
c0ce079
#2 start on json doc
michael-conway Feb 28, 2025
8ac9acb
Create jsonlint.yml
michael-conway Feb 28, 2025
e841fd7
add json lint action
michael-conway Mar 3, 2025
7218d57
add json lint action
michael-conway Mar 3, 2025
8bb5b19
target action to branch
michael-conway Mar 3, 2025
49b65fb
Merge pull request #4 from NIEHS/2-make-dummy-json-doc-and-starter-sc…
michael-conway Mar 3, 2025
4f1004f
#5 adding slot for actions
michael-conway Mar 3, 2025
db619b6
#5 model
michael-conway Mar 3, 2025
ce8c9fc
#5 add schema
michael-conway Mar 4, 2025
8e1ef03
#1 add basics structure
pateldes Mar 4, 2025
69ceda2
Merge branch 'feature/1-create-python-project' into feature/5-feature…
michael-conway Mar 5, 2025
6657ec7
#5 schema val and tests
michael-conway Mar 5, 2025
41e42ec
#5 add pre-commit hooks
michael-conway Mar 5, 2025
3616e23
#5 add formatter as action and pre-commit
michael-conway Mar 5, 2025
8126728
#5 add formatter as action and pre-commit
michael-conway Mar 5, 2025
edd9452
#5 add formatter as action and pre-commit
michael-conway Mar 5, 2025
f7e6027
#5 add pytest to github actions
michael-conway Mar 5, 2025
3112e1c
#5 combine actions
michael-conway Mar 5, 2025
ea39ef6
#5 testing lint action
michael-conway Mar 5, 2025
fd13048
#5 testing lint action
michael-conway Mar 5, 2025
085205d
#5 status badge
michael-conway Mar 5, 2025
0c20c44
#5 status badge
michael-conway Mar 5, 2025
50af817
#5 status badge
michael-conway Mar 5, 2025
047bcb7
#5 status badge
michael-conway Mar 5, 2025
86afa6a
#5 combine actions
michael-conway Mar 5, 2025
0a50b64
Merge branch 'main' into develop
michael-conway Mar 5, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added .github/actions/.gitinclude
Empty file.
43 changes: 43 additions & 0 deletions .github/workflows/lint-and-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
name: CI/CD
on:
push:
branches:
- main
- develop
- feature/**
pull_request:
branches:
- main
- develop
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: psf/black@stable
with:
options: "--check --verbose"
src: "./accelerator_core"
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ "3.10" ]

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Test with pytest
run: |
coverage run -m pytest -v -s
- name: Generate Coverage Report
run: |
coverage report -m
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -169,3 +169,6 @@ cython_debug/

# PyPI configuration file
.pypirc

.idea
.DS_Store
15 changes: 15 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/psf/black
rev: 25.1.0
hooks:
- id: black
language_version: python3.11
18 changes: 16 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,27 @@ Core libraries and classes for accelerator metadata backbone

## Description

This project is the base python package and libraries for the core of the Accellerator Project
This project is the base python package and libraries for the core of the Accelerator Project

[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![CI/CD](https://github.com/NIEHS/accelerator-core/actions/workflows/lint-and-test.yml/badge.svg)](https://github.com/NIEHS/accelerator-core/actions/workflows/lint-and-test.yml)

![System Whiteboard](https://github.com/user-attachments/assets/2a2b07fa-bbed-454c-9050-73eccb7cbf6c)

The project implements all of the core components (accession, dissemination) and provides abstract superclasses and supporting code for all of the source and dissemination components. Addtional tools for test development, etc can be placed here but should be incorporated into specific projects as a python import

Each source and disseination target should be developed in a separate accelerator-source-xxx or accelerator-dissemination-xxx repository



## Developer Notes

This project uses pre-commit hooks to validate code, run tests, and accomplish other tasks.

Loading the requirements.txt into your dev environment will install pre-commit, then you can set up the pre-commit
hooks by running:

```
pre-commit install
```

This should be the first thing you do when cloning this project. More docs on pre-commit are available [here](https://pre-commit.com/)
Empty file added accelerator_core/__init__.py
Empty file.
29 changes: 29 additions & 0 deletions accelerator_core/accession.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
from accelerator_core.crosswalk import Crosswalk


class Accession:
"""Handles validation and CRUD operations for metadata records."""

def __init__(self, data: dict):
"""Initialize Accession with validated data."""
self.data = data

def validate(self) -> bool:
"""Validate JSON output from Crosswalk."""
pass

def create(self) -> str:
"""Create a new record in the database."""
pass

def read(self, record_id: str) -> dict:
"""Retrieve a record from the database."""
pass

def update(self, record_id: str, new_data: dict) -> bool:
"""Update an existing record."""
pass

def delete(self, record_id: str) -> bool:
"""Delete a record from the database."""
pass
10 changes: 10 additions & 0 deletions accelerator_core/crosswalk.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
from abc import ABC, abstractmethod


class Crosswalk(ABC):
"""Abstract superclass for mapping raw data to a structured JSON format."""

@abstractmethod
def transform(self, raw_data: dict) -> dict:
"""Convert raw data into a standardized format."""
pass
Empty file added accelerator_core/db/__init__.py
Empty file.
Empty file.
Empty file added accelerator_core/db/models.py
Empty file.
18 changes: 18 additions & 0 deletions accelerator_core/dissemination.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
class Dissemination:
"""Retrieves data from the database and transforms it into a JSON document for endpoint systems."""

def __init__(self, record_id: str):
"""Initialize Dissemination with a record ID."""
self.record_id = record_id

def fetch_data(self) -> dict:
"""Retrieve data from the database."""
pass

def format_for_endpoint(self, system: str) -> dict:
"""Convert data into a format required by a specific endpoint (CHORDS, Navigator, CEDAR)."""
pass

def export(self, system: str) -> bool:
"""Send the formatted data to the respective system."""
pass
12 changes: 12 additions & 0 deletions accelerator_core/schema/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Accelerator Data Model
## Version: 0.0.1-alpha

## Description

This accel.json file, and the accompanying JSON schema, represent the core data model for Accelerator.


## Development process

The initial accel.json file is derived from the Excel model of the 1.5 CHORDS model
found here: https://nih.sharepoint.com/:f:/r/sites/NIH-DataAcceleratorProgram/Shared%20Documents/General/Design/term%20mappings?csf=1&web=1&e=1JTZT1
Empty file.
Loading