Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding READMEs to all directories #584

Open
wants to merge 49 commits into
base: sk/joss-publication
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
76e35fc
Tweak logging for anonymisation (#528)
stefpiatek Nov 27, 2024
93dc6ba
update main read me
skeating Nov 27, 2024
83cc35a
Remove private DICOM tag for PIXL project name (#527)
p-j-smith Nov 27, 2024
3678f3c
511 - anonymisation for spectroscopy data (#513)
mxochicale Dec 11, 2024
d0f809b
Fix timeout issues when querying instances of a local study in Orthan…
p-j-smith Dec 11, 2024
87edbea
Renovate: ubuntu to v24 (#568)
renovate[bot] Dec 11, 2024
30bbd3b
Renovate: dependency pytest-asyncio to v0.24.0 (#556)
renovate[bot] Dec 11, 2024
b4d5888
Renovate: dependency aiohttp to v3.10.11 [SECURITY] (#533)
renovate[bot] Dec 11, 2024
2abbcfb
Renovate: dependency psycopg2-binary to v2.9.10 (#539)
renovate[bot] Dec 11, 2024
0ebd195
Renovate: dependency jsonpickle to v3.4.2 (#552)
renovate[bot] Dec 11, 2024
0d2f3a9
Renovate: dependency pytest-cov to v6 (#565)
renovate[bot] Dec 11, 2024
6d560f4
Update dependencies (#570)
p-j-smith Dec 11, 2024
f656090
Renovate: dependency dicom-anonymizer to v1.0.13.post1 (#536)
renovate[bot] Dec 12, 2024
9dbb114
Group all non-major updates into a single PR (#569)
p-j-smith Dec 12, 2024
cf96120
Add configuration for IoN Neuro DB project - prognosis ai (#525)
stefpiatek Dec 12, 2024
019032f
Renames file to prevent pytest conflict (#573)
tomaroberts Dec 12, 2024
8ed7655
Renovate: actions/checkout to 11bd719 (#575)
renovate[bot] Dec 16, 2024
83c1140
Renovate: docker/setup-buildx-action to c47758b (#577)
renovate[bot] Dec 16, 2024
f1c63fb
Renovate: actions/setup-python to 0b93645 (#576)
renovate[bot] Dec 16, 2024
6435ebb
Renovate: postgres:16-bookworm Docker digest to 5620f24 (#578)
renovate[bot] Dec 16, 2024
f55891c
Ensure we validate against correct `SPOClassUID` in `pixl_dcmd` tests…
p-j-smith Dec 19, 2024
9fe48f6
Improve anon logging and anonymise all matching studies at once (#582)
stefpiatek Dec 19, 2024
1d62bf7
Start documenting design choices (#574)
stefpiatek Dec 23, 2024
edbf6dc
update main read me (#529)
skeating Dec 24, 2024
235ca58
Merge branch 'sk/joss-1' into sk/joss-pub
skeating Dec 24, 2024
8d7faf6
Merge remote-tracking branch 'origin/sk/joss-publication' into sk/jos…
skeating Dec 24, 2024
65c7b5f
made read mes consistent
skeating Dec 24, 2024
3dcc4fb
Made the read mes more consistent
skeating Dec 24, 2024
a48ec93
made read mes more consistent
skeating Dec 27, 2024
208e74c
Log study info in anonymisation (#587)
stefpiatek Jan 2, 2025
7c777c9
making readmes consistent
skeating Jan 3, 2025
4109db4
making readmes consistent
skeating Jan 3, 2025
08b4ca5
making readmes consistent
skeating Jan 3, 2025
474894e
making readmes consistent
skeating Jan 6, 2025
807f84b
Refactor Python and Orthanc Dockerfiles (#583)
jeremyestein Jan 6, 2025
ecd5abf
Strip whitespace and remove empty identifiers (#586)
stefpiatek Jan 7, 2025
b0360ea
Making readmes more consistent
skeating Jan 7, 2025
ae4df7a
Add default anonymisation config for radiotherapy data (#521)
p-j-smith Jan 8, 2025
b3921b6
always try to create the output folder on the ftps server and catch t…
p-j-smith Jan 9, 2025
2babe4d
Making readmes consistent
skeating Jan 10, 2025
6bf6989
Merge remote-tracking branch 'origin/main' into sk/joss-pub
skeating Jan 10, 2025
415761f
Making readmes consistent
skeating Jan 10, 2025
57f13b9
Making readmes consistent
skeating Jan 10, 2025
e00eeae
Making readmes consistent
skeating Jan 10, 2025
35c34b2
Making readmes consistent
skeating Jan 10, 2025
dca65ca
Make it clear when study already exported (#585)
stefpiatek Jan 14, 2025
4afca8a
Switch to `uv` python tool (#579)
jeremyestein Jan 20, 2025
4af6ed0
Adds test to check modalities not specified in project config are dro…
tomaroberts Jan 23, 2025
95be8f9
Merge remote-tracking branch 'origin/main' into sk/joss-pub
skeating Jan 28, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
**/build
.git
**/.mypy_cache
.venv*
**/.pytest_cache
**/.ruff_cache
**/__pycache__
1 change: 0 additions & 1 deletion .env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ ORTHANC_ANON_URL=http://orthanc-anon:8042
ORTHANC_ANON_USERNAME=
ORTHANC_ANON_PASSWORD=
ORTHANC_ANON_AE_TITLE=
ORTHANC_ANON_HTTP_TIMEOUT=60
ORTHANC_AUTOROUTE_ANON_TO_ENDPOINT=false
ENABLE_DICOM_WEB=true
STUDY_TIME_OFFSET=
Expand Down
53 changes: 27 additions & 26 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ concurrency:

jobs:
lint:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- name: Run pre-commit
uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
Expand All @@ -46,64 +46,65 @@ jobs:
docker compose config --quiet

test:
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
strategy:
fail-fast: false # run all tests if even if one fails
matrix:
package_dir: [pixl_core, hasher, pixl_dcmd, cli, pixl_export, pixl_imaging, pytest-pixl]

steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

- name: Init Python
uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f # v5
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
version: "0.5.10"
python-version: "3.11"
cache: "pip"
enable-cache: true


- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install -e pytest-pixl/
pip install -e pixl_core/[test]
pip install -e ${{ matrix.package_dir }}/[test]
uv sync

- name: Run tests and generate coverage report
working-directory: ${{ matrix.package_dir }}
run: COV_CORE_SOURCE=src COV_CORE_CONFIG=.coveragerc COV_CORE_DATAFILE=.coverage.eager pytest --cov=src --cov-append --cov-report=xml --cov-report=term-missing
run: |
source ../.venv/bin/activate
COV_CORE_SOURCE=src COV_CORE_CONFIG=.coveragerc COV_CORE_DATAFILE=.coverage.eager pytest --cov=src --cov-append --cov-report=xml --cov-report=term-missing
env:
ENV: test
AZURE_KEY_VAULT_NAME: test
AZURE_KEY_VAULT_SECRET_NAME: test

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
uses: codecov/codecov-action@7f8b4b4bde536c465e797be725718b88c5d95e0e # v5.1.1
with:
directory: ${{ matrix.package_dir }}
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

system-test:
if: ${{ ! github.event.pull_request.draft || contains(github.event.pull_request.title, '[force-system-test]') }}
runs-on: ubuntu-22.04
runs-on: ubuntu-24.04
timeout-minutes: 30
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4
- uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3
- name: Init Python
uses: actions/setup-python@39cd14951b08e74b54015e9e001cdefcf80e669f # v5
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
- uses: docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349 # v3

- name: Install uv
uses: astral-sh/setup-uv@v4
with:
version: "0.5.10"
python-version: "3.11"
cache: "pip"
enable-cache: true

- name: Install Python dependencies
# The -e option is required here due to the way the
# Editable install required here due to the way the
# code determines the export directory. See issue #318.
run: |
pip install -e pytest-pixl/
pip install -e pixl_core/[test]
pip install -e cli/[test]
uv sync

- name: Create .secrets.env
run: touch test/.secrets.env
Expand All @@ -129,7 +130,7 @@ jobs:
HASHER_API_AZ_TENANT_ID: ${{ secrets.EXPORT_AZ_TENANT_ID }}
HASHER_API_AZ_KEY_VAULT_NAME: ${{ secrets.EXPORT_AZ_KEY_VAULT_NAME }}
run: |
./run-system-test.sh coverage
uv run ./run-system-test.sh coverage
echo FINISHED SYSTEM TEST SCRIPT

- name: Dump queue docker logs for debugging
Expand Down Expand Up @@ -179,8 +180,8 @@ jobs:


- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@e28ff129e5465c2c0dcc6f003fc735cb6ae0c673 # v4.5.0
uses: codecov/codecov-action@7f8b4b4bde536c465e797be725718b88c5d95e0e # v5.1.1
with:
directory: test
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
6 changes: 6 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,9 @@ repos:
language: script
types_or: [python, shell, yaml, dockerfile]
entry: bin/linters/check_headers_exist.sh

# avoid forgetting to update the lock file after pyproject.toml changes
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.5.15
hooks:
- id: uv-lock
34 changes: 32 additions & 2 deletions .renovaterc.json5
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,36 @@
$schema: "https://docs.renovatebot.com/renovate-schema.json",
extends: [
"github>UCL-ARC/.github//renovate/default-config.json",
"schedule:monthly",
":assignAndReview(team:arc-dev)",
"group:allNonMajor"
],
}
customDatasources: {
dicomSpec: {
defaultRegistryUrlTemplate: "https://dicom.nema.org/medical/dicom",
format: "html",
},
},
customManagers: [
{
customType: "regex",
description: "Update DICOM Spec edition used for validation",
fileMatch: [
"orthanc/orthanc-anon/plugin/download_dicom_spec.py",
"pixl_dcmd/src/pixl_dcmd/main.py",
],
matchStrings: [
'edition\\s?=\\s?"(?<currentValue>.*?)"\n',
'.*\\(edition\\s?=\\s?"(?<currentValue>.*?)"\\)\n',
],
depNameTemplate: "dicomSpec",
datasourceTemplate: "custom.dicomSpec",
},
],
packageRules: [
{
matchDatasources: ["custom.dicomSpec"],
extractVersion: "/medical/dicom/(?<version>\\d{4}[a-z])/",
versioning: "loose",
}
]
}
7 changes: 3 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,17 @@ Please request access to Slab and add further details in a [new blank issue](htt
</details>


## Installation
## Installation in production

Install the PIXL Python modules by running the following commands from the top-level `PIXL/` directory:

```shell
python -m pip install -e pixl_core/
python -m pip install -e cli/
uv sync --package pixl-cli
```

Note, the CLI currently [needs to be installed in editable mode](https://github.com/SAFEHR-data/PIXL/issues/318).

## Development
## Developer setup

[Follow the developer setup instructions](./docs/setup/developer.md).

Expand Down
20 changes: 19 additions & 1 deletion bin/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
## 'PIXL/bin' Directory Contents

### Subdirectories
<details>
<summary>
<h3> Subdirectories with links to the relevant README </h3>

</summary>

[linters](./linters/README.md)

</details>

<details>
<summary>
<h3> Files </h3>

</summary>

| **User docs** |
| :--- |
README.md

</details>

14 changes: 12 additions & 2 deletions bin/linters/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,16 @@
A directory that contains the files used for linting.

## 'PIXL/bin/linters' Directory Contents

### Files
<details>
<summary>
<h3> Files </h3>

</summary>

| **Code** | **User docs** |
| :--- | :--- |
| check_headers_exist.sh | README.md |

check_headers_exist.sh
</details>

30 changes: 21 additions & 9 deletions cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,13 @@ stopped cleanly.
* Python version 3.11 (matching python versions in [pixl-ci](../.github/workflows/main.yml) and [dev](../docs/setup/developer.md#installation-of-pixl-modules)).
* [Docker](https://docs.docker.com/get-docker/) with version `>=27.0.3`
* [Docker Compose](https://docs.docker.com/compose/install/#installation-scenarios) with version `>=v2.28.1-desktop.1`
* We recommend installing PIXL project in specific virtual environment using a environment
management tool such as [conda](https://docs.conda.io/en/latest/) or
[virtualenv](https://virtualenv.pypa.io/en/latest/).
* [uv](https://docs.astral.sh/uv/) to install PIXL in a virtual environment
See detailed instructions [here](../docs/setup/developer.md#setting-up-python-virtual-environment)

## Installation
Activate your python virtual environment and install `PIXL` project in editable mode by running
```bash
python -m pip install -e ../pixl_core -e .
uv sync
```

## Usage
Expand Down Expand Up @@ -130,7 +128,7 @@ are currently available, you can use the `pixl --help` command:
### Local installation
Activate your python environment and install project locally in editable mode with the development and testing dependencies by running
```bash
python -m pip install -e ../pixl_core -e ../pytest-pixl -e ".[test]" -e ".[dev]"
uv sync
```

### Running tests
Expand All @@ -145,15 +143,29 @@ pytest -vs tests/test_docker_commands.py #e.g., for particular tests

## 'PIXL/cli' Directory Contents

### Subdirectories
<details>
<summary>
<h3> Subdirectories with links to the relevant README </h3>

</summary>

[src](./src/README.md)

[tests](./tests/README.md)

### Files
</details>

<details>
<summary>
<h3> Files </h3>

</summary>

| **Configuration** | **User docs** |
| :--- | :--- |
| pyproject.toml | README.md |

</details>

pyproject.toml

README.md

14 changes: 9 additions & 5 deletions cli/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,17 @@ version = "0.2.0rc0"
authors = [{ name = "PIXL authors" }]
description = "PIXL command line interface"
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11"
classifiers = ["Programming Language :: Python :: 3"]
dependencies = [
"core==0.2.0rc0",
"click==8.1.7",
"tqdm==4.66.4",
"tqdm==4.67.1",
]

[project.optional-dependencies]
test = [
"core[test]==0.2.0rc0",
"pytest-mock==3.14.*",
]
dev = [
"core[dev]==0.2.0rc0",
Expand All @@ -25,8 +24,13 @@ dev = [
pixl = "pixl_cli.main:cli"

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
requires = ["hatchling>=1.0.0"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.wheel]
dev-mode-dirs = [
"src"
]

[tool.coverage.report]
exclude_also = [
Expand Down
22 changes: 21 additions & 1 deletion cli/src/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,26 @@
This directory contains the source files for the PIXL commands line interface.

## 'PIXL/cli/src' Directory Contents

### Subdirectories
<details>
<summary>
<h3> Subdirectories with links to the relevant README </h3>

</summary>

[pixl_cli](./pixl_cli/README.md)

</details>

<details>
<summary>
<h3> Files </h3>

</summary>

| **User docs** |
| :--- |
README.md

</details>

Loading