Version 0.1.0 Release #18

kylacochrane · 2024-06-28T14:01:06Z

Version 0.1.0 Release of gasnomenclature

updating test data

Implemented the initial workflow, it is likely that many changes will need to be made. However I am focusing on implementing working unit-tests for the time being

I added some unit tests to the pipeline however these tests are incomplete, and show some current work that needs to be done in terms of altering the output paths as they are quite messy currently. Additionally more tests with the existing data can be added as it woul really improve the robustness of the pipeline

Its likely that the binaries for the individual tools were not being found as the nf-test.config file was not configured to use docker.

Integrating the initial skeleton of the workflow

reverted the `prepareFilePaths` function to use `file` instead of path

Update gas_nomenclature.nf

… samples

… to mlst.json keys

…n provided MLST JSON file(s)

…sponding test

Revise 'input_check' to 'input_assure'; enforce JSON key alteration to match the sample ID if a mismatch is detected

Add final integration tests

Updating tests and documentation to correct 'scaled' distances

Updating documentation for release of pipeline

github-actions · 2024-06-28T14:02:29Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 08582c8

+| ✅ 145 tests passed       |+
#| ❔  22 tests were ignored |#
!| ❗   2 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 0.1.0
schema_lint - Schema $id should be https://raw.githubusercontent.com/phac-nml/gasnomenclature/master/nextflow_schema.json
Found https://raw.githubusercontent.com/phac-nml/gasnomenclature/main/nextflow_schema.json

❔ Tests ignored:

files_exist - File is ignored: assets/nf-core-gasnomenclature_logo_light.png
files_exist - File is ignored: docs/images/nf-core-gasnomenclature_logo_dark.png
files_exist - File is ignored: docs/images/nf-core-gasnomenclature_logo_light.png
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File does not exist: assets/nf-core-gasnomenclature_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-gasnomenclature_logo_light.png
files_unchanged - File does not exist: docs/images/nf-core-gasnomenclature_logo_dark.png
files_unchanged - File ignored due to lint config: docs/README.md
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/gasnomenclature/gasnomenclature/.github/workflows/awstest.yml
actions_awsfulltest - actions_awsfulltest
pipeline_name_conventions - pipeline_name_conventions

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-gasnomenclature_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowGasnomenclature.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.gm_thresholds= 10,5,0
nextflow_config - Config default value correct: params.gm_method= average
nextflow_config - Config default value correct: params.gm_delimiter= .
nextflow_config - Config default value correct: params.pd_distm= hamming
nextflow_config - Config default value correct: params.pd_missing_threshold= 1.0
nextflow_config - Config default value correct: params.pd_sample_quality_threshold= 1.0
nextflow_config - Config default value correct: params.pd_file_type= text
nextflow_config - Config default value correct: params.pd_count_missing= false
nextflow_config - Config default value correct: params.max_cpus= 4
nextflow_config - Config default value correct: params.max_memory= 2.GB
nextflow_config - Config default value correct: params.max_time= 1.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.validate_params= true
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
template_strings - Did not find any Jinja template strings (135 files)
schema_lint - Schema lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: ci.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/base.config and Nextflow scripts.
modules_config - conf/modules.config found and not ignored.
modules_config - INPUT_ASSURE found in conf/modules.config and Nextflow scripts.
modules_config - LOCIDEX_MERGE_REF found in conf/modules.config and Nextflow scripts.
modules_config - LOCIDEX_MERGE_QUERY found in conf/modules.config and Nextflow scripts.
modules_config - PROFILE_DISTS found in conf/modules.config and Nextflow scripts.
modules_config - GAS_CALL found in conf/modules.config and Nextflow scripts.
modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-06-28 14:02:09

Matthew Wells and others added 30 commits April 2, 2024 14:54

updated tracked test data

4a32688

Merge pull request #3 from phac-nml/workflow

377b998

updating test data

Created initial workflow

3ef255b

Implemented the initial workflow, it is likely that many changes will need to be made. However I am focusing on implementing working unit-tests for the time being

removed module tests

f4d2887

updated expected clusters path

bebcbb3

updated test configs

2b8de97

updated nftest.config

e8a7373

Its likely that the binaries for the individual tools were not being found as the nf-test.config file was not configured to use docker.

updated test outpu path

b64bb3d

updated test output paths

f1756e9

fixed test data line endings

7c1e516

updated paths to expected input files

bdfe6c3

updated test conf file input paths

1515c37

updated params in test configs

fbd6117

updated schema, and test file line endings

ca448b5

updated schema types

6893ece

Ran prettier

141b040

updated function data types

7ce6e9f

Merge pull request #4 from phac-nml/workflow

dccc794

Integrating the initial skeleton of the workflow

Update gas_nomenclature.nf

be8a632

reverted the `prepareFilePaths` function to use `file` instead of path

Merge pull request #5 from phac-nml/fix-prepFilePaths

f249833

Update gas_nomenclature.nf

Updated samplesheet to include 'address' column

1e554dd

Updated locidex_merge_ref to include all samples, including new query…

adfe193

… samples

Updated expected test results to accomdate all samples in reference

2a79100

EC error fix

1142788

New module to filter out the new query addresses

b5ee774

Added nf-iridanext plugin

d6f315b

EC error fix

69dd10a

Adding a basic solution for identifying when sample ID's do not match…

7a9eeb9

… to mlst.json keys

Update query filtering process name and add versions

3ec26e2

kylacochrane added 27 commits June 12, 2024 17:08

Removed unexpected character (#) in main.nf.test

ec347e4

Add test data for multiple keyed JSON file

7c1b5dc

Tests added to handle when there are multiple sample entries (keys) i…

8e8ffa4

…n provided MLST JSON file(s)

Updated input_assure to identify when MLST JSON is empty. Added corre…

7909673

…sponding test

EC issue fix

da8c829

Create a new JSON output file in input_assure

6642b72

Ensure MLST JSON files from input_assure are gzipped

348fe95

Merge pull request #13 from phac-nml/input_assure

11167bd

Revise 'input_check' to 'input_assure'; enforce JSON key alteration to match the sample ID if a mismatch is detected

Update samplesheet paths to mlst.json reports

327e46b

Add gas parameter tests

95c0f5c

Add test for hash-missing

0b2ba27

Added test when counting missing data

27ff579

Added test for removing missing loci

23ec449

Added tests with pd_columns

01b1e4b

Update documentation

b08a816

Update CHANGELOG.md

cdbe9ef

Update workflow to allow gm_threshold to be single integers

63e69b3

Add success tests for different thresholds

c57ef6c

Fix error in main_gm_threshold.nf.test

cc8b033

Merge pull request #15 from phac-nml/add_tests

950704c

Add final integration tests

Updating tests and documentation to correct 'scaled' distances

1ed30b8

Added the expected test data files for scaled distances

00a97c9

Updated README to expand on GAS thresholds and linkage methods

d2e2c80

Merge pull request #16 from phac-nml/update_scaled

c6d4ba6

Updating tests and documentation to correct 'scaled' distances

Updating docs for release

df7c748

Add link to release to CHANGELOG.md

32b88c5

Merge pull request #17 from phac-nml/release_version

08582c8

Updating documentation for release of pipeline

kylacochrane marked this pull request as ready for review June 28, 2024 14:14

kylacochrane merged commit e7e73cf into main Jun 28, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Version 0.1.0 Release #18

Version 0.1.0 Release #18

kylacochrane commented Jun 28, 2024

github-actions bot commented Jun 28, 2024 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Version 0.1.0 Release #18

Version 0.1.0 Release #18

Conversation

kylacochrane commented Jun 28, 2024

github-actions bot commented Jun 28, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

github-actions bot commented Jun 28, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️