Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test: expand test coverage for Docker, API, CLI subcommands, and various parameter combos #71

Open
berntpopp opened this issue Dec 25, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@berntpopp
Copy link
Collaborator

We need to broaden our test suite to ensure reliability and consistency across multiple areas of the project. The following tests should be added or expanded:

  1. Docker Tests

    • Validate that running the pipeline or individual commands inside our Docker container works as expected.
    • Include at least one end-to-end test using Docker (e.g., mounting local data, running a pipeline command, verifying output).
  2. API Tests

    • If we have an API layer, confirm endpoint functionality (e.g., job submission, status checking, result retrieval).
    • Ensure JSON responses, error handling, and edge cases are tested.
  3. CLI Tests with adVNTR

    • Confirm that --extra-modules advntr works correctly.
    • Verify that relevant output files (e.g., output_adVNTR_result.tsv) are generated.
    • Test new or existing parameters for the adVNTR module (e.g., --advntr-max-coverage).
  4. Subcommand Tests

    • Cover each subcommand (pipeline, report, cohort, online, etc.) with at least one basic run-through.
    • Test optional arguments (e.g., --keep-intermediates, --archive-results) to confirm files are retained or removed as expected.
  5. Parameter Combinations

    • Run tests that toggle various flags: --keep-intermediates, --delete-intermediates, --fast-mode, etc.
    • Confirm that the required files are present or deleted after each run, depending on the parameters.
  6. Additional FASTQ Cases

    • Add a second FASTQ example to confirm robust handling.
    • Include a FASTQ test without the Shark module (e.g., no --extra-modules shark) to confirm the pipeline still runs normally.

Acceptance Criteria:

  1. Docker

    • At least one test that builds/pulls the Docker image, runs the pipeline, and verifies success.
  2. API

    • Confirm endpoints respond with correct statuses, handle invalid inputs gracefully, and generate valid JSON outputs.
  3. CLI + adVNTR

    • A test scenario that runs vntyper pipeline --extra-modules advntr with a known input and verifies correct adVNTR output.
  4. Subcommands

    • Each subcommand (pipeline, report, cohort, online) has at least one passing test scenario.
  5. Parameter Matrix

    • Tests that toggle various command-line arguments and confirm the presence or absence of intermediate or final files.
  6. Additional FASTQ Scenarios

    • At least one new test using an alternate FastQ dataset.
    • Confirm pipeline runs fine with no Shark filtering if --extra-modules shark is omitted.
@berntpopp berntpopp added the enhancement New feature or request label Dec 25, 2024
@berntpopp berntpopp self-assigned this Dec 25, 2024
@berntpopp berntpopp changed the title test: expand test coverage for Docker, API, CLI subcommands, and various parameter combos Test: expand test coverage for Docker, API, CLI subcommands, and various parameter combos Dec 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant