Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parabricks/applybqsr version updated and tests migrated to nf-test #6881

Merged
merged 63 commits into from
Nov 15, 2024

Conversation

gitanoqevaporelmundoentero
Copy link
Contributor

@gitanoqevaporelmundoentero gitanoqevaporelmundoentero commented Oct 29, 2024

Tools applybqsr is updated to latest version (4.3.2-1) and tests were migrated to new convention (from pytest to nf-test).

PR checklist

Closes #6861

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the module conventions in the contribution docs
  • If necessary, include test data in your PR.
  • Remove all TODO statements.
  • Emit the versions.yml file.
  • Follow the naming conventions.
  • Follow the parameters requirements.
  • Follow the input/output options guidelines.
  • Add a resource label
  • Use BioConda and BioContainers if possible to fulfil software requirements.
  • Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
    • For modules:
      • nf-core modules test <MODULE> --profile docker
      • nf-core modules test <MODULE> --profile singularity
      • nf-core modules test <MODULE> --profile conda
    • For subworkflows:
      • nf-core subworkflows test <SUBWORKFLOW> --profile docker
      • nf-core subworkflows test <SUBWORKFLOW> --profile singularity
      • nf-core subworkflows test <SUBWORKFLOW> --profile conda

Copy link
Contributor

@famosab famosab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far.

.github/workflows/test.yml Show resolved Hide resolved
@famosab
Copy link
Contributor

famosab commented Oct 29, 2024

We need to change the way the GPUs are accessed in the main.nf file. Maybe we can do that here aswell.

so the line
--num-gpus $task.accelerator.request \\

needs to be
$num_gpus \\

and add this above the script block
def num_gpus = task.accelerator ? "--num-gpus $task.accelerator.request" : ''


And we need to add below the label expression on the top of the file
label 'process_gpu'

@gitanoqevaporelmundoentero gitanoqevaporelmundoentero changed the title parabricks/applybqsr version updated and tests migrated to pytest parabricks/applybqsr version updated and tests migrated to nf-test Oct 29, 2024
@famosab famosab self-assigned this Oct 30, 2024
@sateeshperi
Copy link
Contributor

honestly no idea since I didn't test using singularity. I just borrowed that config block from nf-core/scdownstream

@mashehu mashehu mentioned this pull request Nov 8, 2024
@famosab
Copy link
Contributor

famosab commented Nov 11, 2024

Ok small update from me: Our cluster does not support docker, only singularity and for some reason just adding the --nv flag to the singularity options did not work for me.

@sateeshperi
Copy link
Contributor

could you try what we had previously ?

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

could you try what we had previously ?

No, I think the AWS instance was closed. I am trying to run it on other stuff we have locally but each one of those has some limitations.

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

restarted the test instance

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

Snaps are updated. I could not solve the weird fq2bammeth behaviour.
Seems like the test for applybqsr somehow does not pick up and the singularity one for fq2bam fails?

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

singularity hiccup is fixed. the other tests shouldn't be picked up...

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

@sateeshperi Now the fq2bammeth test fails with
.command.sh: line 2: bwameth.py: command not found

any ideas?

@sateeshperi
Copy link
Contributor

that means the container profile is not invoked properly....hmmmm

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

Maybe we can remove it from this PR and tackle it separately. I would like to merge this to have the GPU profile in the CI ;)

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

@sateeshperi if you are fine with that then give us the approval :D

@famosab
Copy link
Contributor

famosab commented Nov 12, 2024

singularity hiccup is fixed. the other tests shouldn't be picked up...

I think the issue is back, back again @mashehu

@mashehu
Copy link
Contributor

mashehu commented Nov 12, 2024

There is a weird ci run, where the profile is empty. No idea how that can be. Will investigate tomorrow

@famosab
Copy link
Contributor

famosab commented Nov 14, 2024

@mashehu ping :)

@famosab
Copy link
Contributor

famosab commented Nov 15, 2024

Now only docker fq2bam is failing with

    >   [PB Error 2024-Nov-15 11:59:36][src/main.cu:997] CUDA_CHECK() failed with no CUDA-capable device is detected (100), exiting.
    >   For technical support visit https://docs.nvidia.com/clara/index.html#parabricks
    >   Exiting...

@famosab
Copy link
Contributor

famosab commented Nov 15, 2024

If we can fix this and merge it all the other parabricks modules can be updated accordingly and a bunch of PRs closed :)

@famosab famosab added this pull request to the merge queue Nov 15, 2024
Merged via the queue into nf-core:master with commit 4a76db9 Nov 15, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update module: parabricks/applybqsr
4 participants