Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

estimate for segmentation.py memory usage #46

Open
mr-c opened this issue May 6, 2020 · 3 comments
Open

estimate for segmentation.py memory usage #46

mr-c opened this issue May 6, 2020 · 3 comments

Comments

@mr-c
Copy link

mr-c commented May 6, 2020

In the CWL tool description for segmentation.py we need memory sizing hints in mebibytes (2**20).

We can provide this number using a bit of JavaScript.

The following numbers are available for the formula:

  • number of inputs JSONs (inputs.bins.length) and the size of each (inputs.bins[N].size)
  • Number of cells per file from --cells-per-file (inputs.cells_per_file)
  • the size of the --fasta file (inputs.pangenome_sequence.size)
  • and the number of CPU cores passed to --parallel-cores (runtime.cores, if that matters)
@mr-c
Copy link
Author

mr-c commented May 12, 2020

Any progress on this @dimatr or others?

@dimatr
Copy link
Collaborator

dimatr commented May 12, 2020

still waiting the deNBI cloud to come up - will test big inputs there to get an impression

@mr-c
Copy link
Author

mr-c commented May 12, 2020

Okay, didn't know that was a blocker. You can also run tests on https://biohackathon.curii.com :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants