Bump initial memory and cpu request for CellAssign

[allyhawkins]

In attempting to run the first two projects through CellAssign, I am noticing a few things:

• The first is that most of the samples are failing in the initial two attempts for running CellAssign, and are more successful on the third attempt (although not always). This tells me we should be initially requesting more memory. I think I would also bump up the cpus so that hopefully this takes less time and is less likely to get killed.
• The second is that, every run has been killed by the external system after ~5-7 hours. In some cases, CellAssign has completed and in others none of the processes have completed. I'm not entirely sure what else we can do other than bump up requests? I wonder if there is a time out that's happening?

Below is a copy of the error that I'm consistently getting:

I've also attached the log file so that you can see the complete output.
nextflow.log

[jashapiro]

I suggested being a bit more conservative with CPU requests, but other than that, this looks fine.

The jobs getting killed could be because these are spot instances, and AWS may be killing them randomly. The best way around that is to make them run faster, so upping the CPUs is probably the right move for that as well. But I am still wary of jumping too high on the CPU count if the payoff isn't there.

[jashapiro]

I'm a little worried that increasing both threads and memory at the same time will mean that you will still hit memory limits. So I might be more conservative here and only up the cpus to 24 for now? Also multithreading tends to be less efficient with more cores, depending how well it is implemented.

[allyhawkins]

Okay yea I wasn't sure how much to bump it up to. I'll change it to 24 and then give it a go 🤞