Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gemini - load chunk error with LSF scheduler #918

Open
Fazulur opened this issue Feb 11, 2019 · 0 comments
Open

gemini - load chunk error with LSF scheduler #918

Fazulur opened this issue Feb 11, 2019 · 0 comments

Comments

@Fazulur
Copy link

Fazulur commented Feb 11, 2019

Dear Team,

I tried to run gemini database using LSF as scheduler and specifying tmp directory to some other location where I have more space.
Command:

gemini load --cores 64 --scheduler lsf --queue normal --tempdir /home/fazulur/tmp/ -t snpEff -v test.vcf.gz test.db

gemini load_chunks function is trying to create chunks in the below path which is not exists. temp directory path is /home/fazulur/tmp/ and my current working directory is /home/fazulur/create-gemini-db/.
/home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db

Here is the Error:

2019-07-17 08:53:41.998 [IPClusterStop] Stopping cluster [pid=5089] with [signal=2]
Traceback (most recent call last):
File "$TOOl_DIR/gemini/v0.30.1/tools/bin/gemini", line 7, in
gemini_main.main()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1249, in main
args.func(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 204, in load_fn
gemini_load.load(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 47, in load
load_ipython(args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 99, in load_ipython
chunks = load_chunks_ipython(grabix_file, args, view)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 339, in load_chunks_ipython
chunk_dbs = view.map(load_chunk, chunk_steps, [load_args] * total_chunks)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-137>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 52, in sync_results
ret = f(self, *args, **kwargs)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-136>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 37, in save_ids
ret = f(self, *args, **kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 1114, in map
return pf.map(*sequences)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 299, in map
return self(*sequences, __ipp_mapping=True)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-119>", line 2, in call
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 80, in sync_view_results
return f(self, *args, **kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 285, in call
return r.get()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 169, in get
raise self.exception()
ipyparallel.error.CompositeError: one or more exceptions from call to method: load_chunk
[18:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 1 123256 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 1 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk0.db' returned non-zero exit status 1
[24:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 123257 246512 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 123257 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk1.db' returned non-zero exit status 1
[22:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 246513 369768 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 246513 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk2.db' returned non-zero exit status 1
[38:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 369769 493024 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 369769 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db' returned non-zero exit status 1
.... 46 more exceptions ...

Could you please help me to resolve this issue.

Thanks In Advance
Fazulur Rehaman

@Fazulur Fazulur changed the title gemini - load chunk error due to incorrect path gemini - load chunk error with LSF scheduler Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant