Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SV module missing file exception #234

Open
ionox0 opened this issue Apr 29, 2020 · 0 comments
Open

SV module missing file exception #234

ionox0 opened this issue Apr 29, 2020 · 0 comments

Comments

@ionox0
Copy link
Member

ionox0 commented Apr 29, 2020

This appears to have been caused by empty VCF and is fixed by Gowtham's fork's change in master branch

WARNING:toil.leader:9/5/jobt4lXkT        C-7FP87A-N002-d \
WARNING:toil.leader:9/5/jobt4lXkT        . \
WARNING:toil.leader:9/5/jobt4lXkT        /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stgcff17905-068f-4b4d-8292-07dbe77f8487/1.5.0 \
WARNING:toil.leader:9/5/jobt4lXkT        /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stgca2ce4e8-9e90-43ba-b35a-851b7449a138/Homo_sapiens_assembly19.fasta \
WARNING:toil.leader:9/5/jobt4lXkT        /home/johnsoni/virtualenvs/pipeline_StructuralVariants/bin/python \
WARNING:toil.leader:9/5/jobt4lXkT        /opt/common/CentOS_6/java/jdk1.8.0_31/bin/java
WARNING:toil.leader:9/5/jobt4lXkT    INFO:cwltool:[job manta_annotation.cwl] /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpH2F4Cy/a/1/out_tmpdirhctDwV$ /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stg016353a5-eb64-4860-ae96-5fc8a26867bc/ACCESS_SV/scripts/iAnnotateSV.sh \
WARNING:toil.leader:9/5/jobt4lXkT        /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stgc3051c28-cb4e-4c9e-99e5-3b049faa3023/somaticSV.vcf.gz \
WARNING:toil.leader:9/5/jobt4lXkT        C-7FP87A-N002-d \
WARNING:toil.leader:9/5/jobt4lXkT        . \
WARNING:toil.leader:9/5/jobt4lXkT        /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stgcff17905-068f-4b4d-8292-07dbe77f8487/1.5.0 \
WARNING:toil.leader:9/5/jobt4lXkT        /juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpQIl4Oj/stgca2ce4e8-9e90-43ba-b35a-851b7449a138/Homo_sapiens_assembly19.fasta \
WARNING:toil.leader:9/5/jobt4lXkT        /home/johnsoni/virtualenvs/pipeline_StructuralVariants/bin/python \
WARNING:toil.leader:9/5/jobt4lXkT        /opt/common/CentOS_6/java/jdk1.8.0_31/bin/java
WARNING:toil.leader:9/5/jobt4lXkT    /ifs/work/bergerm1/Innovation/software/ACCESS_SV/scripts
WARNING:toil.leader:9/5/jobt4lXkT    unpacking vcf gz file from manta
WARNING:toil.leader:9/5/jobt4lXkT    correcting inversions in manta vcf
WARNING:toil.leader:9/5/jobt4lXkT    Adding connection type and making changes to manta VCF
WARNING:toil.leader:9/5/jobt4lXkT    Converting manta VCF into tab file for iAnnotateSV
WARNING:toil.leader:9/5/jobt4lXkT    Traceback (most recent call last):
WARNING:toil.leader:9/5/jobt4lXkT      File "/ifs/work/bergerm1/Innovation/software/ACCESS_SV/scripts/manta_vcf2tab.py", line 134, in <module>
WARNING:toil.leader:9/5/jobt4lXkT        vcf2tab(args.input_vcf, args.outdir, args.verbose)
WARNING:toil.leader:9/5/jobt4lXkT      File "/ifs/work/bergerm1/Innovation/software/ACCESS_SV/scripts/manta_vcf2tab.py", line 52, in vcf2tab
WARNING:toil.leader:9/5/jobt4lXkT        vcf_reader = vcf.Reader(open(vcfFile, 'r'))
WARNING:toil.leader:9/5/jobt4lXkT    IOError: [Errno 2] No such file or directory: './somaticSV_inv_corrected_edited.vcf'
WARNING:toil.leader:9/5/jobt4lXkT    ./somaticSV_inv_corrected_edited.tab
WARNING:toil.leader:9/5/jobt4lXkT    Running iAnnotateSV
WARNING:toil.leader:9/5/jobt4lXkT    Traceback (most recent call last):
WARNING:toil.leader:9/5/jobt4lXkT      File "/ifs/work/bergerm1/Innovation/software/ACCESS_SV/iAnnotateSV/iAnnotateSV/iAnnotateSV.py", line 333, in <module>
WARNING:toil.leader:9/5/jobt4lXkT        main()
WARNING:toil.leader:9/5/jobt4lXkT      File "/ifs/work/bergerm1/Innovation/software/ACCESS_SV/iAnnotateSV/iAnnotateSV/iAnnotateSV.py", line 209, in main
WARNING:toil.leader:9/5/jobt4lXkT        svDF = hp.ReadFile(args.svFilename)
WARNING:toil.leader:9/5/jobt4lXkT      File "/ifs/work/bergerm1/Innovation/software/ACCESS_SV/iAnnotateSV/iAnnotateSV/helper.py", line 16, in ReadFile
WARNING:toil.leader:9/5/jobt4lXkT        dataDF = pd.read_csv(infile, sep='\t', header=0, keep_default_na='True')
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/johnsoni/virtualenvs/pipeline_StructuralVariants/lib/python2.7/site-packages/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 474, in parser_f
WARNING:toil.leader:9/5/jobt4lXkT        return _read(filepath_or_buffer, kwds)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/johnsoni/virtualenvs/pipeline_StructuralVariants/lib/python2.7/site-packages/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 250, in _read
WARNING:toil.leader:9/5/jobt4lXkT        parser = TextFileReader(filepath_or_buffer, **kwds)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/johnsoni/virtualenvs/pipeline_StructuralVariants/lib/python2.7/site-packages/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 566, in __init__
WARNING:toil.leader:9/5/jobt4lXkT        self._make_engine(self.engine)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/johnsoni/virtualenvs/pipeline_StructuralVariants/lib/python2.7/site-packages/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 705, in _make_engine
WARNING:toil.leader:9/5/jobt4lXkT        self._engine = CParserWrapper(self.f, **self.options)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/johnsoni/virtualenvs/pipeline_StructuralVariants/lib/python2.7/site-packages/pandas-0.16.2-py2.7-linux-x86_64.egg/pandas/io/parsers.py", line 1072, in __init__
WARNING:toil.leader:9/5/jobt4lXkT        self._reader = _parser.TextReader(src, **kwds)
WARNING:toil.leader:9/5/jobt4lXkT      File "pandas/parser.pyx", line 350, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:3173)
WARNING:toil.leader:9/5/jobt4lXkT      File "pandas/parser.pyx", line 594, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:5912)
WARNING:toil.leader:9/5/jobt4lXkT    IOError: File ./somaticSV_inv_corrected_edited.tab does not exist
WARNING:toil.leader:9/5/jobt4lXkT    Merging vcf and iAnnotateSV output
WARNING:toil.leader:9/5/jobt4lXkT    Error in fread(tab.filenames) : 
WARNING:toil.leader:9/5/jobt4lXkT      File './C-7FP87A-N002-d_Annotated.txt' does not exist or is non-readable. getwd()=='/juno/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469/tmpH2F4Cy/a/1/out_tmpdirhctDwV'
WARNING:toil.leader:9/5/jobt4lXkT    Calls: %>% -> eval -> eval -> fread
WARNING:toil.leader:9/5/jobt4lXkT    Execution halted
WARNING:toil.leader:9/5/jobt4lXkT    Adding DMP IMPACT fusion frequency
WARNING:toil.leader:9/5/jobt4lXkT    Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/work/access/production/runs/Project_10479_B/structural_variants/Proj_10479_B-1.3.26-1-gedd7803_ikQiCZ_24469
WARNING:toil.leader:9/5/jobt4lXkT    Exception in thread "main" java.io.FileNotFoundException: ./C-7FP87A-N002-d_Annotated_Evidence.txt (No such file or directory)
WARNING:toil.leader:9/5/jobt4lXkT    	at java.io.FileInputStream.open(Native Method)
WARNING:toil.leader:9/5/jobt4lXkT    	at java.io.FileInputStream.<init>(FileInputStream.java:138)
WARNING:toil.leader:9/5/jobt4lXkT    	at java.io.FileReader.<init>(FileReader.java:72)
WARNING:toil.leader:9/5/jobt4lXkT    	at org.mskcc.juber.commands.AnnotateSVResults.main(AnnotateSVResults.java:46)
WARNING:toil.leader:9/5/jobt4lXkT    [job manta_annotation.cwl] Job error:
WARNING:toil.leader:9/5/jobt4lXkT    Error collecting output for parameter 'sv_file_annotated':
WARNING:toil.leader:9/5/jobt4lXkT    :1:1: Did not find output file with glob pattern: '[u'C-7FP87A-N002-d_Annotated_Evidence-annotated.txt']'
WARNING:toil.leader:9/5/jobt4lXkT    ERROR:cwltool:[job manta_annotation.cwl] Job error:
WARNING:toil.leader:9/5/jobt4lXkT    Error collecting output for parameter 'sv_file_annotated':
WARNING:toil.leader:9/5/jobt4lXkT    :1:1: Did not find output file with glob pattern: '[u'C-7FP87A-N002-d_Annotated_Evidence-annotated.txt']'
WARNING:toil.leader:9/5/jobt4lXkT    [job manta_annotation.cwl] completed permanentFail
WARNING:toil.leader:9/5/jobt4lXkT    WARNING:cwltool:[job manta_annotation.cwl] completed permanentFail
WARNING:toil.leader:9/5/jobt4lXkT    WARNING:toil.fileStore:LOG-TO-MASTER: Job used more disk than requested. Consider modifying the user script to avoid the chance of failure due to incorrectly requested resources. Job z/b/jobpUFsyi/g/tmpdEoZaz-_serialiseJob-stream used 112.26% (2.2 GB [2410689536B] used, 2.0 GB [2147483648B] requested) at the end of its run.
WARNING:toil.leader:9/5/jobt4lXkT    Traceback (most recent call last):
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/worker.py", line 324, in workerScript
WARNING:toil.leader:9/5/jobt4lXkT        job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/job.py", line 1351, in _runner
WARNING:toil.leader:9/5/jobt4lXkT        returnValues = self._run(jobGraph, fileStore)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/job.py", line 1296, in _run
WARNING:toil.leader:9/5/jobt4lXkT        return self.run(fileStore)
WARNING:toil.leader:9/5/jobt4lXkT      File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/cwl/cwltoil.py", line 606, in run
WARNING:toil.leader:9/5/jobt4lXkT        raise cwltool.errors.WorkflowException(status)
WARNING:toil.leader:9/5/jobt4lXkT    WorkflowException: permanentFail
WARNING:toil.leader:9/5/jobt4lXkT    ERROR:toil.worker:Exiting the worker because of a failed job on host ju21
WARNING:toil.leader:9/5/jobt4lXkT    WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' 9/5/jobt4lXkT with ID 9/5/jobt4lXkT to 0
WARNING:toil.leader:Job 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' 9/5/jobt4lXkT with ID 9/5/jobt4lXkT is completely failed
INFO:toil.leader:Job ended successfully: 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' n/j/jobZOMdXZ
WARNING:toil.statsAndLogging:Got message from job at time 04-22-2020 00:48:11: Job used more disk than requested. Consider modifying the user script to avoid the chance of failure due to incorrectly requested resources. Job z/b/jobpUFsyi/g/tmpt2BRj4-_serialiseJob-stream used 112.26% (2.2 GB [2410705408B] used, 2.0 GB [2147483648B] requested) at the end of its run.
INFO:toil.leader:Job ended successfully: 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' 8/2/jobHuLk4C
WARNING:toil.statsAndLogging:Got message from job at time 04-22-2020 00:48:35: Job used more disk than requested. Consider modifying the user script to avoid the chance of failure due to incorrectly requested resources. Job z/b/jobpUFsyi/g/tmp_E5utD-_serialiseJob-stream used 112.26% (2.2 GB [2410705408B] used, 2.0 GB [2147483648B] requested) at the end of its run.
INFO:toil.leader:Job ended successfully: 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' B/k/jobuCdA_x
INFO:toil.leader:Finished toil run with 10 failed jobs.
INFO:toil.leader:Failed jobs at end of the run: 'CWLGather' W/R/jobeEAPgE 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_annotation.cwl' 9/5/jobt4lXkT 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/manta/manta_filter.cwl' samtools u/K/job6fe7ry 'CWLScatter' m/6/jobWg4xiV 'file:///work/access/production/workflows/access_workflows/v1/ACCESS-Pipeline/cwl_tools/samtools/index.cwl' samtools index m/w/jobVcS2i1 'CWLGather' B/z/jobglfWzY 'CWLScatter' T/e/jobfnhyrZ 'CWLScatter' R/b/jobCayR0T 'CWLScatter' z/b/jobpUFsyi 'CWLWorkflow' I/U/jobuD50tR
Traceback (most recent call last):
  File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/bin/toil-cwl-runner", line 10, in <module>
    sys.exit(main())
  File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/cwl/cwltoil.py", line 1274, in main
    outobj = toil.start(wf1)
  File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/common.py", line 771, in start
    return self._runMainLoop(rootJobGraph)
  File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/common.py", line 1044, in _runMainLoop
    jobCache=self._jobCache).run()
  File "/home/accessbot/miniconda3/envs/ACCESS_1.3.26/lib/python2.7/site-packages/toil/leader.py", line 245, in run
    raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore)
toil.leader.FailedJobsException
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ionox0