Augustus Training Error: Missing ExonModel File and UnboundLocalError in Funannotate v1.8.17 Despite Pretrained Species Use

[kalonji08]

Are you using the latest release?
Yes, I am using funannotate version v1.8.17, and the issue persists.

Describe the bug
While running the funannotate predict pipeline, I encountered an issue during the Augustus training step. Augustus fails to open the trametes_hisurta_exon_probs.pbl file during ab initio prediction. I tried running the pipeline with a pretrained species model as well, but the same error occurred. This pipeline worked fine with a previous version of funannotate, but with the current version, it results in the following error:

augustus: ERROR
        ExonModel: Couldn't open file /media/external/FA-Trametes/new_trametes_run/prediction_trametes_1/predict_misc/ab_initio_parameters/augustus/species/trametes_hisurta/trametes_hisurta_exon_probs.pbl

Additionally, I found that the following files are present in the species folder but not the one that caused the error:

trametes_hisurta_metapars.cfg  
trametes_hisurta_metapars.cgp.cfg  
trametes_hisurta_metapars.utr.cfg  
trametes_hisurta_parameters.cfg  
trametes_hisurta_weightmatrix.txt

The traceback error in funannotate is:

UnboundLocalError: local variable 'values1' referenced before assignment

What command did you issue?

funannotate predict -i Trametes-hirsuta-S2-contigs_masked.fasta -o prediction_trametes_1 -s "Trametes hisurta" --cpus 22

Logfiles
Here are the relevant portions from the log:

[Sep 20 12:48 PM]: OS: Ubuntu 20.04, 32 cores, ~ 82 GB RAM. Python: 3.8.19
[Sep 20 12:48 PM]: Running funannotate v1.8.17
[Sep 20 12:48 PM]: GeneMark not found and $GENEMARK_PATH environmental variable missing. Will skip GeneMark ab-initio prediction.
[Sep 20 12:48 PM]: Skipping CodingQuarry as no --rna_bam passed
[Sep 20 02:54 PM]: Training Augustus using BUSCO gene models
augustus: ERROR
        ExonModel: Couldn't open file /media/external/FA-Trametes/new_trametes_run/prediction_trametes_1/predict_misc/ab_initio_parameters/augustus/species/trametes_hisurta/trametes_hisurta_exon_probs.pbl

OS/Install Information
Ubuntu 20.04
Python 3.8.19
funannotate v1.8.17

Output of funannotate check --show-versions:

-------------------------------------------------------
Checking dependencies for funannotate v1.8.17
-------------------------------------------------------
You are running Python 3.8.19. Now checking python packages...
biopython: 1.79
goatools: 1.4.12
matplotlib: 3.5.2
natsort: 8.4.0
numpy: 1.24.4
pandas: 1.4.3
psutil: 5.9.4
requests: 2.32.3
scikit-learn: 1.3.2
scipy: 1.8.1
seaborn: 0.11.2
All 11 python packages installed

You are running Perl v5.032001. Now checking perl modules...
All 27 Perl modules installed

Checking Environmental Variables...
$FUNANNOTATE_DB=/home/kalonjilab/funannotate_db
$PASAHOME=/home/kalonjilab/miniconda3/envs/funannotate/opt/pasa-2.5.3
$TRINITY_HOME=/home/kalonjilab/miniconda3/envs/funannotate/opt/trinity-2.15.2
$EVM_HOME=/home/kalonjilab/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0
$AUGUSTUS_CONFIG_PATH=/home/kalonjilab/miniconda3/envs/funannotate/config/
        ERROR: GENEMARK_PATH not set. export GENEMARK_PATH=/path/to/dir
-------------------------------------------------------
Checking external dependencies...
PASA: 2.5.3
CodingQuarry: 2.0
Trinity: 2.15.2
augustus: 3.5.0
bedtools: v2.31.1
diamond: 2.1.8
exonerate: 2.4.0
samtools: 1.21
tbl2asn: 25.8
        ERROR: gmes_petap.pl not installed
        ERROR: signalp not installed

[kalonji08]

I also run this :
funannotate test -t predict --cpus 12

And i got a similar error:
_misc/EVM/CP022973.1/CP022973.1_294245-357021/evm.out.log'],), **{})
error: [Errno 2] No such file or directory: '/home/kalonjilab/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/evidence_modeler.pl' run(*(['/home/kalonjilab/miniconda3/envs/funannotate/opt/evidencemodeler-2.1.0/evidence_modeler.pl', '-G', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084/genome.softmasked.fa', '-g', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084/gene_predictions.gff3', '-w', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/weights.evm.txt', '--min_intron_length', '10', '--exec_dir', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084', '-p', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084/protein_alignments.gff3', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084/evm.out', '/home/kalonjilab/testing_FA/test_mamba/test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_misc/EVM/CP022973.1/CP022973.1_431922-576084/evm.out.log'],), **{})
Progress: 38 complete, 0 failed, 0 remaining
[Sep 20 04:30 PM]: Converting to GFF3 and collecting all EVM results
[Sep 20 04:30 PM]: Evidence modeler has failed, exiting
#########################################################
Traceback (most recent call last):
File "/home/kalonjilab/miniconda3/envs/funannotate/bin/funannotate", line 10, in
sys.exit(main())
File "/home/kalonjilab/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/funannotate.py", line 717, in main
mod.main(arguments)
File "/home/kalonjilab/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 405, in main
runPredictTest(args)
File "/home/kalonjilab/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 160, in runPredictTest
assert 1500 <= countGFFgenes(os.path.join(
File "/home/kalonjilab/miniconda3/envs/funannotate/lib/python3.8/site-packages/funannotate/test.py", line 45, in countGFFgenes
with open(input, 'r') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'test-predict_b960c5d6-9f1a-433b-9fa5-d84404cf30ca/annotate/predict_results/Awesome_testicus.gff3'

[hyphaltip]

hard to say - the test failure seems to be more EVM related not augustus,
do you have a partially failed augustus training run
I would remove
/media/external/FA-Trametes/new_trametes_run/prediction_trametes_1/predict_misc/ab_initio_parameters and rerun predict and see?

[hyphaltip]

The pre-training I wonder reflects different versions of augustus - it may be that pre-trained was for an earlier version of augustus than the one you have installed now since it sounds like the exon probability file not generated.

An alternative problem which the testing step is revealing, suggests augustus or something else is not really working in your install - can you test runnning augustus alone? It is reporting a version number so I assume it is partially ok.