Pipeline fail to open mitochondrion_final.contigs.fa.reformatted.fa for reading

[leytonrotella]

on which platform/server? (Windows? Windows Sublinux? MacOS? Ubuntu? etc.)

Uppmax which is linux based

MitoZ version?

Mitoz3.6

How did you install MitoZ? (e.g. Docker, Udocker, Singularity, Conda-Pack, Conda, or source code)

Using singularity

Did you run a test after your installation, and was the test run okay?

I'm running the test and the error appears

How much data (roughly) did you use for mitogenome assembly? e.g. 5Gbp?

using the "test" dataset provided in this repository

The command you used?

This is the slurm file commands I used

Load necessary modules

module load bioinfo-tools
module load megahit/1.2.9
module load spades/3.15.5

Navigate to the working directory

cd /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ

Define the paths to the FASTQ files and output directory

outprefix=/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test

Define the path to the existing MitoZ Singularity image

mitoz_image=/proj/stylops_storage/nobackup/carlos/MitoZ_v3.6.sif

Run the MitoZ tool within the Singularity container

/usr/bin/apptainer exec
-B /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ:/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ
-B /sw/bioinfo/megahit/1.2.9/rackham:/sw/bioinfo/megahit/1.2.9/rackham
-B /sw/bioinfo/spades/3.15.5/rackham/bin:/sw/bioinfo/spades/3.15.5/rackham/bin
$mitoz_image /bin/bash -c "
export PATH=/app/anaconda/envs/mitoz3.6/bin:$PATH &&
export PYTHONPATH=/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages:$PYTHONPATH &&
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/MitoZ.py all
--genetic_code 5
--clade Arthropoda
--insert_size 250
--thread_number 4
--fq1 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz
--fq2 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz
--o $outprefix
--fastq_read_length 125
--requiring_taxa Arthropoda
1>m.log 2>m.err"

Problem description

The pipeline seems to work, until the creation of "mitochondrion_final.contigs.fa.reformatted.fa" but after that it can't open the target sequence for reading, even if it has the permissions to read and write and apparently it also has the right path to the file.

Log messages from MitoZ (stdout and stderr, e.g., both m.log and m.err files)

m.err

2024-09-02 17:03:47,568 - mitoz.utility.utility - INFO -
Found: /app/anaconda/envs/mitoz3.6/bin/spades.py

2024-09-02 17:03:47,621 - mitoz.utility.utility - INFO -
Found: /app/anaconda/envs/mitoz3.6/bin/megahit

2024-09-02 17:03:47,621 - mitoz.utility.utility - INFO -
all.main() got args:
Namespace(assembler='megahit', clade='Arthropoda', data_size_for_mt_assembly='2,0', debug=False, fastq_read_length='125', filter_by_taxa=True, filter_other_para='', fq1='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz', fq2='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz', func=<function main at 0x2b4f9eec8a60>, genetic_code='5', insert_size='250', kmers=[71], kmers_megahit=['43', '71', '99'], kmers_spades=['auto'], logger=<Logger mitoz.utility.utility (DEBUG)>, memory='50', min_abundance='10', outprefix='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', phred64=False, profiles_dir='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', requiring_relax='0', requiring_taxa='Arthropoda', resume_assembly=False, skip_filter=False, slow_search=False, species_name='Test sp.', template_sbt='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/template.sbt', thread_number='4', tmp_dir=None, workdir='./')

2024-09-02 17:03:47,634 - mitoz.utility.utility - INFO -
all.main() got updated args:
Namespace(assembler='megahit', clade='Arthropoda', data_size_for_mt_assembly='2,0', debug=False, fastq_read_length='125', filter_by_taxa=True, filter_other_para='', fq1='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz', fq2='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz', func=<function main at 0x2b4f9eec8a60>, genetic_code='5', insert_size='250', kmers=[71], kmers_megahit=['43', '71', '99'], kmers_spades=['auto'], logger=<Logger mitoz.utility.utility (DEBUG)>, memory='50', min_abundance='10', outprefix='/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', phred64=False, profiles_dir='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', requiring_relax='0', requiring_taxa='Arthropoda', resume_assembly=False, skip_filter=False, slow_search=False, species_name='Test sp.', template_sbt='/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/template.sbt', thread_number='4', tmp_dir=None, workdir='/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ')

2024-09-02 17:03:47,635 - mitoz.utility.utility - INFO -
filter.main() get args: {'logger': <Logger mitoz.utility.utility (DEBUG)>, 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz', 'phred64': False, 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', 'fastq_read_length': 125, 'data_size_for_mt_assembly': '2,0', 'filter_other_para': '', 'thread_number': '4', 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data', 'workdir_done': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data', 'workdir_log': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data'}

2024-09-02 17:03:47,635 - mitoz.utility.utility - INFO -
filter.main() get updated args: {'logger': <Logger mitoz.utility.utility (DEBUG)>, 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz', 'phred64': False, 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', 'fastq_read_length': 125, 'data_size_for_mt_assembly': '2,0', 'filter_other_para': '', 'thread_number': '4', 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data', 'workdir_done': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data', 'workdir_log': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data'}

2024-09-02 17:03:47,635 - mitoz.utility.utility - INFO -

    Paired-end data:
    fastq_read_length: 125
    required raw data (Gbp): 2.0
    fq_records_per_file: 8,000,000
    lines_per_file * 4: 32,000,000

2024-09-02 17:03:47,636 - mitoz.utility.utility - INFO -

# set -vex
# set -o pipefail
set -e

mkdir -p /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data
mkdir -p /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data
mkdir -p /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data
cd /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data

task_id="FastpTask"
done_file="/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data/$task_id.done"
log_o_file="/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data/$task_id.o.log"
log_e_file="/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/clean_data/$task_id.e.log"

if [ -f $done_file ]; then
    exit 0
fi

if [ -f "/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz" ] && [ -f "/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz" ];
then
    fastp          -i /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz         -I /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz          --length_required 50         --reads_to_process 8000000         -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz         -O /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz         -R "/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test"         -j /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.json         -h /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.html         -w 4 && 1>$log_o_file 2>$log_e_file && touch $done_file   

else
    fastp          -i /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz          --length_required 50         --reads_to_process 8000000         -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz         -R "/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test"         -j /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.json         -h /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.html         -w 4 && 1>$log_o_file 2>$log_e_file && touch $done_file

fi

Read1 before filtering:
total reads: 49392
total bases: 7413168
Q20 bases: 7321860(98.7683%)
Q30 bases: 7127815(96.1507%)

Read2 before filtering:
total reads: 49392
total bases: 7413168
Q20 bases: 6994828(94.3568%)
Q30 bases: 6449373(86.9989%)

Read1 after filtering:
total reads: 49392
total bases: 7413168
Q20 bases: 7321860(98.7683%)
Q30 bases: 7127815(96.1507%)

Read2 after filtering:
total reads: 49392
total bases: 7413168
Q20 bases: 6994828(94.3568%)
Q30 bases: 6449373(86.9989%)

Filtering result:
reads passed filter: 98784
reads failed due to low quality: 0
reads failed due to too many N: 0
reads failed due to too short: 0
reads with adapter trimmed: 0
bases trimmed due to adapters: 0

Duplication rate: 4.32256%

Insert size peak (evaluated by paired-end reads): 253

JSON report: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.json
HTML report: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.html

fastp -i /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R1.fq.gz -I /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/MitoZ-master/test/test.R2.fq.gz --length_required 50 --reads_to_process 8000000 -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz -O /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz -R /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test -j /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.json -h /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.QC.html -w 4
fastp v0.23.2, time used: 2 seconds

2024-09-02 17:03:49,289 - mitoz.utility.utility - INFO -

sequencing: paired end (151 cycles + 151 cycles)

before_filtering:
    total_reads: 98,784
    total_bases: 14,826,336

after_filtering:
    total_reads: 98,784
    total_bases: 14,826,336

2024-09-02 17:03:49,290 - mitoz.utility.utility - INFO -
filter.main() returns:
/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz
/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz

2024-09-02 17:03:49,291 - mitoz.utility.utility - INFO -
assemble.main() got args:
{'logger': <Logger mitoz.utility.utility (DEBUG)>, 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly', 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', 'thread_number': '4', 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz', 'insert_size': '250', 'fastq_read_length': '125', 'assembler': 'megahit', 'tmp_dir': None, 'kmers': [71], 'kmers_megahit': ['43', '71', '99'], 'kmers_spades': ['auto'], 'memory': '50', 'resume_assembly': False, 'profiles_dir': '/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', 'slow_search': False, 'filter_by_taxa': True, 'requiring_taxa': 'Arthropoda', 'requiring_relax': '0', 'abundance_pattern': 'abun\=([0-9]+\.[0-9])', 'min_abundance': '10', 'genetic_code': '5', 'clade': 'Arthropoda', 'skip_read_mapping': True}

2024-09-02 17:03:49,291 - mitoz.utility.utility - INFO -
assemble.main() got updated args:
{'logger': <Logger mitoz.utility.utility (DEBUG)>, 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly', 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test', 'thread_number': '4', 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz', 'insert_size': '250', 'fastq_read_length': '125', 'assembler': 'megahit', 'tmp_dir': None, 'kmers': [71], 'kmers_megahit': ['43', '71', '99'], 'kmers_spades': ['auto'], 'memory': '50', 'resume_assembly': False, 'profiles_dir': '/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', 'slow_search': False, 'filter_by_taxa': True, 'requiring_taxa': 'Arthropoda', 'requiring_relax': '0', 'abundance_pattern': 'abun\=([0-9]+\.[0-9])', 'min_abundance': '10', 'genetic_code': '5', 'clade': 'Arthropoda', 'skip_read_mapping': True}

2024-09-02 17:03:49,291 - mitoz.utility.utility - INFO -
assemble.main() got
fq1: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz
fq2: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz

2024-09-02 17:03:49,292 - mitoz.utility.utility - INFO -
assemble.main() working in /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly

2024-09-02 17:03:49,292 - mitoz.utility.utility - INFO -
use_megahit() workdir: /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly

2024-09-02 17:03:49,292 - mitoz.utility.utility - INFO -
use_megahit() creating: /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit

2024-09-02 17:03:49,294 - mitoz.utility.utility - INFO -
use_megahit() expected assembly_file: /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa

2024-09-02 17:03:49,294 - mitoz.utility.utility - INFO -
megahit --out-dir ./megahit_out --num-cpu-threads 4 --k-list 43,71,99 --memory 500000000000.0 -1 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz -2 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz
2024-09-02 17:03:49 - MEGAHIT v1.2.9
2024-09-02 17:03:49 - Using megahit_core with POPCNT and BMI2 support
2024-09-02 17:03:49 - Convert reads to binary library
2024-09-02 17:03:49 - b'INFO sequence/io/sequence_lib.cpp : 75 - Lib 0 (/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz,/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz): pe, 98784 reads, 151 max length'
2024-09-02 17:03:49 - b'INFO utils/utils.h : 152 - Real: 0.1759\tuser: 0.1011\tsys: 0.0307\tmaxrss: 7412'
2024-09-02 17:03:49 - Start assembly. Number of CPU threads 4
2024-09-02 17:03:49 - k list: 43,71,99
2024-09-02 17:03:49 - Memory used: 500000000000
2024-09-02 17:03:49 - Extract solid (k+1)-mers for k = 43
2024-09-02 17:03:50 - Build graph for k = 43
2024-09-02 17:03:51 - Assemble contigs from SdBG for k = 43
2024-09-02 17:03:51 - Local assembly for k = 43
2024-09-02 17:03:52 - Extract iterative edges from k = 43 to 71
2024-09-02 17:03:52 - Build graph for k = 71
2024-09-02 17:03:52 - Assemble contigs from SdBG for k = 71
2024-09-02 17:03:52 - Local assembly for k = 71
2024-09-02 17:03:53 - Extract iterative edges from k = 71 to 99
2024-09-02 17:03:53 - Build graph for k = 99
2024-09-02 17:03:53 - Assemble contigs from SdBG for k = 99
2024-09-02 17:03:53 - Merging to output final contigs
2024-09-02 17:03:53 - 3 contigs, total 17235 bp, min 313 bp, max 16552 bp, avg 5745 bp, N50 16552 bp
2024-09-02 17:03:53 - ALL DONE. Time elapsed: 4.326991 seconds

2024-09-02 17:03:53,779 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/assemble/script/megahit/reformat_megahit_scafSeq.py /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa.reformatted.fa

2024-09-02 17:03:53,833 - mitoz.utility.utility - INFO -
findmitoscaf.main() got args:
{'logger': <Logger mitoz.utility.utility (DEBUG)>, 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly', 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit', 'thread_number': '4', 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz', 'insert_size': '250', 'fastq_read_length': '125', 'assembler': 'megahit', 'tmp_dir': None, 'kmers': [71], 'kmers_megahit': ['43', '71', '99'], 'kmers_spades': ['auto'], 'memory': '50', 'resume_assembly': False, 'profiles_dir': '/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', 'slow_search': False, 'filter_by_taxa': True, 'requiring_taxa': 'Arthropoda', 'requiring_relax': '0', 'abundance_pattern': 'abun\=([0-9]+\.[0-9])', 'min_abundance': '10', 'genetic_code': '5', 'clade': 'Arthropoda', 'skip_read_mapping': True, 'fastafile': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa.reformatted.fa'}

2024-09-02 17:03:53,833 - mitoz.utility.utility - INFO -
findmitoscaf.main() got updated args:
{'logger': <Logger mitoz.utility.utility (DEBUG)>, 'workdir': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly', 'outprefix': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit', 'thread_number': '4', 'fq1': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz', 'fq2': '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz', 'insert_size': '250', 'fastq_read_length': '125', 'assembler': 'megahit', 'tmp_dir': None, 'kmers': [71], 'kmers_megahit': ['43', '71', '99'], 'kmers_spades': ['auto'], 'memory': '50', 'resume_assembly': False, 'profiles_dir': '/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles', 'slow_search': False, 'filter_by_taxa': True, 'requiring_taxa': 'Arthropoda', 'requiring_relax': '0', 'abundance_pattern': 'abun\=([0-9]+\.[0-9])', 'min_abundance': '10', 'genetic_code': '5', 'clade': 'Arthropoda', 'skip_read_mapping': True, 'fastafile': '/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa.reformatted.fa'}

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() nhmmer_profile:
/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/CDS_HMM/Arthropoda_CDS.hmm

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() defined_gene_length_f:
/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/CDS_HMM/Arthropoda_CDS_length_list

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() MT_annotation_BGI_pro_db:
/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/MT_database/Animal_CDS_protein.fa

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() workdir:
/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() result_wdir:
/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.result

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() got assembly_file:
/crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa.reformatted.fa

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
findmitoscaf.main() got
fq1: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R1.fq.gz
fq2: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.clean_R2.fq.gz

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
We will skip read-mapping during findmitoscaf().

2024-09-02 17:03:53,834 - mitoz.utility.utility - INFO -
tiara mt_expected_result_file: mitochondrion_final.contigs.fa.reformatted.fa

2024-09-02 17:03:53,835 - mitoz.utility.utility - INFO -
tiara --input /crex/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/mt_assembly/megahit/megahit_out/final.contigs.fa.reformatted.fa --output /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.tiara.out.txt --to_fasta mit pla pro --threads 4 --prob_cutoff 0.65 0.60 --probabilities --min_len 1000

2024-09-02 17:04:11,377 - mitoz.utility.utility - INFO -
nhmmer -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmout --tblout /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout --cpu 4 /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/CDS_HMM/Arthropoda_CDS.hmm mitochondrion_final.contigs.fa.reformatted.fa

Error: Failed to open target sequence database mitochondrion_final.contigs.fa.reformatted.fa for reading

2024-09-02 17:04:11,404 - mitoz.utility.utility - ERROR -
Error occured when running command:
nhmmer -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmout --tblout /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout --cpu 4 /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/CDS_HMM/Arthropoda_CDS.hmm mitochondrion_final.contigs.fa.reformatted.fa

2024-09-02 17:04:11,405 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/get_besthit_of_each_CDS_from_nhmmer.py /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit
Traceback (most recent call last):
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/get_besthit_of_each_CDS_from_nhmmer.py", line 45, in
fh_in = open(nhmmer_tbl_result, 'r')
FileNotFoundError: [Errno 2] No such file or directory: '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout'

2024-09-02 17:04:11,449 - mitoz.utility.utility - ERROR -
Error occured when running command:
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/get_besthit_of_each_CDS_from_nhmmer.py /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit

2024-09-02 17:04:11,449 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/simlify_nhmmer_tbl_besthit.py /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim

2024-09-02 17:04:11,479 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/extract_fasta.py -i mitochondrion_final.contigs.fa.reformatted.fa -q /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa
Traceback (most recent call last):
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/extract_fasta.py", line 53, in
with open(args.i, 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'mitochondrion_final.contigs.fa.reformatted.fa'

2024-09-02 17:04:11,525 - mitoz.utility.utility - ERROR -
Error occured when running command:
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/extract_fasta.py -i mitochondrion_final.contigs.fa.reformatted.fa -q /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim -o /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa

2024-09-02 17:04:11,525 - mitoz.utility.utility - INFO -
filter_by_nhmmer() return:
mitochondrion_final.contigs.fa.reformatted.fa, /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim, /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa

2024-09-02 17:04:11,535 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_taxonomy_by_CDS_annotation.py -fa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa -MTsoft /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/MT_annotation_BGI_V1.32/MT_annotation_BGI.pl -db /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/MT_database/Animal_CDS_protein.fa -thread 4 -genetic_code 5 -requiring_taxa 'Arthropoda' -relax 0 -WISECONFIGDIR /app/anaconda/envs/mitoz3.6/share/wise2/wisecfg -outf /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa
Fail test.megahit.hmmtblout.besthit.sim.fa:No such file or directory at /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/MT_annotation_BGI_V1.32/MT_annotation_BGI.pl line 336.
Error occured when running command:
export WISECONFIGDIR=/app/anaconda/envs/mitoz3.6/share/wise2/wisecfg
perl /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/MT_annotation_BGI_V1.32/MT_annotation_BGI.pl -i test.megahit.hmmtblout.besthit.sim.fa -d /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/MT_database/Animal_CDS_protein.fa -cpu 4 -g 5 -o ./

2024-09-02 17:04:12,897 - mitoz.utility.utility - ERROR -
Error occured when running command:
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_taxonomy_by_CDS_annotation.py -fa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa -MTsoft /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/MT_annotation_BGI_V1.32/MT_annotation_BGI.pl -db /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/MT_database/Animal_CDS_protein.fa -thread 4 -genetic_code 5 -requiring_taxa 'Arthropoda' -relax 0 -WISECONFIGDIR /app/anaconda/envs/mitoz3.6/share/wise2/wisecfg -outf /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa

2024-09-02 17:04:12,898 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_hmm-besthit-sim.py /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa
Traceback (most recent call last):
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_hmm-besthit-sim.py", line 38, in
with open(sys.argv[1], 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa'

2024-09-02 17:04:12,939 - mitoz.utility.utility - ERROR -
Error occured when running command:
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_hmm-besthit-sim.py /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa

2024-09-02 17:04:12,940 - mitoz.utility.utility - INFO -
do_filter_by_taxa() return:
hmm_outf_tbl_besthit_sim_filtered_by_taxa:/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa
hmm_besthit_filtered_by_taxa_fa: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa

2024-09-02 17:04:12,940 - mitoz.utility.utility - INFO -
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_by_abundance.py 'abun=([0-9]+.[0-9])' 10 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa
Traceback (most recent call last):
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_by_abundance.py", line 67, in
main()
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_by_abundance.py", line 58, in main
filter_by_abundance(
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_by_abundance.py", line 16, in filter_by_abundance
with open(in_sim, 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa'

2024-09-02 17:04:13,223 - mitoz.utility.utility - ERROR -
Error occured when running command:
/app/anaconda/envs/mitoz3.6/bin/python /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/script/filter_by_abundance.py 'abun=([0-9]+.[0-9])' 10 /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.fa

2024-09-02 17:04:13,225 - mitoz.utility.utility - INFO -
gather_result() creating: /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.result

2024-09-02 17:04:13,226 - mitoz.utility.utility - INFO -
cp -r test.megahit.hmmtblout.besthit.sim.filtered-by-taxa.low_abundance /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.result
Traceback (most recent call last):
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/MitoZ.py", line 102, in
main()
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/MitoZ.py", line 99, in main
args.func(args)
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/all/all.py", line 361, in main
args.fastafiles, assemble_all_result_wdir = mitoz.assemble.main(assemble_args)
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/assemble/assembly.py", line 507, in main
assembly_file, mt_file = use_megahit(
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/assemble/assembly.py", line 405, in use_megahit
mt_file = findmitoscaf.main(args)
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/findmitoscaf.py", line 532, in main
hmm_outf_tbl_besthit_sim_fasta_like = filter_by_abundance(
File "/app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/findmitoscaf/findmitoscaf.py", line 278, in filter_by_abundance
high_abun_sim = glob('{0}.high_abundance_*'.format(hmm_outf_tbl_besthit_sim))[0]
IndexError: list index out of range

m.log

Classification took 9.533689975738525 seconds.
0.10489118091156883 sequences per second.
1736.1588264482875 base pairs per second.
Classification done.
First iteration statistics:
organelle: 1
Second iteration statistics:
mitochondrion: 1

Output saved to /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.tiara.out.txt.
Log file saved to /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/log_test.megahit.tiara.out.txt.

2024-09-02 17:04:12
ln -s /proj/stylops_storage/nobackup/carlos/3_assembly/mitoZ/test.megahit.hmmtblout.besthit.sim.fa

2024-09-02 17:04:12
export WISECONFIGDIR=/app/anaconda/envs/mitoz3.6/share/wise2/wisecfg
perl /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/annotate/script/MT_annotation_BGI_V1.32/MT_annotation_BGI.pl -i test.megahit.hmmtblout.besthit.sim.fa -d /app/anaconda/envs/mitoz3.6/lib/python3.8/site-packages/mitoz/profiles/MT_database/Animal_CDS_protein.fa -cpu 4 -g 5 -o ./