qc broken b.c. it expects auxiliary folder to simultaneously exist and not exist
#160
Comments
bytewife
changed the title
qc broken because it qc broken b.c. it expects auxiliary folder to simultaneously exist and not exist
|
Can you make a pull request with the suggested fix
…On Fri, Dec 8, 2023, 5:37 PM Ivy Raine ***@***.***> wrote:
The qc subcommand requires an 'auxiliary' folder to exist to work. This
folder is intended to be provided by train. However, chrombpnet will not
allow for this folder to exist due to the usage extensive of os.makedirs(...,
exist_ok=False) in chrombpnet_qc(). Thus it's impossible for qc to work
correctly. Since this function is only used when the qc subcommand is
used (and not pipeline), I recommend changing that flag to True within
chrombpnet_qc() and in the block specific to qc.
For completeness, here's the error when the train folders are provided:
Traceback (most recent call last):
File "/opt/conda/bin/chrombpnet", line 33, in <module>
sys.exit(load_entry_point('chrombpnet', 'console_scripts', 'chrombpnet')())
File "/scratch/chrombpnet/chrombpnet/CHROMBPNET.py", line 26, in main
os.makedirs(os.path.join(args.output_dir,"auxiliary"), exist_ok=False)
File "/opt/conda/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/projectTest/output/auxiliary'
Exit code: 1
and here's when they're not provided:
Traceback (most recent call last):
got the model
loading peaks...
File "/opt/conda/bin/chrombpnet", line 33, in <module>
sys.exit(load_entry_point('chrombpnet', 'console_scripts', 'chrombpnet')())
File "/scratch/chrombpnet/chrombpnet/CHROMBPNET.py", line 29, in main
pipelines.chrombpnet_qc(args)
File "/scratch/chrombpnet/chrombpnet/pipelines.py", line 196, in chrombpnet_qc
predict.main(args_copy)
File "/scratch/chrombpnet/chrombpnet/training/predict.py", line 105, in main
test_generator = initializers.initialize_generators(args, mode="test", parameters=None, return_coords=True)
File "/scratch/chrombpnet/chrombpnet/training/data_generators/initializers.py", line 69, in initialize_generators
peak_regions=pd.read_csv(args.peaks,header=None,sep='\t',names=NARROWPEAK_SCHEMA)
File "/opt/conda/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 948, in read_csv
return _read(filepath_or_buffer, kwds)
File "/opt/conda/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 611, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/opt/conda/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1448, in __init__
self._engine = self._make_engine(f, self.engine)
File "/opt/conda/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1705, in _make_engine
self.handles = get_handle(
File "/opt/conda/lib/python3.9/site-packages/pandas/io/common.py", line 863, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: '/projectTest/output/auxiliary/filtered.peaks.bed'
Exit code: 1
Model: "model"
—
Reply to this email directly, view it on GitHub
<#160>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AABDWEKNJT6XQIRO76F4HVLYIO6DXAVCNFSM6AAAAABANNI7D2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGAZTGNJSGY2TAOA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
@akundaje I already made this PR in the link above |
|
@ivyraine is your |
|
Can you provide the exact commands you used to run |
|
I think you are trying to use the same |
|
No- this is from using two different output paths, one for each command. qc cmd: then it leads to the following error: Please allow me to save both your and my time. The reason why the code doesn't work is as I provided in the first comment. |
|
Hello @ivyraine, I appreciate your intention to save both your and my time. But your PR is suggesting a fix that is trying to by-pass a folder existing check which is important to prevent overwriting of existing folders/files. Allow me some time to reproduce this and fix it. |
|
Gotcha- TY |
|
Also your fix wont work - the But chrombpnet qc is looking for it here - Was your fix to change exists_ok to True and just pass the output dir from train to qc ? |
|
I just symlinked the subdirs produced in the |
|
I think |
The
qcsubcommand requires an 'auxiliary' folder to exist to work. This folder is intended to be provided bytrain. However, chrombpnet will not allow for this folder to exist due to the usage ofos.makedirs(..., exist_ok=False)inchrombpnet_qc()and in the block that handlesqcas an input subcommand. Thus it's impossible forqcto work correctly. I recommend changing that flag to True within that block, and inchrombpnet_qc().For completeness, here's the error when the
trainfolders are provided:and here's when they're not provided:
The text was updated successfully, but these errors were encountered: