Backsub is not working with new update?

[hegewisch15]

Hello!

I am having trouble all of a sudden running backsub on ome.tif images which were run through ashlar outside of mcmicro. I was not having issues with this before but when I try to run backsub I get the error below. I did not make any changes to the markers.csv file format or params.yml

I set the start and stop to background since I am running ashlar separately and would like to use the markers_bs.csv to then use the appropriate channel numbers for segmentation. Thank you for the help!

(base) ~ % nextflow run labsyspharm/mcmicro --in /Volumes/Beluga/03232024_CyCIF/ENT057

N E X T F L O W ~ version 24.04.4

Launching https://github.com/labsyspharm/mcmicro [marvelous_austin] DSL2 - revision: f25fcbf [master]

[- ] staging:phenoimager2mc -
[- ] staging:phenoimager2mc -
[- ] illumination -
[- ] staging:phenoimager2mc -
[- ] illumination -
[- ] registration:ashlar -
[- ] background:backsub | 0 of 1
[- ] dearray:coreograph -
[- ] dearray:roadie:runTask -
[- ] segmentation:roadie:runTask -
[- ] segmentation:worker -
[- ] segmentation:s3seg -
[- ] quantification:mcquant -
[- ] downstream:worker -
[- ] viz:autominerva -
No background subtracted image in /Volumes/Beluga/03232024_CyCIF/ENT057/background
No background subtracted markers file in /Volumes/Beluga/03232024_CyCIF/ENT057/background

[ArtemSokolov]

Thanks for catching this, @hegewisch15.
Looks like the latest PR had an off-by-one error, where the pipeline was looking for precomputed backsub images at the wrong step.
Should be fixed in e67733e, but let me know if you're still having issues.

[hegewisch15]

I ran into another issue, which I have not seen before. I pasted it below.

nextflow run labsyspharm/mcmicro --in /Volumes/Beluga/03232024_CyCIF/ENT057

N E X T F L O W ~ version 24.04.4

Launching https://github.com/labsyspharm/mcmicro [focused_panini] DSL2 - revision: e67733e [master]

executor > local (1)
[- ] staging:phenoimager2mc -
[- ] illumination -
[- ] registration:ashlar -
[6d/b2f6fa] background:backsub (1) | 0 of 1
[- ] dearray:coreograph -
[- ] dearray:roadie:runTask -
[- ] segmentation:roadie:runTask -
[- ] segmentation:worker -
[- ] segmentation:s3seg -
[- ] quantification:mcquant -
[- ] downstream:worker -
[- ] viz:autominerva -
ERROR ~ Error executing process > 'background:backsub (1)'

Caused by:
Process background:backsub (1) terminated with an error exit status (1)

Command executed:

python3 /background_subtraction/background_sub.py -r ENT057_crop.ome.tif -m markers.csv -o ENT057_crop_backsub.ome.tif -mo markers_bs.csv

Command exit status:
1

Command output:
Channel CCL3_11 (43) processed, background subtraction
Channel DNA_12 (44) processed, no background subtraction
Channel CCL19 (45) processed, background subtraction
Channel KLRD1 (46) processed, background subtraction
Channel CX3CL1_12 (47) processed, background subtraction
Channel DNA_13 (48) processed, no background subtraction
Channel AF488_23 (49) processed, no background subtraction
Channel CD122_NCR3 (50) processed, background subtraction
Channel CD117_13 (51) processed, background subtraction
Channel DNA_14 (52) processed, no background subtraction
Channel VCAM1_14 (53) processed, background subtraction
Channel ICAM1_14 (54) processed, background subtraction
Channel NCR3 (55) processed, background subtraction
Channel DNA_15 (56) processed, no background subtraction
Channel PanKeratin (57) processed, background subtraction
Channel IL15 (58) processed, background subtraction
Channel CCL4_15 (59) processed, background subtraction
Channel DNA_16 (60) processed, no background subtraction
Channel SMA (61) processed, background subtraction
Channel CD69 (62) processed, background subtraction
Channel ColType1 (63) processed, background subtraction
Channel DNA_17 (64) processed, no background subtraction
Channel CD57 (65) processed, background subtraction
Channel CD103 (66) processed, background subtraction
Channel CD117_17 (67) processed, background subtraction
Channel DNA_18 (68) processed, no background subtraction
Channel Ki67_18 (69) processed, background subtraction
Channel CD11c_18 (70) processed, background subtraction
Channel CD56_18 (71) processed, background subtraction
Channel DNA_19 (72) processed, no background subtraction
Channel NKp46_IP (73) processed, background subtraction
Channel CD11c_19 (74) processed, background subtraction
Channel CX3CL1_19 (75) processed, background subtraction
Channel DNA_20 (76) processed, no background subtraction
Channel CCL19 (77) processed, background subtraction
Channel CD31 (78) processed, background subtraction
Channel NKp80_20 (79) processed, background subtraction
Channel DNA_21 (80) processed, no background subtraction
Channel CD8_CD122 (81) processed, background subtraction
Channel AF568_background (82) processed, no background subtraction
Channel AF647_background (83) processed, no background subtraction
Channel DNA_22 (84) processed, no background subtraction
Channel TBET (85) processed, background subtraction
Channel AF568_background (86) processed, no background subtraction
Channel CD127_2 (87) processed, background subtraction
Channel DNA_23 (88) processed, no background subtraction
Channel AF488_background (89) processed, no background subtraction
Channel CD94_23 (90) processed, background subtraction
Channel CD8 (91) processed, background subtraction
[(1024, 1024), (1024, 1024), (1024, 1024), (1024, 1024), (1024, 1024)]

Command error:
Assembling mosaic 1/ 1 (channel 3/55): 25%|##4 | 24001/96054 [01:22<01:23, 860.25it/s]
Assembling mosaic 1/ 1 (channel 3/55): 25%|##5 | 24091/96054 [01:22<01:31, 789.48it/s]
Assembling mosaic 1/ 1 (channel 3/55): 25%|##5 | 24335/96054 [01:22<00:59, 1204.29it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##5 | 24650/96054 [01:22<00:41, 1713.33it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##5 | 24960/96054 [01:23<00:34, 2083.08it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##6 | 25276/96054 [01:23<00:29, 2383.68it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##6 | 25584/96054 [01:23<00:27, 2582.64it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##6 | 25851/96054 [01:23<00:27, 2554.46it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##7 | 26148/96054 [01:23<00:26, 2673.84it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##7 | 26462/96054 [01:23<00:24, 2803.29it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##7 | 26781/96054 [01:23<00:23, 2911.28it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##8 | 27075/96054 [01:23<00:23, 2915.56it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##8 | 27384/96054 [01:23<00:23, 2953.47it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##8 | 27703/96054 [01:23<00:22, 3015.01it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##9 | 28006/96054 [01:24<00:22, 3015.17it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##9 | 28316/96054 [01:24<00:22, 3018.46it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|##9 | 28623/96054 [01:24<00:22, 3013.40it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|### | 28936/96054 [01:24<00:22, 3032.38it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|### | 29247/96054 [01:24<00:21, 3052.12it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|### | 29557/96054 [01:24<00:21, 3063.31it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|###1 | 29870/96054 [01:24<00:21, 3072.36it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|###1 | 30178/96054 [01:25<00:49, 1335.96it/s]
Assembling mosaic 1/ 1 (channel 3/55): 32%|###1 | 30411/96054 [01:25<01:12, 911.30it/s]
Assembling mosaic 1/ 1 (channel 3/55): 32%|###1 | 30470/96054 [01:25<03:04, 354.63it/s]
Traceback (most recent call last):
File "/background_subtraction/background_sub.py", line 429, in
main(args)
File "/background_subtraction/background_sub.py", line 416, in main
write_pyramid(
File "/background_subtraction/background_sub.py", line 180, in write_pyrami
tif.write(
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 3049, in write
iteritem, dataiter = peek_iterator(dataiter)
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21409, in peek_iterator
first = next(iterator)
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21413, in newiter
yield from iterator
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21413, in newiter
yield from iterator
File "/background_subtraction/background_sub.py", line 241, in tile_from_combined_mosaics
c = da_to_zarr(c) if save_RAM else c.compute()
executor > local (1)
[- ] staging:phenoimager2mc -
[- ] illumination -
[- ] registration:ashlar -
[6d/b2f6fa] background:backsub (1) | 1 of 1, failed: 1 ✘
[- ] dearray:coreograph -
[- ] dearray:roadie:runTask -
[- ] segmentation:roadie:runTask -
[- ] segmentation:worker -
[- ] segmentation:s3seg -
[- ] quantification:mcquant -
[- ] downstream:worker -
[- ] viz:autominerva -
WARN: Got an interrupted exception while taking agent result | java.lang.InterruptedException
ERROR ~ Error executing process > 'background:backsub (1)'

Caused by:
Process background:backsub (1) terminated with an error exit status (1)

Command executed:

python3 /background_subtraction/background_sub.py -r ENT057_crop.ome.tif -m markers.csv -o ENT057_crop_backsub.ome.tif -mo markers_bs.csv

Command exit status:
1

Command output:
Channel CCL3_11 (43) processed, background subtraction
Channel DNA_12 (44) processed, no background subtraction
Channel CCL19 (45) processed, background subtraction
Channel KLRD1 (46) processed, background subtraction
Channel CX3CL1_12 (47) processed, background subtraction
Channel DNA_13 (48) processed, no background subtraction
Channel AF488_23 (49) processed, no background subtraction
Channel CD122_NCR3 (50) processed, background subtraction
Channel CD117_13 (51) processed, background subtraction
Channel DNA_14 (52) processed, no background subtraction
Channel VCAM1_14 (53) processed, background subtraction
Channel ICAM1_14 (54) processed, background subtraction
Channel NCR3 (55) processed, background subtraction
Channel DNA_15 (56) processed, no background subtraction
Channel PanKeratin (57) processed, background subtraction
Channel IL15 (58) processed, background subtraction
Channel CCL4_15 (59) processed, background subtraction
Channel DNA_16 (60) processed, no background subtraction
Channel SMA (61) processed, background subtraction
Channel CD69 (62) processed, background subtraction
Channel ColType1 (63) processed, background subtraction
Channel DNA_17 (64) processed, no background subtraction
Channel CD57 (65) processed, background subtraction
Channel CD103 (66) processed, background subtraction
Channel CD117_17 (67) processed, background subtraction
Channel DNA_18 (68) processed, no background subtraction
Channel Ki67_18 (69) processed, background subtraction
Channel CD11c_18 (70) processed, background subtraction
Channel CD56_18 (71) processed, background subtraction
Channel DNA_19 (72) processed, no background subtraction
Channel NKp46_IP (73) processed, background subtraction
Channel CD11c_19 (74) processed, background subtraction
Channel CX3CL1_19 (75) processed, background subtraction
Channel DNA_20 (76) processed, no background subtraction
Channel CCL19 (77) processed, background subtraction
Channel CD31 (78) processed, background subtraction
Channel NKp80_20 (79) processed, background subtraction
Channel DNA_21 (80) processed, no background subtraction
Channel CD8_CD122 (81) processed, background subtraction
Channel AF568_background (82) processed, no background subtraction
Channel AF647_background (83) processed, no background subtraction
Channel DNA_22 (84) processed, no background subtraction
Channel TBET (85) processed, background subtraction
Channel AF568_background (86) processed, no background subtraction
Channel CD127_2 (87) processed, background subtraction
Channel DNA_23 (88) processed, no background subtraction
Channel AF488_background (89) processed, no background subtraction
Channel CD94_23 (90) processed, background subtraction
Channel CD8 (91) processed, background subtraction
[(1024, 1024), (1024, 1024), (1024, 1024), (1024, 1024), (1024, 1024)]

Command error:
Assembling mosaic 1/ 1 (channel 3/55): 25%|##4 | 24001/96054 [01:22<01:23, 860.25it/s]
Assembling mosaic 1/ 1 (channel 3/55): 25%|##5 | 24091/96054 [01:22<01:31, 789.48it/s]
Assembling mosaic 1/ 1 (channel 3/55): 25%|##5 | 24335/96054 [01:22<00:59, 1204.29it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##5 | 24650/96054 [01:22<00:41, 1713.33it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##5 | 24960/96054 [01:23<00:34, 2083.08it/s]
Assembling mosaic 1/ 1 (channel 3/55): 26%|##6 | 25276/96054 [01:23<00:29, 2383.68it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##6 | 25584/96054 [01:23<00:27, 2582.64it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##6 | 25851/96054 [01:23<00:27, 2554.46it/s]
Assembling mosaic 1/ 1 (channel 3/55): 27%|##7 | 26148/96054 [01:23<00:26, 2673.84it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##7 | 26462/96054 [01:23<00:24, 2803.29it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##7 | 26781/96054 [01:23<00:23, 2911.28it/s]
Assembling mosaic 1/ 1 (channel 3/55): 28%|##8 | 27075/96054 [01:23<00:23, 2915.56it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##8 | 27384/96054 [01:23<00:23, 2953.47it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##8 | 27703/96054 [01:23<00:22, 3015.01it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##9 | 28006/96054 [01:24<00:22, 3015.17it/s]
Assembling mosaic 1/ 1 (channel 3/55): 29%|##9 | 28316/96054 [01:24<00:22, 3018.46it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|##9 | 28623/96054 [01:24<00:22, 3013.40it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|### | 28936/96054 [01:24<00:22, 3032.38it/s]
Assembling mosaic 1/ 1 (channel 3/55): 30%|### | 29247/96054 [01:24<00:21, 3052.12it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|### | 29557/96054 [01:24<00:21, 3063.31it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|###1 | 29870/96054 [01:24<00:21, 3072.36it/s]
Assembling mosaic 1/ 1 (channel 3/55): 31%|###1 | 30178/96054 [01:25<00:49, 1335.96it/s]
Assembling mosaic 1/ 1 (channel 3/55): 32%|###1 | 30411/96054 [01:25<01:12, 911.30it/s]
Assembling mosaic 1/ 1 (channel 3/55): 32%|###1 | 30470/96054 [01:25<03:04, 354.63it/s]
Traceback (most recent call last):
File "/background_subtraction/background_sub.py", line 429, in
main(args)
File "/background_subtraction/background_sub.py", line 416, in main
write_pyramid(
File "/background_subtraction/background_sub.py", line 180, in write_pyrami
tif.write(
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 3049, in write
iteritem, dataiter = peek_iterator(dataiter)
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21409, in peek_iterator
first = next(iterator)
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21413, in newiter
yield from iterator
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/tifffile/tifffile.py", line 21413, in newiter
yield from iterator
File "/background_subtraction/background_sub.py", line 241, in tile_from_combined_mosaics
c = da_to_zarr(c) if save_RAM else c.compute()
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/dask/base.py", line 342, in compute
(result,) = compute(self, traverse=False, **kwargs)
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/dask/base.py", line 628, in compute
results = schedule(dsk, keys, **kwargs)
File "/background_subtraction/background_sub.py", line 337, in scale_background
background_channel = np.rint(ne.evaluate("background_channel * scalar"))
File "/opt/conda/envs/backsub/lib/python3.9/site-packages/numexpr/necompiler.py", line 827, in evaluate
return compiled_ex(*arguments, **kwargs)
ValueError: operands could not be broadcast together with shapes (5000,4160) (2,)

Work dir:
/Users/eveyhegewischsolloa/work/6d/b2f6faeb82a742b185046ac598d946

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

[ArtemSokolov]

That looks like a problem with the module itself.
@kbestak: would you be able to take a look?

[kbestak]

Hi @hegewisch15 ,

I was able to recreate the error you've shown by having multiple rows in the markers.csv with the same marker_name value that is attempting to be subtracted. I have opened an issue to raise an error if this happens in the future: SchapiroLabor/Background_subtraction#18

The line below indicates to me that the first row with a specified background channel matches two channels instead of only one (background value matches two marker_name values)

ValueError: operands could not be broadcast together with shapes (5000,4160) (2,)

Could you please check that values in the marker_name column are unique or share your marker file?