applying cNMF pipeline to denoised data

[wienecke]

Hi.
Is there any reason CaImAn's cNMF pipeline should not be used on 2p calcium data that has already been denoised (by DeepCAD, for example)? I've been motion correcting with CaImAn, then denoising with DeepCAD, then running the CaImAn cNMF pipeline to extract sources and their df/f. I'm skipping deconvolution by setting p = 0 because I'm only imaging at 5 Hz. Do you have any specific recommendations for adapting the CaImAn cNMF pipeline for data that is already denoised? For example, rather than using estimates.C, should I use estimates.C + estimates.R? I'm asking because when I apply CaImAn's cNMF pipeline to DeepCAD-denoised data, the residual traces (estimates.R) are still typically large, relative to estimates.C. Please see screenshot below, where estimates.C is red and estimates.C + estimates.R is blue. I expected the DeepCAD-denoised data to have a much smaller residual component. So these results made me wonder if I'm throwing away a lot of signal by using estimates.C. But, might most of this residual trace be coming from spatially overlapping components (i.e. even if the data were perfectly noiseless, I might still see a large residual in estimates.R)?
Thank you!

[EricThomson]

This is a relatively unexplored territory there are as of yet no consensus answers about denoising before running CNMF. Until recently, we've always said don't denoise, CNMF itself denoises. But I'm not convinced this is true and encourage people to explore and please let us know what you find. Please also be sure to look at the results from deep network interpolated denoising, they can hallucinate neural activity into your movies so be careful because CNMF will discover neural activity there. I don't know much about these denoising methods yet and it is something we need to get up to speed on.

I would not necessarily say that for denoised data you should use C+YrA, with any models you will still have residuals in any model and it is typically assumed to be high frequency biologically unrealistic stuff, and I'm not sure that denoising beforehand would make this any different (though it would change the shape of the noise and magnitude, so I'm not giving any hard and fast rules).

I would play with it, play with params, and see what looks reasonable, and watch for hallucinations 😄

[wienecke]

Thank you! For any one estimates.C, should the residuals contain noise (high frequency biologically unrealistic stuff) and real biological signal from other components (other estimates.C traces from the same source extraction run)?

[kushalkolar]

When working with 2p data, the residuals are due to instrument noise (i.e. the PMT).

…

On Wed, Sep 27, 2023, 18:06 wienecke ***@***.***> wrote: Thank you! For any one estimates.C, should the residuals contain noise (high frequency biologically unrealistic stuff) and real biological signal from other components (other estimates.C traces from the same source extraction run)? — Reply to this email directly, view it on GitHub <#1176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACHXXRFZLJHVUJZ25SI7WZLX4SPPZANCNFSM6AAAAAA5DXHROY> . You are receiving this because you are subscribed to this thread.Message ID: <flatironinstitute/CaImAn/repo-discussions/1176/comments/7128083@ github.com>

[wienecke]

Thank you!