Dandiset-wide visualizations

[pauladkisson]

Many of the visualizations coming from MoSeq like the characteristic "crowd movies" require loading many sessions simultaneously. With this in mind, it would be nice if Neurosift could support some kind of Dandiset-wide visualization structure to accommodate requests like these. I know that Neurosift is still in alpha, so this kind of enhancement may not be an immediate priority, but it could be a very useful extension.

Parallel Issue was raised in NWBWidgets: catalystneuro/nwbwidgets#310

[magland]

Thanks Paul. Yes, I'd like to expand into dandiset-wide visualizations, probably using some sort of scripting or yaml spec to compose the custom figures. Could you describe a minimal example of a visualization that would span more than one NWB file?

[pauladkisson]

I would be happy to, but @kaijfox, @wingillis, and/or @calebweinreb would be able to provide more detail/code.

One minimal example would be syllable usage stats: Each animal has a repertoire of behavioral syllables that they use with different frequencies. These usages are normally computed over many different sessions (and thus many nwb files).

[magland]

Thanks Paul. Is there a dandiset I can look at?

[kaijfox]

I think the central thing to be looking at would be the analysis notebook in dattalab/moseq2-app. Some further docs that might be a good starting point live on the wiki.

A lot of the actual visualization code lives in dattalab/moseq2-viz (docs here).

This all is meant to work in the interactive jupyter environment where someone uses the functions in moseq2_viz to compute some summary dataframes then uses those to run build visualizations. Creating those dataframes isn't especially computationally intensive, but in our offline discussion when we had talked about precomputing some level of session-wise summary information that the dandiset-wide visualizations could work off of and those dataframes might be a good candidate.

Happy to chat more but hopefully this is a start!

[pauladkisson]

Thanks Paul. Is there a dandiset I can look at?

It's still in progress right now (Draft) but this is the one to watch: 000559

[magland]

Thanks. Here's a view of a BehaviorSyllable dataset

and a position plot

I'm going to work on the OnePhotonSeries

[magland]

Just a note on the chunking of the OnePhotonSeries objects in this dandiset... the chunks are way too small to be able to efficiently load this for visualization in the browser.

import h5py
import remfile

url = 'https://dandiarchive.s3.amazonaws.com/blobs/663/731/66373118-1b68-42c4-bf66-1d253bccbc78'

# open the remote file
f = h5py.File(remfile.File(url), 'r')

# load the neurodata object
X = f['/acquisition/Reference1PSeries']

print(X['data'].chunks)

# Output: (15, 33, 86)

[pauladkisson]

Thanks for the catch! Will fix

[pauladkisson]

@magland, chunks are now 14 full frames (9.55MiB). Is that sufficiently large for visualizations?

[magland]

@pauladkisson Sorry for the delay in reply. Do you have some datasets on dandi that have the new chunking? I can check out to see how the visualization performs.

[CodyCBakerPhD]

@magland Those latest DANDI set 59 files also have a raw/processed file split per session, which could additionally constitute a 'dandiset-wide' or other multi-file-per-session-id folder-level approach that could be useful (i.e., viewing the raw acquisition or LFP time series against the behavioral data in the other file)

[magland]

@magland Those latest DANDI set 59 files also have a raw/processed file split per session, which could additionally constitute a 'dandiset-wide' or other multi-file-per-session-id folder-level approach that could be useful (i.e., viewing the raw acquisition or LFP time series against the behavioral data in the other file)

@CodyCBakerPhD
How do you imagine that working from the user's perspective?

[CodyCBakerPhD]

How do you imagine that working from the user's perspective?

Personally I'd prefer some kind of layer that allows navigation of all the files in the DANDI set - similar to how we can currently select multiple data streams to view simultaneously within a single file - which allows me to select multiple files to view simultaneously

That could have two uses cases beyond what @pauladkisson wants

i) select two NWB files that have the same session_id/session_start_time/timestamps_reference and potentially overlapping neurodata objects - takes the union of the two file contents and allows viewing it as if it were a single file, but stored as multiple separate files on DANDI (a lot of users like this pattern so they can easily download all of the lightweight processed data separate from the bulk raw acquisition)

ii) for chronic studies, they often look at how some basic value in a neurodata value changes progressively over sessions, so being able to select a recurring field over several sessions (several NWB files) could prove useful to showing how a trend varies over long timescales (current approach would just be to open multiple neurosift tabs at the same time and move them to different parts of the screen)

[pauladkisson]

@pauladkisson Sorry for the delay in reply. Do you have some datasets on dandi that have the new chunking? I can check out to see how the visualization performs.

This dandiset should have the new chunking.

[CodyCBakerPhD]

Status update: this capability is technically possible with explicit neurosift URLs of the form

https://flatironinstitute.github.io/neurosift/?p=/nwb&url=[first]&url=[second]&url=[third]

however, two things missing before we can close this: (i) documentation of this for individuals that manually know how to get asset URLs [sorry if this exists and is and I missed it] and/or (ii) a simpler interface for enabling this interaction, such as a neurosift landing page that summarizes/navigates the DANDI archive like how Dendro does it and lets you select the files you want to jointly view

[magland]

interface for enabling this interaction, such as a neurosift landing page that summarizes/navigates the DANDI archive like how Dendro does it and lets you select the files you want to jointly view

I agree. Neurosift should have a dandiset view. I'm putting that on my todo list.

[magland]

@CodyCBakerPhD There is now a dandiset page. You can reach that by clicking the dandiset link in the top left part of the nwb page window. It allows browsing of the dandiset assets, just like in dendro. You can select one or more assets and open them in the nwb page.

For example: https://flatironinstitute.github.io/neurosift/?p=/dandiset&dandisetId=000728&dandisetVersion=draft

I think it would be nice if we could ultimately link to this from dandiarchive.

[CodyCBakerPhD]

I think it would be nice if we could ultimately link to this from dandiarchive.

So like a top-level 'Open in neurosift' link from dandiset landing pages? That would be a good request

Otherwise I think the absolute easiest way would be if there's a 'home' button on neurosift itself when viewing assets or a dandiset and that takes you back to a top-level where you can select and browse available dandisets (which have NWB assets)

So I'd also request a top-level dandiset selector for that - again, just like dendro has would be fine

[magland]

Otherwise I think the absolute easiest way would be if there's a 'home' button on neurosift itself when viewing assets or a dandiset and that takes you back to a top-level where you can select and browse available dandisets (which have NWB assets)

@CodyCBakerPhD Thanks for that suggestion!

The home page of neurosift now is a DANDI browser.

[CodyCBakerPhD]

Awesome! Looks and feels great

This can now be closed