[Feature]: Adding stimulus timeseries to NWB file from spikeGLX conversion

[zhiwen10]

What would you like to see added to the NWB GUIDE?

What's the best way to add stimulus or other timeseries to the NWB file generated from NWB GUIDE?
I see a solution here. Is this what you would recommend?

Do you have any interest in helping implement the feature?

No.

Code of Conduct

Yes

Did you confirm this feature was not already reported?

Yes

[CodyCBakerPhD]

Unfortunately, this is not supported yet and is planned for a patch following the official release this summer

What you can do however (and would probably have to do anyway since most behavior streams don't use common proprietary formats) is use NWB GUIDE to convert proprietary formats, then append the same file after that to include additional data streams

We recommend checking out NeuroConv to see if that framework can help in any way (it often helps to break up data streams by DataInterface)

[rly]

@zhiwen10 To append stimulus timeseries to a file created by GUIDE, you can use PyNWB (or MatNWB). You can browse the PyNWB tutorials here: https://pynwb.readthedocs.io/en/stable/tutorials/index.html.

This one is probably useful if you are working with image stimuli:
https://pynwb.readthedocs.io/en/stable/tutorials/domain/images.html#sphx-glr-tutorials-domain-images-py

This one is useful for adding data to an NWB file in general:
https://pynwb.readthedocs.io/en/stable/tutorials/general/add_remove_containers.html#sphx-glr-tutorials-general-add-remove-containers-py

The differences between writing a file from scratch and appending are that:

1. when you create an NWBHDF5IO object, pass mode="r+" or mode="a" instead of mode="w"
2. then call nwbfile = io.read()
3. add your stimulus timeseries to nwbfile
4. then call io.write(nwbfile)

Instead of step 4, you could also export your in-memory NWB file to a new file - this would copy the dataset from the original file into a new file that also contains your stimulus timeseries. See the second tutorial listed above for more details on that.

Feel free to ask any questions about this process on the PyNWB repo: https://github.com/NeurodataWithoutBorders/pynwb

[CodyCBakerPhD]

We'd also be curious what your source data for these data streams looks like to help inspire some generic interfaces to try to meet your use case

[zhiwen10]

Thank you for the links. Very helpful.

I have 2 types of external data to be added to the NWB file:
(1) drift motion signal: [timestamps, analog data]
(2) visual (image) stimulus index series data: [timestamps, duration, image index]. A series of 118 images were represented, at random order with repetition.

I'm adding the drift motion signal in the acquisition field as timeseries data. I know ideally I should add visual image stimulus as indexseries data, like suggested in the tutorial above. But I couldn't work out a easy way to add the 118 images to the template. So I am planning to add the data as timeseries data instead [timestamps, duration, image index], and have the 118 images supplied to users separately.

Attached are code with pyNWB and source data. Any suggestions are welcome. Thanks.

DANDI.zip
Add_timeseries_to_NWB_file.txt

[zhiwen10]

Solved Indexseries problem here.
NeurodataWithoutBorders/pynwb#1884.