Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance ability to reach into a slice without building full data set #197

Open
dylanmcreynolds opened this issue Nov 18, 2020 · 0 comments

Comments

@dylanmcreynolds
Copy link
Contributor

dylanmcreynolds commented Nov 18, 2020

Attempting to view a thumbnail in a web app through databroker. I have a handler that reaches into an h5 file: handler

If I want a frame slice, and I want it lazy, I have to call something like:
frame = run.stream.to_dask()['field_name']

This is nice, however...it ends up calling my handler once for every frame (in my docstream I have one event per frame). While this returns lazy dask arrays. However, there is a significant cost to calling that hanlder once for every frame. This cost isn't a huge issue for a desktop application where a couple seconds of warming up the dataset is a small price to pay for being able to randomly scroll through frames lazily. However, building a stateless web app, this unacceptable. It would be nice to have a metho that lets me dig right in to a small subset of slices to return it to the client quickly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant