[Bug]: Display DANDI in combination with Google drive creates 100 GB file (+resulting sync issue)

[Diirkk]

What happened?

Use 'Panel' to display repository on DANDI (all run on local computer). The jupyter notebook is saved on a Google drive (Windows extension). Opening a repository from dandi pre-allocates ~100 GB on the disc, which Google Drive attempts to sync.

Steps to Reproduce

Run Panel from jupyter notebook file that is stored on Google Drive windows extension and load arbitrary dandi file.

Traceback

No response

Operating System

Windows

Python Version

3.9

Package Versions

No response

Code of Conduct

• I agree to follow this project's Code of Conduct
• Have you ensured this bug was not already reported?

[CodyCBakerPhD]

What we need to do here, at the very least on the Panel side, is allow user selection of the cache folder so they can place it somewhere outside the directory that the notebook is stored

Beyond that though, I'll do some experimentation on my side to see if I can methodically reproduce this problem, which could reveal an interesting behavior I was not previously aware of when locally streaming very large files

If it works as we observed, then we would want to add a caution note to the streaming tutorial as well raise issues upstream to try to determine if its something fsspec is able to resolve on its side, or perhaps an issue with Drive specifically

[luiztauffer]

I agree that the simplest solution would be to add an argument on Panel for the user to choose the cache folder

[bendichter]

You could also just disable the cache

[luiztauffer]

@Diirkk cache is now optional, you can check/uncheck it within the Panel interface
could you please try it and let us know if we can close this Issue? Thanks!