Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove the need for registering an ObjectStore for remote files #899

Open
mesejo opened this issue Oct 5, 2024 · 2 comments
Open

Remove the need for registering an ObjectStore for remote files #899

mesejo opened this issue Oct 5, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@mesejo
Copy link
Contributor

mesejo commented Oct 5, 2024

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Currently, we need to register an ObjectStore to register remote files stored in S3, https, etc. This could be more ergonomic from a DX perspective.

Describe the solution you'd like
Automatically detect if an ObjectStore is needed, like in the datafusion-cli; see here.

Describe alternatives you've considered
keep it as it is now

Additional context
We should increase the test coverage for ObjectStore

@mesejo mesejo added the enhancement New feature or request label Oct 5, 2024
@ion-elgreco
Copy link
Contributor

Just like delta-rs, polars etc we should just take the objectstore from the uri and then parse the storage options either as parameter or env variables

@timsaucer
Copy link
Contributor

Related, though not yet on crates.io: https://github.com/developmentseed/object-store-rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants