Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

May want regular file integrity checks during a backfill for the s3 client. #312

Open
mpawliszyn opened this issue May 12, 2023 · 2 comments

Comments

@mpawliszyn
Copy link
Collaborator

There are several assumptions around the files not changing out from under the s3 client. We might want to check this for a better customer experience.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity.html

We probably need some kind of side channel data for this. The prepare can pull this information perhaps?

@shellderp
Copy link
Collaborator

You could get s3 modified timestamp, return it as a parameter from Prepare, then compare the current value on each operation, failing if it has changed. Prepare is allowed to create new parameters dynamically.

@mpawliszyn
Copy link
Collaborator Author

Yeah that is a good idea both on the timestamp and on the parameter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants