Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fails when filesize is too large #3

Open
ButlerJoseph opened this issue Nov 13, 2019 · 1 comment
Open

Fails when filesize is too large #3

ButlerJoseph opened this issue Nov 13, 2019 · 1 comment

Comments

@ButlerJoseph
Copy link

ButlerJoseph commented Nov 13, 2019 •
edited
Loading

I am using the Cassandra Cain tool (https://github.com/maorfr/cain) to backup a Cassandra datacenter and it is failing when backing up files which are 500 GB in size.

Having looked at the code for this project, I've noticed that this project is using the default UploadPartSize of 5 megabytes (https://github.com/aws/aws-sdk-go/blob/fe72a52350a8962175bb71c531ec9724ce48abd8/service/s3/s3manager/upload.go#L26), this gives us the ability to files up to 52 GB in size.

Please can you consider increasing this value to 64 bytes so SKBN is able to upload files up to 671 GB in size?
@rrusmana
Copy link

Hi @ButlerJoseph
When you say it is failing - did the process fail with an error message or just hangs? What error did you see?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ButlerJoseph @rrusmana