benchmark runner using AWS CLI

[graebm]

This runner skips benchmarks unless it can do them in a single AWS CLI command. If we used multiple commands, one after another, it wouldn't be a fair comparison to the other runners that do everything in parallel. And users probably aren't running multiple CLI commands in parallel, so it doesn't seem worth doing that comparison either.

Here are examples showing how this works:

• Uploading or downloading a single file is simple:
  • benchmark: upload-5GiB
  • cmd: aws s3 cp upload/5GiB/1 s3://graebm-s3-benchmarks/upload/5GiB/1
• A benchmark with multiple files only works if they're in the same directory:
  • benchmark: upload-5GiB-20x
  • cmd: aws s3 cp upload/5GiB s3://graebm-s3-benchmarks/upload/5GiB --recursive
• If the benchmark doesn't use every file in the directory, then we --include the ones we want:
  • benchmark: upload-5GiB-10x
  • cmd: aws s3 cp upload/5GiB s3://graebm-s3-benchmarks/upload/5GiB --recursive --exclude "*" --include 1 --include 2 --include 3 --include 4 --include 5 --include 6 --include 7 --include 8 --include 9 --include 10
• If the benchmark has "filesOnDisk": false then we upload from stdin, or download to stdout. This only works if the benchmark has 1 file.
  • benchmark: upload-5GiB-ram
  • cmd: <5GiB_random_data> | aws s3 cp - s3://graebm-s3-benchmarks/upload/5GiB/1

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.