M3DB Background Repairs (Within a cluster and across clusters)

[richardartoul]

M3DB currently does not have any form of background repairs. This means that unless a manual repair is triggered (stop a node, delete all the data, then turn it back on using the peers bootstrapper) any gaps that created in a replica factor due to a node being down or any other reason are never repaired.

In addition, the design for asynchronous replication between clusters assumes the ability to run background repairs between two separate clusters.

This design will come up with a plan to address both of these needs (with a priority on cross-clusters) first.

Design Doc