-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
CXXCBC-242: SDK Support for Native KV Range Scans (#419)
* Added `prefix_scan` as top-level scan type * Changed scan terms from byte vectors to strings * Seed for sampling scan is randomly generated if not specified * Removed `range_scan_cancelled` error code, returning `request_canceled` instead * Removed sorting * Added `concurrency` option to orchestrator to set the maximum allowed level of concurrency * The constructor of `range_scan_orchestrator` takes the vbucket map instead of the number of vbuckets * Implemented new concurrency approach for scanning vbuckets: * Streams start with the maximum level of concurrency. If a temporary failure occurs (i.e. receiving busy status from the server) the stream is retried a later point and the concurrency is reduced by 1 * When a stream finishes successfully or with a benign error, another stream is initiated to take its place. If the temporary failure occurs, a new stream is not initiated which effectively reduces the concurrency (unless streams are not being executed concurrently in which case that continues) * The number of streams being run per node are kept track of and when a new stream should start, a vbucket on the least busy node is selected (initially random node is selected) * Added `key_value_scan_timeout` to timeout defaults (7.5 seconds) * Removed `batch_time_limit` from options, 90% of the timeout is used instead * Timeouts are now on both range_scan_create and range_scan_continue. There's also a check before retrying a stream if the time since the first attempt exceeds the timeout * The `next` methods of the scan result can return an error code in the case of a fatal error * Added `cancel()` method to `scan_result` that can cancel all streams * Errors on range scan continue or start are now separated into 'fatal' (which takes into account whether the scan is a sampling scan), 'retryable' or 'benign' --------- Co-authored-by: Sergey Avseyev <[email protected]>
- Loading branch information
1 parent
0227652
commit 9394a00
Showing
14 changed files
with
667 additions
and
334 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.