[FEA]: CUB large input support

[jrhemstad]

As a lower-level interface, CUB should optimize for flexibility and performance. As a result, CUB will not guarantee a large input will work by default. However, it should enable users to specify their desired offset type.

This means CUB should not perform any dynamic dispatch based on the input size. Instead, users should have a way to statically specify the offset type. In previous discussion we favored making the type of num_items a template and infer the offset type from the type of num_items.

Design-related research

Beta Give feedback

1. Identify CUB algorithms that do/do not support large inputs #1408
   +0
   
2. Determine and finalize design for large input support in CUB #1454
   3 of 4
   +0
   

Testing large number of items

Beta Give feedback

1. Add tests for large number of items & large number of segments to DeviceSegmentedRadixSort #2139
   +0
   
2. Add tests for large number of items & large number of segments to DeviceSegmentedSort #2140
   +0
   
3. Add tests for large number of items in DeviceScan::*ByKey

Enable large num_items in CUB algorithms that are sensitive to the choice of offset_t

Beta Give feedback

1. Add support for large num_items to device_scan.cuh #2062
   7 of 7
   +0
   
2. Add support for large num_items to DeviceScan::*ByKey #2458
   +0
   
3. Add support for large num_items to device_select.cuh #1422
   8 of 9
   +0
   
4. Add support for large num_items to device_partition.cuh #1437
   3 of 5
   +0
   
5. Add support for large num_items to DevicePartition::ThreeWayPartition #2442
   +0
   
6. Add support for large num_items to DeviceSegmentedReduce::{ArgMin,ArgMax}
7. Add support for large num_items to DeviceReduce::{ArgMin,ArgMax}
8. Add support for large num_items to DeviceRunLengthEncode::Encode
9. Add support for large num_items to DeviceRunLengthEncode::NonTrivialRuns
10. Add support for large num_items to device_merge.cuh
11. Add support for large num_items to device_transform.cuh
12. Add support for large num_items to device_segmented_radix_sort.cuh
13. Add support for large num_items to device_segmented_sort.cuh
14. Add support for large num_items to device_spmv.cuh

Clean up interim testing infrastructure

Beta Give feedback

1. Switch tests for large number of items to use Device* interface in DeviceSelect
2. Switch tests for large number of items to use Device* interface in DeviceScan

Documentation

Beta Give feedback

1. We want to be explicit about supported and tested offset types in the Dispatch interface (see Port thrust::merge[_by_key] to CUB #1817 (comment))

[elstehle]

legend for offset type: ✅ considered done | 🟡 considered lower priority | 🟠 considered higher priority, as it prevents usage for larger-than-INT_MAX number of items | ⏳ in progress

legend for testing columns: ✅ considered done | 🟡 to be done | 🟠 needs to support wider offset types first

algorithm	offset type	tests larger-thanINT_MAX	tests close to [U]INT_MAX
device_adjacent_difference.cuh	✅ choose_offset_t	✅ 2^33, sanity check, iterators	🟡
device_copy.cuh	🟡num_ranges: uint32_t 🟡buffer sizes: iterator_traits<SizeIteratorT>::value_type	🟡	🟡
device_for.cuh	🟡NumItemsT: ForEachN, ForEachCopyN, Bulk difference_type: ForEach, ForEachCopy	🟡	🟡
device_histogram.cuh	🟡 dynamic dispatch: int for (num_rows * row_stride_bytes)<INT_MAX; OffsetT otherwise	🟡	🟡
device_memcpy.cuh	🟡 num_ranges: uint32_t 🟡 buffer sizes: iterator_traits<SizeIteratorT>::value_type	🟡	🟡
device_merge_sort.cuh	🟡 NumItemsT	✅ extensive check	✅ extensive check
device_partition.cuh	⏳ int: Flagged, If 🟠 int: ThreeWayPartition	🟠	🟠
device_radix_sort.cuh	✅ choose_offset_t	✅ extensive check	✅ extensive check
device_reduce.cuh	✅ choose_offset_t: Reduce, Sum, Min, Max, ReduceByKey, TransformReduce ⚠️ (note) int: ArgMin, ArgMax	✅ sanity, 2^{30,31,33)	✅ sanity, 2^32-1
device_run_length_encode.cuh	🟠 int	🟠	🟠
device_scan.cuh	✅ choose_offset_t: DeviceScan ⏳ choose_offset_t: DeviceScanByKey	🟠	🟠
device_segmented_radix_sort.cuh	🟠 num_items & num_segments:int	🟠	🟠
device_segmented_reduce.cuh	🟡 common_iterator_value_t({Begin,End}OffsetIteratorT): Reduce, Sum, Min, Max ⚠️ (note) int: ArgMin, ArgMax num_segments: int	✅ sanity, rnd [2^31; 2^33]	🟡
device_segmented_sort.cuh	🟠 num_items & num_segments:int	🟠	🟠
device_select.cuh	✅ choose_offset_t: UniqueByKey ⏳ int: Flagged, If, Unique
device_spmv.cuh	🟠 int	🟠	🟠
device_merge.cuh	🟠 int	🟡	🟡
device_transform.cuh	🟠 int	🟡	🟡