Skip to content

v0.8.15 scalar index, f16 vector index, and various fixes

Compare
Choose a tag to compare
@chebbyChefNEQ chebbyChefNEQ released this 16 Nov 04:47
· 912 commits to main since this release

What's Changed

  • chore: fix type inference errors in benchmarks by @westonpace in #1556
  • chore: provide a trait to dynamically dispatch different pq based on different vector data type by @eddyxu in #1555
  • chore: update the CI build to check/build all crates in the workspace and not just the lance crate by @westonpace in #1557
  • feat: make it possible to create and load scalar indices for a dataset by @westonpace in #1516
  • feat: generic Product Quantizatoin by @eddyxu in #1560
  • test: add property-based testing for statistics by @wjones127 in #1554
  • feat: ffi to accelerate norm_l2 for f16 if the instruction set is available by @eddyxu in #1562
  • feat: extend FSL with sample by @eddyxu in #1572
  • feat: allow for more advanced storage options in objectstore by @universalmind303 in #1547
  • feat: implement as_slice for bfloat16 array by @eddyxu in #1574
  • perf: add bf16 benchmarks by @eddyxu in #1575
  • feat: f16 for L2 by @eddyxu in #1577
  • chore: update cc dependency to 1.0.83 by @westonpace in #1578
  • feat: make IVF model support f16 and bf16 by @eddyxu in #1573
  • feat: allow the scanner to take advantage of scalar indices by @westonpace in #1543
  • chore: dotprod should be on mac target, not haswell and better randomness for bf16 by @westonpace in #1579
  • feat: make Dataset::nearest() accepts arbitrary query type by @eddyxu in #1582
  • chore(rust): remove extraneous dbg message by @changhiskhan in #1598
  • feat: torch cache-able dataset, with sampling support by @eddyxu in #1591
  • fix: tell writer correct schema when writing index file by @wjones127 in #1518
  • feat: add support for remapping scalar indices during compaction by @westonpace in #1571
  • refactor: switch to using DataFusions physical expr by @wjones127 in #1581
  • chore: various fixes for Python benchmarks by @wjones127 in #1513
  • feat: adaptive cuda allocation for l2/cosine distance computation by @eddyxu in #1601
  • fix: fix a memory leak where a dataset would not be fully deleted by @westonpace in #1606
  • fix: google objectstore uses proper gs configuration by @universalmind303 in #1608

Full Changelog: v0.8.14...v0.8.15