Skip to content

Commit

Permalink
add doc for ascii_folding
Browse files Browse the repository at this point in the history
Signed-off-by: BubbleCal <[email protected]>
  • Loading branch information
BubbleCal committed Oct 17, 2024
1 parent 8de6d50 commit 0101fea
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
4 changes: 4 additions & 0 deletions python/python/lance/dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -1358,6 +1358,10 @@ def create_scalar_index(
remove_stop_words: bool, default False
This is for the ``INVERTED`` index. If True, the index will remove
stop words.
ascii_folding: bool, default False
This is for the ``INVERTED`` index. If True, the index will convert
non-ascii characters to ascii characters if possible.
This would remove accents like "é" -> "e".
Examples
--------
Expand Down
1 change: 1 addition & 0 deletions rust/lance-index/src/scalar/inverted/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,7 @@ impl IndexWorker {
Ok(())
}

#[instrument(level = "debug", skip_all)]
async fn flush_posting_list(&mut self, token: String) -> Result<usize> {
if let Some(posting_list) = self.posting_lists.remove(&token) {
let size = posting_list.size();
Expand Down

0 comments on commit 0101fea

Please sign in to comment.