Skip to content

Commit

Permalink
performance: Optimize ColBERT index free search with torch.topk (#219)
Browse files Browse the repository at this point in the history
* performance: Optimize ColBERT index free search with torch.topk

* Remove typo
  • Loading branch information
Diegi97 committed Aug 7, 2024
1 parent 796b493 commit de3c620
Showing 1 changed file with 2 additions and 3 deletions.
5 changes: 2 additions & 3 deletions ragatouille/models/colbert.py
Original file line number Diff line number Diff line change
Expand Up @@ -477,9 +477,8 @@ def _index_free_search(
for query in embedded_queries:
results_for_query = []
scores = self._colbert_score(query, embedded_docs, doc_mask)
sorted_scores = sorted(enumerate(scores), key=lambda x: x[1], reverse=True)
high_score_idxes = [index for index, _ in sorted_scores[:k]]
for rank, doc_idx in enumerate(high_score_idxes):
sorted_scores = torch.topk(scores, k)
for rank, doc_idx in enumerate(sorted_scores.indices.tolist()):
result = {
"content": documents[doc_idx],
"score": float(scores[doc_idx]),
Expand Down

0 comments on commit de3c620

Please sign in to comment.