Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scaling workspace resources #181

Merged
merged 6 commits into from
Jun 12, 2024
Merged

Scaling workspace resources #181

merged 6 commits into from
Jun 12, 2024

Conversation

achirkin
Copy link
Contributor

@achirkin achirkin commented Jun 6, 2024

Use raft's large workspace resource for large temporary allocations during ANN index build.
This is the port of rapidsai/raft#2194, which didn't make into raft before the algorithms were ported to cuVS.

@achirkin achirkin requested a review from a team as a code owner June 6, 2024 19:14
@github-actions github-actions bot added the cpp label Jun 6, 2024
@achirkin achirkin mentioned this pull request Jun 6, 2024
Merged
10 tasks
@cjnolet cjnolet added improvement Improves an existing functionality non-breaking Introduces a non-breaking change labels Jun 7, 2024
tfeher
tfeher requested changes Jun 10, 2024
View reviewed changes
Copy link
Contributor

@tfeher tfeher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Artem for the PR! It is great to fix potential OOM errors with this PR. Overall this looks good, I have added a few comments.

cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/detail/cagra/utils.hpp Show resolved Hide resolved
cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/detail/cagra/cagra_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/detail/cagra/cagra_build.cuh Outdated Show resolved Hide resolved
@achirkin achirkin requested a review from tfeher June 10, 2024 15:12
tfeher
tfeher requested changes Jun 10, 2024
View reviewed changes
cpp/src/neighbors/detail/cagra/cagra_build.cuh Outdated Show resolved Hide resolved
cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Show resolved Hide resolved
cpp/src/neighbors/ivf_pq/ivf_pq_build.cuh Outdated Show resolved Hide resolved
@achirkin achirkin requested a review from tfeher June 11, 2024 08:47
tfeher
tfeher approved these changes Jun 11, 2024
View reviewed changes
Copy link
Contributor

@tfeher tfeher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Artem for the update, LGTM.

@benfred
Copy link
Member

benfred commented Jun 12, 2024

/merge

@rapids-bot rapids-bot bot merged commit 5668ef0 into rapidsai:branch-24.08 Jun 12, 2024
54 checks passed
@achirkin achirkin mentioned this pull request Jun 13, 2024
Closed
difyrrwrzd added a commit to difyrrwrzd/cuvs that referenced this pull request Aug 10, 2024
@difyrrwrzd
Scaling workspace resources (#181)
6c42aee 
Use raft's large workspace resource for large temporary allocations during ANN index build.
This is the port of rapidsai/raft#2194, which didn't make into raft before the algorithms were ported to cuVS.

Authors:
  - Artem M. Chirkin (https://github.com/achirkin)

Approvers:
  - Tamas Bela Feher (https://github.com/tfeher)

URL: rapidsai/cuvs#181
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tfeher tfeher tfeher approved these changes

Assignees

@achirkin achirkin

Labels
cpp improvement Improves an existing functionality non-breaking Introduces a non-breaking change
Projects
VS/ML/DM Primitives Release Board
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@achirkin @benfred @tfeher @cjnolet