Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RELEASE] cugraph v24.10 #4700

Open
wants to merge 72 commits into
base: main
Choose a base branch
from
Open

[RELEASE] cugraph v24.10 #4700

wants to merge 72 commits into from

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.10 and v24.10 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.10 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.10 into main for the release

raydouglass and others added 30 commits July 19, 2024 17:27
@raydouglass
DOC v24.10 Updates [skip ci]
7926b02
@jameslamb
Merge branch-24.08 into branch-24.10
988b0cd
@AyodeAwe
Merge pull request #4565 from jameslamb/branch-24.10-merge-24.08
f592bdb 
Merge branch-24.08 into branch-24.10
@AyodeAwe
Merge pull request #4546 from rapidsai/branch-24.08
e9709f8 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4566 from rapidsai/branch-24.08
d80a8b6 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4569 from rapidsai/branch-24.08
b915a34 
Forward-merge branch-24.08 into branch-24.10
@bdice
Fix ucx-py version, use UCX 1.17.0 in pip devcontainers (#4562)
4dd8fca 
Fixes ucx-py dependency in `dependencies.yaml` and `update-version.sh`.

Updates to UCX 1.17.0 in pip devcontainers. (context: #4562 (comment)).

Authors:
  - Bradley Dice (https://github.com/bdice)
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #4562
@GPUtester
Merge pull request #4573 from rapidsai/branch-24.08
2a79a83 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4574 from rapidsai/branch-24.08
4a47cae 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4575 from rapidsai/branch-24.08
940e34d 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4576 from rapidsai/branch-24.08
cf8deb0 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4577 from rapidsai/branch-24.08
0440d9d 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4578 from rapidsai/branch-24.08
a7f28e4 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4580 from rapidsai/branch-24.08
5fbc0fc 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4581 from rapidsai/branch-24.08
a20d02b 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4584 from rapidsai/branch-24.08
b64bc3e 
Forward-merge branch-24.08 into branch-24.10
@nv-rliu
Re-configure benchmarking devices & add markers to `bench_cugraph_uni…
1be81a4 
…form_neighbor_sample` (#4561)

This PR re-enables the `benchmarks/cugraph/pytest-based/bench_cugraph_uniform_neighbor_sample.py` benchmark in the MNMG nightlies. 

This benchmark was previously being skipped due to the fact that the benchmarks are configured to run with Pytest markers `"managedmem_off and poolallocator_on"`. Thanks to @jameslamb for spotting this inside `conftest.py`

By adding the missing markers to the test (and removing some `dgx` machine specific dask-client configs), the benchmark should properly run in the nightly jobs.

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - James Lamb (https://github.com/jameslamb)

URL: #4561
@bdice
Use stream_allocator_adaptor constructor instead of factory. (#4557)
8f86c82 
This PR uses the `stream_allocator_adaptor` constructor instead of a factory. With CTAD, we do not need the factory and can use the constructor directly. The factory will be deprecated in rapidsai/rmm#1626.

Authors:
  - Bradley Dice (https://github.com/bdice)
  - Mark Harris (https://github.com/harrism)

Approvers:
  - James Lamb (https://github.com/jameslamb)
  - Seunghwa Kang (https://github.com/seunghwak)

URL: #4557
@GPUtester
Merge pull request #4598 from rapidsai/branch-24.08
19fd1d6 
Forward-merge branch-24.08 into branch-24.10
@GPUtester
Merge pull request #4604 from rapidsai/branch-24.08
c2362c2 
Forward-merge branch-24.08 into branch-24.10
@nv-rliu
Fix test_property_graph_mg Usage of Util Function (#4600)
d92c257 
This PR fixes an issue that recently arose due to a change in constructing a `cudf.Series` object.

`test_property_graph_mg.py` uses a function from `utilities/utils.py` to construct a Series object from 2d arrays. The function is now using the correct `cudf.Series._from_column` API.

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4600
@jameslamb
Use tool.scikit-build.cmake.version, set scikit-build-core minimum-ve…
f935e76 
…rsion (#4597)

Contributes to rapidsai/build-planning#58.

`scikit-build-core==0.10.0` was released today (https://github.com/scikit-build/scikit-build-core/releases/tag/v0.10.0), and wheel-building configurations across RAPIDS are incompatible with it.

This proposes upgrading to that version and fixing configuration here in a way that:

* is compatible with that new `scikit-build-core` version
* takes advantage of the forward-compatibility mechanism (`minimum-version`) that `scikit-build-core` provides, to reduce the risk of needing to do this again in the future

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Mike Sarahan (https://github.com/msarahan)

URL: #4597
@KyleFromNVIDIA
Update pre-commit hooks (#4605)
ce5d3dd 
This PR updates pre-commit hooks to the latest versions that are supported without causing style check errors.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #4605
@seunghwak
Biased sampling primitive bug fix (#4607)
a5cdea2 
Fix bugs in biased sampling with 0 bias values.

Add tests that include 0 bias edges.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4607
@alexbarghi-nv
Add an Explanatory Error Message for uint Types (#4556)
1ef3f56 
Clarify that unsigned integer types are not supported as vertex ids in the error message.  Resolves issue reported by @ogreen.

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4556
@nv-rliu
Updates to cugraph.hypergraph (Duplicate Col Labels Bug) (#4610)
a538745 
cc: @rlratzel @ChuckHastings 

This PR addresses failures seen in certain PRs (like [here](https://github.com/rapidsai/cugraph/actions/runs/10372270389/job/28718471674?pr=4606#step:7:5269)) due to a [recent change](rapidsai/cudf#16514) to `cudf` that disallows selecting duplicate column labels.

---

In `hypergraph.py`, this PR modifies `_create_hyper_edges` and `_create_direct_edges` to ensure that DataFrames are being indexed by non-duplicate column values.

This is done by taking a list that includes duplicates (`fs`), and removing the non-unique values
```python
fs = list(set(fs))
```


_This part requires some attention from the author of the unit test @jnke2016_

In `test_hypergraph.py`, this PR adds the `check_like=True` arg to `assert_frame_equals` function because the ordering of the columns is different for the two DFs.

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Paul Taylor (https://github.com/trxcllnt)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4610
@bdice
Improve update-version.sh (#4599)
3762e6f 
This PR improves `update-version.sh` by clarifying suffix handling and ucxx/ucx-py version handling.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #4599
@KyleFromNVIDIA
Set CUDA_STATIC_MATH_LIBRARIES in Python builds (#4612)
de81819 
In #4526, we introduced the CUDA_STATIC_MATH_LIBRARIES option and set it to OFF by default. We did not set it to ON in the Python builds, and as a result ended up bundling dynamic CUDA libraries. Set the option to ON in the Python build.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #4612
@nv-rliu
Add Additional Check for NetworkX Release Candidate Versions (#4613)
a6a84eb 
Part of rapidsai/graph_dl#579

This PR allows `nx-cugraph` to operate with release candidate (under development) versions of `networkx`

What This Pattern Allows:
- Just a single digit: `3.4`, `3.9`, etc

- Release Candidate format: `3.4rc0`. `3.7rc2`

This is needed to complete adding nightly test coverage for development branches of `nx`

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Erik Welch (https://github.com/eriknw)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4613
@alexbarghi-nv
[FEA] Biased Sampling in cuGraph-PyG (#4586)
4c26e7a 
Enables biased sampling in PylibcuGraph and cuGraph-PyG.  Also fixes a memory issue that was revealed by calling biased sampling instead of uniform neighbor sampling; the loader was pre-allocating more memory than needed.

- [x] Update pylibcugraph
- [x] Update dist sampler in cugraph
- [x] Correctness testing
- [x] Make issue for PyG upstream ([here](pyg-team/pytorch_geometric#9566))

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4586
KyleFromNVIDIA and others added 14 commits September 25, 2024 15:09
@KyleFromNVIDIA
Install mg test executables (#4656)
87987af 
`rapids_test_add()` does not install the executables we need, because it's checking the value of `${MPIEXEC_EXECUTABLE}` and seeing that it's not a target. Install the target separately.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - Bradley Dice (https://github.com/bdice)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4656
@nv-rliu
Fix cit-patents Dataset for nx-cugraph Benchmark (#4666)
1e5b328 
Replace `_` with `-`

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4666
@alexbarghi-nv
[FEA] Biased Sampling in cuGraph-DGL (#4595)
36c190a 
Adds support for biased sampling to cuGraph-DGL.

Resolves rapidsai/cugraph-gnn#25
Merge after #4583, #4586, #4607

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - Ralph Liu (https://github.com/nv-rliu)
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - Tingyu Wang (https://github.com/tingyu66)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4595
@jameslamb
reduce pip verbosity in wheel builds (#4651)
54b03da 
Proposes reducing the verbosity of `pip wheel` called in wheel builds from `-vvv` to `-v`.

This eliminates the 1000s of lines like this in CI logs:

![image](https://github.com/user-attachments/assets/4dc2b024-c9f1-4618-9c94-8166c6aa9a13)

to hopefully make it easier to view those logs interactively

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Vyas Ramasubramani (https://github.com/vyasr)

URL: #4651
@seberg
Remove NumPy <2 pin (#4615)
0f4fe8f 
This PR removes the NumPy<2 pin which is expected to work for
RAPIDS projects once CuPy 13.3.0 is released (CuPy 13.2.0 had
some issues preventing the use with NumPy 2).

Authors:
  - Sebastian Berg (https://github.com/seberg)
  - https://github.com/jakirkham
  - Rick Ratzel (https://github.com/rlratzel)
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - James Lamb (https://github.com/jameslamb)
  - Philip Hyunsu Cho (https://github.com/hcho3)

Approvers:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - James Lamb (https://github.com/jameslamb)

URL: #4615
@alexbarghi-nv
Support Negative Sampling in pylibcugraph and cuGraph-PyG (#4660)
4dfd3b3 
Adds support for negative sampling in `pylibcugraph` and `cugraph-pyg`.

Closes rapidsai/cugraph-gnn#39
Merge after #4641

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4660
@galipremsagar
Swtch traceback to --native in cugraph (#4663)
9b107b9 
In cudf we have observed a ~10% speed up of pytest suite execution by switching pytest traceback to `--native`:

```
currently:

102474 passed, 2117 skipped, 902 xfailed in 892.16s (0:14:52)

--tb=short:

102474 passed, 2117 skipped, 902 xfailed in 898.99s (0:14:58)

--tb=no:

102474 passed, 2117 skipped, 902 xfailed in 815.98s (0:13:35)

--tb=native:

102474 passed, 2117 skipped, 902 xfailed in 820.92s (0:13:40)
```

This PR makes similar change to `cugraph` repo.

xref: rapidsai/cudf#16851

Authors:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)

URL: #4663
@nv-rliu @acostadon
@rlratzel @eriknw
Add nx-cugraph Docs Pages (#4669)
a936327 
Closes rapidsai/graph_dl#606 and [another
issue]

## Proposed Changes
In preparation for GA release, this PR adds a landing page for
`nx-cugraph` in the cugraph API documentation site. The new pages can be
viewed by clicking `nx-cugraph` in the navigation bar at the top of the
page.

### New pages
nx-cugraph
 └─ How it works
 └─ Supported Algorithms
 └─ Getting Started
 └─ Benchmarks
 └─ FAQ

## Notes for Reviewers

- In order to build and test these docs, I modified the `build.sh` file
to use `sphinx-autobuild`.

```bash
122
123     cd ${REPODIR}/docs/cugraph-docs
124     #make html
125     sphinx-autobuild source build/html
126 fi
127
```

- For now, I believe the best way to view these changes is to clone the
PR branch, then run `build.sh` in order to host the webserver locally..

---------

Co-authored-by: Don Acosta <[email protected]>
Co-authored-by: rlratzel <[email protected]>
Co-authored-by: Erik Welch <[email protected]>
@nv-rliu @eriknw
Add nx-cugraph introduction notebook to repo (#4677)
f8b9ac3 
## Proposed Changes

This PR adds an introduction notebook to the `notebooks/demo` directory
of the repository. Click the link to view the files directory in the dev
branch.
* [The introduction to `nx-cugraph`
notebook](https://github.com/rapidsai/cugraph/blob/1a70b7ad588bcc1dc6f65af8fe07149ed8083ef0/notebooks/demo/accelerating_networkx.ipynb)

---------

Co-authored-by: Erik Welch <[email protected]>
@acostadon @rlratzel
Implementing some of the VDR feedback (#4674)
e6be367 
Adding some content and navigation options per VDR,
resolves  rapidsai/graph_dl#594
resolves rapidsai/graph_dl#593

---------

Co-authored-by: rlratzel <[email protected]>
@jnke2016
Refactor the python function symmetrizing the edgelist (#4649)
5fad435 
This PR updates the python API to symmetrize the edge list through the
CAPI for PLC algorithms. This PR also deprecates the legacy python
function symmetrizing the edge list

closes #4588
@alexbarghi-nv
Constrain versions of PyTorch and CI artifacts in CI Runs, upgrade to…
3789b70 
… dgl 2.4 (#4690)

We were pulling the wrong packages because the PyTorch version constraint wasn't tight enough.  Hopefully these sorts of issues will be resolved in the `cugraph-gnn` repository going forward, where we can pin a specific pytorch version for testing.

Authors:
  - Alex Barghi (https://github.com/alexbarghi-nv)
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - https://github.com/jakirkham
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4690
@nv-rliu @acostadon
@rlratzel @jameslamb
Updates docs to describe nx-cugraph based on latest updates for 24.10 (…
7733017 
…#4694)

## Proposed Changes

This PR contains critical fixes to the cugraph docs pages.

---------

Co-authored-by: acostadon <[email protected]>
Co-authored-by: rlratzel <[email protected]>
Co-authored-by: James Lamb <[email protected]>
@eriknw
nx-cugraph: add NX_CUGRAPH_AUTOCONFIG=True env var to enable full…
acf5029 
… zero-code change (#4685)

This is for convenience and sets or updates `NETWORKX` environment variables.

Do we like `NX_CUGRAPH` as the env var name? What should we consider a true value: "True" (case-insensitive) or a non-empty value?

This works with the latest dev version of NetworkX. I have not yet tried it with older NetworkX versions.

Authors:
  - Erik Welch (https://github.com/eriknw)
  - https://github.com/jakirkham

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Ralph Liu (https://github.com/nv-rliu)

URL: #4685
@raydouglass raydouglass requested review from a team as code owners October 7, 2024 15:10
@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

alexbarghi-nv
alexbarghi-nv approved these changes Oct 7, 2024
View reviewed changes
Copy link
Member

@alexbarghi-nv alexbarghi-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexbarghi-nv alexbarghi-nv alexbarghi-nv approved these changes

@msarahan msarahan Awaiting requested review from msarahan msarahan is a code owner automatically assigned from rapidsai/packaging-codeowners

At least 3 approving reviews are required to merge this pull request.

Assignees
No one assigned
Labels
benchmarks ci CMake conda cuGraph datasets python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.