Enable conda-build to produce .conda packages

[jakirkham]

Fixes rapidsai/build-planning#98

Switch from .tar.bz2 packages to .conda packages. This will allow the metadata of packages to be read without decompression. Also .conda packages allow changing the compression used under-the-hood (currently this is Zstd based).

[jakirkham]

Nearly all jobs passed. One failed due to a network error (snippet below)

Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): ...working... failed

CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/conda-forge/noarch/repodata.json>
Elapsed: -

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https//conda.anaconda.org/conda-forge/noarch'

Restarted the failed job

[jameslamb]

I support this!

But note there will need to also be some changes to CI tools / scripts which are currently listing files via wildcards like *.tar.bz2.

search: https://github.com/search?q=org%3Arapidsai+%22tar.bz2%22&type=code&p=2

examples:

• https://github.com/rapidsai/gha-tools/blob/2b3d27e693e3bb96802023a7de4ab477597b1135/tools/rapids-find-anaconda-uploads.py#L67
• https://github.com/rapidsai/dependency-file-generator/blob/4e32ded33b3df70e88cff06bac59488fca74ae7f/ci/publish/conda.sh#L12
• https://github.com/rapidsai/cugraph-pg/blob/d0d101a53c89984841199c3555bde6ee5d8658be/ci/build_docs.sh#L40

[jakirkham]

Adding DO NOT MERGE label while we work through those cases James identified

There are a list of xref'd PRs below James' other comment: rapidsai/build-planning#98 (comment)

Perhaps folks can review these as they are able

[jameslamb]

Perhaps folks can review these as they are able

Thanks! I just reviewed all of those and did one more GitHub search (link). I think you got all the cases, so once those open PRs are merged (at least the ones in the active repos), I think this PR here in ci-imgs could be merged.

[jakirkham]

Thanks James! 🙏

Appreciate you going through those so quickly. Please let me know if I've missed any feedback that needs addressing

Am noticing the label checker is not enabled here. So the DO NOT MERGE label, doesn't fail CI

To avoid accidental merges before those other items are addressed, will mark this as draft. We can mark it ready for review after

[jakirkham]

Think we have cleared all the requirements for this change

Would propose that we add this after 24.10 ships. So leaving as draft for now

Though think we should do this at the start of 24.12 so we have more time to get comfortable with this change