Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Atomics backend refactor #1631

Merged
merged 74 commits into from
May 10, 2024
Merged

Atomics backend refactor #1631

merged 74 commits into from
May 10, 2024

Commits on Apr 18, 2024

  1. Delete <cuda/std/atomic> header

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    32a0227 View commit details
    Browse the repository at this point in the history

  2. Move atomic from libcxx to top-level

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    4be887d View commit details
    Browse the repository at this point in the history

  3. Move PTX backends from libcxx to <cuda/std/__atomic/...>

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    b36fec6 View commit details
    Browse the repository at this point in the history

  4. Delete remaining atomics backends. Move MSVC backend

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    52a60bb View commit details
    Browse the repository at this point in the history

  5. First pass at making atomic use new backends

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    76294d1 View commit details
    Browse the repository at this point in the history

  6. Change atomic_storage operator()() to get()

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    a780c26 View commit details
    Browse the repository at this point in the history

  7. Fixup: Change desired of compexch to accept by value. 

    * This matches other implementations.
    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    217527d View commit details
    Browse the repository at this point in the history

  8. Fix merge conflicts (LIBCUDACXX->CCCL)

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    eaaa670 View commit details
    Browse the repository at this point in the history

  9. Fix another merge conflict (LIBCUDACXX->CCCL)

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    452fc3b View commit details
    Browse the repository at this point in the history

  10. Simplify tag dispatch in the atomic backend

    @wmaxey
    wmaxey committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    91f8b11 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Make tests work when full path is specified to lit.

    @wmaxey
    wmaxey committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0e6c0c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. Update barrier, latch, and semaphore, to use new atomic_impl.

    @wmaxey
    wmaxey committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    a370a02 View commit details
    Browse the repository at this point in the history

  2. Make changes to atomic work.

    @wmaxey
    wmaxey committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    bf801ce View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. Rearrange headers and update latch/barrier.

    @wmaxey
    wmaxey committed May 2, 2024
    Configuration menu
    Copy the full SHA
    64b31af View commit details
    Browse the repository at this point in the history

  2. Update codegen to reflect new header layout.

    @wmaxey
    wmaxey committed May 2, 2024
    Configuration menu
    Copy the full SHA
    a625d3f View commit details
    Browse the repository at this point in the history

  3. Make platform.h define LIBCUDACXX_ATOMIC_BLAH_LOCK_FREE. 

     * We previously defined or *clobbered* the existing STL definitions.
 * See: `ATOMIC_BOOL_LOCK_FREE`->`LIBCUDACXX_ATOMIC_BOOL_LOCK_FREE`
    @wmaxey
    wmaxey committed May 2, 2024
    Configuration menu
    Copy the full SHA
    4a4782b View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Fix missing <cstdint> in generated ptx file.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    fc97437 View commit details
    Browse the repository at this point in the history

  2. __cuda_std__ mode does not require use of host atomics checks.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    f91a7c1 View commit details
    Browse the repository at this point in the history

  3. Fix missing _If in types.h.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    8fb4c6e View commit details
    Browse the repository at this point in the history

  4. Fix missing <cstdint> in derived PTX file.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    c4546f5 View commit details
    Browse the repository at this point in the history

  5. Remove uneeded headers from base.h.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    077e3d3 View commit details
    Browse the repository at this point in the history

  6. Fix type mixup in __atomic_wait.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    b38a43a View commit details
    Browse the repository at this point in the history

  7. Change heterogeneous tests to permutate over H/D launcher combinations. 

    * However this is restricted to *one* device launch per suite to prevent deadlocks.
* Tests are much slower, but extremely thorough.
* Concurrent H/D coverage is particularly exemplified in latch.pass.cpp.
    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    6e4e947 View commit details
    Browse the repository at this point in the history

  8. Change tests to use validate_pinned API.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    936daaf View commit details
    Browse the repository at this point in the history

  9. Fix a couple issues with __atomic_underlying_t in __atomics/types

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    6a76eb5 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into fea/atomic_refactor_simplify

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    639f41f View commit details
    Browse the repository at this point in the history

  11. [pre-commit.ci] auto code formatting

    @pre-commit-ci
    pre-commit-ci[bot] committed May 3, 2024
    Configuration menu
    Copy the full SHA
    2b3ed47 View commit details
    Browse the repository at this point in the history

  12. Fix <atomic> header include guard.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    5164484 View commit details
    Browse the repository at this point in the history

  13. Move thread_count trait around and remove unused sink.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    9241238 View commit details
    Browse the repository at this point in the history

  14. Fix mistakes from merging clang-format changes.

    @wmaxey
    wmaxey committed May 3, 2024
    Configuration menu
    Copy the full SHA
    835cea3 View commit details
    Browse the repository at this point in the history

  15. [pre-commit.ci] auto code formatting

    @pre-commit-ci
    pre-commit-ci[bot] committed May 3, 2024
    Configuration menu
    Copy the full SHA
    bff1381 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. Merge branch 'main' into fea/atomic_refactor_simplify

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    608e63c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into fea/atomic_refactor_simplify

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    4a478b9 View commit details
    Browse the repository at this point in the history

  3. Add system header guards to every new __atomic header

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    198490e View commit details
    Browse the repository at this point in the history

  4. Delete <atomic> header synopsis.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    321706f View commit details
    Browse the repository at this point in the history

  5. Fix push/pop macros in <atomic>.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    a5e3d88 View commit details
    Browse the repository at this point in the history

  6. ATOMIC_VAR_INIT->LIBCUDACXX_ATOMIC_VAR_INIT 

    * This avoids conflicting with the host's definition.
    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    836addb View commit details
    Browse the repository at this point in the history

  7. Make <cuda/atomic> tests include the correct header.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    137b854 View commit details
    Browse the repository at this point in the history

  8. Fix typing with volatile atomic types.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    38f188a View commit details
    Browse the repository at this point in the history

  9. Include correct header for cuda::atomic.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    634dfae View commit details
    Browse the repository at this point in the history

  10. Fix underlying_t in notify_wait.h

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    e417e9b View commit details
    Browse the repository at this point in the history

  11. Revert using volatile in latch.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    90a182c View commit details
    Browse the repository at this point in the history

  12. Make helpers more useful in tests.

    @wmaxey
    wmaxey committed May 8, 2024
    Configuration menu
    Copy the full SHA
    8929189 View commit details
    Browse the repository at this point in the history

Commits on May 9, 2024

  1. Prevent non-CUDA compilers from seeing PTX.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    6cdbf6b View commit details
    Browse the repository at this point in the history

  2. Make the MSVC atomic header a little more friendly.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    a886bae View commit details
    Browse the repository at this point in the history

  3. Make the derived PTX header only visible to CUDA compilers.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    38d5f36 View commit details
    Browse the repository at this point in the history

  4. Remove the defaulted scope specifier on the atomic type layer.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    80978c3 View commit details
    Browse the repository at this point in the history

  5. Remove the defaulted scope specifier from the atomics API layer.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    52bbcf1 View commit details
    Browse the repository at this point in the history

  6. Fix missing cassert in several tests.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    8e9cd55 View commit details
    Browse the repository at this point in the history

  7. Revert mistaken LIBCUDACXX_ATOMIC_FLAG_INIT change.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    ef66a51 View commit details
    Browse the repository at this point in the history

  8. Fix bad atomic alignment errors.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    83d3414 View commit details
    Browse the repository at this point in the history

  9. [pre-commit.ci] auto code formatting

    @pre-commit-ci
    pre-commit-ci[bot] committed May 9, 2024
    Configuration menu
    Copy the full SHA
    55cd1ec View commit details
    Browse the repository at this point in the history

  10. Make internal owned memory atomic APIs have a default ctor.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    b3b8e60 View commit details
    Browse the repository at this point in the history

  11. Fully qualify the atomics APIs in the cuda/atomic header.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    50e37d7 View commit details
    Browse the repository at this point in the history

  12. Add missing type_traits to host.h 

    Co-authored-by: Michael Schellenberger Costa <[email protected]>
    @wmaxey @miscco
    wmaxey and miscco authored May 9, 2024
    Configuration menu
    Copy the full SHA
    1cdffdb View commit details
    Browse the repository at this point in the history

  13. Use _LIBCUDACXX_INLINE_VISIBILITY for API functions.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    082137f View commit details
    Browse the repository at this point in the history

  14. Reorder derived PTX functions attribute declarations.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    45a75d2 View commit details
    Browse the repository at this point in the history

  15. traits fixups in __atomic/types

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    aaf5e94 View commit details
    Browse the repository at this point in the history

  16. Default ctors and sprinkle noexcept around on some __atomic/types A… 

    …PIs.
    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    a015830 View commit details
    Browse the repository at this point in the history

  17. Apply suggestions to common.h.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    8d90f56 View commit details
    Browse the repository at this point in the history

  18. Remove full namespace qualifier in atomic storage trait.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    63b6f5d View commit details
    Browse the repository at this point in the history

  19. modernization fixes to order.h.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    6620229 View commit details
    Browse the repository at this point in the history

  20. Move includes cuda/atomic.h and cuda/barrier.h.

    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    e00ebf3 View commit details
    Browse the repository at this point in the history

  21. Adjust header error block and add missing includes in `<cuda/std/atom… 

    …ic>`
    @wmaxey
    wmaxey committed May 9, 2024
    Configuration menu
    Copy the full SHA
    310f06a View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Add missing system header block in cuda/atomic

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    1398c3c View commit details
    Browse the repository at this point in the history

  2. Fix invalid use of typename.

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    678006c View commit details
    Browse the repository at this point in the history

  3. _LIBCUDACXX_TRAITS->_CCCL_TRAIT

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    b3a24fe View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto code formatting

    @pre-commit-ci
    pre-commit-ci[bot] committed May 10, 2024
    Configuration menu
    Copy the full SHA
    923f61e View commit details
    Browse the repository at this point in the history

  5. Fix return type of host atomics.

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    3ade90a View commit details
    Browse the repository at this point in the history

  6. Fix cassert missing from generated atomic header.

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    b297715 View commit details
    Browse the repository at this point in the history

  7. Fix visibility of host atomics in NVRTC build.

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    f40a59f View commit details
    Browse the repository at this point in the history

  8. Add more tests to bad_atomic_alignment.pass.cpp

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    b841304 View commit details
    Browse the repository at this point in the history

  9. Fix alignment warnings in host compare_exchange layer.

    @wmaxey
    wmaxey committed May 10, 2024
    Configuration menu
    Copy the full SHA
    52788ff View commit details
    Browse the repository at this point in the history

  10. [pre-commit.ci] auto code formatting

    @pre-commit-ci
    pre-commit-ci[bot] committed May 10, 2024
    Configuration menu
    Copy the full SHA
    044b350 View commit details
    Browse the repository at this point in the history