C2H cuda::buffer support - #11344
Conversation
📝 SummarySummary by CodeRabbit
WalkthroughC2H now provides checked device and host buffers, shared allocation validation, typed environment parsing, and stream-aware random-data generation. Default-stream wrappers remain available. Tests cover allocation, parsing, generated values, and concurrent stream isolation. ChangesC2H buffers and generators
Assessment against linked issues
Possibly related PRs
Priority: ➖ Normal Change: Feature Merge Risk: 🟡 Moderate · up to The stream-aware generators can retain excessive device memory, return invalid random data after CURAND failures, or race across per-thread streams. These issues should be fixed before merge. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
c2h/include/c2h/detail/checked_memory.cuh (1)
45-46: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valuesuggestion:
std::strtollaccepts negative and malformed input.C2H_DEVICE_MEMORY_LIMIT=-1casts to a near-SIZE_MAXlimit, which silently setsinfo.override = truewhile changing nothing. Reject values that do not parse to a positive number.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 6fa497b9-b4bb-4c98-8a0a-89132e3d0d2c
📒 Files selected for processing (16)
c2h/generators.cuc2h/generators_gen_values.cuc2h/generators_vector.cuc2h/include/c2h/buffer_generators.cuhc2h/include/c2h/catch2_test_helper.hc2h/include/c2h/checked_allocator.cuhc2h/include/c2h/checked_memory_resource.cuhc2h/include/c2h/detail/catch2_nvtx.hc2h/include/c2h/detail/catch2_seed.hc2h/include/c2h/detail/checked_memory.cuhc2h/include/c2h/detail/env.cuhc2h/include/c2h/detail/generators.cuhc2h/include/c2h/generator_common.hc2h/include/c2h/generators.hc2h/include/c2h/vector_generators.hcub/test/catch2_test_c2h_buffers.cu
Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
85602af to
0611eea
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 669cbb1c-537b-4557-92c7-151ed2dceda5
📒 Files selected for processing (4)
c2h/generators.cuc2h/include/c2h/detail/checked_memory.cuhc2h/include/c2h/detail/generators.cuhcub/test/catch2_test_c2h_buffers.cu
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.
0611eea to
be96c1c
Compare
This comment has been minimized.
This comment has been minimized.
|
The fix for NVHPC issue is filed in #11366 |
This comment has been minimized.
This comment has been minimized.
Simplified preprocessor guards accordingly
…ibution Add a multi-stream test that demonstrates race condition, confirm failure as per CodeRabbit review claim. Save generation/distribution per stream/device to resolve the race and fix the test.
…in prepare_random_generator(seed, num_items)
- Device-scoped cleanup remains the primary path. - If device selection throws, the state is reset without switching devices.
MSVC requires constexpr constants to be caputed, but Clang insists they should not be. Solution: do not use any_of, use count instead which does not need lambda to begin with.
- The stream-aware overload now delegates null streams to the safe current-device path in c2h/generators.cu:187 - The regression test now exercises both implicit and explicit legacy-default-stream calls in catch2_test_c2h_buffers.cu
Porting one missing change from NVIDIA#10898, to protect from overflow in integral addition of `bytes + padding`. Added a regression test using size_t::max() as bytes.
7dba45a to
0f2e3d2
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: ed651d59-a16b-44fc-9970-b1a9f9df0b07
📒 Files selected for processing (3)
c2h/generators.cuc2h/include/c2h/detail/checked_memory.cuhcub/test/catch2_test_c2h_buffers.cu
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.
| [[nodiscard]] bool matches(int device, ::cudaStream_t stream) const noexcept | ||
| { | ||
| m_distribution.resize(num_items); | ||
| return m_device == device && m_stream == stream; |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🤖 get_repo_knowledge executed:
get_repo_knowledge NVIDIA/cccl /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/conventions /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/learnings
Length of output: 47633
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed hunk ---'
git diff -- c2h/generators.cu
printf '%s\n' '--- generators.cu relevant source ---'
sed -n '1,130p' c2h/generators.cu
printf '%s\n' '--- generator symbols and tests ---'
rg -n -C 3 'generator_state_t|prepare_random_generator|state_for|generator_t|curandSetStream|cudaStreamPerThread|cudaStreamLegacy' c2h --glob '!*.json'Repository: NVIDIA/cccl
Length of output: 8589
🏁 Script executed:
#!/bin/bash
set -eu
sed -n '1,130p' c2h/generators.cu
sed -n '130,320p' c2h/generators.cuRepository: NVIDIA/cccl
Length of output: 8700
🌐 Web query:
NVIDIA CUDA Runtime API cudaStreamPerThread each host thread stream handle identity
💡 Result:
The handle cudaStreamPerThread (or CU_STREAM_PER_THREAD in the Driver API) serves as a special identifier that explicitly references an implicit, per-thread default stream [1][2][3]. When used, this handle provides each host thread with its own independent default stream, which is local to both that specific host thread and the current CUDA context [1][3]. Because these streams are local to the thread, they do not synchronize with other streams, including the default streams of other threads or the legacy default stream (unless explicitly synchronized) [1][2][4]. This independence allows commands issued to the default stream by different host threads to execute concurrently on the GPU [2][4]. In contrast to the legacy default stream, which is shared across all host threads and introduces implicit synchronization, the per-thread default stream behaves like a regular, non-blocking stream [2][4]. While the handle cudaStreamPerThread always identifies this specific per-thread stream, it is important to note that the behavior of the default stream (the implicit stream used when passing 0) can be configured to use this per-thread behavior globally for a compilation unit by using the --default-stream per-thread compiler option or by defining the CUDA_API_PER_THREAD_DEFAULT_STREAM preprocessor macro before including CUDA headers [1][2][4]. In summary: - Handle identity: cudaStreamPerThread is a special constant that maps to an implicit, thread-local stream [1][3]. - Concurrency: Because each thread gets its own instance of this stream, operations queued from different threads do not serialize against each other [2][4]. - Compatibility: It is a valid stream handle that can be used with all CUDA Runtime APIs that accept a stream parameter [5][6].
Citations:
- 1: https://docs.nvidia.com/cuda/cuda-runtime-api/stream-sync-behavior.html
- 2: https://developer.nvidia.com/blog/gpu-pro-tip-cuda-7-streams-simplify-concurrency/
- 3: https://docs.nvidia.com/cuda/archive/13.2.1/pdf/CUDA_Runtime_API.pdf
- 4: https://docs.nvidia.com/cuda/archive/13.1.0/cuda-programming-guide/02-basics/asynchronous-execution.html
- 5: https://docs.nvidia.com/cuda/cuda-runtime-api/group%5F%5FCUDART%5F%5FSTREAM.html
- 6: https://docs.nvidia.com/cuda/archive/12.2.1/cuda-runtime-api/group__CUDART__STREAM.html
important: Separate generator_state_t for each cudaStreamPerThread host thread
cudaStreamPerThread uses a different implicit stream for each host thread, although the handle value is the same. generator_state_t::matches compares only m_device and m_stream, so generator_t merges these streams into one state. stream.sync() then synchronizes only the calling thread's stream before m_distribution is resized or overwritten. A consumer on another thread can read shared device storage while it is being regenerated or reallocated.
Include host-thread identity in the state key, reject cudaStreamPerThread, or use independent storage. Add a regression test with two host threads and an in-flight first consumer.
Source: Path instructions
| void generate(::cuda::stream_ref stream) | ||
| { | ||
| #if C2H_HAS_CURAND | ||
| curandSetStream(m_gen, stream.get()); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🤖 get_repo_knowledge executed:
get_repo_knowledge NVIDIA/cccl /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/learnings /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/conventions
Length of output: 47929
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- changed file context ---'
sed -n '1,320p' c2h/generators.cu
printf '%s\n' '--- related declarations and call sites ---'
rg -n -C 4 'prepare_random_generator|curandSetStream|curandGenerateUniform|generator_t|generator_state_t|distribution' c2hRepository: NVIDIA/cccl
Length of output: 18011
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- targeted source map ---'
ast-grep outline c2h/generators.cu
printf '%s\n' '--- CUDA error/status handling in nearby c2h code ---'
rg -n -C 3 'curand[A-Za-z]+|CUDA_TRY|cuda[A-Za-z]+\(' c2h --glob '*.{cu,cc,h,cpp,hpp}'Repository: NVIDIA/cccl
Length of output: 4294
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
from pathlib import Path
p = Path("c2h/generators.cu")
lines = p.read_text().splitlines()
for n in range(1, min(len(lines), 310) + 1):
if 90 <= n <= 270:
print(f"{n:4}: {lines[n-1]}")
PYRepository: NVIDIA/cccl
Length of output: 6213
🏁 Script executed:
#!/bin/bash
set -eu
sed -n '90,270p' c2h/generators.cuRepository: NVIDIA/cccl
Length of output: 5127
important: Handle the return status from both curandSetStream and curandGenerateUniform. prepare_random_generator unconditionally marks the distribution as generated and returns its pointer after generate returns. If either call fails, the pointer can refer to data that was not generated on the requested stream or contains invalid values.
Source: Path instructions
|
|
||
| auto state = std::make_unique<generator_state_t>(device, stream); | ||
| auto& result = *state; | ||
| m_states.push_back(std::move(state)); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
set -euo pipefail
ast-grep outline c2h/generators.cu --match 'generator_t|init_generator|cleanup_generator' --view expanded
rg -n -C 5 '\b(init_generator|cleanup_generator|prepare_random_data)\s*\(' \
--type=cpp --type-add 'cuda:*.cu' --type-add 'cuh:*.cuh' -tcuda -tcuh .Repository: NVIDIA/cccl
Length of output: 10029
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- c2h/generators.cu ---'
sed -n '60,245p' c2h/generators.cu
printf '%s\n' '--- catch2_main.h ---'
sed -n '1,65p' c2h/include/c2h/catch2_main.h
printf '%s\n' '--- stream creation/cleanup in relevant tests ---'
sed -n '140,215p' cub/test/catch2_test_c2h_buffers.cu
rg -n -C 3 'make_stream|stream_ref|cudaStreamCreate|cudaStreamDestroy|\.get\(\)' cub/test c2h --glob '*.cu' --glob '*.cuh' | head -240Repository: NVIDIA/cccl
Length of output: 26022
🤖 get_repo_knowledge executed:
get_repo_knowledge NVIDIA/cccl /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/conventions /tmp/coderabbit-repo-knowledge/nvidia-cccl-2f773195/learnings
Length of output: 47786
important: Bound the generator_state_t cache. Each stream creates a retained state and device-vector allocation. init_generator() runs before session.run(), while cleanup_generator() runs only afterward. Transient streams can therefore accumulate device memory until the test process ends. Add state eviction or use bounded stream-state ownership.
Source: Path instructions
🥳 CI Workflow Results🟩 Finished in 2h 53m: Pass: 100%/174 | Total: 7d 17h | Max: 2h 52m | Hits: 38%/472571See results here. |
1 participant
Description
This PR contributes a portion of #10898
Adds a synchronous, device-accessible checked memory resource using
cudaMalloc/cudaFree, plusc2h::make_device_bufferfor creatingcuda::device_bufferinstances backed by it.Adds a synchronous, host-accessible memory resource backed by aligned new/delete. On integrated devices, it checks available device memory before allocating. Includes
c2h::make_host_bufferfor creatingcuda::host_bufferinstances backed by it.Adds stream-aware C2H generator support and the
gen_device_buffer,gen_buffers, andgen_host_bufferhelpers for generating data into checked CUDA buffers.Adds focused tests covering the resources, factories, allocation failures, and generated buffer contents.
closes #11343
gated by #11338Checklist