CUDA Backend¶

Clifft's CPU implementation is the stable reference. The experimental CUDA backend shares circuit parsing, compilation, and symbolic planning with it, then lowers the prepared plan into a private GPU executable. It follows the same contract as the HIP backend: a backend-specific ExecutablePlan and Sampler keep precision, workspace, and launch controls outside the stable API, and the CPU executor remains the semantic oracle.

Current capabilities¶

Unsupported programs are rejected during lowering; there is no automatic CPU fallback.

Execution tiers¶

The CUDA backend selects one of three tiers per program and device:

In the cooperative tiers every thread of a block evaluates the scalar control flow (random draws, branch selection, expressions) redundantly from identical inputs, so branch decisions need no broadcast storage. Coefficient sweeps are strided across the block, measurement probabilities are tree-reduced through a fixed shared-memory scratch, and byte outputs are written by lane 0.

The per-shot RNG derives from the seed root and global shot index, so a shot draws the same random stream whatever the batch size and concurrency cap. Rows are reproducible for a fixed tier, precision, and block size. Rows are not guaranteed to be bit-identical across tiers or block sizes: coefficient sweeps are lane-strided and probabilities tree-reduce across the block, so both the tier and the block size change the floating-point summation order, which can move a probability across a measurement threshold. Compare different tiers or block sizes statistically, or through forced replay, rather than row for row.

The shared-memory budget is the device's opt-in limit minus a 16 KB reduction scratch. On an H100 or H200 (227 KB opt-in) FP64 states fit through k = 13 and FP32 through k = 14; devices with a 96 KB or 164 KB opt-in cover one or two fewer coordinates. selected_tier() reports the automatic choice for an executable on the current device without allocating a workspace.

Hardware and source build¶

The documented target is Linux x86_64 with the CUDA toolkit and a Hopper-class sm_90 device (H100 or H200), where the backend was validated. Other architectures can be development targets by setting CMAKE_CUDA_ARCHITECTURES; the interpreter has no architecture-specific branch, but their shared-memory budgets and conformance coverage differ.

CUDA builds require a CUDA toolkit with C++20 support (CUDA 12 or newer). Build an editable installation from a checkout:

git clone https://github.com/unitaryfoundation/clifft.git
cd clifft

uv venv
CMAKE_ARGS="-DCLIFFT_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=90" \
    uv pip install -e .

For standalone C++ development, configure and build the CUDA targets directly:

cmake -S . -B build-cuda -G Ninja \
    -DCLIFFT_ENABLE_CUDA=ON \
    -DCMAKE_CUDA_ARCHITECTURES=90
cmake --build build-cuda -j

The build compiles device code without a visible GPU. Sampling requires a compatible device at runtime:

from clifft.experimental import cuda

print(cuda.is_built())
print(cuda.is_available())
print(cuda.backend_info())

Compile and reuse a sampler¶

from clifft.experimental import cuda

program = cuda.compile("""
    H 0
    T 0
    H 0
    M 0
    OBSERVABLE_INCLUDE(0) rec[-1]
""")

sampler = cuda.Sampler(program)
result = sampler.sample(100_000, seed=1234)
print(sampler.tier)
print(result.measurements.shape)

cuda.Program and clifft.Program are not interchangeable. cuda.compile() currently accepts Stim circuit text and does not expose the CPU input_format switch. cuda.selected_tier(program) reports the tier automatic selection would pick without allocating a workspace.

The same contract is available from C++:

#include "clifft/circuit/parser.h"
#include "clifft/frontend/frontend.h"
#include "clifft/sampling/cuda/executable_plan.h"
#include "clifft/sampling/cuda/sampler.h"
#include "clifft/sampling/planner.h"

using namespace clifft;

const sampling::SamplingPlan plan = sampling::plan_sampling(trace(parse(R"(
    H 0
    T 0
    H 0
    M 0
    OBSERVABLE_INCLUDE(0) rec[-1]
)")));
const sampling::cuda::ExecutablePlan executable(plan);

sampling::cuda::Sampler sampler(executable);
const sampling::SamplingResult result = sampler.sample(100000, uint64_t{1234});

Construct one sampler per concurrently active caller and reuse it. Construction resolves the execution tier, uploads the program, and allocates a bounded workspace on the device that is current at that time. Calls on one sampler are synchronous and must not overlap.

Each shared kernel opts into the device's maximum shared-memory budget once during construction. Individual launches request only the bytes their program needs, so samplers with different widths can safely use the same kernel concurrently.

For post-selection, compile the detector mask into the plan and call sampler.sample_survivors(). Fixed-row sample() rejects a post-selected program. Survivor sampling always returns aggregate counts; pass keep_records = true to retain survivor rows.

Precision and launch controls¶

sampler = cuda.Sampler(
    program,
    precision="fp32",
    max_batch_shots=16_384,
    tier="auto",
    max_concurrent_shots=0,
)
result = sampler.sample(100_000, seed=42, block_size=256)
print(sampler.allocated_device_bytes)

The C++ constructor takes the same values in the same order:

sampling::cuda::Sampler sampler(executable,
                                sampling::cuda::CoefficientPrecision::FP32,
                                /*max_batch_shots=*/16384,
                                sampling::cuda::ExecutionTier::Auto,
                                /*max_concurrent_shots=*/0);

• FP64 coefficient evolution is the default. FP32 halves coefficient storage and widens the shared-memory tier by one coordinate. Probability reductions, normalization factors, aggregate statistics, replay log-probabilities, and EXP_VAL outputs remain FP64 in both modes.
• max_batch_shots bounds the retained per-shot output rows. Larger requests are split into synchronous launches that reuse the workspace.
• tier forces one execution tier for experiments ("thread_per_shot", "block_shared", or "block_global" from Python); block_shared is rejected when the program does not fit the device. Sampler.tier reports the resolved choice.
• max_concurrent_shots caps how many shots the cooperative tiers keep resident per launch, which bounds BlockGlobal slab memory. Zero derives the cap from the multiprocessor count and free device memory.
• block_size must be a power of two between 1 and 1024. In the cooperative tiers it is the number of lanes that share one shot.
• allocated_device_bytes() exposes the retained workspace size.

CPU and CUDA use separate random-stream domains, so compare deterministic branches directly and stochastic results statistically.

Performance considerations¶

Tier selection balances coefficient storage and the amount of work per shot. The width-four thread-per-shot cutoff is conservative: its global coefficient pool grows with both batch size and active width, while the shared tier keeps coefficients on chip. A different crossover on one circuit does not establish a better default for all workloads.

The cooperative tiers launch up to 32 blocks per multiprocessor by default, subject to batch size and, for global coefficient storage, available memory. This oversubscribes the device so its scheduler can fill available capacity; it does not imply that all launched blocks reside on the multiprocessors at once. Each global block needs its own coefficient slab, so increasing the cap also increases retained storage.

Early postselection failures can make per-shot control overhead dominate the coefficient work. The automatic policy uses peak active width and memory fit, so it does not account for discard depth. Measure the relevant workload and consider forcing ExecutionTier::ThreadPerShot for discard-heavy programs. GPU speedups are workload-dependent.

See the benchmark instructions for measuring workloads with the maintained harness.

Architecture¶

HIR -> SamplingPlan -> CPU ExecutablePlan -> trusted CPU sampling oracle
                    -> private CUDA executable -> tiered device interpreter

The CUDA executable is a backend-specific packing of prepared SamplingAction alternatives: host-computed Pauli phases and pairings, active-width transitions, expressions, and noise distributions. The device executes the plan without topology planning or allocation in its dispatch loop. CUDA lowering shares execution-ready Pauli preparation and result containers with the CPU backend; its executable layout, workspace, and kernels remain backend-specific.

When changing the kernels, preserve these invariants:

• no allocation inside a kernel or ordinary dispatch loop;
• every lane of a block advances the same RNG and takes the same branches;
• a collapse stages through scratch, because lanes write packed outputs while other lanes still read overlapping sources;
• reductions, normalization, statistics, replay likelihoods, and expectation values remain FP64; and
• the CPU ExecutablePlan remains the semantic oracle.

Testing and contribution boundary¶

Ordinary CPU builds compile the host-side CUDA lowering tests. They check packed actions, expressions, noise tables, prepared Pauli data, width limits, and rejection of unsupported plans. Adding a SamplingAction without CUDA lowering fails during this build.

CUDA-enabled CI additionally compiles sm_90 device code and runs GPU-free conformance cases:

cmake --build build-cuda --target clifft_tests clifft_cuda_tests -j
ctest --test-dir build-cuda --output-on-failure -R CUDA

On NVIDIA hardware, run the shared sampling tests, focused API/replay tests, and common CPU references together:

uv run --no-sync pytest tests/python --require-gpu=cuda -k 'cuda or gpu_reference' -v

GPU-free CI skips kernel-launch tests, so it does not establish runtime correctness on hardware. The Python command above fails if CUDA has no available device. The hardware suite exercises FP64 and FP32 repeatability, every tier against the CPU executor on forced branches and expectation values, cross-tier agreement on a wide program, noisy distributions, post-selection, retained output rows, the cooperative concurrency cap, separate samplers called concurrently, and automatic selection on both sides of the device's shared-memory boundary (located at runtime through selected_tier()), where forced replay pins the collapse and the state after it against the CPU and survivor rows are checked for completeness and alignment. Run it on a machine with a device:

./build-cuda/tests/clifft_cuda_tests -d yes