fix(local-ai): use CUDA for allocatable GPU memory

[joelagnel]

Summary

• query the CUDA driver directly for NVIDIA GPU identity and allocator-visible total/free memory
• remove NVML and DXGI shared-memory accounting from Local AI qualification
• use CUDA-major compatibility and allocator-visible capacity for both discrete and unified-memory NVIDIA devices
• preserve retryable partial GPU facts and restrict nvcuda.dll loading to System32

Closes #1191.

Related investigation: #1237 and #1239. CUDA itself supplies the capacity its allocator can use, avoiding DXGI shared-memory overcounting on discrete GPUs while supporting unified-memory devices through the same source of truth.

Commit structure

1. fix(local-ai): use CUDA for allocatable GPU memory - Joel Fernandes
2. refactor(local-ai): capture CUDA GPUs with a functional pipeline - Pedro Larroy

The branch contains exactly these two commits and no merge commit.

Validation

• ./build.ps1 - passed
• dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj --no-restore - 3,836 passed, 32 environment-gated skips
• dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore - 2,808 passed
• dotnet test ./tests/OpenClaw.SetupEngine.Tests/OpenClaw.SetupEngine.Tests.csproj --no-restore - 1,027 passed
• Local AI GPU proof filter via scripts/run-proof-tests.ps1 - 8 passed
• git diff --check upstream/main...HEAD - passed

Real behavior proof

Current head was probed on Windows x64 with an NVIDIA GeForce RTX 5090:

• CUDA driver API major: 13
• stable selector: GPU-cc66bca6-b5ff-dd70-995c-d81a07add980, matching nvidia-smi
• allocator-visible total memory: 34,190,458,880 bytes
• allocator-visible free memory: 32,432,455,680 bytes at capture time
• a second probe launched with that UUID in CUDA_VISIBLE_DEVICES succeeded and returned the same GPU

Focused setup proof verified full-offload log parsing, CUDA model-buffer evidence, allocator-memory delta evidence, and endpoint refresh behavior. A WinUI screenshot was not captured because interactive computer-use is unavailable in this session; the changed hardware path is covered by the live probe and focused tests above.

Security and compatibility

• every CUDA driver import is restricted to DllImportSearchPath.System32
• CUDA UUID formatting matches the existing GPU-... NVML manifest identity, preserving existing managed installations
• an identified GPU with a transient context or memory-query failure remains an incomplete, retryable result instead of being misreported as absent

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

[clawsweeper]

Codex review: needs real behavior proof before merge. Reviewed August 31, 2026, 5:12 PM ET / 21:12 UTC.

ClawSweeper review

What this changes

The PR replaces NVML and DXGI Local AI GPU discovery with direct CUDA-driver probing, then updates qualification, setup verification, manifest handling, and tests for CUDA UUIDs and memory values.

Merge readiness

⛔ Blocked until stronger real behavior proof is added - 8 items remain

Keep this PR open. The CUDA probe’s reported capacity is used directly for eligibility, but the related WDDM investigation records a CUDA-reported 46.3 GiB device that failed llama-server allocation at 15.81 GiB, so the PR can still approve an unusable model and reproduce the costly setup failure it aims to prevent.

Priority: P1
Reviewed head: fa285061e37c4082d5815955d7a9e7d379bd4f64
Owner decision: Required. See Decision needed.

Review scores

Measure	Result	What it means
Overall readiness	🧂 unranked krab (1/6)	The patch has a concrete WDDM false-positive eligibility defect and lacks end-to-end proof for the hardware behavior it changes.
Proof confidence	🦪 silver shellfish (2/6)	Needs stronger real behavior proof before merge: The changed production owner is CudaHostHardwareProbe, whose memory result feeds Local AI eligibility and then llama-server setup. The supplied RTX 5090 terminal probe demonstrates discovery and UUID selection, but not the claimed allocation-safe outcome: it lacks a current-head full model allocation and successful inference on the WDDM hardware class where related evidence records an allocation failure. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality	🧂 unranked krab (1/6)	1 actionable review finding remain.

Verification

Check	Result	Evidence
Real behavior	Needs proof	Needs stronger real behavior proof before merge: The changed production owner is CudaHostHardwareProbe, whose memory result feeds Local AI eligibility and then llama-server setup. The supplied RTX 5090 terminal probe demonstrates discovery and UUID selection, but not the claimed allocation-safe outcome: it lacks a current-head full model allocation and successful inference on the WDDM hardware class where related evidence records an allocation failure. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed	7 items	Introduced capacity source: The new CUDA probe records the current context's cuMemGetInfo total and free values as GPU-visible memory. Introduced eligibility trigger: Qualification now treats that GPU-visible total as the complete capacity used to decide whether a model fits. Contrary WDDM allocation evidence: The related investigation at #1237 records CUDA reporting 46.3 GiB on Windows ARM64 while llama-server ran out of memory at a 15.81 GiB allocation, so CUDA-visible capacity is not yet proven to be a safe WDDM eligibility bound.
Findings	1 actionable finding	[P1] Retain a WDDM-safe eligibility capacity bound
Security	None	None.

How this fits together

Local AI setup probes an NVIDIA GPU, chooses a fitting runtime and model, then downloads and launches llama-server. The probe feeds eligibility in both the setup UI and tray, so its capacity value determines whether users are offered a Local AI installation.

flowchart LR
A[Windows NVIDIA driver] --> B[CUDA hardware probe]
B --> C[Local AI memory qualification]
C --> D[Model and runtime selection]
D --> E[Managed llama-server]
E --> F[Setup verification]

Loading

Decision needed

Question	Recommendation
Should Local AI retain a WDDM-conservative eligibility bound until direct CUDA capacity is demonstrated to predict llama-server allocations on unified-memory Windows hardware?	Require a conservative WDDM policy: Keep or introduce a capacity signal that matches actual WDDM allocations and prove it on the affected hardware before landing.

Why: The evidence establishes that replacing DXGI aggregation is necessary, but it does not establish that CUDA’s reported total is a safe cross-hardware admission limit.

Before merge

• Add real behavior proof - Needs stronger real behavior proof before merge: The changed production owner is CudaHostHardwareProbe, whose memory result feeds Local AI eligibility and then llama-server setup. The supplied RTX 5090 terminal probe demonstrates discovery and UUID selection, but not the claimed allocation-safe outcome: it lacks a current-head full model allocation and successful inference on the WDDM hardware class where related evidence records an allocation failure. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
• Retain a WDDM-safe eligibility capacity bound (P1) - cuMemGetInfo now supplies the number that decides whether a model can be installed, but the recorded WDDM case in Local AI unusable on DGX Spark (Windows ARM64): shared GPU memory is not CUDA-allocatable, and a capability failure rolls back the whole setup #1237 reports 46.3 GiB from CUDA and an out-of-memory failure at 15.81 GiB in llama-server. This can again qualify a model that will fail after download. Preserve or derive a conservative WDDM-capacity policy and cover that case before using this value for admission.
• Resolve merge risk (P1) - On affected WDDM systems, merging can turn an over-reported CUDA total into an eligible model download followed by llama-server allocation or setup failure.
• Resolve merge risk (P1) - The PR body does not declare the repository-required custom proof pools, despite changing CUDA hardware behavior and persisted-install reconciliation.
• Complete next step (P2) - A maintainer must choose the supported WDDM and unified-memory capacity policy before a repair can safely proceed.
• Improve patch quality - Adopt and test a WDDM-conservative capacity policy that still defines the intended unified-memory support boundary.
• Improve patch quality - Add redacted current-head proof of selection, full model allocation, and fixed-prompt inference on affected WDDM hardware, then update the PR body to declare the applicable proof pools.

Findings

• [P1] Retain a WDDM-safe eligibility capacity bound — src/OpenClaw.Shared/Inference/CudaHostHardwareProbe.cs:72-81

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Change balance	production +262/-600, tests +126/-117	The PR replaces the platform hardware-discovery implementation across 18 files, making hardware-specific proof more important than unit coverage alone.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #1191
Summary: This PR is an open candidate fix for the canonical Local AI false-capacity report, but its CUDA-only policy remains unproven on the reported WDDM hardware.

Members:

• canonical: Do not count DXGI shared system memory as CUDA capacity on discrete GPUs #1191 - The PR formally targets this report's DXGI shared-memory capacity defect.
• same_root_cause: Local AI unusable on DGX Spark (Windows ARM64): shared GPU memory is not CUDA-allocatable, and a capability failure rolls back the whole setup #1237 - Its recorded WDDM experiment directly evaluates the same question of CUDA-reported versus usable allocation capacity.
• partial_overlap: fix(inference,setup): shared-memory capacity semantics and destructive rollback (PR #1237 follow-ups) #1239 - It addresses the same qualification defect with a different capacity source and also includes separate rollback work.

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

1. Make capacity conservative on WDDM (recommended)
   Use a capacity policy that is validated against real llama-server allocations on the affected hardware, with regression coverage for the over-reporting case.
2. Accept the unsupported-hardware risk
   Land CUDA-only qualification only if maintainers explicitly accept possible false-positive Local AI setup on WDDM unified-memory systems.
3. Pause the CUDA-only swap
   Close or pause this branch if a safe unified-memory policy cannot be established without broader platform investigation.

Technical review

Best possible solution:

Use a capacity policy proven conservative for WDDM while retaining supported unified-memory hardware, then demonstrate selection, full model allocation, and successful inference on the affected hardware class.

Do we have a high-confidence way to reproduce the issue?

No. The source path and related WDDM allocation trace establish a credible scenario, but the supplied proof does not show a current-head model allocation and inference on the affected hardware.

Is this the best way to solve the issue?

No. Removing DXGI aggregation addresses the reported bug, but using cuMemGetInfo as the final capacity bound conflicts with the recorded WDDM allocation result and needs a conservative policy.

Full review comments:

• [P1] Retain a WDDM-safe eligibility capacity bound — src/OpenClaw.Shared/Inference/CudaHostHardwareProbe.cs:72-81
cuMemGetInfo now supplies the number that decides whether a model can be installed, but the recorded WDDM case in Local AI unusable on DGX Spark (Windows ARM64): shared GPU memory is not CUDA-allocatable, and a capability failure rolls back the whole setup #1237 reports 46.3 GiB from CUDA and an out-of-memory failure at 15.81 GiB in llama-server. This can again qualify a model that will fail after download. Preserve or derive a conservative WDDM-capacity policy and cover that case before using this value for admission.
  Confidence: 0.95

Overall correctness: patch is incorrect
Overall confidence: 0.95

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f46400aab24e.

Labels

Label changes:

• remove merge-risk: 🚨 security-boundary: Current PR review merge-risk labels are merge-risk: 🚨 compatibility.
• remove merge-risk: 🚨 availability: Current PR review merge-risk labels are merge-risk: 🚨 compatibility.

Label justifications:

• P1: A false-positive capacity decision can make an active Local AI setup download a large model and then fail at runtime.
• merge-risk: 🚨 compatibility: The replacement changes the admission rule for existing NVIDIA Windows hardware, including unified-memory WDDM configurations.
• rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦪 silver shellfish and patch quality is 🧂 unranked krab.
• status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: The changed production owner is CudaHostHardwareProbe, whose memory result feeds Local AI eligibility and then llama-server setup. The supplied RTX 5090 terminal probe demonstrates discovery and UUID selection, but not the claimed allocation-safe outcome: it lacks a current-head full model allocation and successful inference on the WDDM hardware class where related evidence records an allocation failure. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

What I checked:

• Introduced capacity source: The new CUDA probe records the current context's cuMemGetInfo total and free values as GPU-visible memory. (src/OpenClaw.Shared/Inference/CudaHostHardwareProbe.cs:72, fa285061e37c)
• Introduced eligibility trigger: Qualification now treats that GPU-visible total as the complete capacity used to decide whether a model fits. (src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:141, fa285061e37c)
• Contrary WDDM allocation evidence: The related investigation at Local AI unusable on DGX Spark (Windows ARM64): shared GPU memory is not CUDA-allocatable, and a capability failure rolls back the whole setup #1237 records CUDA reporting 46.3 GiB on Windows ARM64 while llama-server ran out of memory at a 15.81 GiB allocation, so CUDA-visible capacity is not yet proven to be a safe WDDM eligibility bound.
• Coverage gap: The added eligibility test proves that legacy synthetic shared-memory fields are ignored, but it does not model a CUDA-reported total exceeding the allocation limit of the selected llama-server model. (tests/OpenClaw.Shared.Tests/LocalInferenceQualificationTests.cs:121, fa285061e37c)
• Native-load boundary: Each new CUDA P/Invoke declares DllImportSearchPath.System32, and focused coverage checks that invariant. (src/OpenClaw.Shared/Inference/CudaHostHardwareProbe.cs:144, fa285061e37c)
• Feature provenance: The current Local AI selector dates to the merged managed-Local-AI feature commit, whose raw recorded parent and patch show this selector was introduced there. (src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:64, ccd64bbb7d68)

Likely related people:

• Joel: Raw commit ccd64bb adds src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs:64 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: ccd64bbb7d68; files: src/OpenClaw.Shared/Inference/Catalog/LocalInferenceSelector.cs)
• larroy: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (5 earlier review cycles)

• reviewed 2026-08-28T03:41:51.892Z sha c6765a7 :: needs real behavior proof before merge. :: [P1] Keep WDDM allocation capacity conservative
• reviewed 2026-08-28T07:33:01.892Z sha aff93c1 :: needs real behavior proof before merge. :: [P1] Keep WDDM eligibility capacity conservative
• reviewed 2026-08-28T14:40:08.771Z sha f84912d :: needs real behavior proof before merge. :: [P1] Keep WDDM eligibility capacity conservative | [P1] Preserve existing Local AI GPU identifiers on upgrade
• reviewed 2026-08-28T15:34:25.782Z sha effc11a :: needs real behavior proof before merge. :: [P1] Keep WDDM eligibility capacity conservative | [P1] Migrate existing NVML UUID GPU selectors
• reviewed 2026-08-31T20:09:20.168Z sha 8b66f30 :: needs real behavior proof before merge. :: [P1] Keep WDDM eligibility capacity conservative | [P1] Migrate existing NVML UUID selectors | [P2] Load the CUDA driver through a trusted resolution path

[larroy]

Approach looks good.

We should add a test if possible for the case where Cuda detection fails or can't be initialized.

[larroy]

we should surface an error to the user here if cuda is not found or can't be initialized

[larroy]

Suggest to refactor this in a more functional idiom:

private static IReadOnlyList<GpuInfo> CaptureCudaGpus()
{
    if (!OperatingSystem.IsWindows() || CuInit(0) != CudaSuccess ||
        CuDeviceGetCount(out int count) != CudaSuccess)
    {
        return [];
    }

    int? cudaMajorVersion =
        CuDriverGetVersion(out int driverVersion) == CudaSuccess && driverVersion > 0
            ? driverVersion / 1000
            : null;

    return Enumerable.Range(0, count)
        .Select(ordinal => TryCaptureGpu(ordinal, cudaMajorVersion))
        .Where(gpu => gpu is not null)
        .Select(gpu => gpu!)
        .ToList();
}

private static GpuInfo? TryCaptureGpu(int ordinal, int? cudaMajorVersion)
{
    if (CuDeviceGet(out int device, ordinal) != CudaSuccess)
        return null;

    string? name = ReadDeviceName(device);
    string? pciBusId = ReadPciBusId(device);
    if (string.IsNullOrWhiteSpace(name) || string.IsNullOrWhiteSpace(pciBusId))
        return null;

    return WithCudaContext(device, () =>
    {
        if (CuMemGetInfo(out nuint freeBytes, out nuint totalBytes) != CudaSuccess ||
            totalBytes == 0 || totalBytes > long.MaxValue || freeBytes > totalBytes)
        {
            return null;
        }

        return new GpuInfo(
            GpuVendor.Nvidia,
            name,
            GpuVisibleMemoryBytes: (long)totalBytes,
            FreeGpuVisibleMemoryBytes: (long)freeBytes,
            CudaMajorVersion: cudaMajorVersion,
            StableId: ToCudaVisibleDevicesSelector(pciBusId));
    });
}

private static GpuInfo? WithCudaContext(int device, Func<GpuInfo?> action)
{
    if (CuCtxCreate(out IntPtr context, 0, device) != CudaSuccess)
        return null;

    try
    {
        return action();
    }
    finally
    {
        _ = CuCtxDestroy(context);
    }
}