Commit ccd64bb
feat: add managed local AI with llama-server (#1178)
* refactor(device): centralize physical memory probing
Move physical-memory discovery into one shared implementation.
Give setup and inference code a consistent RAM capacity value.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(inference): add host hardware snapshot
Capture CPU, memory, and GPU facts in a reusable host snapshot.
Provide the hardware input used by Local AI eligibility selection.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* fix(setup): roll back cancelled setup transactions
Roll back both interrupted and completed steps when setup is cancelled.
Use an independent cleanup token so cancellation cannot strand partial state.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(setup): manage mirrored WSL networking safely
Configure mirrored networking only with explicit consent and restore it on rollback.
Preserve BOMs, line endings, unterminated final lines, and concurrent user edits.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* refactor(setup): add immutable Local AI path policy
Centralize and validate runtime, model, manifest, preset, and staging paths.
Reject reparse points and delete only prevalidated app-owned trees during cleanup.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* refactor(setup): add verified native artifact installer
Download, verify, extract, and install native archives atomically.
Follow only bounded credential-free HTTPS redirects to approved GitHub release hosts.
Clean staging only through no-follow app-owned path validation.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* refactor(local-ai): add durable runtime manifests
Persist installed runtime and model metadata with atomic file updates.
Give setup, startup, and cleanup one durable source of installation truth.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* refactor(local-ai): add native Windows managed-process host
Host native Local AI processes with bounded startup and shutdown behavior.
Capture output and terminate owned process trees reliably on Windows.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(inference): add pinned runtime and model catalogs
Define immutable llama.cpp, CUDA, and GGUF catalog entries with recipe capacity metadata.
Pin versions, hashes, 262K KV topology, and runtime workspace requirements for reproducible installs.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* feat(inference): qualify NVIDIA GPUs by capability
Select any NVML-backed NVIDIA adapter with a stable ID, compatible driver, and CUDA runtime.
Require memory for model weights, recipe KV cache, and workspace without GPU SKU or CPU/GPU pairing.
Recommend the largest fitting model without silently changing explicit choices.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* feat(inference): probe NVIDIA hardware through trusted NVML
Load NVML from trusted locations and collect NVIDIA GPU capabilities.
Avoid unsafe library resolution while providing selector-grade hardware data.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* improve(inference): account for shared NVIDIA GPU memory
Correlate NVML adapters with unambiguous DXGI memory observations and count shared memory for any NVIDIA GPU.
Fail closed on duplicate or ambiguous adapter names so model selection cannot borrow another device's budget.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(huggingface): download verified GGUF models
Resume interrupted GGUF downloads with strict Range and Content-Range handling.
Verify size and hash before atomic promotion, restarting safely when resume is unsupported.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(llama-server): install verified native CUDA runtime
Install and inspect the pinned llama-server and CUDA runtime components.
Safely reconcile exact orphan runtime paths so interrupted promotion can retry.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(llama-server): add request-driven managed router
Launch llama-server on demand with an OS-assigned or validated fixed port.
Prove listener ownership by child PID and start time before health checks or persistence.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(setup): select and acquire qualified Local AI plan
Make Local AI opt-in and inspect hardware plus WSL viability without mutation.
Verify the WSL platform before large downloads, then acquire native inference before gateway provisioning.
Resolve bundled defaults reliably from RID-specific output.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* feat(setup): persist and start managed Local AI
Reconcile and reuse only exact manifest-owned runtime and model artifacts after interruption.
Persist a proven endpoint, clean durable state on uninstall, and test trusted runtime redirect boundaries.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* feat(setup): verify local inference readiness
Probe health, execute real inference, and verify GPU activity on the owned endpoint.
Fail setup cleanly when the installed Local AI stack is not operational.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(setup): configure IPv4 llama-server gateway provider
Use explicit 127.0.0.1 for Windows-to-WSL gateway connections and persisted setup state.
Remove only exact managed Local AI provider state during uninstall, preserving drift.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(local-ai): own llama router in companion lifecycle
Bind Local AI to the singleton setup-managed distro resolved from the loaded gateway registry.
Quiesce exact provider state before transitions, then publish only healthy owned endpoints and compensate failures.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* feat(setup-ui): add Local AI onboarding
Surface Local AI eligibility, explicit consent, review, and setup progress.
Show WSL platform readiness before downloads, then native inference before gateway provisioning.
Describe dynamic multi-gigabyte disk use accurately.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
* feat(tray): add Local AI status and controls
Add Local AI navigation, status, controls, logs, and localized resources.
Wire the page through application services and retain focused UI contracts.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* improve(setup): explain unavailable Local AI
Keep a compact Local AI review visible when hardware, WSL, or networking blocks setup.
Hide unavailable controls and show every detected dependency reason through See why.
Co-authored-by: Karen Lai <7976322+karkarl@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* fix(local-ai): verify GPU model placement
Accept full CUDA offload when either NVML memory movement or the parsed CUDA model buffer proves placement.
Reject startup when neither signal accounts for the selected model.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* style(setup-ui): pad WSL networking consent content
The consent InfoBar content sat flush against the bottom edge of the
warning, leaving the message and checkbox visually cramped. Add the
same bottom content margin the sandbox InfoBar already uses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L8LRNTZ5fBgjGwmtBCrSp3
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* fix(setup): stop WSL probes stalling on pipe EOF
Wait for the direct child process independently of redirected-pipe EOF, then bound the final output drain. Existing-distro inspection now fails closed when WSL times out or returns an unknown error.
Show progress during the Welcome-page inspection so a cold WSL service does not make setup look frozen.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
* fix(setup): explain WSL platform install failures
After an elevated WSL install fails, run a best-effort GitHub quota diagnostic and distinguish likely quota exhaustion from other download failures. Diagnostic timeouts never replace the original failure.
Offer Store, winget, and elevated PowerShell recovery routes, and allow the platform step to be retried.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
---------
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Co-authored-by: Pedro Larroy <plarroy@nvidia.com>
Co-authored-by: Dallin Romney <dallinromney@gmail.com>
Co-authored-by: Jacob Tomlinson <jtomlinson@nvidia.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Karen Lai <7976322+karkarl@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent d3ed6c7 commit ccd64bb
93 files changed
Lines changed: 14681 additions & 208 deletions
File tree
- docs
- src
- OpenClaw.Connection/LocalAi
- OpenClaw.SetupEngine.UI
- Pages
- OpenClaw.SetupEngine
- OpenClaw.Tray.WinUI
- Pages
- Presentation
- Services
- Strings
- en-us
- fr-fr
- nl-nl
- zh-cn
- zh-tw
- Windows
- tests
- OpenClaw.Connection.Tests
- OpenClaw.E2ETests/Setup
- OpenClaw.SetupEngine.Tests
- OpenClaw.Tray.Tests
- Presentation
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| 69 | + | |
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| |||
160 | 161 | | |
161 | 162 | | |
162 | 163 | | |
| 164 | + | |
163 | 165 | | |
164 | 166 | | |
165 | 167 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
0 commit comments