@@ -21,13 +21,24 @@ packing boxes into trucks. If everything fits, the bid is valid. If not, the pro
2121The match is ** all-or-nothing** : every replica of every service must find a home, or the whole
2222deployment is rejected. Inventory is only updated if all placements succeed.
2323
24+ > ** Scope note — leased IPs (AC18–AC21).** One part of the decision happens * outside* this
25+ > engine. ** Leased IPs** (public IP addresses a tenant leases for a service) are checked by the
26+ > ** inventory service** ([ cluster/inventory.go] ( ../cluster/inventory.go ) ) as a ** cluster-level
27+ > pre-check, before** ` tryAdjust ` /` Adjust ` is ever called. It is not part of the per-node engine,
28+ > but it * is* part of the same overall "can my cluster run this?" question and obeys the same
29+ > all-or-nothing outcome: if the provider lacks enough free leased IPs, the deployment is rejected
30+ > and the per-node engine never runs. The leased-IP criteria below (AC18–AC21) document that
31+ > pre-check.
32+
2433---
2534
2635## Core Concepts
2736
2837- ** Node** — one machine with its own CPU, RAM, GPUs, and local disk.
2938- ** Cluster storage** — shared persistent disk pools (e.g., ` beta2 ` , ` beta3 ` ) usable by any node
3039 that supports that class.
40+ - ** Leased IP** — a dedicated public IP a tenant leases for a service. Drawn from a single, finite
41+ ** provider-wide** pool (managed by MetalLB), not tied to any node.
3142- ** Resource group** — one service definition + a replica count.
3243- ** Available capacity** — ` Allocatable − Allocated ` , never negative. An ` Allocatable ` of ` -1 ` is
3344 treated as effectively unlimited.
@@ -239,6 +250,7 @@ caller can inspect what would have been allocated. The engine's inventory does n
239250- Recoverable example: Node 1 has no GPU → engine tries Node 2.
240251- Fatal example: Cluster ` beta2 ` pool is empty when any replica needs it → stop;
241252 ` ErrInsufficientCapacity ` .
253+ - Fatal example: Not enough free leased IPs (AC19) → stop before any node is tried.
242254
243255### AC16 — "Unlimited" allocatable
244256** WHAT:** When a node's ` Allocatable ` for a resource is ` -1 ` , that dimension is treated as
@@ -258,6 +270,50 @@ failure: **CPU → GPU → Memory → Storage volumes (in declaration order)**.
258270 class-not-supported / RAM-volume-over-memory; cluster-level for pool exhaustion or bad
259271 attributes).
260272
273+ ### AC18 — Leased IP counting
274+ ** WHAT:** The number of leased IPs a deployment needs is the count of ** unique** ` LEASED_IP `
275+ endpoint sequence numbers across the whole group spec. The same endpoint referenced by several
276+ services is ** one** IP, not many.
277+ ** HOW:** The provider counts via ` GetEndpointQuantityOfResourceGroup(gspec, Endpoint_LEASED_IP) ` ,
278+ which collects distinct endpoint sequence numbers into a set and returns the set size. This is
279+ computed once per reservation and stored as the reservation's endpoint quantity.
280+
281+ - ✅ ** Counts as 1:** Three services all reference leased-IP endpoint ` #1 ` → requested = 1.
282+ - ✅ ** Counts as 2:** Group references leased-IP endpoints ` #1 ` and ` #2 ` → requested = 2.
283+
284+ ### AC19 — Leased IP availability gate (cluster-level, pre-engine)
285+ ** WHAT:** Before any node is examined, the provider checks it has enough free leased IPs for the
286+ whole deployment. This is a single cluster-wide pool — never per-node, never per-replica.
287+ ** HOW:** ` available = leased_ip.allocatable − leased_ip.allocated ` , where
288+ ` allocated = in-use + reserved ` . ` reserved ` counts IPs held by reservations that the IP operator
289+ has ** not yet confirmed** , so pending bids still consume availability. If ` requested > available ` ,
290+ the reservation is rejected with ** ` insufficient number of IPs ` ** and the per-node engine never
291+ runs.
292+
293+ - ✅ ** Match:** Pool of 10, 4 in use, 2 reserved → available 4; request 3 → passes; engine proceeds
294+ to fit CPU/memory/GPU/storage.
295+ - ❌ ** Reject (cluster-level):** Same pool, request 5 > available 4 → ` insufficient number of IPs ` ;
296+ no node is tried.
297+ - ❌ ** No double-claiming:** Reserved-but-unconfirmed IPs still count as allocated, so two
298+ concurrent requests cannot both be offered the same free IPs — the second sees lower availability.
299+
300+ ### AC20 — Provider without leased IP support
301+ ** WHAT:** A provider that has no IP operator configured cannot serve leased IPs at all. Any
302+ deployment that needs at least one leased IP is rejected, regardless of node capacity.
303+ ** HOW:** If the requested leased IP count is non-zero and the inventory service has no IP client,
304+ the reservation is rejected with ** ` no leased IPs available ` ** .
305+
306+ - ❌ ** Reject:** Request 1 leased IP against a provider with no IP operator → `no leased IPs
307+ available`, even on an otherwise empty, fully-capable cluster.
308+ - ✅ ** Unaffected:** Same provider, deployment requesting 0 leased IPs → IP check skipped (AC21).
309+
310+ ### AC21 — Zero leased IPs requested
311+ ** WHAT:** A deployment requesting no leased IPs skips the IP gate entirely and proceeds straight to
312+ the per-node engine (parallels AC8 for GPUs). The reservation's IPs are treated as already
313+ confirmed.
314+
315+ - ✅ ** Match:** Requested leased IPs = 0 → no IP check; placement depends only on CPU/memory/GPU/storage.
316+
261317---
262318
263319## Worked Example — Mixed GPU Cluster, 3 Replicas
@@ -288,6 +344,8 @@ Variations:
288344| -----------------------------| ---------------------------------------------------------------------------------------------------|
289345| ` ErrInsufficientCapacity ` | After exhausting all nodes, one or more replicas could not be placed; or a cluster-level resource (persistent pool, storage class, GPU constraint) blocks placement. |
290346| ` ErrGroupResourceMismatch ` | Defensive guardrail — a subsequent replica produced different adjusted resources or scheduler params than the first replica of the same service. Rare in practice because request freezing usually converts heterogeneity into capacity failures. |
347+ | ` no leased IPs available ` | A deployment needs ≥1 leased IP but the provider has no IP operator configured (AC20). Wraps the inventory reservation error. |
348+ | ` insufficient number of IPs ` | Requested leased IPs exceed ` leased_ip.allocatable − leased_ip.allocated ` (AC19). Cluster-level — no node is tried. Wraps the inventory reservation error. |
291349
292350---
293351
@@ -299,3 +357,10 @@ Variations:
299357- Attribute parsing: [ ParseGPUAttributes / ParseStorageAttributes] ( ../cluster/types/v1beta3/clients/inventory/metrics.go )
300358- Interface normalization: [ FilterGPUInterface] ( ../cluster/types/v1beta3/types.go#L160 )
301359- Test fixtures (4-node CPU example): [ client_test.go:609-704] ( ../cluster/kube/operators/clients/inventory/client_test.go#L609-L704 )
360+
361+ Leased IP (AC18–AC21), enforced in the inventory service, not the per-node engine:
362+
363+ - Leased IP gate: [ handleRequest] ( ../cluster/inventory.go#L457 )
364+ - Availability math: [ availableLeasedIPs / countReservedIPs / leasedIPStatus] ( ../cluster/inventory.go#L411-L444 )
365+ - Request counting: [ GetEndpointQuantityOfResourceGroup] ( ../cluster/util/endpoint_quantity.go#L8 )
366+ - Status reporting: ` provider.Inventory.LeasedIP ` (` leased_ip.{allocatable, allocated} ` in the provider status stream)
0 commit comments