santisation

Author: eldiablolives

src/zingcore/2.5/MIGRATION.md

@@ -184,7 +184,11 @@ int got = read_frame_wait(loop, aio, watch_id, ack, sizeof(ack));
 uint8_t done[65536];
 got = read_frame_wait(loop, aio, watch_id, done, sizeof(done));
 
-// See abi/FILE_AIO_PROTOCOL.md for normative framing and parsing recipes.
+// Backpressure note:
+// - file/aio has a bounded submission queue.
+// - If you receive an immediate ERROR with msg "queue full", stop submitting new jobs,
+//   drain completions/events, and retry after observing progress.
+// See abi/FILE_AIO_PROTOCOL.md for normative framing, parsing recipes, and the backpressure contract.
 ```
 
 Sandboxing is still via env vars (`ZI_FS_ROOT`, `ZI_NET_ALLOW`), but the path is in params.

src/zingcore/2.5/Makefile

@@ -3,6 +3,9 @@ CFLAGS  ?= -std=c11 -O2 -Wall -Wextra -Wpedantic
 AR      ?= ar
 ARFLAGS ?= rcs
 
+# Auto header dependencies (prevents stale objects when headers change).
+DEPFLAGS ?= -MMD -MP
+
 BUILD       ?= build
 DIST        ?= dist
 DIST_DEBUG  ?= $(DIST)/debug
@@ -69,6 +72,8 @@ OBJ := \
 	$(BUILD)/zi_hopabi25.o \
 	$(BUILD)/zi_telemetry.o
 
+TESTOBJ = $(addprefix $(BUILD)/,$(addsuffix .o,$(TESTS)))
+
 TESTS := test_caps test_async_registry test_zingcore25_api test_problem test_telemetry_jsonl test_sysabi25_min_core test_sysabi25_ctl_caps_list test_sysabi25_file_aio_cap test_sysabi25_tcp_cap test_sysabi25_tcp_loop_connect_cap test_sysabi25_http_cap test_sysabi25_http_loop_cap test_sysabi25_argv_cap test_sysabi25_env_cap test_sysabi25_hopper_cap test_sysabi25_event_bus_cap test_sysabi25_sys_info_cap test_sysabi25_sys_loop_cap test_bus_rpc_v1 test_hopabi25_basic
 
 EXAMPLES := stdio_caps_demo all_caps_demo hopabi_guest_demo
@@ -87,23 +92,25 @@ dist-dirs:
 
 $(BUILD)/%.o: $(SRCDIR)/%.c | dirs
 	mkdir -p $(dir $@)
-	$(CC) $(CFLAGS) -I$(INCDIR) -c $< -o $@
+	$(CC) $(CFLAGS) $(DEPFLAGS) -I$(INCDIR) -MF $(@:.o=.d) -MT $@ -c $< -o $@
 
 $(BUILD)/test_%.o: $(TESTDIR)/test_%.c | dirs
 	mkdir -p $(dir $@)
-	$(CC) $(CFLAGS) -I$(INCDIR) -c $< -o $@
+	$(CC) $(CFLAGS) $(DEPFLAGS) -I$(INCDIR) -MF $(@:.o=.d) -MT $@ -c $< -o $@
 
 $(LIB): $(OBJ)
 	$(AR) $(ARFLAGS) $@ $^
 
-test: $(LIB) $(addprefix $(BUILD)/,$(addsuffix .o,$(TESTS)))
+test: $(LIB) $(TESTOBJ)
 	@set -e; \
 	for t in $(TESTS); do \
 	  bin="$(BUILD)/$$t"; \
 	  $(CC) $(CFLAGS) "$(BUILD)/$$t.o" $(LIB) -o "$$bin"; \
 	  "$$bin"; \
 	done
 
+-include $(OBJ:.o=.d) $(TESTOBJ:.o=.d)
+
 examples: $(LIB)
 	@set -e; \
 	for e in $(EXAMPLES); do \

src/zingcore/2.5/abi/FILE_AIO_PROTOCOL.md

@@ -250,6 +250,74 @@ OK payload:
 - Submission errors (bad payload, out-of-bounds pointers, queue full) are returned as an immediate ERROR response.
 - Execution errors (open/read/write failures, unknown `file_id`, sandbox denial) are returned as an `EV_DONE` ERROR frame.
 
+## Backpressure (guest-level contract)
+
+`file/aio` has a bounded internal submission queue. When the queue is full, the host MUST reject additional job submissions.
+
+### What the guest observes
+
+- The job submission itself is still a normal ZCL1 request written to the `file/aio` handle.
+- If the queue is full, the host MUST return an **immediate** ZCL1 ERROR response for that `(op, rid)`.
+
+In the zingcore 2.5 reference implementation, this error uses:
+
+- `trace_len/trace` = `"file.aio"`
+- `msg_len/msg` = `"queue full"`
+
+These strings are not a general ZCL1 standard, but they are the stable shape used by zingcore 2.5 today.
+Guests SHOULD treat this condition as a retryable backpressure signal.
+
+### How to wait (no “queue has space” readiness)
+
+The intended guest strategy is:
+
+- Guests SHOULD keep a bounded number of in-flight jobs.
+- On a `queue full` submission error, guests SHOULD stop submitting new jobs for that queue.
+- Guests SHOULD drain completions/events from the same `file/aio` handle.
+- Guests SHOULD use `sys/loop` to block until the queue becomes **readable** (meaning: there are frames to read), then read and process them.
+- After observing progress (typically one or more `EV_DONE` frames), guests SHOULD retry the submission.
+
+This avoids busy-looping while still fitting the single-wait model.
+
+### Submission writable readiness (queue has space)
+
+Guests MAY also watch the `file/aio` handle for `writable` readiness to wait for submission space.
+
+Contract:
+
+- When the submission queue is full, the host SHOULD report the handle as not-writable.
+- When the queue transitions from full to having at least one free slot, the host SHOULD report the handle as writable.
+
+This is a readiness hint and is still subject to races:
+
+- A guest might observe `writable` and still lose a race and get `queue full`.
+- Guests MUST handle `queue full` as the authoritative signal.
+
+Intended guest pattern:
+
+- On `queue full`, WATCH `writable` and `POLL`.
+- Retry submission on READY(writable).
+
+Pseudocode sketch:
+
+```c
+for (;;) {
+	submit_job(aio, rid, ...);
+	frame ack = read_frame_wait(loop, aio, watch_id);
+
+	if (ack.status == ERROR && ack.trace == "file.aio" && ack.msg == "queue full") {
+		// Backpressure: wait for progress (completions) before retrying.
+		do {
+			frame f = read_frame_wait(loop, aio, watch_id);
+			// process f (ACK/EV_DONE/ERROR)
+		} while (!saw_ev_done_for_any_job());
+		continue; // retry submit
+	}
+
+	break;
+}
+```
+
 ## Notes
 
 - This version returns READ data inline in completion frames. Large reads may be truncated by the runtime.

src/zingcore/2.5/abi/HTTP_PROTOCOL.md

@@ -108,9 +108,15 @@ If an environment variable is unset or invalid, implementations SHOULD fall back
 ### Default transfer semantics (v1)
 
 Implementations SHOULD default to conservative semantics:
-- Reject `Transfer-Encoding: chunked` requests (treat as `ZI_E_INVALID`) unless explicitly supported.
+- If `Transfer-Encoding: chunked` is not supported, reject such requests (treat as `ZI_E_INVALID`).
+- If `Transfer-Encoding: chunked` is supported, the runtime MUST decode the chunked framing and expose the decoded bytes as a request body `STREAM` (i.e., `body_kind=2`).
 - Close the connection after completing a response, unless keep-alive is explicitly supported.
 
+Notes (reference runtime behavior):
+- Server-side incoming requests with `Transfer-Encoding: chunked` are supported and are exposed as `body_kind=STREAM`.
+- Multipart (Option A) advertisement is conservative; runtimes MAY choose not to advertise `MULTIPART` for chunked bodies.
+- Client-side `FETCH` supports `Transfer-Encoding: chunked` responses by decoding and exposing the decoded bytes as `body_kind=STREAM`.
+
 ## ZCL1 Operations
 
 All frames use ZCL1 v1 (see `ZCL1_PROTOCOL.md`).