`oneOf` schema-validator over-reports "must have required property" from non-matching variants — storyboard CI sees false negatives that local doesn't

[bokelley]

Context

When running the storyboard CLI (npx -y -p @adcp/client@5.21.1 adcp storyboard run) against a server that returns AdCP 3.0.1 get_products responses, validation fails in CI but passes locally — same code, same client version, same Python version.

The downstream Python SDK has confirmed the response shapes are 3.0.1-compliant (Python jsonschema validates them clean). Tracking issue with full investigation: adcontextprotocol/adcp-client-python#324.

Symptoms

Repeated stderr from @adcp/client/dist/lib/core/TaskExecutor.js:1119:

Schema validation failed for get_products: [
  "/products/2/name: must have required property 'name'",
  "/products/2/description: must have required property 'description'",
  "/products/2/publisher_properties: must have required property 'publisher_properties'",
  "/products/2/reporting_capabilities: must have required property 'reporting_capabilities'",
  "/products/2/format_ids/0/agent_url: must have required property 'agent_url'",
  ...
]

The dumped response body in the same log shows products[2] does have all those fields populated. Validator reports them missing anyway.

Likely root cause

The 3.0.1 Product schema is a oneOf discriminated union (per delivery_type / pricing_options[*].pricing_model). It looks like the client's Ajv setup iterates each oneOf variant and surfaces must have required property errors from variants that didn't match — even when one variant DID match and overall validation should pass.

Suggested fix: configure Ajv with allErrors: false for oneOf variant errors, OR collect + filter so only the best-matching-variant's residual errors surface, OR check if any variant validates and short-circuit before emitting the cross-variant error union.

Source under suspicion: @adcp/client/dist/lib/validation/index.js (Ajv setup) and dist/lib/core/TaskExecutor.js:1119 (validation-failure logger).

Reproduction

git clone https://github.com/adcontextprotocol/adcp-client-python
cd adcp-client-python
python3 -m venv .venv && .venv/bin/pip install -e .[dev]
ADCP_PORT=3001 .venv/bin/python examples/seller_agent.py &
npx -y -p @adcp/client@5.21.1 adcp storyboard run http://127.0.0.1:3001/mcp media_buy_seller --json --allow-http

Local: overall_status: passing (45/47 pass).
Fresh CI install: overall_status: partial, ~13 storyboards fail with the validation errors above.

The structural difference between passing products (indices 0, 1) and failing products (2-5 — injected via seed_product controller scenarios) is the pricing_option_id value (po-cpm-homepage vs cpm_standard) and product_id (hyphenated vs underscored). Neither has a schema-level pattern restriction visible in the cache, so the divergence likely originates in the validator's error-aggregation logic rather than in the schemas themselves.

Why this matters downstream

The Python SDK's storyboard CI job is continue-on-error: true because of this divergence — but it makes every PR (e.g., adcontextprotocol/adcp-client-python#323) ship with one red check that's purely reporter-side, masking any real future regression. A fix here un-masks the signal.

References

• Downstream tracking: adcontextprotocol/adcp-client-python#324
• Python-side regression anchor: adcontextprotocol/adcp-client-python#327 (confirmed all six static products validate clean against AdCP 3.0.1 via Python jsonschema)
• Source of the validation-failure log: @adcp/client/dist/lib/core/TaskExecutor.js:1119

[bokelley]

Triage

Classification: Bug — library (schema-validator) + conformance (storyboard CI false negatives)

Root cause confirmed. schema-loader.ts initialises Ajv with allErrors: true. When a oneOf validation fails, Ajv emits the top-level node error and one sub-error per property mismatch in every branch it evaluated. Those sub-branch errors surfaced as spurious "must have required property" failures in storyboard CI; they do not appear locally when NODE_ENV=production because defaultResponseMode() in client-hooks.ts downgrades to 'warn' in production (separate, related vector — distinct issue).

Fix: filterOneOfSubErrors() added to src/lib/validation/schema-validator.ts, applied in both validateRequest and validateResponse. Drops any AJV error whose schemaPath is a strict descendant of a failing oneOf node's path; always keeps oneOf node errors (including nested ones); deliberately excludes anyOf (partial-match semantics — sub-errors carry disambiguation signal).

Pre-PR expert reviews: code-reviewer ✅ · ad-tech-protocol-expert ✅

Draft PR: #1112 — ready for human review assignment before merge.

Out of scope here:

• NODE_ENV-sensitive defaultResponseMode() CI/local discrepancy → separate issue
• anyOf sub-error flooding in get_adcp_capabilities → separate scope

---

Generated by Claude Code

[bokelley]

Closing as not-a-bug — the validator was correct.

Root cause was downstream in adcp-client-python: examples/seller_agent.py::seed_product was appending fixture-shaped skeleton products to its PRODUCTS list without filling schema-required defaults. @adcp/client@5.21.1's validator was correctly flagging real missing fields (publisher_properties: minItems 1, format_ids[].agent_url, available_reporting_frequencies: minItems 1), not over-reporting from non-matching oneOf variants.

Fixed on the seller side: adcontextprotocol/adcp-client-python#321 (merged). After that landed, storyboard CI is fully green against the same @adcp/client@5.21.1 validator. Sorry for the false alarm — the original triage hypothesis (Ajv oneOf aggregation) didn't survive contact with the actual seller fixture shape.

Downstream tracking closed: adcontextprotocol/adcp-client-python#324