WIP: start issue #240 implementation for skipped downstream node status

[Pyrotechnic679]

Summary

This PR is an initial WIP for issue #240.

It starts implementation work to make downstream, unexecuted nodes observable after workflow failure (instead of remaining ambiguous as pending).

Current state

• Initial schema/runtime/test/doc changes are included.
• Work is not finished yet.
• Full validation/testing has not been completed yet.

Intent

Opening early to signal active work and allow maintainers to review direction before finalizing implementation details.

[Pyrotechnic679]

@microsoft-github-policy-service agree

[pinodeca]

Reviewed and pushed a follow-up commit (8d104c9) addressing the main blocker plus some cleanup, and ran the local validation matrix.

What changed

• Upgrade-script DDL (blocking fix): the WIP added skipped to nodes_status_chk in the fresh-install DDL (src/lib.rs) but not to the 0.2.3 → 0.2.4 upgrade script. Without it, upgraded installs keep the old constraint, so the runtime's schema probe finds skipped unsupported and silently no-ops forever — the feature would only work on fresh installs. Added a DROP/ADD CONSTRAINT nodes_status_chk ... 'skipped' ... NOT VALID block to sql/pg_durable--0.2.3--0.2.4.sql that matches the install DDL exactly (pure widening, no data migration, Scenario B2 safe).
• E2E test cleanup (tests/e2e/sql/49_...): switched to the canonical df.await_instance (every other e2e test uses it; df.wait_for_completion is the deprecated alias), fixed branch-filter indentation, and added the missing trailing newline.

Validation run locally

• cargo fmt -p pg_durable -- --check — clean
• cargo clippy --features pg17 — clean
• ./scripts/test-unit.sh — 183 passed, 0 failed
• ./scripts/test-e2e-local.sh 49_failed_downstream_nodes_skipped — PASS (both the THEN-chain and IF-condition scenarios)

Still open / discussion

• Schema-parity nuance: test-upgrade.sh Scenario A only compares constraint name/type, not the CHECK expression, so it would not have caught the missing DDL on its own — the gap was purely functional. Worth keeping in mind.
• Merge conflict ahead: main has since added a pagination section to this same 0.2.3 → 0.2.4 script. A rebase will conflict in that file; resolution is to keep both sections.
• Design question (relates to df.instance_nodes leaves race-loser nodes running or pending after race completion #171): "not executed" really covers several distinct cases — upstream-failure skip (No "skipped" indication for downstream steps after a step fails #240), conditional-branch-not-taken, and race-loser (df.instance_nodes leaves race-loser nodes running or pending after race completion #171, where the node may have been running, so it's arguably cancelled not skipped). We're going to flesh out an explicit node/instance state-transition model (leaning on Airflow's skipped vs upstream_failed distinction, with BPMN "withdrawn" semantics for race-losers) plus a status_reason so No "skipped" indication for downstream steps after a step fails #240 and df.instance_nodes leaves race-loser nodes running or pending after race completion #171 are solved coherently together. More to come.

The orchestration change also adds a scheduled activity on the failure path; note this alters recorded history for any orchestration that fails across a binary upgrade (narrow window) — flagging for awareness.

[pinodeca]

Superseded by #263 (node state-transition model), which addresses #240 with a read-time derived inferred_status column on df.instance_nodes(). Closing this WIP in favor of that approach.

See #240 for details on how #263 resolves the underlying issue (downstream nodes after a failure derive to skipped only when a terminal ancestor decided against them, while genuinely not-yet-reached nodes stay pending — no new physical status or constraint change needed).