chore: "no fail fast" test mode (#367)

KSXGitHub · claude · web-flow · commit ad1ee65c0f4f · 2026-03-23T09:00:31.000+07:00
* feat(test): add `PDU_NO_FAIL_FAST` mode to `test.sh` Add a no-fail-fast mode activated by `PDU_NO_FAIL_FAST=true` that continues running all checks instead of stopping at the first failure, then reports a summary at the end. This helps AI environments that may fail on some checks but need to see which ones pass. Update AI instruction templates to document the new flag. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * fix(test): move no-fail-fast logic into `run_if` and use single quotes https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * fix(test): add comment and error message in no-fail-fast mode Add a comment explaining why a temporary file is used instead of a variable for the failure marker, and log the failed command with its exit code when no-fail-fast mode catches a failure. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * docs(ai): remove conditional preamble from no-fail-fast instruction https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * docs(ai): remove conditional preamble from no-fail-fast instruction in template Update the source template and regenerate all AI instruction files. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * fix(test): address review feedback on test.sh - Use mktemp -d with a marker file inside to avoid path reuse - Quote $failure_marker in the final check for consistency - Use $* instead of $@ in error message for unambiguous logging https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * fix(test): use EXIT trap to clean up temporary directory Replace explicit rm -rf calls with a trap to ensure cleanup happens even when errexit terminates the script early in fail-fast mode. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe * fix(test): improve error logging for failed commands in no-fail-fast mode Use `printf '%q'` with `"$@"` to preserve argument quoting and capture exit status into a variable for reliability. https://claude.ai/code/session_01VHRbHsz8VJFdNioSj6dyxe --------- Co-authored-by: Claude <noreply@anthropic.com>
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
@@ -17,3 +17,4 @@ Read and follow the CONTRIBUTING.md file in this repository for all code style c
 - Install toolchain before running tests: `rustup toolchain install "$(< rust-toolchain)" && rustup component add --toolchain "$(< rust-toolchain)" rustfmt clippy`
 - Run `FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh` to validate changes. If a test fails with a hint about `RUSTFLAGS` and `--cfg pdu_test_skip_*`, follow the hint and rerun with the suggested flags.
 - **ALWAYS run the full test suite** (`FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh`) before committing, regardless of how trivial the change seems — this includes documentation-only changes, comment edits, config changes, and refactors. The test suite checks formatting, linting, building, tests, and docs across multiple feature combinations; any type of change can break any of these checks.
+- Set `PDU_NO_FAIL_FAST=true` to run all checks instead of stopping at the first failure — this lets you see which checks pass and which fail
diff --git a/AGENTS.md b/AGENTS.md
@@ -17,3 +17,4 @@ Read and follow the CONTRIBUTING.md file in this repository for all code style c
 - Install toolchain before running tests: `rustup toolchain install "$(< rust-toolchain)" && rustup component add --toolchain "$(< rust-toolchain)" rustfmt clippy`
 - Run `FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh` to validate changes. If a test fails with a hint about `RUSTFLAGS` and `--cfg pdu_test_skip_*`, follow the hint and rerun with the suggested flags.
 - **ALWAYS run the full test suite** (`FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh`) before committing, regardless of how trivial the change seems — this includes documentation-only changes, comment edits, config changes, and refactors. The test suite checks formatting, linting, building, tests, and docs across multiple feature combinations; any type of change can break any of these checks.
+- Set `PDU_NO_FAIL_FAST=true` to run all checks instead of stopping at the first failure — this lets you see which checks pass and which fail
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -17,4 +17,5 @@ Read and follow the CONTRIBUTING.md file in this repository for all code style c
 - Install toolchain before running tests: `rustup toolchain install "$(< rust-toolchain)" && rustup component add --toolchain "$(< rust-toolchain)" rustfmt clippy`
 - Run `FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh` to validate changes. If a test fails with a hint about `RUSTFLAGS` and `--cfg pdu_test_skip_*`, follow the hint and rerun with the suggested flags.
 - **ALWAYS run the full test suite** (`FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh`) before committing, regardless of how trivial the change seems — this includes documentation-only changes, comment edits, config changes, and refactors. The test suite checks formatting, linting, building, tests, and docs across multiple feature combinations; any type of change can break any of these checks.
+- Set `PDU_NO_FAIL_FAST=true` to run all checks instead of stopping at the first failure — this lets you see which checks pass and which fail
 - `gh` (GitHub CLI) is not installed — do not attempt to use it
diff --git a/template/ai-instructions/shared.md b/template/ai-instructions/shared.md
@@ -17,3 +17,4 @@ Read and follow the CONTRIBUTING.md file in this repository for all code style c
 - Install toolchain before running tests: `rustup toolchain install "$(< rust-toolchain)" && rustup component add --toolchain "$(< rust-toolchain)" rustfmt clippy`
 - Run `FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh` to validate changes. If a test fails with a hint about `RUSTFLAGS` and `--cfg pdu_test_skip_*`, follow the hint and rerun with the suggested flags.
 - **ALWAYS run the full test suite** (`FMT=true LINT=true BUILD=true TEST=true DOC=true ./test.sh`) before committing, regardless of how trivial the change seems — this includes documentation-only changes, comment edits, config changes, and refactors. The test suite checks formatting, linting, building, tests, and docs across multiple feature combinations; any type of change can break any of these checks.
+- Set `PDU_NO_FAIL_FAST=true` to run all checks instead of stopping at the first failure — this lets you see which checks pass and which fail
diff --git a/test.sh b/test.sh
@@ -1,6 +1,22 @@
 #! /bin/bash
 set -o errexit -o pipefail -o nounset
 
+# Validate PDU_NO_FAIL_FAST
+no_fail_fast="${PDU_NO_FAIL_FAST:-false}"
+case "$no_fail_fast" in
+true | false) ;;
+*)
+  echo "error: Invalid value for PDU_NO_FAIL_FAST: $no_fail_fast (expected 'true' or 'false')" >&2
+  exit 1
+  ;;
+esac
+
+# A temporary file is used instead of a variable because run_if and unit are
+# subshells, so variable assignments inside them don't propagate to the parent.
+failure_dir=$(mktemp -d)
+trap 'rm -rf "$failure_dir"' EXIT
+failure_marker="$failure_dir/failed"
+
 run() (
   echo >&2
   echo "exec> $*" >&2
@@ -16,7 +32,19 @@ run_if() (
   condition="$1"
   shift
   case "$condition" in
-  true) run "$@" ;;
+  true)
+    if [[ $no_fail_fast == 'true' ]]; then
+      run "$@" || {
+        exit_status=$?
+        printf 'error: Command failed with exit code %d: ' "$exit_status" >&2
+        printf '%q ' "$@" >&2
+        printf '\n' >&2
+        touch "$failure_marker"
+      }
+    else
+      run "$@"
+    fi
+    ;;
   false) skip "$@" ;;
   *)
     echo "error: Invalid condition: $condition" >&2
@@ -39,3 +67,9 @@ unit --all-features "$@"
 unit --features cli "$@"
 unit --features cli-completions "$@"
 unit --features ai-instructions "$@"
+
+if [[ -f "$failure_marker" ]]; then
+  echo >&2
+  echo 'error: Some checks have failed. Review the output above for details.' >&2
+  exit 1
+fi
