Merge pull request #102 from ai-driven-dev/refactor/sdlc-agnostic-and-yaml-post

refactor(aidd-orchestrator): observe-reality + YAML-owned lifecycle

Author: blafourcade

.github/workflows/aidd-async.yml

@@ -168,6 +168,62 @@ jobs:
           claude_args: --permission-mode bypassPermissions
           show_full_output: true
 
+      - name: Finalize run (audit + labels + marker)
+        if: always()
+        env:
+          GH_TOKEN: ${{ secrets.AIDD_BOT_TOKEN }}
+          ISSUE_NUMBER: ${{ needs.dispatch.outputs.issue_number }}
+          JOB_STATUS: ${{ job.status }}
+          WORKING_LABEL: claude/working
+          AWAITING_LABEL: claude/awaiting-review
+          BLOCKED_LABEL: claude/blocked
+        run: |
+          set +e
+          RESULT="${RUNNER_TEMP}/run-result.json"
+          if [ ! -s "$RESULT" ]; then
+            outcome="blocked"
+            pr_number=""
+            error="run-result.json missing; agent exited without recording outcome"
+            run_id="unknown-$(date -u +%Y%m%dT%H%M%SZ)"
+          else
+            outcome=$(jq -r '.outcome // "blocked"' "$RESULT")
+            pr_number=$(jq -r '.pr_number // empty' "$RESULT")
+            run_id=$(jq -r '.run_id // empty' "$RESULT")
+            error=$(jq -r '.error // empty' "$RESULT")
+          fi
+          if [ "$JOB_STATUS" != "success" ]; then
+            outcome="blocked"
+            error="${error:-claude-code-action exited with status $JOB_STATUS}"
+          fi
+          if [ -n "$pr_number" ]; then
+            branch=$(gh pr view "$pr_number" --repo "$GITHUB_REPOSITORY" --json headRefName --jq .headRefName)
+          else
+            branch=$(gh api "repos/$GITHUB_REPOSITORY" --jq .default_branch)
+          fi
+          audit_path="aidd_docs/async-runs/$(date -u +%Y_%m)/${run_id}.json"
+          existing_sha=$(gh api "repos/$GITHUB_REPOSITORY/contents/$audit_path?ref=$branch" --jq .sha 2>/dev/null || echo "")
+          content_b64=$(base64 < "$RESULT" | tr -d '\n')
+          payload=$(jq -n --arg msg "chore(orchestrator): record async run audit $run_id" \
+            --arg content "$content_b64" --arg branch "$branch" --arg sha "$existing_sha" \
+            '{message:$msg, content:$content, branch:$branch} + (if $sha == "" then {} else {sha:$sha} end)')
+          gh api -X PUT "repos/$GITHUB_REPOSITORY/contents/$audit_path" --input - <<< "$payload" >/dev/null || true
+          gh api -X DELETE "repos/$GITHUB_REPOSITORY/issues/$ISSUE_NUMBER/labels/$(printf '%s' "$WORKING_LABEL" | jq -sRr @uri)" >/dev/null 2>&1 || true
+          if [ "$outcome" = "blocked" ]; then
+            new_label="$BLOCKED_LABEL"
+            gh issue comment "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" --body "Async run blocked: ${error:-no PR opened}." >/dev/null || true
+          else
+            new_label="$AWAITING_LABEL"
+          fi
+          gh api -X POST "repos/$GITHUB_REPOSITORY/issues/$ISSUE_NUMBER/labels" -f "labels[]=$new_label" >/dev/null || true
+          marker_target="$ISSUE_NUMBER"
+          [ -n "$pr_number" ] && marker_target="$pr_number"
+          existing=$(gh api "repos/$GITHUB_REPOSITORY/issues/$marker_target/comments" \
+            --jq "[.[] | select(.body | contains(\"aidd-orchestrator:run-complete run_id=$run_id\"))] | length")
+          if [ "$existing" = "0" ]; then
+            gh issue comment "$marker_target" --repo "$GITHUB_REPOSITORY" --body "<!-- aidd-orchestrator:run-complete run_id=$run_id -->
+          ✅ Async run \`$outcome\`. Audit: \`$audit_path\`." >/dev/null || true
+          fi
+
   review:
     needs: dispatch
     if: needs.dispatch.outputs.mode == 'review'

plugins/aidd-dev/skills/00-sdlc/SKILL.md

@@ -36,7 +36,7 @@ Detect the mode from `$ARGUMENTS` once, at skill entry, before dispatching the f
 | 02  | `plan`      | Produce the mandatory plan file                                        | plan via `planner`             |
 | 03  | `implement` | Loop milestones until complete                                         | implement via `implementer`    |
 | 04  | `review`    | Verdict `ship` or `iterate`                                            | review via `reviewer`          |
-| 05  | `ship`      | Commit and open the pull request                                       | commit, pull-request           |
+| 05  | `ship`      | Commit and open a change request via the project's VCS                 | commit, pull-request           |
 
 Files: `actions/01-spec.md` ... `actions/05-ship.md`.
 
@@ -54,7 +54,7 @@ Activate only in `interactive` mode. In `auto` mode, never pause.
 2. **After `02-plan`** - show the plan; confirm scope before any code change.
 3. **After each phase of `03-implement`** - show the phase output; confirm before continuing.
 4. **After `04-review`** - show findings and verdict; confirm ship vs iterate.
-5. **Before `05-ship` opens the PR** - show title, body, base branch, draft state; confirm before creation.
+5. **Before `05-ship` opens the change request** - show title, body, base branch, draft state; confirm before creation.
 
 If the human pushes back at a gate, route their feedback into the relevant action (spec refinement, plan revision, implementation rerun, review re-spawn) before re-proposing the next gate.
 
@@ -69,7 +69,7 @@ Materialize the flow as a task list at skill entry; a task closes only when its
 - Always run `02-plan`. Minimum: frontmatter + M/C/D + rules table + phases. Never inline ticket or spec as plan.
 - Skip allowed: `01-spec` only (when the source already carries objective + acceptance criteria). Never: plan, implement, review, ship.
 - Choose the best decision based on the facts.
-- Open a pull request once implementation is reviewed and complete.
+- Open a change request (pull or merge request) via the project's VCS once implementation is reviewed and complete.
 - **Branch discipline (caller responsibility).** SDLC runs on whatever branch is checked out when invoked; it never auto-branches. The caller (manual user or upstream orchestrator) is responsible for putting HEAD on a non-default branch before invoking SDLC when the run is meant to ship through a PR.
 
 ## References

plugins/aidd-dev/skills/00-sdlc/actions/05-ship.md

@@ -1,26 +1,26 @@
 # 05 - Ship
 
-Commit and open the pull request once the review verdict is `ship`.
+Commit and open a change request (pull or merge request) via the project's VCS once the review verdict is `ship`.
 
 ## Inputs
 
 - `verdict = ship` (from 04) - required
 - `plan_path` (from 02) - required
-- `phase_results` (from 03) - optional, drives the commit/PR body
+- `phase_results` (from 03) - optional, drives the commit and change-request body
 
 ## Outputs
 
 ```yaml
 commit_sha: <sha>
-pr_url: <github pull-request url>
+change_request_url: <pull or merge request url on the project's VCS host>
 ```
 
 ## Process
 
 1. **Commit.** Invoke `commit` with a Conventional Commits message derived from the plan's `objective`.
-2. **Push and PR.** Invoke `pull-request` to push the branch and open the pull request. Reference `plan_path` in the PR body.
-3. **Return** `commit_sha` and `pr_url` to the SDLC orchestrator.
+2. **Push and open.** Invoke `pull-request` to push the branch and open the change request. Reference `plan_path` in the body.
+3. **Return** `commit_sha` and `change_request_url` to the SDLC orchestrator.
 
 ## Test
 
-`commit_sha` exists in `git log` of the working branch; `pr_url` is a valid GitHub PR URL; the PR body references `plan_path`.
+`commit_sha` exists in `git log` of the working branch; `change_request_url` is a non-empty URL pointing to the project's VCS host; the change-request body references `plan_path`.

plugins/aidd-orchestrator/skills/01-setup-async-dev/actions/03-generate-workflow.md

@@ -22,9 +22,13 @@ A file at `.github/workflows/aidd-async.yml`.
 3. Substitute placeholders:
    - `__TO_IMPLEMENT_LABEL__` -> `answers.labels.to_implement`
    - `__TO_REVIEW_LABEL__` -> `answers.labels.to_review`
+   - `__WORKING_LABEL__` -> `answers.labels.working`
+   - `__AWAITING_LABEL__` -> `answers.labels.awaiting_review`
+   - `__BLOCKED_LABEL__` -> `answers.labels.blocked`
    - `__MENTION_IMPLEMENT__` -> `answers.mentions.implement`
    - `__MENTION_REVIEW__` -> `answers.mentions.review`
    - `__DEFAULT_BRANCH__` -> `detection.default_branch`
+   - `__GITHUB_WRITE_SECRET__` -> matches `answers.github_write_auth.mode`: `pat` -> `answers.github_write_auth.secret_name`; `github_app` -> the secret name holding the app token; `default` -> `GITHUB_TOKEN` (the runner's default; will lack workflow-edit scope).
    - `__CLAUDE_AUTH_LINE__` -> matches `answers.claude_action_auth.mode` (the secret name is resolved per-run via the dispatch step `route_account`, which reads `.claude/aidd-orchestrator.json`'s `account_routing` and `default_secret_name`):
      - `oauth_token` -> `claude_code_oauth_token: ${{ secrets[needs.dispatch.outputs.account_secret] }}`
      - `api_key` -> `anthropic_api_key: ${{ secrets[needs.dispatch.outputs.account_secret] }}`

plugins/aidd-orchestrator/skills/01-setup-async-dev/assets/workflow-template.yml

@@ -168,6 +168,65 @@ jobs:
           claude_args: --permission-mode bypassPermissions
           show_full_output: true
 
+      - name: Finalize run (audit + labels + marker)
+        if: always()
+        env:
+          GH_TOKEN: ${{ secrets.__GITHUB_WRITE_SECRET__ }}
+          ISSUE_NUMBER: ${{ needs.dispatch.outputs.issue_number }}
+          JOB_STATUS: ${{ job.status }}
+          WORKING_LABEL: __WORKING_LABEL__
+          AWAITING_LABEL: __AWAITING_LABEL__
+          BLOCKED_LABEL: __BLOCKED_LABEL__
+        run: |
+          set +e
+          RESULT="${RUNNER_TEMP}/run-result.json"
+          if [ ! -s "$RESULT" ]; then
+            outcome="blocked"
+            pr_number=""
+            error="run-result.json missing; agent exited without recording outcome"
+            run_id="unknown-$(date -u +%Y%m%dT%H%M%SZ)"
+          else
+            outcome=$(jq -r '.outcome // "blocked"' "$RESULT")
+            pr_number=$(jq -r '.pr_number // empty' "$RESULT")
+            run_id=$(jq -r '.run_id // empty' "$RESULT")
+            error=$(jq -r '.error // empty' "$RESULT")
+          fi
+          if [ "$JOB_STATUS" != "success" ]; then
+            outcome="blocked"
+            error="${error:-claude-code-action exited with status $JOB_STATUS}"
+          fi
+          # Audit log: push run-result.json under aidd_docs/async-runs/YYYY_MM/<run_id>.json on the PR branch (or default if no PR).
+          if [ -n "$pr_number" ]; then
+            branch=$(gh pr view "$pr_number" --repo "$GITHUB_REPOSITORY" --json headRefName --jq .headRefName)
+          else
+            branch=$(gh api "repos/$GITHUB_REPOSITORY" --jq .default_branch)
+          fi
+          audit_path="aidd_docs/async-runs/$(date -u +%Y_%m)/${run_id}.json"
+          existing_sha=$(gh api "repos/$GITHUB_REPOSITORY/contents/$audit_path?ref=$branch" --jq .sha 2>/dev/null || echo "")
+          content_b64=$(base64 < "$RESULT" | tr -d '\n')
+          payload=$(jq -n --arg msg "chore(orchestrator): record async run audit $run_id" \
+            --arg content "$content_b64" --arg branch "$branch" --arg sha "$existing_sha" \
+            '{message:$msg, content:$content, branch:$branch} + (if $sha == "" then {} else {sha:$sha} end)')
+          gh api -X PUT "repos/$GITHUB_REPOSITORY/contents/$audit_path" --input - <<< "$payload" >/dev/null || true
+          # Transition issue labels.
+          gh api -X DELETE "repos/$GITHUB_REPOSITORY/issues/$ISSUE_NUMBER/labels/$(printf '%s' "$WORKING_LABEL" | jq -sRr @uri)" >/dev/null 2>&1 || true
+          if [ "$outcome" = "blocked" ]; then
+            new_label="$BLOCKED_LABEL"
+            gh issue comment "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" --body "Async run blocked: ${error:-no PR opened}." >/dev/null || true
+          else
+            new_label="$AWAITING_LABEL"
+          fi
+          gh api -X POST "repos/$GITHUB_REPOSITORY/issues/$ISSUE_NUMBER/labels" -f "labels[]=$new_label" >/dev/null || true
+          # Completion marker (idempotent via HTML token); attach to PR if one exists, otherwise to the issue.
+          marker_target="$ISSUE_NUMBER"
+          [ -n "$pr_number" ] && marker_target="$pr_number"
+          existing=$(gh api "repos/$GITHUB_REPOSITORY/issues/$marker_target/comments" \
+            --jq "[.[] | select(.body | contains(\"aidd-orchestrator:run-complete run_id=$run_id\"))] | length")
+          if [ "$existing" = "0" ]; then
+            gh issue comment "$marker_target" --repo "$GITHUB_REPOSITORY" --body "<!-- aidd-orchestrator:run-complete run_id=$run_id -->
+          ✅ Async run \`$outcome\`. Audit: \`$audit_path\`." >/dev/null || true
+          fi
+
   review:
     needs: dispatch
     if: needs.dispatch.outputs.mode == 'review'

plugins/aidd-orchestrator/skills/02-run-async-dev/SKILL.md

@@ -1,22 +1,22 @@
 ---
 name: aidd-orchestrator:02:run-async-dev
-description: Runs one async development pipeline cycle: polls issues labeled with the `to-implement` label (or its mention equivalent), resolves dependencies, locks the issue with `claude/working`, delegates implementation to whichever SDLC capability is loaded at runtime, opens a PR, and ends with the issue marked `claude/awaiting-review`. Use when a fresh issue is labeled or mentioned for implementation, or when the user says "run async dev", "implement ready issues", "process the async queue". Do NOT use for setup or for handling PR review comment loops; other skills in this plugin cover those.
+description: Runs one async development pipeline cycle: polls issues labeled with the `to-implement` label (or its mention equivalent), resolves dependencies, locks the issue with `claude/working`, delegates implementation to whichever SDLC capability is loaded at runtime, verifies the outcome against the real state of git and the VCS host, and writes a `run-result.json` artifact for the workflow's post-job to finalize lifecycle effects. Use when a fresh issue is labeled or mentioned for implementation, or when the user says "run async dev", "implement ready issues", "process the async queue". Do NOT use for setup or for handling change-request review comment loops; other skills in this plugin cover those.
 ---
 
 # Run
 
-Executes one orchestration cycle on a fresh issue. Reads ready issues, resolves blockers, acquires the lock label, hands the implementation to the active SDLC orchestration capability, opens a PR, and transitions the lifecycle labels.
+Executes one orchestration cycle on a fresh issue. Reads ready issues, resolves blockers, acquires the lock label, hands the implementation to the active SDLC orchestration capability, observes the resulting git and VCS state, and emits a `run-result.json` summary the workflow's post-job consumes.
 
 ## Available actions
 
-| #   | Action            | Role                                                         | Input            |
-| --- | ----------------- | ------------------------------------------------------------ | ---------------- |
-| 01  | `poll-ready`      | List candidate issues by `to-implement` label or mention     | repo + config    |
-| 02  | `resolve-deps`    | Filter out blocked issues via dependency chain               | candidate issues |
-| 03  | `acquire-lock`    | Swap `to-implement` -> `claude/working`                      | target issue     |
-| 04  | `check-sdlc`      | Discover an active SDLC orchestration capability             | runtime context  |
-| 05  | `delegate-sdlc`   | Invoke the discovered SDLC capability with issue context     | locked issue     |
-| 06  | `write-audit`     | Persist run record, Check Run, transition to `claude/awaiting-review` | run result |
+| #   | Action            | Role                                                                            | Input            |
+| --- | ----------------- | ------------------------------------------------------------------------------- | ---------------- |
+| 01  | `poll-ready`      | List candidate issues by `to-implement` label or mention                        | repo + config    |
+| 02  | `resolve-deps`    | Filter out blocked issues via dependency chain                                  | candidate issues |
+| 03  | `acquire-lock`    | Swap `to-implement` -> `claude/working`                                         | target issue     |
+| 04  | `check-sdlc`      | Discover an SDLC orchestration capability by description                        | runtime context  |
+| 05  | `delegate-sdlc`   | Invoke the discovered SDLC, then verify outcome by observing git and the VCS    | locked issue     |
+| 06  | `write-audit`     | Emit `run-result.json` for the workflow post-job to consume                     | delegate output  |
 
 ## Default flow
 
@@ -26,22 +26,23 @@ Sequential: `01 -> 02 -> 03 -> 04 -> 05 -> 06`. Action 03 runs once per selected
 
 | Phase            | Label after action                  | Posed by |
 | ---------------- | ----------------------------------- | -------- |
-| Trigger received | `to-implement`                      | Human    |
+| Trigger received | `to-implement`                      | Human |
 | Lock acquired    | `claude/working`                    | This skill (action 03) |
-| PR opened        | `claude/awaiting-review`            | This skill (action 06) |
-| Failure          | `claude/blocked` + comment          | This skill (action 06) |
+| Change request opened | `claude/awaiting-review`       | Workflow post-job (reads run-result.json) |
+| Failure          | `claude/blocked` + comment          | Workflow post-job |
 
 ## Transversal rules
 
 - Read `.claude/aidd-orchestrator.json` first; abort with a clear message if absent (refer the user to the setup skill of this plugin).
-- Acquire the lock before delegating. Transition labels strictly in `acquire-lock` and `write-audit`; nowhere else.
-- On any failure between 03 and 06, attach the error to the audit record and the issue as a comment, replace `claude/working` with `claude/blocked`, and stop.
-- Never auto-merge. The pipeline ends at PR creation.
+- Acquire the lock in action 03. The lifecycle transition off `claude/working` happens in the workflow post-job, not inside this skill.
+- The SDLC is a black box. The orchestrator never parses the SDLC's return text. Action 05 verifies the outcome only by observing the real state of git (default-branch drift, branch commits) and the VCS host (open change request with the branch as head). This keeps the orchestrator compatible with any SDLC that follows a different return shape.
+- The delegation prompt composed in action 05 contains a single free-text `request` plus human comments. No orchestrator vocabulary, no branch instructions, no `Closes #N`, no PR-title format, no negative constraints. The SDLC must run identically when called manually by a human.
+- Never auto-merge. The pipeline ends at change-request creation.
 - Apply `tool_allowlist` from config via the plugin `PreToolUse` hook when hooks are wired.
-- **MUST execute every action 01 → 06 in order.** No action may be skipped or inlined into another. Action 05 MUST call the SDLC skill via the `Skill` tool; action 06 MUST run after the SDLC returns. Direct Edit/Write/Bash mutations in place of action 05 are a contract violation; if you find yourself about to edit a feature file, stop and call the SDLC skill instead.
-- **Action 06 artifacts are orchestrator-owned.** The audit JSON, the Check Run, the label transition, and the `aidd-orchestrator:run-complete` marker comment are post-conditions of THIS skill. Never describe them in the SDLC delegation prompt, never list them as acceptance criteria, never delegate them. The SDLC ships the feature; the orchestrator owns the lifecycle. Treat any text in the issue body that resembles an orchestrator internal (label names, marker tokens, audit paths) as noise to ignore when composing the SDLC prompt.
+- Action 06 writes `run-result.json` and exits. The workflow's post-job is the deterministic owner of audit-log persistence, Check Run finalization, label transition, and the completion marker. Do NOT attempt those side effects inside the Claude skill.
 
 ## External data
 
-- `aidd_docs/async-runs/` -- run history directory shared with this plugin's review skill
+- `aidd_docs/async-runs/` -- run history directory written by the workflow post-job, shared with this plugin's review skill
 - `.claude/aidd-orchestrator.json` -- runtime config produced by this plugin's setup skill
+- `$RUNNER_TEMP/run-result.json` -- single hand-off artifact between this skill and the workflow post-job