[PHASE 4] Create CI Error Diagnosis Skill

[ashleyshaw]

Phase 4b: CI Error Diagnosis Skill (Future Enhancement)

Create a new skill to automatically diagnose and document pre-existing CI failures (STEP 11 enhancement).

🎯 Objectives

• Automatically test failure on develop branch
• Extract failure logs and error messages
• Suggest root cause categories
• Recommend fixes from related issues
• Generate pre-filled GitHub issue

✅ Features

Automatic Diagnosis

• Run failed check on develop branch
• Extract error logs
• Parse error messages
• Identify error patterns

Root Cause Analysis

• Categorize errors (lint, type, test, build, etc)
• Search related issues for similar errors
• Suggest probable causes
• Recommend investigations

Fix Recommendations

• Search for related PRs that fixed similar issues
• Extract fixes from commit history
• Suggest inline patch
• Recommend related skills/agents

Issue Generation

• Pre-fill GitHub issue template
• Include:
  • Error category
  • Root cause analysis
  • Reproduction steps
  • Suggested fix (if found)
  • Related issues
• Ready for immediate creation

💻 Implementation Plan

Phase 1: Error Extraction

• Clone repo at commit
• Run failed check
• Parse error output
• Extract structured data

Phase 2: Pattern Matching

• Build error pattern database
• Match current error to patterns
• Suggest likely causes
• Search related issues

Phase 3: Fix Recommendations

• Search commit history for fixes
• Identify related PRs
• Extract patches
• Suggest solutions

Phase 4: Integration

• Integrate with STEP 11
• Add to PR finalization workflow
• Create GitHub issue automatically
• Test on sample failures

🧪 Testing

• Test on 10+ real CI failures
• Verify diagnosis accuracy
• Verify fix recommendations are relevant
• Test issue generation
• Measure automation rate (goal: > 80%)

📊 Effort & Timeline

Timeline: Post-Phase 3 (target: Oct 2026)
Effort: 6–8 hours (implementation + testing)
Owner: TBD

🔗 Related Issues

Epic: #2681
Phase 4a: #2689 (Issue Agent skills)
Phase 4c: #2691 (Scheduled label audit)

📚 References

• Current STEP 11 Process: prompts/03-pr-finalization-workflow-improved.md (STEP 11)
• Example Skill: skills/audit-label-coverage/
• Changelog Agent: agents/changelog/ (validation example)

📝 Nice-to-Have Features

• Slack notification with diagnosis
• Auto-assign to appropriate owner
• Link to related documentation
• Track fix success rates

[linear-code]

LS-3129

[github-actions]

⚠️ This issue does not follow a required issue template.

Missing required section(s): Definition of Ready (DoR), Definition of Done (DoD)

The issue will remain open and flagged until the body is corrected.
Please update the issue with one of the official templates:

• https://github.com/lightspeedwp/.github/issues/new/choose

If this was opened by automation, update the automation to preserve the template sections before creating the issue.