Skip to content

Add triage-issues-local skill for OS issue triage#170

Open
rachaelrenk wants to merge 2 commits into
mainfrom
docs/issue-triage-automation
Open

Add triage-issues-local skill for OS issue triage#170
rachaelrenk wants to merge 2 commits into
mainfrom
docs/issue-triage-automation

Conversation

@rachaelrenk
Copy link
Copy Markdown
Contributor

Summary

Add automated issue triage for the newly open-sourced docs repo. This PR adds a triage-issues-local skill that the Oz OSS repo management agent uses to triage community issues, plus creates the supporting label taxonomy on the repo.

Changes

New: .agents/skills/triage-issues-local/SKILL.md

  • Skill for the Oz management agent to triage new GitHub issues
  • Classifies issues (site bug, content issue, feature request, support question)
  • Identifies affected docs pages by cross-referencing the site structure
  • Assesses priority using a defined rubric (high/medium/low)
  • Suggests fix direction for each issue
  • Checks for duplicate open issues
  • Applies triage + priority labels automatically
  • Posts a structured comment with welcome preamble, triage analysis, and reviewer instructions
  • References AGENTS.md, terminology glossary, and source repos (warp-internal, warp-server) for validation

Labels created on the repo (already live)

  • priority/high (red) — Factually incorrect info, broken page, security-related
  • priority/medium (yellow) — Outdated content, confusing instructions, incomplete coverage
  • priority/low (green) — Typos, minor formatting, nice-to-have improvements
  • triage (gray) — Applied automatically; removed once a human reviews
  • improve or update documentation (blue) — Fixes the missing label referenced by the CONTENT.yml issue template

Context

Co-Authored-By: Oz oz-agent@warp.dev

Add a new skill for the Oz management agent to triage community
issues on the docs repo. The skill classifies issues, identifies
affected pages, assesses priority, checks for duplicates, applies
labels (triage + priority), and posts a structured comment with
a welcome preamble and reviewer instructions.

Labels (priority/high, priority/medium, priority/low, triage, and
improve or update documentation) were created directly on the repo.

Co-Authored-By: Oz <oz-agent@warp.dev>
@cla-bot cla-bot Bot added the cla-signed label Jun 2, 2026
@vercel
Copy link
Copy Markdown

vercel Bot commented Jun 2, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jun 2, 2026 8:42pm

Request Review

@oz-for-oss
Copy link
Copy Markdown
Contributor

oz-for-oss Bot commented Jun 2, 2026

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@rachaelrenk rachaelrenk self-assigned this Jun 2, 2026
Copy link
Copy Markdown
Contributor

@oz-for-oss oz-for-oss Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR adds a repo-local docs issue triage skill. The main concerns are that the skill is not located/named where the triage workflow resolves repo-local companion skills, and its instructions redefine the core workflow by asking the agent to mutate GitHub directly instead of returning triage_result.json for the workflow to apply.

Concerns

  • The added skill uses .agents/skills/triage-issues-local/SKILL.md / triage-issues-local, but the dispatch resolver looks for .agents/skills/triage-issue-local/SKILL.md, so this guidance will not be loaded.
  • The skill asks the agent to apply labels and post comments directly, which conflicts with the core triage workflow's JSON-artifact contract and can produce duplicate or unauthorized GitHub mutations.
  • The duplicate search instruction uses an unpaginated gh issue list call instead of the shared repository-wide dedupe workflow, so it can miss older duplicate issues.

Verdict

Found: 0 critical, 4 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@@ -0,0 +1,124 @@
---
name: triage-issues-local
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] The repo-local resolver only looks for .agents/skills/triage-issue-local/SKILL.md and name: triage-issue-local when dispatching the core triage-issue workflow, so this pluralized companion will be ignored. Rename the directory/frontmatter to triage-issue-local and mark it as specializing triage-issue.

### 6. Check for duplicates

Search open issues for potential duplicates:
- Use `gh issue list --repo warpdotdev/docs --state open` to list open issues
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] The core duplicate workflow requires a repository-wide search that excludes the incoming issue; this default gh issue list call only returns the first page, so duplicate detection will silently miss older issues. Delegate to the shared dedupe-issue workflow or require a paginated search.

- **`triage`** — Always apply this. It signals the issue needs human review.
- **One priority label** — `priority/high`, `priority/medium`, or `priority/low` based on your assessment.

Use the GitHub CLI or API to apply labels.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] The dispatch workflow expects the agent to write triage_result.json and then applies labels itself. Calling GitHub directly from the companion violates the core triage-issue output contract and can create duplicate or unauthorized label mutations.


### 8. Post the triage comment

Post a single comment on the issue with three sections:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] The workflow posts or updates the issue comment from the uploaded triage_result.json; instructing the agent to post another comment directly can duplicate comments and bypass the maintainer-visible triage format. Describe the desired issue_body content instead.

…nation

- Rename triage-issues-local -> triage-issue-local (singular) to match
  the management agent's resolver convention
- Switch from direct label/comment application to triage_result.json
  artifact output, consistent with the review-pr workflow pattern
- Fix unpaginated gh issue list by adding --limit 100
- Add JSON validation step
- Update example to show JSON schema instead of raw markdown comment

Co-Authored-By: Oz <oz-agent@warp.dev>
@rachaelrenk
Copy link
Copy Markdown
Contributor Author

Review request — questions for @captainsafia

This PR adds a triage-issue-local skill for the Oz management agent to triage docs issues. A few open questions about how the skill integrates with the management system:

  1. triage_result.json contract — The skill currently outputs a triage_result.json file (consistent with the review.json pattern in review-pr.yml). Is this the expected artifact contract for the management agent's triage workflow, or does the agent apply labels/comments directly?

  2. Comment posting — The skill includes an issue_body field in the JSON for the workflow to post as a comment. Does the management workflow handle posting from the JSON artifact, or should the skill instruct the agent to post directly? We want to avoid duplicate comments.

  3. Skill name — We went with triage-issue-local (singular). Can you confirm this matches the resolver's expected path?

@@ -0,0 +1,117 @@
---
name: triage-issue-local
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually skip most of the content that tells the agent to read the issue and write out the triage_result.json. The OSS workflows will handle that. You can keep this one focused strictly on the rules.

The version in the warp repo is a good example of what the target structure would look like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants