|
| 1 | += LLM Onboarding Improvements for Issuer |
| 2 | +:toc: macro |
| 3 | +:toclevels: 3 |
| 4 | + |
| 5 | +[horizontal] |
| 6 | +Mission:: Sub-mission for ReleaseHx 0.1.2 release |
| 7 | +Agent Role:: Product Engineer + Technical Writer |
| 8 | +Date:: 2026-01-24 |
| 9 | +Status:: Completed |
| 10 | + |
| 11 | +toc::[] |
| 12 | + |
| 13 | +== Executive Summary |
| 14 | + |
| 15 | +After learning Issuer to create GitHub issues for ReleaseHx 0.1.2 patch release, I documented observations about the tool's LLM-friendliness and propose specific improvements to make Issuer more accessible to AI agents. |
| 16 | + |
| 17 | +*Key Finding:* Issuer's documentation is comprehensive but could benefit from restructuring specifically for LLM consumption, with clearer hierarchies, machine-readable schemas, and quick-reference patterns. |
| 18 | + |
| 19 | +== What Worked Well |
| 20 | + |
| 21 | +=== Comprehensive README.adoc |
| 22 | + |
| 23 | +The README is thorough and well-structured with: |
| 24 | + |
| 25 | +* Clear feature list up front |
| 26 | +* Installation instructions for both Ruby and Docker users |
| 27 | +* Extensive CLI usage documentation |
| 28 | +* IMYML format specification with property reference |
| 29 | + |
| 30 | +=== Rich Example Files |
| 31 | + |
| 32 | +The `examples/` directory provides excellent learning material: |
| 33 | + |
| 34 | +* `basic-example.yml` - Simple, clear demonstration |
| 35 | +* `advanced-stub-example.yml` - Shows sophisticated features (stub processing, tag prefixes) |
| 36 | +* Multiple examples covering different use cases |
| 37 | + |
| 38 | +=== AsciiDoc Tagged Sections |
| 39 | + |
| 40 | +The use of `// tag::ai-prompt[]` markers shows awareness of AI agent needs and makes it easier to extract relevant sections. |
| 41 | + |
| 42 | +=== Dry-Run Mode |
| 43 | + |
| 44 | +The `--dry` flag is invaluable for validation without API calls - perfect for LLM testing and learning. |
| 45 | + |
| 46 | +=== Clear Property Naming |
| 47 | + |
| 48 | +Despite being abbreviated (summ, vrsn, user, tags), the property names are intuitive enough when seen in context. |
| 49 | + |
| 50 | +== What Was Confusing or Required Extra Effort |
| 51 | + |
| 52 | +=== IMYML Format Discovery |
| 53 | + |
| 54 | +*Issue:* The IMYML format specification is embedded deep in the README (lines 160-350). An LLM agent needs to scroll through installation and usage sections first. |
| 55 | + |
| 56 | +*Impact:* Required reading ~500 lines of documentation before understanding the core data format. |
| 57 | + |
| 58 | +*What I Did:* Read README sequentially, then jumped to examples to understand format through demonstration. |
| 59 | + |
| 60 | +=== Property Reference Buried |
| 61 | + |
| 62 | +*Issue:* The `[[imyml-ref]]` section with complete property documentation starts at line 233, after the format examples. |
| 63 | + |
| 64 | +*Impact:* Had to read examples first, then circle back to understand all available properties. |
| 65 | + |
| 66 | +=== No Machine-Readable Schema |
| 67 | + |
| 68 | +*Issue:* No JSON Schema, SGYML, or other machine-readable format definition. |
| 69 | + |
| 70 | +*Impact:* LLMs must parse AsciiDoc prose to understand validation rules, types, and constraints. |
| 71 | + |
| 72 | +=== Tag Prefix Logic Not Immediately Clear |
| 73 | + |
| 74 | +*Issue:* The `+` and `-` prefix notation for tags is powerful but requires careful reading to understand: |
| 75 | + |
| 76 | +* Regular tags: `[bug, docs]` |
| 77 | +* Append tags: `[+urgent]` |
| 78 | +* Removal tags: `[-needs:docs]` |
| 79 | + |
| 80 | +*Impact:* Could easily misunderstand and create invalid IMYML files. |
| 81 | + |
| 82 | +=== Authentication Setup |
| 83 | + |
| 84 | +*Issue:* Authentication section is late in README (line 410+), after CLI usage. |
| 85 | + |
| 86 | +*Impact:* An agent might try to execute without realizing authentication is required. |
| 87 | + |
| 88 | +=== Type Property Constraint |
| 89 | + |
| 90 | +*Issue:* The `type` property "must already be registered" but there's no clear guidance on what types are available or how to check. |
| 91 | + |
| 92 | +*Impact:* Uncertainty about valid values for the `type` field. |
| 93 | + |
| 94 | +== Top Priority Documentation Improvements |
| 95 | + |
| 96 | +=== Create AI Agent Quick Start Section ⭐⭐⭐ |
| 97 | + |
| 98 | +*Location:* Add near top of README after Overview |
| 99 | + |
| 100 | +*Proposed Content:* |
| 101 | + |
| 102 | +[source,asciidoc] |
| 103 | +---- |
| 104 | +[[ai-agent-quickstart]] |
| 105 | +=== Quick Start for AI Agents |
| 106 | +
|
| 107 | +For LLM agents learning to use Issuer: |
| 108 | +
|
| 109 | +. *Core Concept:* IMYML (Issue Management YAML) format defines issues in YAML, Issuer posts them to GitHub |
| 110 | +. *Required Reading Order:* |
| 111 | + - <<imyml-format,IMYML Format>> - Understand the data structure |
| 112 | + - <<imyml-ref,Property Reference>> - Complete field documentation |
| 113 | + - <<cli-usage,CLI Usage>> - Command-line options |
| 114 | + - <<authentication,Authentication>> - Token setup |
| 115 | +. *Examples:* Study `examples/basic-example.yml` and `examples/advanced-stub-example.yml` |
| 116 | +. *Testing:* Always use `--dry` flag first to validate without posting |
| 117 | +. *Authentication:* Set `ISSUER_API_TOKEN` environment variable before posting |
| 118 | +
|
| 119 | +*Minimal Valid IMYML:* |
| 120 | +
|
| 121 | +[source,yaml] |
| 122 | +---- |
| 123 | +$meta: |
| 124 | + proj: org/repo |
| 125 | +issues: |
| 126 | + - summ: Issue title here |
| 127 | +---- |
| 128 | +
|
| 129 | +=== Create IMYML Cheat Sheet ⭐⭐⭐ |
| 130 | +
|
| 131 | +*Location:* New file `docs/IMYML-REFERENCE.adoc` (or section in README): Issue title |
| 132 | +
|
| 133 | +
|
| 134 | +== Complete Template |
| 135 | +
|
| 136 | +[source,yaml] |
| 137 | +---- |
| 138 | +$meta: |
| 139 | + proj: org/repo # Required: org/repo or user/repo |
| 140 | + defaults: |
| 141 | + vrsn: 1.0.0 # String: milestone/version |
| 142 | + user: username # String: GitHub username |
| 143 | + type: Task # String: must exist in repo |
| 144 | + tags: [label1, +append] # Array: see tag logic below |
| 145 | + stub: false # Boolean: enable stub processing |
| 146 | + body: Default text # String: default body text |
| 147 | + head: Header text # String: prepend when stub=true |
| 148 | + tail: Footer text # String: append when stub=true |
| 149 | + |
| 150 | +issues: # Array: list of issue records |
| 151 | + - summ: Title # Required: one-line summary |
| 152 | + body: Description # String: full description |
| 153 | + type: Bug # String: issue type |
| 154 | + vrsn: 2.0.0 # String: milestone |
| 155 | + user: assignee # String: GitHub username |
| 156 | + tags: [label, +add] # Array: see tag logic |
| 157 | + stub: false # Boolean: override default |
| 158 | +---- |
| 159 | +
|
| 160 | +== Tag Logic Rules |
| 161 | +
|
| 162 | +* Regular tags: `[bug, docs]` - Applied based on default logic |
| 163 | +* Append tags: `[+urgent]` - Always added to all issues |
| 164 | +* Removal tags: `[-needs:docs]` - Remove from defaults |
| 165 | +* Can combine: `[feature, +critical, -auto]` |
| 166 | +
|
| 167 | +== Property Types |
| 168 | +
|
| 169 | +summ:: String (required) - Issue title |
| 170 | +body:: String (optional) - Issue description (Markdown formatted) |
| 171 | +type:: String (optional) - Must already exist in target repo |
| 172 | +vrsn:: String (optional) - Milestone name (auto-created if missing) |
| 173 | +user:: String (optional) - GitHub username for assignee |
| 174 | +tags:: Array[String] (optional) - Labels with prefix notation |
| 175 | +stub:: Boolean (optional) - Enable header/footer injection |
| 176 | +---- |
| 177 | + |
| 178 | +=== Add Machine-Readable Schema ⭐⭐ |
| 179 | + |
| 180 | +*Location:* New file `schemas/imyml-schema.json` or `schemas/imyml-schema.sgyml` |
| 181 | + |
| 182 | +*Benefit:* Allows LLMs to validate IMYML structure programmatically without parsing prose. |
| 183 | + |
| 184 | +*Consider:* Since SchemaGraphy exists in the DocOps Lab ecosystem, use SGYML format for the schema definition. |
| 185 | + |
| 186 | +=== Reorganize README Sections ⭐⭐ |
| 187 | + |
| 188 | +*Proposed New Order:* |
| 189 | + |
| 190 | +. Overview (keep as-is) |
| 191 | +. *AI Agent Quick Start* (NEW) |
| 192 | +. *IMYML Format* (move up from line 160) |
| 193 | +. *IMYML Property Reference* (keep with format section) |
| 194 | +. *Examples* (link to examples/ directory) |
| 195 | +. Installation (keep as-is) |
| 196 | +. Authentication (move up before CLI usage) |
| 197 | +. CLI Usage (after auth) |
| 198 | +. Advanced Usage (keep as-is) |
| 199 | +. Development (keep as-is) |
| 200 | + |
| 201 | +*Rationale:* Put the data format first so agents understand what they're creating before learning how to execute. |
| 202 | + |
| 203 | +=== Expand Type Property Documentation ⭐ |
| 204 | + |
| 205 | +*Location:* In IMYML Reference section |
| 206 | + |
| 207 | +*Add:* |
| 208 | + |
| 209 | +[source,asciidoc] |
| 210 | +---- |
| 211 | +type::: |
| 212 | +(String) |
| 213 | +The type of issue, which must already be registered in the target project or repository. |
| 214 | ++ |
| 215 | +Common GitHub issue types include: `Bug`, `Feature`, `Task`, `Enhancement`, `Question`, `Documentation`. |
| 216 | +
|
| 217 | +[NOTE] |
| 218 | +Issue types vary by repository. Check the target repository's issue type configuration before using this field. If a type doesn't exist, issue creation will fail. |
| 219 | +
|
| 220 | +[TIP] |
| 221 | +Use `--dry` mode to test without posting, then check GitHub repo settings for available types if validation fails. |
| 222 | +---- |
| 223 | + |
| 224 | +== Additional Improvement Ideas |
| 225 | + |
| 226 | +=== Inline Examples in Property Reference |
| 227 | + |
| 228 | +*Enhancement:* Add mini-examples next to each property definition. |
| 229 | + |
| 230 | +*Example:* |
| 231 | + |
| 232 | +[source,asciidoc] |
| 233 | +---- |
| 234 | +tags::: |
| 235 | +(Array of Strings) |
| 236 | +Labels to apply to the issue. |
| 237 | ++ |
| 238 | +.Example |
| 239 | +[source,yaml] |
| 240 | +---- |
| 241 | +tags: [bug, priority:high, +urgent, -needs:review] |
| 242 | +---- |
| 243 | ++ |
| 244 | +Supports prefix notation: |
| 245 | ++ |
| 246 | +* `bug` - regular label |
| 247 | +* `+urgent` - append to all issues |
| 248 | +* `-needs:review` - remove from defaults |
| 249 | +
|
| 250 | +=== Visual Hierarchy Markers |
| 251 | +
|
| 252 | +*Enhancement:* Use consistent AsciiDoc markers for LLM parsing: |
| 253 | +
|
| 254 | +* `[[property-name]]` anchors for all properties |
| 255 | +* `[.required]` and `[.optional]` role markers |
| 256 | +* `[.example]` markers for code blocks |
| 257 | +
|
| 258 | +=== Create docs/agent/ Directory |
| 259 | +
|
| 260 | +*Proposed Structure:* |
| 261 | +
|
| 262 | +.... |
| 263 | +docs/ |
| 264 | +├── agent/ |
| 265 | +│ ├── README.adoc # AI agent overview |
| 266 | +│ ├── quickstart.adoc # Fast onboarding |
| 267 | +│ ├── imyml-reference.adoc # Complete property docs |
| 268 | +│ ├── common-patterns.adoc # Recipe-style examples |
| 269 | +│ └── troubleshooting.adoc # FAQ and error handling |
| 270 | +└── schemas/ |
| 271 | + └── imyml-schema.sgyml # Machine-readable schema |
| 272 | +.... |
| 273 | +
|
| 274 | +=== Error Message Improvements |
| 275 | +
|
| 276 | +*Enhancement:* When validation fails, suggest solutions: |
| 277 | +
|
| 278 | +.... |
| 279 | +Error: Unknown type 'Improvement' |
| 280 | +Available types in DocOps/releasehx: Bug, Feature, Task, Enhancement |
| 281 | +Tip: Use --dry mode to validate before posting |
| 282 | +.... |
| 283 | +
|
| 284 | +=== Authentication Quick Check Command |
| 285 | +
|
| 286 | +*Enhancement:* Add a test command: |
| 287 | +
|
| 288 | +.... |
| 289 | +issuer --test-auth |
| 290 | +# Output: |
| 291 | +# ✓ GitHub token found (ISSUER_API_TOKEN) |
| 292 | +# ✓ Successfully authenticated as: briandominick |
| 293 | +# ✓ API rate limit: 4,987/5,000 remaining |
| 294 | +.... |
| 295 | +
|
| 296 | +== Mission-Specific Observations |
| 297 | +
|
| 298 | +=== What I Learned About Issuer |
| 299 | +
|
| 300 | +IMYML is intuitive once understood:: The abbreviated property names (summ, vrsn, user, tags) are actually helpful for conciseness |
| 301 | +
|
| 302 | +Dry-run is essential: Perfect safety mechanism for validation |
| 303 | +
|
| 304 | +Tag prefix logic is powerful:: The `+`/`-` notation is elegant for label management |
| 305 | +
|
| 306 | +Stub functionality is sophisticated:: head/body/tail system is well-designed for template-based issue creation |
| 307 | +
|
| 308 | +Docker-first approach is smart:: Lowers barrier to entry for non-Ruby users |
| 309 | +
|
| 310 | +=== Blockers Encountered |
| 311 | +
|
| 312 | +Need to run from issuer directory:: |
| 313 | +Had to execute from `/home/brian/Documents/work/issuer/` since issuer wasn't in releasehx bundle |
| 314 | +
|
| 315 | +Type property uncertainty:: |
| 316 | +Not sure if "Bug" and "Task" are valid without checking GitHub repo |
| 317 | +
|
| 318 | +=== Successful Outcomes |
| 319 | +
|
| 320 | +* ✅ Created valid IMYML file on first attempt after reading docs |
| 321 | +* ✅ Dry-run passed validation |
| 322 | +* ✅ Two issues correctly structured with all metadata |
| 323 | +* ✅ Ready for actual posting (awaiting Operator approval) |
| 324 | +
|
| 325 | +== Recommendations for Implementation |
| 326 | +
|
| 327 | +=== Immediate Actions (High ROI, Low Effort) |
| 328 | +
|
| 329 | +. *Add AI Agent Quick Start section* to README (30 minutes) |
| 330 | +. *Move IMYML format section up* in README (15 minutes) |
| 331 | +. *Move Authentication before CLI Usage* (5 minutes) |
| 332 | +. *Add inline examples to Property Reference* (1 hour) |
| 333 | +
|
| 334 | +=== Short-Term Actions (High ROI, Medium Effort) |
| 335 | +
|
| 336 | +[start=5] |
| 337 | +. *Create IMYML Quick Reference cheat sheet* (2 hours) |
| 338 | +. *Create docs/ai-agents/ directory structure* (4 hours) |
| 339 | +. *Expand type property documentation* (30 minutes) |
| 340 | +
|
| 341 | +=== Long-Term Actions (High ROI, High Effort) |
| 342 | +
|
| 343 | +[start=8] |
| 344 | +. *Create machine-readable schema* (SGYML format) (8 hours) |
| 345 | +. *Implement --test-auth command* (4 hours) |
| 346 | +. *Enhance error messages with suggestions* (ongoing) |
| 347 | +
|
| 348 | +== Conclusion |
| 349 | +
|
| 350 | +Issuer is already well-documented and functional. The proposed improvements focus on *restructuring and augmenting existing documentation* rather than changing core functionality. |
| 351 | +
|
| 352 | +Key Insight:: |
| 353 | +LLM agents benefit most from: |
| 354 | ++ |
| 355 | +* *Hierarchical navigation* (quick start → detailed reference) |
| 356 | +* *Format-first documentation* (show IMYML before CLI) |
| 357 | +* *Machine-readable schemas* (programmatic validation) |
| 358 | +* *Inline examples* (see-and-understand patterns) |
| 359 | +
|
| 360 | +These improvements would make Issuer significantly more accessible to AI agents while also benefiting human users through clearer documentation structure. |
0 commit comments