Pre release cleanup#4
Merged
Merged
Conversation
brando-dill
approved these changes
Jun 9, 2026
brando-dill
left a comment
brando-dill
left a comment
Collaborator
There was a problem hiding this comment.
It looks good. I think we'll have to release and then get feedback to refine at this point.
…lease - Remove evals/scorecards/gpt-5x-improvement-plan.md and remaining-issues-plan.md — internal working docs (score snapshots, open questions, "input needed") not meant for a public repo. Legitimate eval methodology docs (routing-eval, anchor-selection-eval, plan-quality-eval) are retained. - Harden .gitignore: add *-PLAN.md / PLAN-*.md / HELIX-SETUP.md / helix-*.md / .pytest_cache patterns and section comments so internal planning files cannot accidentally ship. Verified eval methodology files (*-eval.md) are NOT caught. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Complete the customer-facing product rename and repair drift left by the PR #3 merge and the MCP-config cleanup commit: - .well-known/agent-skills/index.json: PingOne MT/ST → PingOne / PingOne Advanced Identity Cloud (AIC) in ping-foundation and ping-orchestration descriptions (this is the index marketplaces and agents consume) - Finish MT/ST → customer-facing rename in SKILL.md bodies/descriptions for ping-orchestration, ping-quickstart, ping-foundation; and in ping-marketplace.json display descriptions + .cursor-plugin description. Machine routing slugs (product_family/products arrays) intentionally unchanged. - Repair 5 dead links to deleted references/runtime/docs-mcp-routing.md: inline the docs-MCP fallback guidance in 4 SKILL.md files; point app-integration-overview.md to the existing mcp-preflight.md - Standardize contact email to developer-experience@pingidentity.com across all 4 plugin manifests (was split with devex@) - Fix 8 generated shortlist stubs to conform to reference-manifest-schema.json ({skill, branch, generated_at, max_docs, docs}) instead of malformed placeholder Validator clean; Layer 1 + Layer 2 mock evals 100%. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ck; email → devex@ Pre-release cleanup of the skills and public manifests: - Email: standardize all 6 plugin/marketplace manifests to devex@pingidentity.com - Generated tier: the references/generated/ shortlists were empty placeholders never populated by CI. Remove them entirely: * delete 8 stub JSON files + generated dirs in ping-foundation and ping-orchestration * strip all "generated" blocks from references/index.json * remove "Generated shortlist" fallback steps from all SKILL.md retrieval sections * remove generated_path from all 5 ping-marketplace.json files * update 2 curated-anchor cross-references, routing-hints.md, plugin README - Docs MCP: the Ping docs MCP server does not exist yet. Remove the docs-MCP fallback step from ping-foundation, ping-app-integration, ping-universal-services, and ping-identity-for-ai retrieval sections. Retrieval is now curated-anchors-only (plus the companion SDK skills for app integration). Validator clean; all manifests valid JSON; SKILL.md files ≤120 lines. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ayer Follow-on to the skills/manifests cleanup — bring the authoring spec, taxonomies, templates, and CONTRIBUTING in line with the curated-only retrieval model: - rules/routing-rules.md + shared/taxonomies/routing-rules.md: drop the generated shortlist and Docs MCP tiers from the reference-tier list; keep curated anchors only - rules/runtime-selection.md: remove generated/docs-MCP tier discipline + anti-patterns; fix dangling refs to docs-mcp-routing.md and PLAN.md - rules/authoring-rules.md + shared/templates/AUTHORING-RULES.md: remove the "generated stubs" authoring section; retarget the directory-path table to curated paths - rules/ping-identity.mdc: curated-only tier discipline; customer-facing platform names - shared/templates/SKILL.template.md: drop generated-shortlist blocks; customer-facing branch names (PingOne / PingOne Advanced Identity Cloud) - shared/templates/SKILL.reference-authoring.md: remove generated-stub guidance - shared/templates/ping-marketplace.template.json: drop generated_path; remove dead runtime docs-mcp-routing.md path; drop max_shortlist_docs - CONTRIBUTING.md: remove the docs-mcp-routing.md step and generated/ mkdir from the new-skill checklist Builder tooling (scripts/build_reference_manifests.py, reference-manifest-schema.json) kept untouched per decision — latent, not in CI. Eval rubrics left as-is. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…d references
The earlier rename only covered SKILL.md routing files. The curated reference
bodies and plugin-orientation files still used internal shorthand ("PingOne MT",
"PingOne ST") that no customer recognizes. This converts all 59 shipped files to
public product names with a context-aware convention:
- PingOne MT -> PingOne (or "PingOne (multi-tenant cloud)" in comparison contexts)
- PingOne ST -> PingOne Advanced Identity Cloud (AIC) on first mention, AIC after
- Collapsed circular disambiguators: "PingOne ST (AIC)" -> "AIC", etc.
Machine routing slugs (pingone-mt/pingone-st in product_family/products arrays,
directory paths, cross-reference paths) are canonical taxonomy and left untouched.
Real product names (PingAM, PingIDM, PingDS, DaVinci) and node class identifiers
preserved.
Validator clean, Layer 1 + Layer 2 mock evals at 100%, plugin validate passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tier language - Delete two drifted copies of routing-eval.md (shared/evals/, evals/scorecards/); keep the richest canonical one at evals/routing-eval.md (has the Layer 2 section). - Repoint the two AUTHORING-RULES references to evals/routing-eval.md. - Remove "generated shortlist" tier language from the retrieval-path rubric (that tier was removed earlier this release). - Narrow prompt-set.schema expected_tier enum to ["curated"] — all 92 prompts already use "curated"; "generated"/"docs-mcp" were dead enum values. - Scrub generated/docs-mcp from plan-quality-eval tier_discipline rubric. Builder scaffolding (scripts/build_reference_manifests.py, shared/schemas/ reference-manifest-schema.json) left as-is per prior decision — dormant, not in CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After renaming PingOne ST to "PingOne Advanced Identity Cloud (AIC)", bare "PingOne" is now a prefix of the AIC name. Routing tables that listed "PingOne" -> pingone-mt before "PingOne Advanced Identity Cloud" -> pingone-st were ambiguous: a user saying "I have PingOne Advanced Identity Cloud" would match the MT row first. Fix: put the more-specific AIC signals first in all four routing surfaces: - plugins/ping-identity/routing-hints.md - plugins/ping-identity/platform-scope.md - plugins/ping-identity/skills/ping-quickstart/SKILL.md - shared/taxonomies/routing-rules.md Add an explicit "match most specific first" note at each signal table. The pingone-mt "PingOne" signal now carries the qualifier "(bare, without Advanced Identity Cloud)" to make the intent unambiguous. All mock evals still 100%. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…y-for-ai skills Both skills were emitting brief advisory prose when tasks required output artifacts (Markdown guides, Python code). The "Stop if sufficient" retrieval rule was treated as a full-stop after loading a curated anchor, leaving check-required strings unwritten. Changes: - Add "Output discipline" section to ping-universal-services/SKILL.md and ping-identity-for-ai/SKILL.md: curated anchors are accuracy context, not a substitute for required artifacts. - Fix misleading scope line in agent-security-patterns.md: "Does NOT cover: human-in-the-loop delegation" caused the model to bail on the CIBA task even though Pattern 6 covers it fully. Replaced with a precise exclusion (workforce helpdesk delegation) and added an explicit pointer to CIBA. - Add T-60–T-65 platform-declaration test cases to ping-foundation.yaml: covers bare "I have PingOne" → pingone-mt and AIC signals → pingone-st routing without clarification. Diagnosed from Layer 3 eval regression: with_skill output tokens were 3–7× lower than without_skill on affected tasks; pass rates dropped -40% (universal-services) and -20% (identity-for-ai). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Test commit to confirm ed25519 key A31A6637CEDE9FF1 signs commits correctly. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
george-bafaloukas-forgerock
force-pushed
the
pre-release-cleanup
branch
from
e78ccd1 to
21fd404
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.