Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"description": "Agent plugins for Ping Identity — teach any AI coding agent how to build with Ping Identity.",
"owner": {
"name": "Ping Identity",
"email": "developer-experience@pingidentity.com"
"email": "devex@pingidentity.com"
},
"plugins": [
{
Expand All @@ -14,7 +14,7 @@
"version": "1.0.0",
"author": {
"name": "Ping Identity",
"email": "developer-experience@pingidentity.com"
"email": "devex@pingidentity.com"
},
"homepage": "https://developer.pingidentity.com",
"repository": "https://github.com/pingidentity/agent-plugins",
Expand Down
2 changes: 1 addition & 1 deletion .cursor-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "ping-identity",
"version": "1.0.0",
"description": "Six-umbrella agent skill suite for Ping Identity: PingOne MT, PingOne ST (AIC), Ping Software Suite, DaVinci, universal services, app integration, identity for AI.",
"description": "Six-umbrella agent skill suite for Ping Identity: PingOne, PingOne Advanced Identity Cloud (AIC), Ping Software Suite, DaVinci, universal services, app integration, identity for AI.",
"author": { "name": "Ping Identity", "email": "devex@pingidentity.com" },
"homepage": "https://developer.pingidentity.com",
"repository": "https://github.com/pingidentity/agent-plugins",
Expand Down
12 changes: 12 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,24 @@
# Local agent / tooling state
.playwright-mcp/
.claude/
.pytest_cache/

# Eval run outputs (keep dir, drop contents)
evals/results/**/
!evals/results/.gitkeep

# Internal planning / working docs — never ship
PLAN.md
PLAN-*.md
*-PLAN.md
CLAUDE.md
PHASE-3-PLAN.md
SUMMARY.md
HELIX-SETUP.md
helix-*.md
docs/

# Secrets and local config
*.local.json
.env.local
.env
4 changes: 2 additions & 2 deletions .well-known/agent-skills/index.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@
},
{
"name": "ping-foundation",
"description": "Platform setup, administration, and core configuration for PingOne MT, PingOne ST (AIC), PingFederate, PingAccess, PingDirectory, and PingID. Covers tenant setup, app registration, directories, policies, and branding.",
"description": "Platform setup, administration, and core configuration for PingOne, PingOne Advanced Identity Cloud (AIC), PingFederate, PingAccess, PingDirectory, and PingID. Covers tenant setup, app registration, directories, policies, and branding.",
"source": "https://github.com/pingidentity/agent-plugins/tree/main/plugins/ping-identity/skills/ping-foundation",
"tags": ["ping", "pingone", "aic", "pingfederate", "foundation", "setup", "admin", "configuration"]
},
{
"name": "ping-orchestration",
"description": "Design and build authentication flows, journeys, and orchestration logic across DaVinci (PingOne MT), AIC/PingAM journeys (PingOne ST), and PingFederate authentication policies.",
"description": "Design and build authentication flows, journeys, and orchestration logic across DaVinci (PingOne), AIC/PingAM journeys (PingOne Advanced Identity Cloud), and PingFederate authentication policies.",
"source": "https://github.com/pingidentity/agent-plugins/tree/main/plugins/ping-identity/skills/ping-orchestration",
"tags": ["ping", "davinci", "aic", "journey", "authentication", "mfa", "registration", "orchestration"]
},
Expand Down
13 changes: 6 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,20 +62,19 @@ The pre-commit hook enforces these checks locally on every commit.
```bash
cp shared/templates/SKILL.template.md plugins/ping-identity/skills/<name>/SKILL.md
cp shared/templates/ping-marketplace.template.json plugins/ping-identity/skills/<name>/ping-marketplace.json
mkdir -p plugins/ping-identity/skills/<name>/references/{curated,generated,runtime}
mkdir -p plugins/ping-identity/skills/<name>/references/curated
```
3. Fill in `SKILL.md`:
- Keep it ≤120 lines — routing decision tree only, no prose
- `name:` must exactly match the directory name
- `description:` must be specific enough that an agent can decide to activate this skill without reading the body
- Add a `## When to use` section, a `## When NOT to use` section, and a routing table
4. Add ≥3 curated anchors in `references/curated/` — see Authoring a curated anchor below
5. Add `references/runtime/docs-mcp-routing.md` — use `shared/templates/runtime-routing-stub.md` as a starting point
6. Add the skill to `plugins/ping-identity/references/index.json`
7. Add `evals/prompts/<skill>.yaml` — see Eval prompt requirements below
8. Add the skill to `plugins/ping-identity/plugin-map.md` and `.well-known/agent-skills/index.json`
9. Run `python3 scripts/validate_skills.py --root .` — must exit 0
10. Run all mock evals — must exit 0
5. Add the skill to `plugins/ping-identity/references/index.json`
6. Add `evals/prompts/<skill>.yaml` — see Eval prompt requirements below
7. Add the skill to `plugins/ping-identity/plugin-map.md` and `.well-known/agent-skills/index.json`
8. Run `python3 scripts/validate_skills.py --root .` — must exit 0
9. Run all mock evals — must exit 0

**Decision rule** (from strategy doc § 4): A new skill is created only if it has a distinct trigger, a distinct user outcome, enough reference depth that routing materially improves results, and low overlap with an existing umbrella. Otherwise add it as a reference pack inside an existing skill.

Expand Down
45 changes: 45 additions & 0 deletions evals/prompts/ping-foundation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,51 @@ trigger_prompts:
expected_tier: curated
notes: "feedback: SSO workforce trigger"

# ----- Platform-declaration routing (bare PingOne → pingone-mt, AIC signals → pingone-st) -----

- id: T-60
prompt: "I have PingOne. How do I set up my first OIDC app?"
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-mt/app-registration.md
expected_tier: curated
notes: "Bare 'I have PingOne' must resolve to pingone-mt without asking for clarification."

- id: T-61
prompt: "I have PingOne AIC. How do I register an OAuth 2.0 client for my web app?"
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-st/app-setup.md
expected_tier: curated
notes: "'PingOne AIC' is an AIC signal → pingone-st branch."

- id: T-62
prompt: "We're using PingOne Advanced Identity Cloud and need to set up a new realm with a custom identity store."
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-st/foundation-overview.md
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-st/directory-setup.md
expected_tier: curated
notes: "'PingOne Advanced Identity Cloud' full name → pingone-st. Must NOT treat as bare PingOne MT."

- id: T-63
prompt: "I'm on PingOne (not AIC). How do I configure a sign-on policy that requires MFA for all users?"
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-mt/sign-on-policies.md
expected_tier: curated
notes: "Explicit parenthetical disambiguation '(not AIC)' → must route pingone-mt."

- id: T-64
prompt: "My company has PingOne. I need to add a user population and a custom attribute to the directory."
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-mt/directory-and-populations.md
expected_tier: curated
notes: "'My company has PingOne' — bare PingOne → pingone-mt."

- id: T-65
prompt: "We run AIC — I need to configure the hosted login page themes."
expected_anchors:
- plugins/ping-identity/skills/ping-foundation/references/curated/pingone-st/themes-and-customization.md
expected_tier: curated
notes: "Bare 'AIC' → pingone-st."

non_trigger_prompts:
- id: N-01
prompt: "Build a DaVinci flow for customer login with risk-based step-up MFA."
Expand Down
5 changes: 1 addition & 4 deletions evals/routing-eval.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,20 +70,17 @@ Choose the smallest tier that resolves the prompt:
| Tier | Use when |
|---|---|
| **Curated refs only** | The curated anchor(s) for the skill + platform fully answer the question |
| **Curated refs + bounded shortlist** | Curated anchors orient but do not cover the specific task; a generated shortlist provides the gap |
| **Docs fallback only if still necessary** | Neither curated nor shortlist is sufficient; a targeted external query is required |
| **Docs fallback only if still necessary** | Curated anchors are insufficient; a targeted external query is required |

**Retrieval discipline rules:**
- Load the smallest trusted context first
- Do not load more than 3 curated refs unless the task explicitly spans multiple domains
- Do not open a generated shortlist if a curated anchor already resolves the task
- Never load external docs if curated refs are sufficient
- Stop as soon as the answer is good enough

State:
- Retrieval tier used
- Curated refs you would load (by path)
- Shortlist refs you would load (by path), if any
- Whether external docs are needed: Yes / No
- Estimated retrieval token spend
- Estimated total token spend
Expand Down
2 changes: 1 addition & 1 deletion evals/schemas/prompt-set.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"id": { "type": "string", "pattern": "^T-[0-9]{2,3}$" },
"prompt": { "type": "string", "minLength": 10 },
"expected_anchors": { "type": "array", "items": { "type": "string" } },
"expected_tier": { "type": "string", "enum": ["curated", "generated", "docs-mcp"] },
"expected_tier": { "type": "string", "enum": ["curated"] },
"notes": { "type": "string" }
}
}
Expand Down
220 changes: 0 additions & 220 deletions evals/scorecards/gpt-5x-improvement-plan.md

This file was deleted.

Loading