Skip to content

docs(examples): add nemoguards cache configuration example - #1459

Merged
Pouyanpi merged 2 commits into
developfrom
docs/cache-nemoguards-example
Oct 29, 2025
Merged

docs(examples): add nemoguards cache configuration example#1459
Pouyanpi merged 2 commits into
developfrom
docs/cache-nemoguards-example

Conversation

@Pouyanpi

@Pouyanpi Pouyanpi commented Oct 17, 2025

Copy link
Copy Markdown
Collaborator

Description

This is PR 5/5 (final) in the NeMoGuards caching feature stack.

Test Plan

results:

======================================================================
MODEL CACHE CONFIGURATION
======================================================================
✅ content_safety: maxsize=10000, track_stats=True
✅ topic_control: maxsize=10000, track_stats=True
✅ jailbreak_detection: maxsize=10000, track_stats=True

======================================================================
GENERATING 1000 TEST PROMPTS
======================================================================
Generated 1000 prompts (100 unique × 10 repetitions)

======================================================================
PROCESSING PROMPTS
======================================================================
Processed 100/1000 prompts - Hits: 270, Misses: 30, Errors: 0
Processed 200/1000 prompts - Hits: 540, Misses: 60, Errors: 0
Processed 300/1000 prompts - Hits: 810, Misses: 90, Errors: 0
Processed 400/1000 prompts - Hits: 1080, Misses: 120, Errors: 0
Processed 500/1000 prompts - Hits: 1350, Misses: 150, Errors: 0
Processed 600/1000 prompts - Hits: 1620, Misses: 180, Errors: 0
Processed 700/1000 prompts - Hits: 1890, Misses: 210, Errors: 0
Processed 800/1000 prompts - Hits: 2160, Misses: 240, Errors: 0
Processed 900/1000 prompts - Hits: 2430, Misses: 270, Errors: 0
Processed 1000/1000 prompts - Hits: 2700, Misses: 300, Errors: 0

Completed processing 1000 prompts

======================================================================
CACHE STATISTICS FROM LOGS
======================================================================
content_safety       - Hits:  900, Misses:  100, Hit Rate:  90.0%
topic_safety         - Hits:  900, Misses:  100, Hit Rate:  90.0%
jailbreak            - Hits:  900, Misses:  100, Hit Rate:  90.0%

======================================================================
CACHE STATISTICS FROM CACHE OBJECTS
======================================================================
content_safety       - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%
topic_control        - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%
jailbreak_detection  - Size: 100/10000, Hits: 900, Misses: 100, Hit Rate: 90.00%

======================================================================
VALIDATING traces.jsonl
======================================================================
Total LLM spans in traces: 3000
Cached LLM spans (llm.cache.hit=true): 2700

content_safety       - Total: 1000, Cached:  900, Hit Rate:  90.0%
topic_safety         - Total: 1000, Cached:  900, Hit Rate:  90.0%
jailbreak            - Total: 1000, Cached:  900, Hit Rate:  90.0%

Traces file: /Users/prezakhani/git/nemoguardrails-project/nemo-guardrails-review/.traces/trace.jsonl

======================================================================
SUMMARY
======================================================================
✅ Processed 1000 prompts with input rails (content safety, topic safety, jailbreak)
✅ Total cache hits: 2700
✅ Total cache misses: 300
✅ Overall cache hit rate: 90.0%
✅ Traces validated for llm.cache.hit attribute across all safety checks

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@tgasser-nv tgasser-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a description of how this config is different to the regular hero-workflow one (i.e. caching) and how they're configured. I can update the PR as you have so many others on your plate

Comment thread examples/configs/nemoguards_cache/README.md Outdated
@Pouyanpi
Pouyanpi force-pushed the feat/cache-jailbreak-detection branch from 9dbb323 to d38eb0b Compare October 19, 2025 10:26
Base automatically changed from feat/cache-jailbreak-detection to develop October 19, 2025 10:38
@Pouyanpi
Pouyanpi force-pushed the docs/cache-nemoguards-example branch from 9649c8a to 9cbb265 Compare October 20, 2025 10:54
@Pouyanpi
Pouyanpi force-pushed the docs/cache-nemoguards-example branch from 9cbb265 to ebb5f09 Compare October 20, 2025 10:55
@github-actions

Copy link
Copy Markdown
Contributor

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1459

@Pouyanpi

Copy link
Copy Markdown
Collaborator Author

@greptileai

@Pouyanpi
Pouyanpi requested a review from tgasser-nv October 21, 2025 13:10

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@tgasser-nv tgasser-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks for the updates

@Pouyanpi
Pouyanpi merged commit 28826ec into develop Oct 29, 2025
18 checks passed
@Pouyanpi
Pouyanpi deleted the docs/cache-nemoguards-example branch October 29, 2025 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants