Skip to content

docs: Possible update to injection detection - #1144

Merged
Pouyanpi merged 4 commits into
NVIDIA-NeMo:developfrom
mikemckiernan:mmck-inj-detect
May 28, 2025
Merged

docs: Possible update to injection detection#1144
Pouyanpi merged 4 commits into
NVIDIA-NeMo:developfrom
mikemckiernan:mmck-inj-detect

Conversation

@mikemckiernan

@mikemckiernan mikemckiernan commented Apr 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

Possible update to injection detection.

Review HTML:
https://nvidia.github.io/NeMo-Guardrails/review/pr-1144/user-guides/guardrails-library.html#injection-detection

Related Issue(s)

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@mikemckiernan
docs: Possible update to injection detection
9cb9335 
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
@github-actions

github-actions Bot commented Apr 24, 2025 •
edited by mikemckiernan
Loading

Copy link
Copy Markdown
Contributor

Documentation preview

https://nvidia.github.io/NeMo-Guardrails/review/pr-1144/user-guides/guardrails-library.html#injection-detection

@mikemckiernan mikemckiernan self-assigned this Apr 24, 2025
@mikemckiernan mikemckiernan added the documentation Improvements or additions to documentation label Apr 24, 2025
erickgalinkin
erickgalinkin requested changes May 1, 2025
View reviewed changes

@erickgalinkin erickgalinkin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor notes. This is really great, Mike! Thank you!

Comment thread docs/user-guides/guardrails-library.md Outdated
Comment thread docs/user-guides/guardrails-library.md Outdated
Comment thread docs/user-guides/guardrails-library.md Outdated
* `sanitize` attempts to "de-fang" the malicious content, returning the output in a way that is less likely to result exploitation. This action is generally considered unsuitable for production use.
- Code injection (Python): Recommended if the LLM output is used as an argument to downstream functions or passed to a code interpreter.
- SQL injection: Recommended if the LLM output is used as part of a SQL query to a database.
- Template injection (Jinja): Recommended for use if LLM output is rendered using templating languages like Jinja.

@erickgalinkin erickgalinkin May 1, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Template injection (Jinja): Recommended for use if LLM output is rendered using templating languages like Jinja.
- Template injection (Jinja): Recommended for use if LLM output is rendered using templating languages. Currently, only Jinja is supported.

The syntax used is specific to Jinja, so even if you're using something Jinja-like, it won't work.

@mikemckiernan mikemckiernan May 2, 2025

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clarification about "Jinja-like" not working. I'm not a fan of "currently" because, depending on priorities, it might read "currently" for three or four years. How about...

- Template injection (Jinja): Recommended for use if LLM output is rendered using the Jinja templating language.
  This rule is usually paired with code injection rules.

When we add more languages, we'll just have to update this section--and would have to even if "currently" was in the text.

Comment thread docs/user-guides/guardrails-library.md Outdated
@mikemckiernan
Feedback from Erick
e89d95c 
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
@codecov-commenter

codecov-commenter commented May 2, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (develop@2a4f576). Learn more about missing BASE report.

Additional details and impacted files 
@@            Coverage Diff             @@
##             develop    #1144   +/-   ##
==========================================
  Coverage           ?   68.65%           
==========================================
  Files              ?      161           
  Lines              ?    15978           
  Branches           ?        0           
==========================================
  Hits               ?    10969           
  Misses             ?     5009           
  Partials           ?        0
Flag Coverage Δ
python 68.65% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mikemckiernan
Remove errant import and streaming output
09d01a6 
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
@Pouyanpi

Pouyanpi commented May 26, 2025

Copy link
Copy Markdown
Collaborator

@mikemckiernan thank you for the changes 👍🏻 We had update to injection detection and added yara_rules field that could be used instead of yara_path in #1164. Would you kindly include that too? Thanks!

@mikemckiernan
docs: Add inline YARA rules
9e37187 
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
@Pouyanpi Pouyanpi added this to the v0.14.0 milestone May 28, 2025
@Pouyanpi
Pouyanpi self-requested a review May 28, 2025 16:57
Pouyanpi
Pouyanpi approved these changes May 28, 2025
View reviewed changes

@Pouyanpi Pouyanpi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @mikemckiernan for making the changes 👍🏻

@Pouyanpi
Pouyanpi merged commit 4bcedf5 into NVIDIA-NeMo:develop May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erickgalinkin erickgalinkin erickgalinkin requested changes
@Pouyanpi Pouyanpi Pouyanpi approved these changes
@tgasser-nv tgasser-nv Awaiting requested review from tgasser-nv

Assignees

@mikemckiernan mikemckiernan

Labels

documentation Improvements or additions to documentation

Projects

None yet

Milestone

v0.14.0

Development

Successfully merging this pull request may close these issues.

4 participants

@mikemckiernan @codecov-commenter @Pouyanpi @erickgalinkin