Skip to content

fix(library): detect_regex_pattern() matches during output streaming but does not block - #1937

Merged
Pouyanpi merged 3 commits into
NVIDIA-NeMo:developfrom
m-misiura:fix/regex-streaming-output-not-blocking
May 28, 2026
Merged

fix(library): detect_regex_pattern() matches during output streaming but does not block#1937
Pouyanpi merged 3 commits into
NVIDIA-NeMo:developfrom
m-misiura:fix/regex-streaming-output-not-blocking

Conversation

@m-misiura

@m-misiura m-misiura commented May 27, 2026

Copy link
Copy Markdown
Contributor

Introduction

This PR attempts to provide a minimal fix for this GH issue

Root cause

The built-in detect_regex_pattern action correctly detects forbidden regex patterns during streaming output rail execution (logs confirm the match), but the streaming framework does not block the content. The matched output is streamed to the client as if no violation occurred.

In the server logs, we would see

INFO:nemoguardrails.actions.action_dispatcher:Executing registered action: detect_regex_pattern
INFO:nemoguardrails.library.regex.actions:Regex pattern matched: \bfight\s+club\b

The pattern matches — but the response is not blocked. The full LLM output streams to the client.

When the regex pattern matches in a streaming output chunk, the stream should be terminated and a guardrails violation error should be returned:

{"error": {"message": "Blocked by regex check output rails.", "type": "guardrails_violation", "param": "regex check output", "code": "content_blocked"}}

What seems to happen is:

  • is_output_blocked looks for an output_mapping function attached to the action's metadata. If none is found, it falls back to default_output_mapping
  • detect_regex_pattern returns a RegexDetectionResult (a TypedDict / dict)
  • Since a dict is not bool and not int/float, default_output_mapping returns False — meaning "not blocked" — regardless of what is_match says.

Closes #1936

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@m-misiura
m-misiura marked this pull request as ready for review May 27, 2026 10:57
@greptile-apps

greptile-apps Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a streaming output rail regression where detect_regex_pattern correctly identified forbidden patterns but the streaming framework never blocked the response, because is_output_blocked fell back to default_output_mapping which returns False for any dict result. The fix adds a dedicated _regex_blocked_mapping function and wires it as output_mapping on the @action decorator so is_output_blocked can correctly interpret the RegexDetectionResult.

  • Adds _regex_blocked_mapping that reads is_match from the RegexDetectionResult and returns it as a boolean, and registers it via output_mapping=_regex_blocked_mapping in the @action decorator.
  • Adds two unit tests: one verifying the end-to-end is_output_blocked behavior, and another ensuring output_mapping is always present in action_meta as a regression guard.

Confidence Score: 5/5

Safe to merge — the change is minimal, targeted, and directly fixes the root cause without altering existing call paths.

The fix correctly routes RegexDetectionResult through a dedicated mapping function instead of the generic fallback, and the new unit tests verify both the registration and the blocking behavior. No existing functionality is changed for non-streaming or non-regex paths.

No files require special attention.

Important Files Changed

Filename Overview
nemoguardrails/library/regex/actions.py Adds _regex_blocked_mapping and wires it as output_mapping on detect_regex_pattern so the streaming rail framework can correctly treat a dict-typed result as blocked.
tests/test_regex_detection.py Two new unit tests verify that is_output_blocked returns the right value and that output_mapping is registered on the action; stale #1932 issue reference in the assertion message was already flagged in a previous review thread.

Sequence Diagram

sequenceDiagram
    participant Client
    participant StreamingFramework
    participant ActionDispatcher
    participant detect_regex_pattern
    participant is_output_blocked
    participant _regex_blocked_mapping

    Client->>StreamingFramework: stream LLM output chunk
    StreamingFramework->>ActionDispatcher: "execute detect_regex_pattern(source=output, text=chunk)"
    ActionDispatcher->>detect_regex_pattern: call action
    detect_regex_pattern-->>ActionDispatcher: "RegexDetectionResult{is_match: true, ...}"
    ActionDispatcher-->>StreamingFramework: RegexDetectionResult

    note over StreamingFramework: Before fix: falls back to default_output_mapping(dict) → False (not blocked)
    note over StreamingFramework: After fix: reads action_meta[output_mapping]

    StreamingFramework->>is_output_blocked: is_output_blocked(result, detect_regex_pattern)
    is_output_blocked->>_regex_blocked_mapping: result.get(is_match, False)
    _regex_blocked_mapping-->>is_output_blocked: True
    is_output_blocked-->>StreamingFramework: True (blocked)
    StreamingFramework-->>Client: "{error: {type: guardrails_violation, ...}}"
Loading

Reviews (2): Last reviewed commit: "Update tests/test_regex_detection.py" | Re-trigger Greptile

Comment thread tests/test_regex_detection.py
Comment thread nemoguardrails/library/regex/actions.py Outdated
@coderabbitai

coderabbitai Bot commented May 27, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Added a _regex_blocked_mapping helper function to the detect_regex_pattern action that derives output blocking from the is_match field, enabling streaming output rails to automatically block matched patterns. Includes unit tests validating the blocking behavior and output_mapping registration.

Changes

Regex Detection Output Mapping

Layer / File(s) Summary
Output mapping implementation
nemoguardrails/library/regex/actions.py
Introduces _regex_blocked_mapping helper that returns True when is_match is truthy, and wires it into the detect_regex_pattern decorator via output_mapping parameter to enable streaming output blocking.
Output mapping validation tests
tests/test_regex_detection.py
Adds test_regex_output_mapping_blocks_on_match to verify output is blocked when regex matches and unblocked when it doesn't, and test_regex_output_mapping_is_registered to ensure the output_mapping is present in action metadata.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related issues

  • NVIDIA-NeMo/Guardrails#1936: Fixes detect_regex_pattern streaming output blocking by adding output_mapping that derives blocked status from is_match field.

Possibly related PRs

Suggested labels

bug

Suggested reviewers

  • Pouyanpi
🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the bug being fixed: detect_regex_pattern() matches patterns during output streaming but fails to block the output.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes ✅ Passed Minor bug fix (7 lines main code, 2 tests added). PR explicitly calls it "minimal fix" with clear root cause. Per instructions, minor changes satisfy the test results requirement.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov

codecov Bot commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Pouyanpi and others added 2 commits May 28, 2026 09:39
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>

@Pouyanpi Pouyanpi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @m-misiura , LGTM!

@Pouyanpi Pouyanpi changed the title bug: detect_regex_pattern() matches during output streaming but does not block fix(library): detect_regex_pattern() matches during output streaming but does not block May 28, 2026
@Pouyanpi
Pouyanpi merged commit a4fab4e into NVIDIA-NeMo:develop May 28, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: detect_regex_pattern() matches during output streaming but does not block

2 participants