fix(library): detect_regex_pattern() matches during output streaming but does not block - #1937
Conversation
Greptile SummaryThis PR fixes a streaming output rail regression where
|
| Filename | Overview |
|---|---|
| nemoguardrails/library/regex/actions.py | Adds _regex_blocked_mapping and wires it as output_mapping on detect_regex_pattern so the streaming rail framework can correctly treat a dict-typed result as blocked. |
| tests/test_regex_detection.py | Two new unit tests verify that is_output_blocked returns the right value and that output_mapping is registered on the action; stale #1932 issue reference in the assertion message was already flagged in a previous review thread. |
Sequence Diagram
sequenceDiagram
participant Client
participant StreamingFramework
participant ActionDispatcher
participant detect_regex_pattern
participant is_output_blocked
participant _regex_blocked_mapping
Client->>StreamingFramework: stream LLM output chunk
StreamingFramework->>ActionDispatcher: "execute detect_regex_pattern(source=output, text=chunk)"
ActionDispatcher->>detect_regex_pattern: call action
detect_regex_pattern-->>ActionDispatcher: "RegexDetectionResult{is_match: true, ...}"
ActionDispatcher-->>StreamingFramework: RegexDetectionResult
note over StreamingFramework: Before fix: falls back to default_output_mapping(dict) → False (not blocked)
note over StreamingFramework: After fix: reads action_meta[output_mapping]
StreamingFramework->>is_output_blocked: is_output_blocked(result, detect_regex_pattern)
is_output_blocked->>_regex_blocked_mapping: result.get(is_match, False)
_regex_blocked_mapping-->>is_output_blocked: True
is_output_blocked-->>StreamingFramework: True (blocked)
StreamingFramework-->>Client: "{error: {type: guardrails_violation, ...}}"
Reviews (2): Last reviewed commit: "Update tests/test_regex_detection.py" | Re-trigger Greptile
📝 WalkthroughWalkthroughAdded a ChangesRegex Detection Output Mapping
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Signed-off-by: Pouyan <13303554+Pouyanpi@users.noreply.github.com>
Pouyanpi
left a comment
There was a problem hiding this comment.
Thanks @m-misiura , LGTM!
detect_regex_pattern() matches during output streaming but does not blockdetect_regex_pattern() matches during output streaming but does not block
Introduction
This PR attempts to provide a minimal fix for this GH issue
Root cause
The built-in
detect_regex_patternaction correctly detects forbidden regex patterns during streaming output rail execution (logs confirm the match), but the streaming framework does not block the content. The matched output is streamed to the client as if no violation occurred.In the server logs, we would see
The pattern matches — but the response is not blocked. The full LLM output streams to the client.
When the regex pattern matches in a streaming output chunk, the stream should be terminated and a guardrails violation error should be returned:
{"error": {"message": "Blocked by regex check output rails.", "type": "guardrails_violation", "param": "regex check output", "code": "content_blocked"}}What seems to happen is:
Closes #1936
Checklist