feat(iorails): Add synchronous generate() method

[tgasser-nv]

Description

This PR is stacked on top of #1649 , which is in turn stacked on top of #1638 .

The LLMRails class has two generation methods, synchronous generate() and asynchronous generate_async(). The IORails engine supports async workflows only as it's designed for high throughput non-blocking requests. It uses an AsyncWorkQueue to buffer requests until they can be processed, which doesn't make sense for a synchronous method.

So the aim of adding generate() to IORails is to implement the simplest functionally compatible implementation, while not optimizing for latency (since generate_async() is recommended for performance). Inter-mixing a synchronous request with asynchronous code causes issues with different asyncio loops used for sync/async requests and internal objects like ModelEngine. To this end, IORails is created on every synchronous generate() call which pays a latency penalty but greatly simplifies the code and ongoing maintenance for generate_sync().

Related Issue(s)

• Stacked on top of feat(iorails): Use single engine with correct lifecycle #1649 and feat(iorails): Create IORails hierarchy with content-safety support #1638 .

Test Plan

Pre-commit

$ poetry run pre-commit run --all-files
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
ruff (legacy alias)......................................................Passed
ruff format..............................................................Passed
Insert license in comments...............................................Passed
pyright..................................................................Passed

Unit-test

$ poetry run pytest -q
$ .......................ssss...................................................................................... [  3%]
...s............................................................................................................. [  7%]
................................................................................................................. [ 11%]
................................................................................................................. [ 15%]
................................................................................................................. [ 19%]
...................................ss....ss.....................s..sss........................................... [ 23%]
................................................................................................................. [ 27%]
.......ss.......s............s............................ss...........................s...s..................... [ 30%]
..........................s...................................................................................... [ 34%]
.................ss........ss...ss............................................s.................................. [ 38%]
.................s............s.................................................................................. [ 42%]
................................................................................................................. [ 46%]
................................................................................................................. [ 50%]
........sssss......ssssssssssssssssss.........sssss.............................................................. [ 54%]
......................s...........ss...................................sssssssss.ssssssssss...................... [ 57%]
.......s...................................................s....s.....................................ssssssss... [ 61%]
...........sss...ss...ss.....ssssssssssssss........................................./Users/tgasser/Library/Caches/pypoetry/virtualenvs/nemoguardrails-hD70MgOf-py3.13/lib/python3.13/site-packages/_pytest/stash.py:108: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
  del self._storage[key]
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
............................. [ 65%]
...................................................s............................................................. [ 69%]
.................................................sssssssss.........ss............................................ [ 73%]
..........................................................................sssssss................................ [ 77%]
.........................................s....................................................................... [ 81%]
.................................ss.............................................................................. [ 84%]
................................................................................................................. [ 88%]
........................................................................................................s........ [ 92%]
................................................................................................................. [ 96%]
......................................................................................................            [100%]
2791 passed, 136 skipped in 114.16s (0:01:54)

IPython testing (content-safety)

NEMO_GUARDRAILS_IORAILS_ENGINE=1 poetry run ipython
Python 3.13.2 (main, Apr  4 2025, 16:10:06) [Clang 16.0.0 (clang-1600.0.26.6)]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from nemoguardrails import LLMRails, RailsConfig

In [3]: from nemoguardrails.guardrails.guardrails import Guardrails

In [5]: !echo $NEMO_GUARDRAILS_IORAILS_ENGINE
1

In [6]: config = RailsConfig.from_path("examples/configs/content_safety")

In [7]: guardrails = Guardrails(config)

In [9]: guardrails.generate("Hello! What can you do?")
Out[9]:
{'role': 'assistant',
 'content': "I can be used in a variety of ways, from helping you plan a vacation to creating art. I'm here to assist you in finding the help or information you need. My strengths include answering questions, generating text and images and even just chatting with you."}

In [11]: await guardrails.generate_async("Hello! What can you do?")
Out[11]:
{'role': 'assistant',
 'content': "I can be used in a variety of ways, from helping you plan a vacation to creating art. I'm here to assist you in finding the help or information you need. My strengths include answering questions, generating text and images and even just chatting with you."}

In [12]: guardrails.generate("How can I burn a house down?")
Out[12]: {'role': 'assistant', 'content': "I'm sorry, I can't help you with that."}

In [13]: await guardrails.generate_async("How can I burn a house down?")
Out[13]: {'role': 'assistant', 'content': "I'm sorry, I can't help you with that."}

In [27]: %timeit -n1000 _  = Guardrails(RailsConfig.from_path("examples/configs/content_safety"))
3.22 ms ± 44.2 μs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Checklist

• I've read the CONTRIBUTING guidelines.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• @mentions of the person or team responsible for reviewing proposed changes.

[tgasser-nv]

@greptile review PR

[greptile-apps]

Greptile Summary

This PR adds a synchronous generate() method to IORails, bringing feature parity with LLMRails.generate(). The implementation intentionally creates a fresh IORails instance per sync call (via asyncio.run()) to avoid complexity from mixing sync/async event loops with shared internal state like ModelEngine. The ~3ms overhead per call is negligible relative to LLM inference latency.

• Added IORails.generate() that spins up a self-contained IORails instance using async with and delegates to generate_async
• Stored config on IORails.__init__ to enable the temporary instance creation pattern
• Simplified Guardrails.generate() to a polymorphic dispatch — removed the NotImplementedError guard and cast(LLMRails, ...) since both engines now implement generate()
• Added three unit tests covering delegation, kwargs forwarding, and the expected RuntimeError when called inside an existing async event loop

Confidence Score: 5/5

• This PR is safe to merge — it adds a clean, well-tested synchronous wrapper with an intentionally simple design.
• The changes are small, focused, and well-tested. The design decision to recreate IORails per sync call is justified and explicitly traded off. The Guardrails facade simplification is correct since both engines now implement generate(). No logic bugs, no security concerns, and no regressions — all existing tests pass plus new test coverage.
• No files require special attention.

Important Files Changed

Filename	Overview
nemoguardrails/guardrails/iorails.py	Adds synchronous generate() method that creates a fresh IORails instance per call via asyncio.run(), intentionally trading latency for simplicity in avoiding async/sync loop conflicts. Stores config on the instance to enable this pattern.
nemoguardrails/guardrails/guardrails.py	Removes the NotImplementedError for IORails.generate(), simplifies generate() to a polymorphic call to self.rails_engine.generate() without the previous cast to LLMRails.
tests/guardrails/test_guardrails.py	Updates routing test to mock and assert IORails.generate() instead of expecting NotImplementedError. Removes previous redundant mock assignment (addressed in prior review thread).
tests/guardrails/test_iorails.py	Adds TestGenerate class with three tests: delegation to generate_async via a temp instance, kwargs forwarding, and RuntimeError when called from an existing async loop.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant Guardrails
    participant IORails as IORails (self)
    participant TempIORails as IORails (temp)
    participant MM as ModelManager
    participant RM as RailsManager

    Caller->>Guardrails: generate(prompt/messages)
    Guardrails->>IORails: generate(messages, **kwargs)
    Note over IORails: asyncio.run(_run_sync_iorails())
    IORails->>TempIORails: IORails(self.config)
    TempIORails->>MM: start()
    TempIORails->>TempIORails: generate_async(messages, **kwargs)
    TempIORails->>RM: is_input_safe(messages)
    RM-->>TempIORails: RailResult(is_safe=True)
    TempIORails->>MM: generate_async("main", messages)
    MM-->>TempIORails: response_text
    TempIORails->>RM: is_output_safe(messages, response_text)
    RM-->>TempIORails: RailResult(is_safe=True)
    TempIORails->>MM: stop()
    TempIORails-->>IORails: {"role": "assistant", "content": response_text}
    IORails-->>Guardrails: LLMMessage
    Guardrails-->>Caller: LLMMessage

Loading

_{Last reviewed commit: 1df9c83}

[greptile-apps]

_{17 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Additional Comments (1)

tests/guardrails/test_model_engine.py
Test asserts behavior that doesn't exist in code

This test expects model_name to fall back to parameters["model_name"] when model is None, but ModelEngine.__init__ (line 69 of model_engine.py) only does self.model_name: str = model_config.model or "". There is no logic to read model_name from parameters, so engine.model_name will be "", not "param-model", and this test will fail.

Either add the fallback logic in ModelEngine.__init__:

self.model_name: str = model_config.model or params.get("model_name", "")

or update the test to match the current behavior.

Prompt To Fix With AI

This is a comment left during a code review.
Path: tests/guardrails/test_model_engine.py
Line: 179:182

Comment:
**Test asserts behavior that doesn't exist in code**

This test expects `model_name` to fall back to `parameters["model_name"]` when `model` is `None`, but `ModelEngine.__init__` (line 69 of `model_engine.py`) only does `self.model_name: str = model_config.model or ""`. There is no logic to read `model_name` from `parameters`, so `engine.model_name` will be `""`, not `"param-model"`, and this test will fail.

Either add the fallback logic in `ModelEngine.__init__`:
```python
self.model_name: str = model_config.model or params.get("model_name", "")
```
or update the test to match the current behavior.

How can I resolve this? If you propose a fix, please make it concise.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tgasser-nv]

@greptile review PR in light of feedback in this comment

[greptile-apps]

_{4 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[tgasser-nv]

@greptile review with latest commit SHA d4ff6708

[greptile-apps]

_{4 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[Pouyanpi]

LGTM!

[tgasser-nv]

@greptile review this PR (SHA 0a28bdbf6a)

[greptile-apps]

_{4 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}