docs: Update Benchmark README with updated configs

[tgasser-nv]

Description

This PR updates the top-level README with instructions on using the Mock LLMs, self-contained Procfile for server code, and AIPerf installation and use on the client-side. It more clearly describes where server-side and client-side dependencies are installed and run.

It also updates configs after merging #1862

Related Issue(s)

#1862 Fixes duplication of /v1 URLs, the configs in this PR are updated to reflect that.

Checklist

• I've read the CONTRIBUTING guidelines.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

• Documentation

  • Updated benchmarking quickstart guide with revised server-side CPU benchmarking workflow and client-side AIPerf testing instructions.

• Refactor

  • Restructured configuration to organize model-specific settings per model.
  • Updated process configuration to explicitly pass model endpoint parameters.
  • Simplified dependency management.

[github-actions]

Documentation preview

https://nvidia-nemo.github.io/Guardrails/review/pr-1905

[greptile-apps]

Greptile Summary

This PR updates the benchmark quickstart guide to replace the old standalone requirements.txt / virtualenv workflow with a poetry-based server setup and a new AIPerf client-side benchmarking section. It also appends /v1 to the base_url fields in both the Procfile and examples/configs/content_safety_local/config.yml to resolve the duplicate /v1 path issue fixed in #1862.

• README restructured: old 4-step flow (venv → start servers → validate → curl) is replaced with a cleaner 3-step flow (install via poetry + honcho → validate → AIPerf sweep), including an explicit CWD note for client-side commands and a fix to the latency-clamping documentation (corrected "less than" → "greater than" for LATENCY_MAX_SECONDS).
• Config/Procfile alignment: both base_url values and MAIN_MODEL_BASE_URL now consistently include /v1, matching the upstream URL-handling fix.
• requirements.txt deleted: server-side and client-side dependencies are now managed separately (poetry for the server, a dedicated aiperf venv for the client).

Confidence Score: 5/5

Documentation and configuration-only changes; no runtime logic is affected.

All changes are confined to documentation, a Procfile env-var update, and YAML config base_url corrections. The config changes are consistent with each other and with the upstream fix. No application code is touched.

No files require special attention.

Important Files Changed

Filename	Overview
benchmark/README.md	Restructured quickstart guide: replaces the old venv/requirements.txt workflow with a poetry-based server setup and a new AIPerf client section; also fixes a documentation bug in the latency clamping description (less → greater than LATENCY_MAX_SECONDS)
benchmark/Procfile	Appended /v1 to MAIN_MODEL_BASE_URL to align with the upstream fix for duplicated /v1 URL paths from #1862
benchmark/requirements.txt	Deleted; server-side dependencies now installed via poetry, and the aiperf client tool is installed separately in its own venv
examples/configs/content_safety_local/config.yml	Both model base_urls updated to include /v1 (localhost:8000/v1 and localhost:8001/v1), consistent with the Procfile change and the #1862 URL-path fix

Sequence Diagram

sequenceDiagram
    participant AIPerf as AIPerf Client (aiperf_env)
    participant GR as Guardrails Server :9000
    participant AppLLM as Mock App LLM :8000
    participant CSLLM as Mock CS LLM :8001

    Note over GR,CSLLM: Terminal 1 - honcho start (benchmark dir)
    Note over AIPerf: Terminal 2 - repo root (aiperf_env)

    AIPerf->>GR: POST /v1/chat/completions
    GR->>CSLLM: POST /v1/chat/completions (content safety)
    CSLLM-->>GR: safe/unsafe response
    GR->>AppLLM: POST /v1/chat/completions (main inference)
    AppLLM-->>GR: mock response
    GR-->>AIPerf: final response
    Note over AIPerf: results saved to aiperf_results/

Loading

_{Reviews (4): Last reviewed commit: "Fix copy-paste error with LATENCY_MAX_SE..." | Re-trigger Greptile}

[coderabbitai]

Walkthrough

This PR updates benchmark infrastructure and model configuration. The Procfile now explicitly passes MAIN_MODEL_BASE_URL to the server process, the benchmark README is restructured to document server-side setup and client-side AIPerf benchmarking workflows, and the content safety config moves endpoint configuration into per-model parameter blocks.

Changes

Benchmark Setup and Model Configuration

Layer / File(s)	Summary
Benchmark server process configuration benchmark/Procfile	The gr process command now includes explicit MAIN_MODEL_BASE_URL environment variable passed to the nemoguardrails server.
Benchmark quickstart workflow documentation benchmark/README.md	Restructured quickstart guide with three sequential steps: (1) server-side setup with ulimit and honcho start, (2) service validation via mock endpoint checks, and (3) client-side benchmarking with AIPerf sweep and concurrent load configuration including example log output.
Per-model endpoint configuration examples/configs/content_safety_local/config.yml	Model base_url configuration moved from shared top-level placement into per-model parameters blocks, allowing main and content_safety models to independently target their respective local endpoints.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Suggested reviewers

• Pouyanpi
• cparisien
• miyoungc

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title 'docs: Update Benchmark README with updated configs' is partially related to the changeset but overstates the scope—the PR modifies the Benchmark README and touches several config files, but also includes non-documentation changes (Procfile update, requirements.txt clearing) that the title does not reflect.	Consider whether the title should cover the actual scope of changes (e.g., including the Procfile and requirements.txt modifications) or whether this PR should be split into separate documentation and configuration management changes.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes	✅ Passed	PR contains only minor changes: documentation/config restructuring with no functional code or breaking changes. Per check rule, test results not required for minor changes.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/benchmark-readme

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

benchmark/README.md (1)

179-179: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Correct the latency max-bound condition description.

The max clamp condition is inverted in prose. For a max bound, the sampled value should be clamped when it is greater than LATENCY_MAX_SECONDS.

Suggested doc fix

-* If the sampled value is less than `LATENCY_MAX_SECONDS`, it is set to `LATENCY_MAX_SECONDS`.
+* If the sampled value is greater than `LATENCY_MAX_SECONDS`, it is set to `LATENCY_MAX_SECONDS`.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmark/README.md` at line 179, The documentation's description of the
latency max clamp is inverted: update the sentence that references
LATENCY_MAX_SECONDS so it reads that if a sampled latency is greater than
LATENCY_MAX_SECONDS it is clamped to LATENCY_MAX_SECONDS (i.e., replace "less
than `LATENCY_MAX_SECONDS`" with "greater than `LATENCY_MAX_SECONDS`" in the
line describing the max clamp).

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@benchmark/README.md`:
- Around line 125-126: The sentence fragment "This makes requests against" in
benchmark/README.md is incomplete; update that sentence to state the target of
the requests and any relevant context (e.g., "This makes requests against the
local benchmark server at http://localhost:8080" or "This makes requests against
the deployed API endpoint to measure latency and throughput"), ensuring the
sentence follows surrounding instructions and clarifies what is being requested
and why.
- Around line 29-30: Replace the incorrect occurrences of the string "set ulimit
-n65536" in the README with the proper shell command "ulimit -n 65536" (remove
the "set" prefix and add a space between -n and the value); search for the exact
token "set ulimit -n65536" to find both occurrences and update them so the
quickstart steps correctly increase the file-descriptor limit.

---

Outside diff comments:
In `@benchmark/README.md`:
- Line 179: The documentation's description of the latency max clamp is
inverted: update the sentence that references LATENCY_MAX_SECONDS so it reads
that if a sampled latency is greater than LATENCY_MAX_SECONDS it is clamped to
LATENCY_MAX_SECONDS (i.e., replace "less than `LATENCY_MAX_SECONDS`" with
"greater than `LATENCY_MAX_SECONDS`" in the line describing the max clamp).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 56daaf89-9f3a-41db-a13f-2be16c376b32

📥 Commits

Reviewing files that changed from the base of the PR and between 6115f6a and 558801a.

📒 Files selected for processing (4)

• benchmark/Procfile
• benchmark/README.md
• benchmark/requirements.txt
• examples/configs/content_safety_local/config.yml

💤 Files with no reviewable changes (1)

• benchmark/requirements.txt

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[h-network]

Happy to see the README is updated
https://github.com/h-network/Reports/tree/main/nvidiaGuardrails