Skip to content

Commit 91102c8

Browse files
svenchiltonclaude
andcommitted
fix(examples): correct nim_base_url to avoid doubled /v1/ in combined notebook
Address Greptile feedback on PR #1906 (combined_guardrails_nim.ipynb). The local-deployment override set `nim_base_url = "http://localhost:8125/v1/"`, which collides with `nim_server_endpoint = "/v1/security/nvidia/nemoguard-jailbreak-detect"` (set in the YAML and not overridden). The `join_nim_url` helper in `nemoguardrails/library/jailbreak_detection/request.py` rstrips the base's trailing slash, lstrips the endpoint's leading slash, and urljoins the two relatively — producing `http://localhost:8125/v1/v1/security/.../nemoguard-jailbreak-detect` (double `/v1/`, 404 from the NIM). Local jailbreak detection was silently bypassed. Fix: drop the `/v1/` suffix from the local base, matching the remote pattern (`https://ai.api.nvidia.com`) and the canonical NemoGuard NIM examples in the repo docs (caching/model-memory-cache.md, the tracing tutorial). The `/v1` already lives in nim_server_endpoint; both deployments now follow the same convention. The cell 7 markdown is updated with a "Note on the jailbreak-detection NIM URL convention" paragraph explaining the rule and the join_nim_url mechanism so future readers don't trip on this. Note: this is a pre-existing bug on `develop` (combined_guardrails_nim.ipynb was inherited unchanged for PR #1906). The same incorrect pattern also lives in jailbreak_detection_nim.ipynb on dev/schilton/update-docs-and-tutorials and will need the same fix when that notebook ships. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent f394aa4 commit 91102c8

1 file changed

Lines changed: 2 additions & 189 deletions

File tree

examples/notebooks/combined_guardrails_nim.ipynb

Lines changed: 2 additions & 189 deletions
Original file line numberDiff line numberDiff line change
@@ -130,201 +130,14 @@
130130
{
131131
"cell_type": "markdown",
132132
"metadata": {},
133-
"source": [
134-
"## Combined Configuration\n",
135-
"\n",
136-
"All four guardrails are configured in a single `RailsConfig`. The `prompts` section provides the safety classification templates for the content safety and topic control NIMs."
137-
]
133+
"source": "## Combined Configuration\n\nAll four guardrails are configured in a single `RailsConfig`. The `prompts` section provides the safety classification templates for the content safety and topic control NIMs.\n\n**Note on the jailbreak-detection NIM URL convention.** For the NemoGuard NIM, `nim_base_url` should NOT include a `/v1/` suffix — the `/v1` lives in `nim_server_endpoint` (set to `/v1/security/nvidia/nemoguard-jailbreak-detect` in the YAML below). The [`join_nim_url`](../../nemoguardrails/library/jailbreak_detection/request.py) helper rstrips the base's trailing slash, lstrips the endpoint's leading slash, and `urljoin`s the two relatively — so a `/v1/` in the base would produce a doubled-path 404 (e.g., `http://localhost:8125/v1/v1/security/...`). The convention applies to both remote and local deployments; the local override below preserves it.\n"
138134
},
139135
{
140136
"cell_type": "code",
141137
"execution_count": null,
142138
"metadata": {},
143139
"outputs": [],
144-
"source": [
145-
"# For remote deployment:\n",
146-
"# import os\n",
147-
"# os.environ[\"NVIDIA_API_KEY\"] = \"nvapi-...\"\n",
148-
"\n",
149-
"YAML_CONFIG = \"\"\"\n",
150-
"models:\n",
151-
" - type: main\n",
152-
" engine: nim\n",
153-
" model: meta/llama-3.1-8b-instruct\n",
154-
"\n",
155-
" - type: content_safety\n",
156-
" engine: nim\n",
157-
" model: nvidia/llama-3.1-nemotron-safety-guard-8b-v3\n",
158-
"\n",
159-
" - type: topic_control\n",
160-
" engine: nim\n",
161-
" model: nvidia/llama-3.1-nemoguard-8b-topic-control\n",
162-
"\n",
163-
"rails:\n",
164-
" input:\n",
165-
" flows:\n",
166-
" - jailbreak detection model\n",
167-
" - content safety check input $model=content_safety\n",
168-
" - topic safety check input $model=topic_control\n",
169-
" - gliner detect pii on input\n",
170-
" output:\n",
171-
" flows:\n",
172-
" - content safety check output $model=content_safety\n",
173-
" - gliner detect pii on output\n",
174-
" config:\n",
175-
" gliner:\n",
176-
" threshold: 0.5\n",
177-
" input:\n",
178-
" entities:\n",
179-
" - first_name\n",
180-
" - last_name\n",
181-
" - ssn\n",
182-
" - date_of_birth\n",
183-
" - phone_number\n",
184-
" - street_address\n",
185-
" - email\n",
186-
" output:\n",
187-
" entities:\n",
188-
" - first_name\n",
189-
" - last_name\n",
190-
" - ssn\n",
191-
" - date_of_birth\n",
192-
" - phone_number\n",
193-
" - email\n",
194-
" jailbreak_detection:\n",
195-
" nim_base_url: \"https://ai.api.nvidia.com\"\n",
196-
" nim_server_endpoint: \"/v1/security/nvidia/nemoguard-jailbreak-detect\"\n",
197-
" api_key_env_var: NVIDIA_API_KEY\n",
198-
"\n",
199-
"prompts:\n",
200-
" - task: content_safety_check_input $model=content_safety\n",
201-
" content: |\n",
202-
" Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n",
203-
"\n",
204-
" <BEGIN UNSAFE CONTENT CATEGORIES>\n",
205-
" S1: Violence.\n",
206-
" S2: Sexual.\n",
207-
" S3: Criminal Planning/Confessions.\n",
208-
" S4: Guns and Illegal Weapons.\n",
209-
" S5: Controlled/Regulated Substances.\n",
210-
" S6: Suicide and Self Harm.\n",
211-
" S7: Sexual (minor).\n",
212-
" S8: Hate/Identity Hate.\n",
213-
" S9: PII/Privacy.\n",
214-
" S10: Harassment.\n",
215-
" S11: Threat.\n",
216-
" S12: Profanity.\n",
217-
" S13: Needs Caution.\n",
218-
" S14: Other.\n",
219-
" S15: Manipulation.\n",
220-
" S16: Fraud/Deception.\n",
221-
" S17: Malware.\n",
222-
" S18: High Risk Gov Decision Making.\n",
223-
" S19: Political/Misinformation/Conspiracy.\n",
224-
" S20: Copyright/Trademark/Plagiarism.\n",
225-
" S21: Unauthorized Advice.\n",
226-
" S22: Illegal Activity.\n",
227-
" S23: Immoral/Unethical.\n",
228-
" <END UNSAFE CONTENT CATEGORIES>\n",
229-
"\n",
230-
" <BEGIN CONVERSATION>\n",
231-
"\n",
232-
" user: {{ user_input }}\n",
233-
"\n",
234-
" <END CONVERSATION>\n",
235-
"\n",
236-
" Provide your safety assessment for the above conversation in the below output JSON format:\n",
237-
" {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n",
238-
"\n",
239-
" Do not include anything other than the output JSON in your response.\n",
240-
" Output JSON:\n",
241-
" output_parser: nemoguard_parse_prompt_safety\n",
242-
" max_tokens: 50\n",
243-
"\n",
244-
" - task: content_safety_check_output $model=content_safety\n",
245-
" content: |\n",
246-
" Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n",
247-
"\n",
248-
" <BEGIN UNSAFE CONTENT CATEGORIES>\n",
249-
" S1: Violence.\n",
250-
" S2: Sexual.\n",
251-
" S3: Criminal Planning/Confessions.\n",
252-
" S4: Guns and Illegal Weapons.\n",
253-
" S5: Controlled/Regulated Substances.\n",
254-
" S6: Suicide and Self Harm.\n",
255-
" S7: Sexual (minor).\n",
256-
" S8: Hate/Identity Hate.\n",
257-
" S9: PII/Privacy.\n",
258-
" S10: Harassment.\n",
259-
" S11: Threat.\n",
260-
" S12: Profanity.\n",
261-
" S13: Needs Caution.\n",
262-
" S14: Other.\n",
263-
" S15: Manipulation.\n",
264-
" S16: Fraud/Deception.\n",
265-
" S17: Malware.\n",
266-
" S18: High Risk Gov Decision Making.\n",
267-
" S19: Political/Misinformation/Conspiracy.\n",
268-
" S20: Copyright/Trademark/Plagiarism.\n",
269-
" S21: Unauthorized Advice.\n",
270-
" S22: Illegal Activity.\n",
271-
" S23: Immoral/Unethical.\n",
272-
" <END UNSAFE CONTENT CATEGORIES>\n",
273-
"\n",
274-
" <BEGIN CONVERSATION>\n",
275-
"\n",
276-
" user: {{ user_input }}\n",
277-
"\n",
278-
" response: agent: {{ bot_response }}\n",
279-
"\n",
280-
" <END CONVERSATION>\n",
281-
"\n",
282-
" Provide your safety assessment for the above conversation in the below output JSON format:\n",
283-
" {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n",
284-
"\n",
285-
" Do not include anything other than the output JSON in your response.\n",
286-
" Output JSON:\n",
287-
" output_parser: nemoguard_parse_response_safety\n",
288-
" max_tokens: 50\n",
289-
"\n",
290-
" - task: topic_safety_check_input $model=topic_control\n",
291-
" content: |\n",
292-
" You are a patient support assistant for a healthcare organization. Your role is to\n",
293-
" provide general health information, help patients understand symptoms and conditions,\n",
294-
" explain medications and treatments, and assist with appointment and administrative questions.\n",
295-
"\n",
296-
" Guidelines:\n",
297-
" - Only answer questions related to health, wellness, medical conditions, treatments,\n",
298-
" medications, and healthcare administration.\n",
299-
" - Do not answer questions about finance, investments, politics, law, or any topic\n",
300-
" unrelated to healthcare.\n",
301-
" - Do not provide specific diagnoses or prescribe medications — always recommend\n",
302-
" consulting a healthcare professional for personalized medical advice.\n",
303-
" - Do not answer questions asking for personal details about the agent or its creators.\n",
304-
" - Allow health-related small talk and greetings.\n",
305-
" - For off-topic requests, politely redirect the conversation.\n",
306-
"\"\"\"\n",
307-
"\n",
308-
"config = RailsConfig.from_content(yaml_content=YAML_CONFIG)\n",
309-
"\n",
310-
"# models order: [main, content_safety, topic_control]\n",
311-
"if DEPLOYMENT == \"local\":\n",
312-
" config.models[0].parameters[\"base_url\"] = \"http://localhost:8001/v1\"\n",
313-
" config.models[1].parameters[\"base_url\"] = \"http://localhost:8123/v1\"\n",
314-
" config.models[1].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemotron-safety-guard-8b-v3\"\n",
315-
" config.models[2].parameters[\"base_url\"] = \"http://localhost:8124/v1\"\n",
316-
" config.models[2].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemoguard-8b-topic-control\"\n",
317-
" config.rails.config.gliner.server_endpoint = \"http://localhost:8000/v1/chat/completions\"\n",
318-
" config.rails.config.jailbreak_detection.nim_base_url = \"http://localhost:8125/v1/\"\n",
319-
"elif DEPLOYMENT == \"remote\":\n",
320-
" config.models[0].api_key_env_var = \"NVIDIA_API_KEY\"\n",
321-
" config.models[1].api_key_env_var = \"NVIDIA_API_KEY\"\n",
322-
" config.models[2].api_key_env_var = \"NVIDIA_API_KEY\"\n",
323-
" config.rails.config.gliner.server_endpoint = \"https://integrate.api.nvidia.com/v1/chat/completions\"\n",
324-
" config.rails.config.gliner.api_key_env_var = \"NVIDIA_API_KEY\"\n",
325-
"\n",
326-
"rails = LLMRails(config)"
327-
]
140+
"source": "# For remote deployment:\n# import os\n# os.environ[\"NVIDIA_API_KEY\"] = \"nvapi-...\"\n\nYAML_CONFIG = \"\"\"\nmodels:\n - type: main\n engine: nim\n model: meta/llama-3.1-8b-instruct\n\n - type: content_safety\n engine: nim\n model: nvidia/llama-3.1-nemotron-safety-guard-8b-v3\n\n - type: topic_control\n engine: nim\n model: nvidia/llama-3.1-nemoguard-8b-topic-control\n\nrails:\n input:\n flows:\n - jailbreak detection model\n - content safety check input $model=content_safety\n - topic safety check input $model=topic_control\n - gliner detect pii on input\n output:\n flows:\n - content safety check output $model=content_safety\n - gliner detect pii on output\n config:\n gliner:\n threshold: 0.5\n input:\n entities:\n - first_name\n - last_name\n - ssn\n - date_of_birth\n - phone_number\n - street_address\n - email\n output:\n entities:\n - first_name\n - last_name\n - ssn\n - date_of_birth\n - phone_number\n - email\n jailbreak_detection:\n nim_base_url: \"https://ai.api.nvidia.com\"\n nim_server_endpoint: \"/v1/security/nvidia/nemoguard-jailbreak-detect\"\n api_key_env_var: NVIDIA_API_KEY\n\nprompts:\n - task: content_safety_check_input $model=content_safety\n content: |\n Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n\n <BEGIN UNSAFE CONTENT CATEGORIES>\n S1: Violence.\n S2: Sexual.\n S3: Criminal Planning/Confessions.\n S4: Guns and Illegal Weapons.\n S5: Controlled/Regulated Substances.\n S6: Suicide and Self Harm.\n S7: Sexual (minor).\n S8: Hate/Identity Hate.\n S9: PII/Privacy.\n S10: Harassment.\n S11: Threat.\n S12: Profanity.\n S13: Needs Caution.\n S14: Other.\n S15: Manipulation.\n S16: Fraud/Deception.\n S17: Malware.\n S18: High Risk Gov Decision Making.\n S19: Political/Misinformation/Conspiracy.\n S20: Copyright/Trademark/Plagiarism.\n S21: Unauthorized Advice.\n S22: Illegal Activity.\n S23: Immoral/Unethical.\n <END UNSAFE CONTENT CATEGORIES>\n\n <BEGIN CONVERSATION>\n\n user: {{ user_input }}\n\n <END CONVERSATION>\n\n Provide your safety assessment for the above conversation in the below output JSON format:\n {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n\n Do not include anything other than the output JSON in your response.\n Output JSON:\n output_parser: nemoguard_parse_prompt_safety\n max_tokens: 50\n\n - task: content_safety_check_output $model=content_safety\n content: |\n Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n\n <BEGIN UNSAFE CONTENT CATEGORIES>\n S1: Violence.\n S2: Sexual.\n S3: Criminal Planning/Confessions.\n S4: Guns and Illegal Weapons.\n S5: Controlled/Regulated Substances.\n S6: Suicide and Self Harm.\n S7: Sexual (minor).\n S8: Hate/Identity Hate.\n S9: PII/Privacy.\n S10: Harassment.\n S11: Threat.\n S12: Profanity.\n S13: Needs Caution.\n S14: Other.\n S15: Manipulation.\n S16: Fraud/Deception.\n S17: Malware.\n S18: High Risk Gov Decision Making.\n S19: Political/Misinformation/Conspiracy.\n S20: Copyright/Trademark/Plagiarism.\n S21: Unauthorized Advice.\n S22: Illegal Activity.\n S23: Immoral/Unethical.\n <END UNSAFE CONTENT CATEGORIES>\n\n <BEGIN CONVERSATION>\n\n user: {{ user_input }}\n\n response: agent: {{ bot_response }}\n\n <END CONVERSATION>\n\n Provide your safety assessment for the above conversation in the below output JSON format:\n {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n\n Do not include anything other than the output JSON in your response.\n Output JSON:\n output_parser: nemoguard_parse_response_safety\n max_tokens: 50\n\n - task: topic_safety_check_input $model=topic_control\n content: |\n You are a patient support assistant for a healthcare organization. Your role is to\n provide general health information, help patients understand symptoms and conditions,\n explain medications and treatments, and assist with appointment and administrative questions.\n\n Guidelines:\n - Only answer questions related to health, wellness, medical conditions, treatments,\n medications, and healthcare administration.\n - Do not answer questions about finance, investments, politics, law, or any topic\n unrelated to healthcare.\n - Do not provide specific diagnoses or prescribe medications — always recommend\n consulting a healthcare professional for personalized medical advice.\n - Do not answer questions asking for personal details about the agent or its creators.\n - Allow health-related small talk and greetings.\n - For off-topic requests, politely redirect the conversation.\n\"\"\"\n\nconfig = RailsConfig.from_content(yaml_content=YAML_CONFIG)\n\n# models order: [main, content_safety, topic_control]\nif DEPLOYMENT == \"local\":\n config.models[0].parameters[\"base_url\"] = \"http://localhost:8001/v1\"\n config.models[1].parameters[\"base_url\"] = \"http://localhost:8123/v1\"\n config.models[1].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemotron-safety-guard-8b-v3\"\n config.models[2].parameters[\"base_url\"] = \"http://localhost:8124/v1\"\n config.models[2].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemoguard-8b-topic-control\"\n config.rails.config.gliner.server_endpoint = \"http://localhost:8000/v1/chat/completions\"\n config.rails.config.jailbreak_detection.nim_base_url = \"http://localhost:8125\"\nelif DEPLOYMENT == \"remote\":\n config.models[0].api_key_env_var = \"NVIDIA_API_KEY\"\n config.models[1].api_key_env_var = \"NVIDIA_API_KEY\"\n config.models[2].api_key_env_var = \"NVIDIA_API_KEY\"\n config.rails.config.gliner.server_endpoint = \"https://integrate.api.nvidia.com/v1/chat/completions\"\n config.rails.config.gliner.api_key_env_var = \"NVIDIA_API_KEY\"\n\nrails = LLMRails(config)"
328141
},
329142
{
330143
"cell_type": "markdown",

0 commit comments

Comments
 (0)