|
130 | 130 | { |
131 | 131 | "cell_type": "markdown", |
132 | 132 | "metadata": {}, |
133 | | - "source": [ |
134 | | - "## Combined Configuration\n", |
135 | | - "\n", |
136 | | - "All four guardrails are configured in a single `RailsConfig`. The `prompts` section provides the safety classification templates for the content safety and topic control NIMs." |
137 | | - ] |
| 133 | + "source": "## Combined Configuration\n\nAll four guardrails are configured in a single `RailsConfig`. The `prompts` section provides the safety classification templates for the content safety and topic control NIMs.\n\n**Note on the jailbreak-detection NIM URL convention.** For the NemoGuard NIM, `nim_base_url` should NOT include a `/v1/` suffix — the `/v1` lives in `nim_server_endpoint` (set to `/v1/security/nvidia/nemoguard-jailbreak-detect` in the YAML below). The [`join_nim_url`](../../nemoguardrails/library/jailbreak_detection/request.py) helper rstrips the base's trailing slash, lstrips the endpoint's leading slash, and `urljoin`s the two relatively — so a `/v1/` in the base would produce a doubled-path 404 (e.g., `http://localhost:8125/v1/v1/security/...`). The convention applies to both remote and local deployments; the local override below preserves it.\n" |
138 | 134 | }, |
139 | 135 | { |
140 | 136 | "cell_type": "code", |
141 | 137 | "execution_count": null, |
142 | 138 | "metadata": {}, |
143 | 139 | "outputs": [], |
144 | | - "source": [ |
145 | | - "# For remote deployment:\n", |
146 | | - "# import os\n", |
147 | | - "# os.environ[\"NVIDIA_API_KEY\"] = \"nvapi-...\"\n", |
148 | | - "\n", |
149 | | - "YAML_CONFIG = \"\"\"\n", |
150 | | - "models:\n", |
151 | | - " - type: main\n", |
152 | | - " engine: nim\n", |
153 | | - " model: meta/llama-3.1-8b-instruct\n", |
154 | | - "\n", |
155 | | - " - type: content_safety\n", |
156 | | - " engine: nim\n", |
157 | | - " model: nvidia/llama-3.1-nemotron-safety-guard-8b-v3\n", |
158 | | - "\n", |
159 | | - " - type: topic_control\n", |
160 | | - " engine: nim\n", |
161 | | - " model: nvidia/llama-3.1-nemoguard-8b-topic-control\n", |
162 | | - "\n", |
163 | | - "rails:\n", |
164 | | - " input:\n", |
165 | | - " flows:\n", |
166 | | - " - jailbreak detection model\n", |
167 | | - " - content safety check input $model=content_safety\n", |
168 | | - " - topic safety check input $model=topic_control\n", |
169 | | - " - gliner detect pii on input\n", |
170 | | - " output:\n", |
171 | | - " flows:\n", |
172 | | - " - content safety check output $model=content_safety\n", |
173 | | - " - gliner detect pii on output\n", |
174 | | - " config:\n", |
175 | | - " gliner:\n", |
176 | | - " threshold: 0.5\n", |
177 | | - " input:\n", |
178 | | - " entities:\n", |
179 | | - " - first_name\n", |
180 | | - " - last_name\n", |
181 | | - " - ssn\n", |
182 | | - " - date_of_birth\n", |
183 | | - " - phone_number\n", |
184 | | - " - street_address\n", |
185 | | - " - email\n", |
186 | | - " output:\n", |
187 | | - " entities:\n", |
188 | | - " - first_name\n", |
189 | | - " - last_name\n", |
190 | | - " - ssn\n", |
191 | | - " - date_of_birth\n", |
192 | | - " - phone_number\n", |
193 | | - " - email\n", |
194 | | - " jailbreak_detection:\n", |
195 | | - " nim_base_url: \"https://ai.api.nvidia.com\"\n", |
196 | | - " nim_server_endpoint: \"/v1/security/nvidia/nemoguard-jailbreak-detect\"\n", |
197 | | - " api_key_env_var: NVIDIA_API_KEY\n", |
198 | | - "\n", |
199 | | - "prompts:\n", |
200 | | - " - task: content_safety_check_input $model=content_safety\n", |
201 | | - " content: |\n", |
202 | | - " Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n", |
203 | | - "\n", |
204 | | - " <BEGIN UNSAFE CONTENT CATEGORIES>\n", |
205 | | - " S1: Violence.\n", |
206 | | - " S2: Sexual.\n", |
207 | | - " S3: Criminal Planning/Confessions.\n", |
208 | | - " S4: Guns and Illegal Weapons.\n", |
209 | | - " S5: Controlled/Regulated Substances.\n", |
210 | | - " S6: Suicide and Self Harm.\n", |
211 | | - " S7: Sexual (minor).\n", |
212 | | - " S8: Hate/Identity Hate.\n", |
213 | | - " S9: PII/Privacy.\n", |
214 | | - " S10: Harassment.\n", |
215 | | - " S11: Threat.\n", |
216 | | - " S12: Profanity.\n", |
217 | | - " S13: Needs Caution.\n", |
218 | | - " S14: Other.\n", |
219 | | - " S15: Manipulation.\n", |
220 | | - " S16: Fraud/Deception.\n", |
221 | | - " S17: Malware.\n", |
222 | | - " S18: High Risk Gov Decision Making.\n", |
223 | | - " S19: Political/Misinformation/Conspiracy.\n", |
224 | | - " S20: Copyright/Trademark/Plagiarism.\n", |
225 | | - " S21: Unauthorized Advice.\n", |
226 | | - " S22: Illegal Activity.\n", |
227 | | - " S23: Immoral/Unethical.\n", |
228 | | - " <END UNSAFE CONTENT CATEGORIES>\n", |
229 | | - "\n", |
230 | | - " <BEGIN CONVERSATION>\n", |
231 | | - "\n", |
232 | | - " user: {{ user_input }}\n", |
233 | | - "\n", |
234 | | - " <END CONVERSATION>\n", |
235 | | - "\n", |
236 | | - " Provide your safety assessment for the above conversation in the below output JSON format:\n", |
237 | | - " {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n", |
238 | | - "\n", |
239 | | - " Do not include anything other than the output JSON in your response.\n", |
240 | | - " Output JSON:\n", |
241 | | - " output_parser: nemoguard_parse_prompt_safety\n", |
242 | | - " max_tokens: 50\n", |
243 | | - "\n", |
244 | | - " - task: content_safety_check_output $model=content_safety\n", |
245 | | - " content: |\n", |
246 | | - " Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n", |
247 | | - "\n", |
248 | | - " <BEGIN UNSAFE CONTENT CATEGORIES>\n", |
249 | | - " S1: Violence.\n", |
250 | | - " S2: Sexual.\n", |
251 | | - " S3: Criminal Planning/Confessions.\n", |
252 | | - " S4: Guns and Illegal Weapons.\n", |
253 | | - " S5: Controlled/Regulated Substances.\n", |
254 | | - " S6: Suicide and Self Harm.\n", |
255 | | - " S7: Sexual (minor).\n", |
256 | | - " S8: Hate/Identity Hate.\n", |
257 | | - " S9: PII/Privacy.\n", |
258 | | - " S10: Harassment.\n", |
259 | | - " S11: Threat.\n", |
260 | | - " S12: Profanity.\n", |
261 | | - " S13: Needs Caution.\n", |
262 | | - " S14: Other.\n", |
263 | | - " S15: Manipulation.\n", |
264 | | - " S16: Fraud/Deception.\n", |
265 | | - " S17: Malware.\n", |
266 | | - " S18: High Risk Gov Decision Making.\n", |
267 | | - " S19: Political/Misinformation/Conspiracy.\n", |
268 | | - " S20: Copyright/Trademark/Plagiarism.\n", |
269 | | - " S21: Unauthorized Advice.\n", |
270 | | - " S22: Illegal Activity.\n", |
271 | | - " S23: Immoral/Unethical.\n", |
272 | | - " <END UNSAFE CONTENT CATEGORIES>\n", |
273 | | - "\n", |
274 | | - " <BEGIN CONVERSATION>\n", |
275 | | - "\n", |
276 | | - " user: {{ user_input }}\n", |
277 | | - "\n", |
278 | | - " response: agent: {{ bot_response }}\n", |
279 | | - "\n", |
280 | | - " <END CONVERSATION>\n", |
281 | | - "\n", |
282 | | - " Provide your safety assessment for the above conversation in the below output JSON format:\n", |
283 | | - " {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n", |
284 | | - "\n", |
285 | | - " Do not include anything other than the output JSON in your response.\n", |
286 | | - " Output JSON:\n", |
287 | | - " output_parser: nemoguard_parse_response_safety\n", |
288 | | - " max_tokens: 50\n", |
289 | | - "\n", |
290 | | - " - task: topic_safety_check_input $model=topic_control\n", |
291 | | - " content: |\n", |
292 | | - " You are a patient support assistant for a healthcare organization. Your role is to\n", |
293 | | - " provide general health information, help patients understand symptoms and conditions,\n", |
294 | | - " explain medications and treatments, and assist with appointment and administrative questions.\n", |
295 | | - "\n", |
296 | | - " Guidelines:\n", |
297 | | - " - Only answer questions related to health, wellness, medical conditions, treatments,\n", |
298 | | - " medications, and healthcare administration.\n", |
299 | | - " - Do not answer questions about finance, investments, politics, law, or any topic\n", |
300 | | - " unrelated to healthcare.\n", |
301 | | - " - Do not provide specific diagnoses or prescribe medications — always recommend\n", |
302 | | - " consulting a healthcare professional for personalized medical advice.\n", |
303 | | - " - Do not answer questions asking for personal details about the agent or its creators.\n", |
304 | | - " - Allow health-related small talk and greetings.\n", |
305 | | - " - For off-topic requests, politely redirect the conversation.\n", |
306 | | - "\"\"\"\n", |
307 | | - "\n", |
308 | | - "config = RailsConfig.from_content(yaml_content=YAML_CONFIG)\n", |
309 | | - "\n", |
310 | | - "# models order: [main, content_safety, topic_control]\n", |
311 | | - "if DEPLOYMENT == \"local\":\n", |
312 | | - " config.models[0].parameters[\"base_url\"] = \"http://localhost:8001/v1\"\n", |
313 | | - " config.models[1].parameters[\"base_url\"] = \"http://localhost:8123/v1\"\n", |
314 | | - " config.models[1].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemotron-safety-guard-8b-v3\"\n", |
315 | | - " config.models[2].parameters[\"base_url\"] = \"http://localhost:8124/v1\"\n", |
316 | | - " config.models[2].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemoguard-8b-topic-control\"\n", |
317 | | - " config.rails.config.gliner.server_endpoint = \"http://localhost:8000/v1/chat/completions\"\n", |
318 | | - " config.rails.config.jailbreak_detection.nim_base_url = \"http://localhost:8125/v1/\"\n", |
319 | | - "elif DEPLOYMENT == \"remote\":\n", |
320 | | - " config.models[0].api_key_env_var = \"NVIDIA_API_KEY\"\n", |
321 | | - " config.models[1].api_key_env_var = \"NVIDIA_API_KEY\"\n", |
322 | | - " config.models[2].api_key_env_var = \"NVIDIA_API_KEY\"\n", |
323 | | - " config.rails.config.gliner.server_endpoint = \"https://integrate.api.nvidia.com/v1/chat/completions\"\n", |
324 | | - " config.rails.config.gliner.api_key_env_var = \"NVIDIA_API_KEY\"\n", |
325 | | - "\n", |
326 | | - "rails = LLMRails(config)" |
327 | | - ] |
| 140 | + "source": "# For remote deployment:\n# import os\n# os.environ[\"NVIDIA_API_KEY\"] = \"nvapi-...\"\n\nYAML_CONFIG = \"\"\"\nmodels:\n - type: main\n engine: nim\n model: meta/llama-3.1-8b-instruct\n\n - type: content_safety\n engine: nim\n model: nvidia/llama-3.1-nemotron-safety-guard-8b-v3\n\n - type: topic_control\n engine: nim\n model: nvidia/llama-3.1-nemoguard-8b-topic-control\n\nrails:\n input:\n flows:\n - jailbreak detection model\n - content safety check input $model=content_safety\n - topic safety check input $model=topic_control\n - gliner detect pii on input\n output:\n flows:\n - content safety check output $model=content_safety\n - gliner detect pii on output\n config:\n gliner:\n threshold: 0.5\n input:\n entities:\n - first_name\n - last_name\n - ssn\n - date_of_birth\n - phone_number\n - street_address\n - email\n output:\n entities:\n - first_name\n - last_name\n - ssn\n - date_of_birth\n - phone_number\n - email\n jailbreak_detection:\n nim_base_url: \"https://ai.api.nvidia.com\"\n nim_server_endpoint: \"/v1/security/nvidia/nemoguard-jailbreak-detect\"\n api_key_env_var: NVIDIA_API_KEY\n\nprompts:\n - task: content_safety_check_input $model=content_safety\n content: |\n Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n\n <BEGIN UNSAFE CONTENT CATEGORIES>\n S1: Violence.\n S2: Sexual.\n S3: Criminal Planning/Confessions.\n S4: Guns and Illegal Weapons.\n S5: Controlled/Regulated Substances.\n S6: Suicide and Self Harm.\n S7: Sexual (minor).\n S8: Hate/Identity Hate.\n S9: PII/Privacy.\n S10: Harassment.\n S11: Threat.\n S12: Profanity.\n S13: Needs Caution.\n S14: Other.\n S15: Manipulation.\n S16: Fraud/Deception.\n S17: Malware.\n S18: High Risk Gov Decision Making.\n S19: Political/Misinformation/Conspiracy.\n S20: Copyright/Trademark/Plagiarism.\n S21: Unauthorized Advice.\n S22: Illegal Activity.\n S23: Immoral/Unethical.\n <END UNSAFE CONTENT CATEGORIES>\n\n <BEGIN CONVERSATION>\n\n user: {{ user_input }}\n\n <END CONVERSATION>\n\n Provide your safety assessment for the above conversation in the below output JSON format:\n {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n\n Do not include anything other than the output JSON in your response.\n Output JSON:\n output_parser: nemoguard_parse_prompt_safety\n max_tokens: 50\n\n - task: content_safety_check_output $model=content_safety\n content: |\n Task: Check if there is unsafe content in the user or agent messages in the below conversation in accordance with our safety policy with the below categories. Provide a 'safe' or 'unsafe' rating for both the user message and the agent response. If there is unsafe content, please also provide a list of violated categories according to our safety policy below.\n\n <BEGIN UNSAFE CONTENT CATEGORIES>\n S1: Violence.\n S2: Sexual.\n S3: Criminal Planning/Confessions.\n S4: Guns and Illegal Weapons.\n S5: Controlled/Regulated Substances.\n S6: Suicide and Self Harm.\n S7: Sexual (minor).\n S8: Hate/Identity Hate.\n S9: PII/Privacy.\n S10: Harassment.\n S11: Threat.\n S12: Profanity.\n S13: Needs Caution.\n S14: Other.\n S15: Manipulation.\n S16: Fraud/Deception.\n S17: Malware.\n S18: High Risk Gov Decision Making.\n S19: Political/Misinformation/Conspiracy.\n S20: Copyright/Trademark/Plagiarism.\n S21: Unauthorized Advice.\n S22: Illegal Activity.\n S23: Immoral/Unethical.\n <END UNSAFE CONTENT CATEGORIES>\n\n <BEGIN CONVERSATION>\n\n user: {{ user_input }}\n\n response: agent: {{ bot_response }}\n\n <END CONVERSATION>\n\n Provide your safety assessment for the above conversation in the below output JSON format:\n {\"User Safety\": safe or unsafe rating for the user message, \"Response Safety\": safe or unsafe rating for the agent response. Omit if no agent response present. \"Safety Categories\": a comma-separated list of applicable safety categories from the provided taxonomy. Omit if all safe.}\n\n Do not include anything other than the output JSON in your response.\n Output JSON:\n output_parser: nemoguard_parse_response_safety\n max_tokens: 50\n\n - task: topic_safety_check_input $model=topic_control\n content: |\n You are a patient support assistant for a healthcare organization. Your role is to\n provide general health information, help patients understand symptoms and conditions,\n explain medications and treatments, and assist with appointment and administrative questions.\n\n Guidelines:\n - Only answer questions related to health, wellness, medical conditions, treatments,\n medications, and healthcare administration.\n - Do not answer questions about finance, investments, politics, law, or any topic\n unrelated to healthcare.\n - Do not provide specific diagnoses or prescribe medications — always recommend\n consulting a healthcare professional for personalized medical advice.\n - Do not answer questions asking for personal details about the agent or its creators.\n - Allow health-related small talk and greetings.\n - For off-topic requests, politely redirect the conversation.\n\"\"\"\n\nconfig = RailsConfig.from_content(yaml_content=YAML_CONFIG)\n\n# models order: [main, content_safety, topic_control]\nif DEPLOYMENT == \"local\":\n config.models[0].parameters[\"base_url\"] = \"http://localhost:8001/v1\"\n config.models[1].parameters[\"base_url\"] = \"http://localhost:8123/v1\"\n config.models[1].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemotron-safety-guard-8b-v3\"\n config.models[2].parameters[\"base_url\"] = \"http://localhost:8124/v1\"\n config.models[2].parameters[\"model_name\"] = \"nvidia/llama-3.1-nemoguard-8b-topic-control\"\n config.rails.config.gliner.server_endpoint = \"http://localhost:8000/v1/chat/completions\"\n config.rails.config.jailbreak_detection.nim_base_url = \"http://localhost:8125\"\nelif DEPLOYMENT == \"remote\":\n config.models[0].api_key_env_var = \"NVIDIA_API_KEY\"\n config.models[1].api_key_env_var = \"NVIDIA_API_KEY\"\n config.models[2].api_key_env_var = \"NVIDIA_API_KEY\"\n config.rails.config.gliner.server_endpoint = \"https://integrate.api.nvidia.com/v1/chat/completions\"\n config.rails.config.gliner.api_key_env_var = \"NVIDIA_API_KEY\"\n\nrails = LLMRails(config)" |
328 | 141 | }, |
329 | 142 | { |
330 | 143 | "cell_type": "markdown", |
|
0 commit comments