Skip to content

Auto update review: evidence and benchmark candidates - #16

Open
github-actions[bot] wants to merge 1 commit into
mainfrom
auto/update-review-31993608564
Open

Auto update review: evidence and benchmark candidates#16
github-actions[bot] wants to merge 1 commit into
mainfrom
auto/update-review-31993608564

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown

Auto Update Review PR

This PR was generated by the Agentic AI Security update-review pipeline. It proposes reviewable evidence changes only; maintainers should verify claims before merging.

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: d22ed58f915a7821eadf9467
  • Source: https://genai.owasp.org/ai-security-solutions-landscape/
  • Proposed claim: Solutions Directory - OWASP Gen AI Security Project Skip to content Join us in Orlando, FL 10/11 – 10/15 @ InfoSec World 2026 | Register Now! GETTING STARTED Introduction MEETINGS CONTRIBUTING EVENTS GLOSSARY RESOURCES All LLM TOP 10 LLM TOP 10 FOR 2025 LLM TOP 10 FOR 2023/24 CHEAT SHEETS WHITEPAPERS TOOLS LEARNING VIDEOS SOLUTIONS DIRECTORY ROADMAP NEWSLETTER PROJECT INITIATIVES Top 10 for LLM and GenAI AGENTIC SECURITY AI Bill of Materials Ai Data Security AI Security Governance AI Security Solutions AI Threat Intel & Resp AI Red Teaming BLOG ABOUT Mission and Charter Governance LEADERSHIP INDUSTRY RECOGNITION CONTRIBUTORS SPONSORS SUPPORTERS SPONSORSHIP NEWSROOM CONTACT BRANDING GEN AI SECURITY Solution landscape AI Security Landscape – Solutions Directory The landscape directory includes traditional and emerging security controls addressing Generative AI, Agentic and AI Red Teaming. It is not a comprehensive list or an endorsement but a community resource of open source and proprietary solutions. Submit an Entry Search Simple Search content Clear Learn More about the AI Solutions Landscape Initiative Filters: LLMTop10 2025 Select content LLM Risks 2023 LLM01:25 (53) LLM02:25 (49) LLM06:25 (45) LLM07:25 (44) LLM03:25 (36) LLM04:25 (36) LLM05:25 (36) LLM09:25 (36) LLM10:25 (34) LLM08:25 (33) LLMTop10 2023 Select content LLM Risks 2025 LLM01:25 (53) LLM02:25 (49) LLM06:25 (45) LLM07:25 (44) LLM03:25 (36) LLM04:25 (36) LLM05:25 (36) LLM09:25 (36) LLM10:25 (34) LLM08:25 (33) A
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI01-005, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: 56caf9ff3c84cf6183b9e125
  • Source: https://genai.owasp.org/initiatives/agentic-security-initiative/
  • Proposed claim: Agentic Security Initiative - OWASP Gen AI Security Project Skip to content Join us in Orlando, FL 10/11 – 10/15 @ InfoSec World 2026 | Register Now! GETTING STARTED Introduction MEETINGS CONTRIBUTING EVENTS GLOSSARY RESOURCES All LLM TOP 10 LLM TOP 10 FOR 2025 LLM TOP 10 FOR 2023/24 CHEAT SHEETS WHITEPAPERS TOOLS LEARNING VIDEOS SOLUTIONS DIRECTORY ROADMAP NEWSLETTER PROJECT INITIATIVES Top 10 for LLM and GenAI AGENTIC SECURITY AI Bill of Materials Ai Data Security AI Security Governance AI Security Solutions AI Threat Intel & Resp AI Red Teaming BLOG ABOUT Mission and Charter Governance LEADERSHIP INDUSTRY RECOGNITION CONTRIBUTORS SPONSORS SUPPORTERS SPONSORSHIP NEWSROOM CONTACT BRANDING GEN AI SECURITY Initiatives Agentic AI Security Initiative Securing autonomous agents and multi-step AI workflows The Agentic Security Research Initiative explores the emerging security implications of agentic systems, particularly those utilizing advanced frameworks (e.g., LangGraph, AutoGPT, CrewAI) and novel capabilities like Llama 3’s agentic features. Join the Initiative Resource Links: #team-genai-agentic-security-initiative Github Initiative Charter What’s New Resources Learning Videos Blog State of Agentic AI Security and Governance 2.01 The State of Agentic AI Security and Governance provides a comprehensive view of today’s landscape for securing and governing autonomous AI systems. It explores the frameworks, Download Now AIUC-1: Crosswalks OWASP Top 10 For Agentic Applications Th
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI01-005, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for openai

  • Tool: openai
  • Update ID: f3f68bb93d513222ac7c3b9f
  • Source: https://platform.openai.com/docs/guides/safety-best-practices
  • Proposed claim: Safety best practices | OpenAI API For the complete documentation index, see llms.txt . Markdown versions of documentation pages are available by appending .md to the page URL. ChatGPT Home API Codex Docs Guides, concepts, and product docs for Codex Use cases Example workflows and tasks teams can take on with ChatGPT or Codex Docs Use cases Resources ChatGPT Plugins Extend ChatGPT and Codex Workspace Agents Trigger published ChatGPT workspace agents Commerce Build commerce flows in ChatGPT Ads Publish and measure ads in ChatGPT Resources Showcase Demo apps to get inspired Blog Learnings and experiences from developers Cookbook Notebook examples for building with OpenAI models Learn Docs, videos, and demo apps for building with OpenAI Community Programs, meetups, and support for builders Start searching API Dashboard Try ChatGPT Overview Models Agents Tools Voice & Audio Production API reference Search the API docs Search docs Suggested responses create reasoning_effort realtime prompt caching Primary navigation API Codex ChatGPT Docs Use cases Resources Resources Search docs Suggested responses create reasoning_effort realtime prompt caching Overview Models Agents Tools Voice & Audio Production API reference Overview Models Agents Tools Voice & Audio Production API reference Docs section Production Home Get started Quickstart Using GPT-5.6 Key concepts Core concepts Responses API Conversation state Background mode Streaming WebSocket mode Multi-agent Webhooks File inputs Comp
  • Supports: ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for anthropic

  • Tool: anthropic
  • Update ID: c03e04d0f5b12be1ea523f55
  • Source: https://www.anthropic.com/news
  • Proposed claim: Newsroom \ Anthropic Skip to main content Skip to footer Research Policy Commitments Learn News Try Claude Newsroom Press inquiries press@anthropic.com Non-media inquiries How to get support Media assets Download press kit Introducing Claude Opus 5 Product Jul 24, 2026 Opus 5 is a step change improvement for the Opus tier powering long-running agents while delivering improvements in coding and professional work. Announcements Jul 9, 2026 Inviting hard questions We’re asking the public for their hardest questions about AI, and committing to show our work as we address them. Features Jul 6, 2026 The Making of Claude Code The inside story of how Claude Code went from an internal CLI to Anthropic's coding agent, told by researchers, engineers and early users who built it. Announcements Jun 30, 2026 Redeploying Fable 5 Fable 5 returns globally July 1. We're also proposing an industry-wide framework for scoring jailbreak severity, together with Amazon, Microsoft, Google, and other Glasswing partners. Product Jun 30, 2026 Introducing Claude Sonnet 5 Sonnet 5 delivers frontier performance across coding, agents, and professional work at scale. News Search Date Category Title Aug 14, 2026 Announcements How Claude’s text watermark works Aug 7, 2026 Product Improving Fable 5's biology safeguards Aug 4, 2026 Announcements Mariano-Florentino (Tino) Cuéllar to join Anthropic as Chief Global Affairs Officer Jul 30, 2026 Investigating three real-world incidents in our cybersecurity evaluation
  • Supports: ASI01
  • Recommended benchmarks: MCP-ASI01-005

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for anthropic

  • Tool: anthropic
  • Update ID: e7319e154baf5c78224b22a0
  • Source: https://docs.anthropic.com/
  • Proposed claim: Documentation - Claude Platform Docs Claude Platform Docs Messages Managed Agents Admin Resources  Best practices Models & pricing CLI, SDKs, and libraries Claude API skill Release notes  API reference  English   Console Log in  Claude Platform Docs   Solutions AI agents Code modernization Coding Customer support Financial services Government Higher education K-12 teachers Life sciences Partners Claude on AWS Claude on Google Cloud Learn Blog Courses Use cases Connectors Customer stories Engineering at Anthropic Events Powered by Claude Service partners Startups program Company Anthropic Careers Economic Futures Research News Responsible Scaling Policy Security and compliance Transparency Learn Blog Courses Use cases Connectors Customer stories Engineering at Anthropic Events Powered by Claude Service partners Startups program Help and security Availability Status Support Discord Terms and policies Privacy policy Responsible disclosure policy Terms of service: Commercial Terms of service: Consumer Usage policy Claude Platform Start building with Claude Everything you need to integrate Claude into your applications. From first API call to production.  Search Ctrl K  Quickstart  Get API key  API reference Python TypeScript Go Java Ruby PHP C# cURL CLI  import anthropic client = anthropic.Anthropic() message = client.messages.create( model = "claude-opus-5" , max_tokens = 1024 , messages = [{ "role" : "user" , "content" : "Hello, Claude" }] ) for block in message.co
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for azure-content-safety

  • Tool: azure-content-safety
  • Update ID: 69356c1a654f3e05623aed52
  • Source: https://learn.microsoft.com/en-us/azure/ai-services/content-safety/
  • Proposed claim: Azure AI Content Safety documentation - Quickstarts, Tutorials, API Reference - Foundry Tools | Microsoft Learn Skip to main content This browser is no longer supported. Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support. Download Microsoft Edge More info about Internet Explorer and Microsoft Edge Table of contents Read in English Edit Azure AI Content Safety documentation The cloud-based Azure AI Content Safety API provides developers with access to advanced algorithms for processing images and text and flagging content that is potentially offensive, risky, or otherwise undesirable. About Azure AI Content Safety Overview What is Azure AI Content Safety? Get started Content Safety Studio Concept Harm categories What's new What's new in Azure AI Content Safety Image moderation Concept Harm categories Custom categories (preview) Quickstart Using Content Safety Studio Using the REST API or client SDKs multimodal (/azure/machine-learning/concept-retrieval-augmented-generation How-To Guide Use custom categories (standard) (preview) Use custom categories (rapid) (preview) Text moderation Concept Harm categories Custom categories (preview) Groundedness detection Quickstart Using Content Safety Studio Using the REST API or client SDKs Detect groundedness in LLM responses How-To Guide Use custom categories (standard) (preview) Use custom categories (rapid) (preview) Use a blocklist User input risk detection Concept Prompt Shield
  • Supports: ASI02
  • Recommended benchmarks: RAG-ASI06-007, HITL-ASI09-010, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for aws-bedrock-guardrails

  • Tool: aws-bedrock-guardrails
  • Update ID: 247228a09d6bbe4b70d6ba99
  • Source: https://aws.amazon.com/bedrock/guardrails/
  • Proposed claim: Generative AI Data Governance – Amazon Bedrock Guardrails – AWS Skip to main content Filter: All English Contact us AWS Marketplace Support My account re:Invent Discover AWS Products Solutions Pricing Resources Search Filter: All Sign in to console Create account Explore topics re:Invent 2026 The session catalog is live. Explore what's waiting for you Customer Stories See how customers innovate with AWS data & AI Independent Software Vendors AI for software & tech: build, scale, and monetize AI for Small Businesses Solutions designed for your business. Delivered by AWS Partners Agent Toolkit Give AI coding agents up-to-date docs and AWS resource access Amazon Quick Answers grounded in your actual business data News & announcements AWS blog About AWS AWS is the world's most comprehensive cloud, enabling organizations to accelerate innovation, reduce costs, and scale more efficiently Why AWS Getting Started Security Compliance Trust Center Sustainability Global Infrastructure Featured Products Analytics Application Integration Artificial Intelligence Business Applications Compute Customer Experience Databases Developer Tools End User Computing Game Tech Management Tools Media Services Migration & Modernization Multicloud & Hybrid Networking & Content Delivery Operations Security & Identity Storage Supply Chain Browse all products Featured Products Get started with one of these featured services or browse all Browse all products Amazon Quick AI-powered assistant for work with re
  • Supports: ASI02, ASI03, ASI05, ASI06
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, HITL-ASI09-010, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for aws-bedrock-guardrails

  • Tool: aws-bedrock-guardrails
  • Update ID: d03f151a7d22c491bf6f8a37
  • Source: https://aws.amazon.com/blogs/machine-learning/
  • Proposed claim: Artificial Intelligence Skip to Main Content Filter: All English Contact us AWS Marketplace Support My account re:Invent Discover AWS Products Solutions Pricing Resources Search Filter: All Sign in to console Create account Explore topics Event Register now for AWS re:Invent 2026 Data Migration On-prem databases weren't built for agentic AI. Amazon RDS is Independent Software Vendors AI for software & tech: build, scale, and monetize AI for Small Businesses Solutions designed for your business. Delivered by AWS Partners Artificial Intelligence (AI) Accelerate AI from experimentation to production with AWS AI agents AgentCore: One platform to build, connect and optimize agents News & announcements AWS blog About AWS AWS is the world's most comprehensive cloud, enabling organizations to accelerate innovation, reduce costs, and scale more efficiently Why AWS Getting Started Security Compliance Trust Center Sustainability Global Infrastructure Featured Products Analytics Application Integration Artificial Intelligence Business Applications Compute Customer Experience Databases Developer Tools End User Computing Game Tech Management Tools Media Services Migration & Modernization Multicloud & Hybrid Networking & Content Delivery Operations Security & Identity Storage Supply Chain Browse all products Featured Products Get started with one of these featured services or browse all Browse all products Amazon Quick AI-powered assistant for work with research, business insights, automati
  • Supports: ASI02, ASI03, ASI05, ASI06
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, HITL-ASI09-010, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: a62ad3c1047966f12a22fa53
  • Source: https://cloud.google.com/security/products/model-armor
  • Proposed claim: Model Armor | Google Cloud Page Contents New: Model Armor now offers in-line protection for Gemini Enterprise Agent Platform, Langchain, and more. Learn more. Model Armor Protect AI prompts, responses, and agent interactions Runtime security for generative and agentic AI, Model Armor provides comprehensive protections against prompt injection, sensitive data leaks, and harmful content. Try Model Armor free Read documentation Model Armor offers a free tier; try it today . Product highlights Screens AI prompts, responses, and agent interactions Detects and prevents sensitive data leaks Blocks prompt injection, jailbreaking, and malware Get an executive summary on Model Armor Features AI model and agent threat detection Proactively identifies and blocks prompt injection and jailbreaking techniques designed to manipulate or compromise LLMs and agents. It also detects malicious URLs embedded in prompts or responses before they can cause harm. Granular content safety Provides fine-grained control of harmful, unethical, or undesirable content, such as hate speech, harassment, sexually explicit material, and dangerous topics. Use adjustable confidence thresholds to allow organizations to precisely tune enforcement based on specific application context, user base, and risk tolerance. Integrated sensitive data protection Integrated with Google Cloud's Sensitive Data Protection service and specifically adapted for the unpredictability of AI-generated text. Model Armor helps prevent the
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, MCP-ASI08-008, MCP-ASI01-005, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: b98b6a47921d3827a745f029
  • Source: https://cloud.google.com/vertex-ai/generative-ai/docs/release-notes
  • Proposed claim: Vertex AI release notes | Generative AI on Vertex AI | Google Cloud Documentation Skip to main content Technology areas close AI and ML Application development Application hosting Compute Data analytics and pipelines Databases Distributed, hybrid, and multicloud Industry solutions Migration Networking Observability and monitoring Security Storage Cross-product tools close Access and resources management Costs and usage management Infrastructure as code SDK, languages, frameworks, and tools / Console English Deutsch Español – América Latina Français Português – Brasil 中文 – 简体 日本語 한국어 Sign in Vertex AI Generative AI on Vertex AI Start free Guides API reference Vertex AI Cookbook Prompt gallery Resources FAQ Pricing Technology areas More Guides API reference Vertex AI Cookbook Prompt gallery Resources FAQ Pricing Cross-product tools More Console Resources Getting help Quotas and limits Throughput quota Release notes Current Archive Security bulletins Locations Deployments and endpoints Data residency Supported capabilities Vertex AI and zero data retention Service Level Agreement Deprecations and migrations Deprecations Generative AI module in Vertex AI SDK AI and ML Application development Application hosting Compute Data analytics and pipelines Databases Distributed, hybrid, and multicloud Industry solutions Migration Networking Observability and monitoring Security Storage Access and resources management Costs and usage management Infrastructure as code SDK, languages, framew
  • Supports: ASI02, ASI05, ASI06, ASI08, ASI10
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, MCP-ASI08-008, MCP-ASI02-004, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: 11bd47f08a8838185fe81a63
  • Source: https://github.com/NVIDIA/garak/releases/tag/v0.16.0
  • Proposed claim: ## What's Changed ### New features * Feature: technique and intent annotation and initial IntentProbe iteration by garak-maintainers in Feature: technique and intent annotation and initial IntentProbe iteration NVIDIA/garak#1984 "Power is in tearing attacks to pieces and putting them together again in shapes of your own choosing." This initial feature represents the first step in enabling users to provided their own context and requirements for target expectations and enable identification of attack vectors that show the edges of the underlying systems safeguards and runtime posture from new perspectives. This feature includes the initial groundwork for bringing user context into account during evaluation of a target. In its first iteration, it provides a new facet of information while keeping the existing probes primarily unchanged. The trait and intent concepts are explored, with policy introduced only as a reference definition. Further iteration and community feedback will guide how these new concepts are consumed, and influence how the broader Context Aware Scanning feature evolves. * arch: support & data for trait/intent, and for policy datatype by @leondz in arch: support & data for trait/intent, and for policy datatype NVIDIA/garak#1421 * cas feature: int
  • Supports:
  • Recommended benchmarks: RAG-ASI06-007

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: a59ba35a99a110db1ece22f2
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.23.0
  • Proposed claim: ## What's Changed This release expands tool calling and observability in IORails. Tool calling now works for streaming and non-streaming requests, including local rails that validate model-emitted tool calls and application-returned tool results. The OpenAI-compatible server also supports tool calling and adds a new /v1/checks endpoint for running input or output rails without generating a new model response. NeMo Guardrails 0.23.0 also adds lightweight Hugging Face classifier rails, context bloat detection, and a Polygraf integration for PII detection and masking. Exact NumPy search replaces Annoy as the default embedding index, removing the native C++ dependency while preserving existing similarity-threshold semantics. Distribution wheels are now approximately ten times smaller. IORails OpenTelemetry support now includes opt-in content capture and richer request, response, and token-usage attributes. LangChain integrations add support for the OpenAI Responses API and Harmony response format models. This release requires Pydantic >=2.5, =2.5,<3.0 and migrate validators and model APIs to Pydantic 2 (#967) ## New Contri
  • Supports: ASI02, ASI08, ASI10
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, RAG-ASI06-007, MCP-ASI08-008, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: bc756b11f43e657c4c2d2980
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.22.0
  • Proposed claim: ## What's changed The three major features in this release are: * Anonymous usage reporting: Basic usage reporting is documented with clear privacy boundaries and opt-out controls. The telemetry reference explains what fields are collected, what data is excluded, how local audit files work, and how to opt out with NEMO_GUARDRAILS_NO_USAGE_STATS=1, DO_NOT_TRACK=1, or the ~/.config/nemoguardrails/do_not_track file. * LangChain decoupling: We have had many customers ask us to be less dependent on LangChain. Now, LangChain is optional. For OpenAI-compatible LLMs, we now ship with a built-in client over httpx that offers better direct support. For others, you can bring LangChain back with NEMOGUARDRAILS_LLM_FRAMEWORK=langchain, and install the matching provider package. * IORails milestone 2: The second phase of our new, more efficient orchestration engine. This brings us closer to feature parity to LLMRails by including streaming, OpenTelemetry, and reasoning-model support. We also added speculative generation in non-streaming mode, which runs input rails in parallel with the main application LLM response generation. This hides the latency of input rails while still guara
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: c978da0e3fade029802b96ed
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.21.0
  • Proposed claim: ## What's Changed This release introduces IORails, a new optimized Input/Output rail engine that supports parallel execution of NemoGuard rails (content-safety, topic-safety, and jailbreak detection) with logging and unique request IDs. A new check_async method in LLMRails enables standalone input/output rails validation without requiring a full conversation flow. The guardrails server is now fully OpenAI-compatible (including a new v1/models endpoint), and a new GuardrailsMiddleware enables seamless integration with LangChain agents. New community integrations include PolicyAI for content moderation, CrowdStrike AIDR, and regex-based detection rails. Embedding index initialization is now lazy, improving startup performance. Streaming internals have been cleaned up along with a major documentation revamp. ### 🚀 Features - (library) Update Trend Micro Vision One AI Guard official endpoint (#1546) - (llmrails) Add check_async method for input/output rails validation (#1605) - (server) Make guardrails server OpenAI compatible ([#1340](https://github.com/NVIDI
  • Supports: ASI01
  • Recommended benchmarks: MCP-ASI01-005

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: 44cc7370ed14748551408f78
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.20.0
  • Proposed claim: ## What's Changed This release adds support for reasoning-capable content safety models like Nemotron-Content-Safety-Reasoning-4B (with configurable /think mode for explainable moderation), GLiNER for open-source PII detection, and multilingual refusal messages in content safety rails, along with a major documentation restructuring. Streaming configuration has been simplified (note: the streaming field has been removed from the config). ### 🚀 Features - (llm) Propagate model and base URL in LLMCallException; improve error handling (#1502) - (content_safety) Add support to auto select multilingual refusal bot messages (#1530) - (library) Adding GLiNER for PII detection (open alternative to PrivateAI) (#1545) - (benchmark) Implement Mock LLM streaming (#1564) - (library) Add reasoning guardrail connector (#1565) ### 🐛 Bug Fixes - (models) Surface relevant exception when initializing langchain
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: 24a621b456556f8617b84273
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.19.0
  • Proposed claim: ## What's Changed LangChain v1.x Support Is Here! This release brings the highly requested LangChain 1.x compatibility to NeMo Guardrails. Many users in the community have been asking for this upgrade, and we're excited to deliver it. NeMo Guardrails now works with the latest LangChain ecosystem, including support for the new content blocks API that enables better handling of reasoning traces and tool calls. This release also introduces several important bug fixes including an important fix related to async streaming for the nim or nvidia_ai_endpoints provider. ### 🚀 Features - Support langchain v1 (#1472) - (llm) Add LangChain 1.x content blocks support for reasoning and tool calls (#1496) - (benchmark) Add Procfile to run Guardrails and mock LLMs (#1490) - (benchmark): Add AIPerf run script ((#1501)) ### 🐛 Bug Fixes - (llm) Add async streaming support to ChatNVIDIA provider patch ([#1504]([Question] caching issues? NVIDIA-NeMo/Guardrails#150
  • Supports: ASI02
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for cortex-cloud-ai-security

  • Tool: cortex-cloud-ai-security
  • Update ID: f951e937bd3fe32738d896e7
  • Source: https://www.paloaltonetworks.com/blog/
  • Proposed claim: Palo Alto Networks Blog | Stay protected in an AI world You are using an outdated browser. Please upgrade your browser to improve your experience. Protect Against Russia-Ukraine Cyber Activity --> Putting OpenAI Cyber Models to Work for Defenders Unit 42 is putting the latest frontier cyber models to work across customer environments to find, validate and help remediate the attack paths that matter most. Announcement Products and Services Threat Intelligence Unit 42 Aug 12, 2026 By Sam Rubin Latest Blogs Announcement , Products and Services Palo Alto Networks Recognized as the Only Vendor to be Named a 4X Leader in... Aug 10, 2026 By Anupam Upadhyaya Announcement , Products and Services Strengthening Security of AI Coding: Prisma AIRS API Integration with OpenA... Aug 07, 2026 By Srikanth Hanumanula and Spencer Thellmann AI and Cybersecurity , Must-Read Articles , News and Events , Product Features , Uncategorized , Use-Cases Bridging the Gap: An Unprecedented Approach to Browser and Endpoint Securit... Aug 06, 2026 By Maxim Shifrin , Kritika Singhal and Gal Shalev AI Security , Products and Services Prisma AIRS - Unified Data Protection for Claude Aug 05, 2026 By Spencer Thellmann and Srikanth Hanumanula Announcement , Products and Services Redefining Network Security for the Frontier AI Era Aug 04, 2026 By Anand Oswal Announcement Introducing Unit 42 Threat Intelligence: Know What Matters, Understand the ... Aug 03, 2026 By Sam Rubin and Sherrod DeGrippo POPULAR BLOGS The 3
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for wiz-ai-spm

  • Tool: wiz-ai-spm
  • Update ID: 633d6487c5b26dcf3f6be032
  • Source: https://www.wiz.io/blog
  • Proposed claim: Wiz Blog | Latest stories about Cloud Security Sign in Experiencing an incident? Wiz Pricing Get a demo Platform Solutions Pricing Resources Customers Company Get a demo All Research AI Product & Company News Security Product Public Sector Data Security CIRT Wiz Agents Blog Featured Wiz on Wiz: How the Wiz FinOps Team Uses Wiz Cloud Cost + 2 Ron Tzrouya , Guy Aharon , Noa Manor and 2 more August 14, 2026 Powering cost investigation and optimization with deep cloud context Securing Data in the AI era Snegha Ramnarayanan , Shachar Horvitz , Noa Azaria , Chad Knipschild August 14, 2026 AI is changing the context around data risk, making it critical to understand what’s connected, what’s exposed, and why. How to Investigate GitHub PAT Compromise: Lessons From a Multi-Organization Campaign Eden Abergil August 13, 2026 A practical playbook for investigating GitHub token compromise, drawn from Wiz CIRT's response to a coordinated multi-organization campaign. Closing the Blind Spot: Securing Personal Repositories in the Software Supply Chain Karin Sukonik , Salman Ladha , Ziad Ghalleb August 13, 2026 Personal repositories are where corporate secrets quietly escape. Wiz correlates them to your developers, validates the real risk, and drives the fix. Inside the Metabase SQLi: Exploited in the Wild Rami McCarthy August 10, 2026 Reverse engineering Metabase CVE-2026-72898 with AI to accelerate defense. Cloud Threat Highlights: H1 2026 Wiz Threat Research August 6, 2026 Cloud and AI threa
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: 6a39da2223d845f1e4395049
  • Source: https://www.crowdstrike.com/en-us/blog/
  • Proposed claim: Cybersecurity Blog | CrowdStrike Blog Featured August 2026 Patch Tuesday: One Exploited Zero-Day and 62 Critical Vulnerabilities Among 415 CVEs Aug 11, 2026 CrowdStrike Threat Hunts for Shell Command Obfuscation on VMware ESX Aug 07, 2026 Expanding AI Benchmarks in Cybersecurity Beyond Vulnerability Discovery Aug 06, 2026 Secure Agent Harness Execution: Preventing Escape Aug 04, 2026 Recent August 2026 Patch Tuesday: One Exploited Zero-Day and 62 Critical Vulnerabilities Among 415 CVEs Aug 11, 2026 CrowdStrike Threat Hunts for Shell Command Obfuscation on VMware ESX Aug 07, 2026 Expanding AI Benchmarks in Cybersecurity Beyond Vulnerability Discovery Aug 06, 2026 Secure Agent Harness Execution: Preventing Escape Aug 04, 2026 Video Video Highlights the 4 Key Steps to Successful Incident Response Dec 02, 2019 Helping Non-Security Stakeholders Understand ATT&CK in 10 Minutes or Less [VIDEO] Feb 21, 2019 Analyzing Targeted Intrusions Through the ATT&CK Framework Lens [VIDEO] Jan 22, 2019 Qatar’s Commercial Bank Chooses CrowdStrike Falcon®: A Partnership Based on Trust [VIDEO] Aug 20, 2018 Category Agentic SOC Agentic SOC How AI-leading Security Teams Are Building the Agentic SOC 07/06/26 New Claude Integration Brings Audit Data into the Falcon Platform 05/21/26 How Charlotte AI AgentWorks Fuels Security's Agentic Ecosystem 03/25/26 CrowdStrike Services and Agentic MDR Put the Agentic SOC in Reach 03/24/26 Cloud & Application Security Cloud & Application Security Falcon Cloud Secur
  • Supports:
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, A2A-ASI07-006, HITL-ASI09-010

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for promptfoo

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for promptfoo

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for phoenix

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for phoenix

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for phoenix

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

  • Tool: langchain-langgraph
  • Update ID: 3778885d1aa4ad61dfe2c09e
  • Source: https://github.com/langchain-ai/langchain/releases/tag/langchain-core%3D%3D1.5.5
  • Proposed claim: Changes since langchain-core==1.5.4 release(core): 1.5.5 (#39655) fix(core): make abatch_iterate consistent with batch_iterate for None and zero size (#39367) fix(core): respect pydantic aliases when validating tool inputs (#39572) fix(core): issues in merging chunks (#39535) fix(core): handle v1 base model validation in async path (#39576) fix(core): handle tool descriptions for infer_schema=False (#39573) fix(core): clear usage metadata callback on exceptions in context manager (#39616) fix(core): handle falsy LLM and chat model caches (#39283) chore(core): add httpx as an explicit dep (#39612) fix(core): preserve non-str/non-dict items in DictPromptTemplate list values (#39588) fix(core): raise ValueError when explicit tool_outputs length mismatches tool_calls in tool_example_to_messages (#39142) fix(core): guard malformed Anthropic content blocks (#38670)
  • Supports: ASI02
  • Recommended benchmarks: RAG-ASI06-007, MCP-ASI08-008, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: 1e0daeba326f4df5259a41e7
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.22
  • Proposed claim: # Release Notes ## [2026-05-14] ### llama-index-agent-agentmesh [0.2.0] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-agentops [0.5.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) - mass uv lock --upgrade (#21638) ### llama-index-callbacks-aim [0.4.1] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) ### llama-index-callbacks-arize-phoenix [0.7.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) - mass uv lock --upgrade (#21638) ### llama-index-callbacks-honeyhive [0.5.0] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-langfuse [0.5.0] - chore(deps): bump
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: f8bbfc0a2d559be721845ddc
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.20
  • Proposed claim: # Release Notes ## [2026-04-03] ### llama-index-agent-agentmesh [0.2.0] - fix vulnerability with nltk (#21275) ### llama-index-callbacks-agentops [0.5.0] - chore(deps): bump the uv group across 50 directories with 2 updates (#21164) - chore(deps): bump the uv group across 24 directories with 1 update (#21219) - chore(deps): bump the uv group across 21 directories with 2 updates (#21221) - fix vulnerability with nltk (#21275) ### llama-index-callbacks-aim [0.4.1] - fix vulnerability with nltk (#21275) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the uv group across 58 directories with 1 update (#21166) - chore(deps): bump the uv group across 24 directories with 1 update (#21219) - chore(deps): bump the uv group across 21 directories w
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: 8dab09487e0d30ef0cb6badf
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.19
  • Proposed claim: # Release Notes ## [2026-03-25] ### llama-index-agent-agentmesh [0.2.0] - chore(deps): bump the uv group across 49 directories with 1 update (#21083) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the uv group across 3 directories with 1 update (#21069) ### llama-index-core [0.14.19] - fix: pass delete_from_docstore parameter in BaseIndex.delete_ref_doc (#20990) - fix(core): preserve CTE names during schema prefixing in SQLDatabase.run_sql (#21028) - fix(core): align sync retrieval dedup key with async (hash + ref_doc_id) (#21034) - fix(core): raise ValueError instead of returning string from structured_predict (#21036) - fix(core): remove incorrect per-node delete calls in index helpers (#21050) - chore(deps): bump the uv group across 49 directories with 1 update ([#21083](https://git
  • Supports:
  • Recommended benchmarks: RAG-ASI06-007

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: 79f70c72fa4dd006b5118b4c
  • Source: https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices
  • Proposed claim: Security Best Practices - Model Context Protocol Documentation Index Fetch the complete documentation index at: /llms.txt Use this file to discover all available pages before exploring further. Skip to main content Model Context Protocol home page Version 2026-07-28 (latest) Search... ⌘ K Ask Assistant Blog GitHub Search... Navigation Security Security Best Practices Documentation Specification Extensions Registry SEPs Community Get started What is MCP? About MCP Architecture Servers Clients Versioning Develop with MCP Connect to local MCP servers Connect to remote MCP Servers Build with Agent Skills Build an MCP server Clients SDKs Security Understanding Authorization in MCP Security Best Practices Developer tools Inspector Debugging Examples Example Servers On this page Introduction Purpose and Scope Attacks and Mitigations Confused Deputy Problem Terminology Vulnerable Conditions Architecture and Attack Flows Attack Description Mitigation Token Passthrough Risks Mitigation Server-Side Request Forgery (SSRF) Attack Description Risks Mitigation SSRF Against Authorization Servers Resources and Tools State Handle Hijacking Attack Description Mitigation Local MCP Server Compromise Attack Description Risks Mitigation OAuth Authorization URL Validation Attack Description Risks Mitigation stdio Transport Security in Proxy Scenarios Attack Description Risks Mitigation Mix-Up Attacks Attack Description Mitigation Localhost Redirect URI Impersonation Attack Description Mitigation CIM
  • Supports: ASI02, ASI03, ASI04, ASI05, ASI07
  • Recommended benchmarks: MCP-ASI04-001, MCP-ASI02-004, MCP-ASI07-003, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

automated-update needs-review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@icecreamyoubest