Skip to content

Auto update review: evidence and benchmark candidates - #10

Open
github-actions[bot] wants to merge 1 commit into
mainfrom
auto/update-review-28776898063
Open

Auto update review: evidence and benchmark candidates#10
github-actions[bot] wants to merge 1 commit into
mainfrom
auto/update-review-28776898063

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Auto Update Review PR

This PR was generated by the Agentic AI Security update-review pipeline. It proposes reviewable evidence changes only; maintainers should verify claims before merging.

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: d22ed58f915a7821eadf9467
  • Source: https://genai.owasp.org/ai-security-solutions-landscape/
  • Proposed claim: Solutions Landscape - OWASP Gen AI Security Project Skip to content Join us in London, 6/2 – 6/4 InfoSecurity Europe – OWASP GenAI and Agentic Security Summit | Register Now! GETTING STARTED Introduction MEETINGS CONTRIBUTING EVENTS GLOSSARY RESOURCES All LLM TOP 10 LLM TOP 10 FOR 2025 LLM TOP 10 FOR 2023/24 CHEAT SHEETS WHITEPAPERS TOOLS LEARNING VIDEOS SOLUTIONS DIRECTORY ROADMAP NEWSLETTER PROJECT INITIATIVES AI Security Landscape AIBOM Generator GOVERNANCE CHECKLIST Threat Intelligence AGENTIC APP SECURITY Secure AI Adoption AI Red Teaming Data Security BLOG ABOUT Mission and Charter Governance LEADERSHIP INDUSTRY RECOGNITION CONTRIBUTORS SPONSORS SUPPORTERS SPONSORSHIP NEWSROOM CONTACT BRANDING GEN AI SECURITY Solution landscape AI Security Solutions Landscape The landscape includes traditional and emerging security controls addressing LLM and Generative AI risks in the OWASP Top 10. It is not a comprehensive list or an endorsement but a community resource of open source and proprietary solutions. Contributions are open and reviewed for accuracy. Download Guide Get Cheat Sheets Watch the video See a Solution Missing? Submit - LLM/GenAI Solution Submit - Agentic AI Solution All Scope & Plan Augment/Fine Tune Data Develop & Experiment Test & Evaluate Deploy Operate Monitor Govern Open Source Commercial All LLM10:23 LLM09:23 LLM08:23 LLM07:23 LLM06:23 LLM05:23 LLM04:23 LLM03:23 LLM02:23 LLM01:23 All Commercial Stage: Develop & Experiment , Monitor , Deploy , Test & Evaluate
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, MCP-ASI08-008, MCP-ASI01-005

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: 56caf9ff3c84cf6183b9e125
  • Source: https://genai.owasp.org/initiatives/agentic-security-initiative/
  • Proposed claim: Agentic Security Initiative - OWASP Gen AI Security Project Skip to content Join us in London, 6/2 – 6/4 InfoSecurity Europe – OWASP GenAI and Agentic Security Summit | Register Now! GETTING STARTED Introduction MEETINGS CONTRIBUTING EVENTS GLOSSARY RESOURCES All LLM TOP 10 LLM TOP 10 FOR 2025 LLM TOP 10 FOR 2023/24 CHEAT SHEETS WHITEPAPERS TOOLS LEARNING VIDEOS SOLUTIONS DIRECTORY ROADMAP NEWSLETTER PROJECT INITIATIVES AI Security Landscape AIBOM Generator GOVERNANCE CHECKLIST Threat Intelligence AGENTIC APP SECURITY Secure AI Adoption AI Red Teaming Data Security BLOG ABOUT Mission and Charter Governance LEADERSHIP INDUSTRY RECOGNITION CONTRIBUTORS SPONSORS SUPPORTERS SPONSORSHIP NEWSROOM CONTACT BRANDING GEN AI SECURITY Initiatives Agentic Security Initiative Securing autonomous agents and multi-step AI workflows The Agentic Security Research Initiative explores the emerging security implications of agentic systems, particularly those utilizing advanced frameworks (e.g., LangGraph, AutoGPT, CrewAI) and novel capabilities like Llama 3’s agentic features. Join the Initiative Resource Links: #team-genai-agentic-security-initiative Github Initiative Charter What’s New Resources Learning Videos Blog AI Security Solutions Landscape for Agentic AI Q2 2026 The Solutions Landscape monitors and maps the full Agentic AI lifecycle, focusing on the DevOps–SecOps intersection to meet evolving security needs. Guided by the Agentic AI Download Now A Practical Guide for Secure MCP Server D
  • Supports: ASI01, ASI02, ASI04, ASI05, ASI07
  • Recommended benchmarks: MCP-ASI04-001, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, MCP-ASI01-005, A2A-ASI07-006, MCP-ASI07-003

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for openai

  • Tool: openai
  • Update ID: f3f68bb93d513222ac7c3b9f
  • Source: https://platform.openai.com/docs/guides/safety-best-practices
  • Proposed claim: Safety best practices | OpenAI API Home API Docs Guides and concepts for the OpenAI API API reference Endpoints, parameters, and responses Codex Docs Guides, concepts, and product docs for Codex Use cases Example workflows and tasks teams hand to Codex ChatGPT Apps SDK Build apps to extend ChatGPT Workspace Agents Trigger published ChatGPT workspace agents Commerce Build commerce flows in ChatGPT Ads Publish and measure ads in ChatGPT Resources Showcase Demo apps to get inspired Blog Learnings and experiences from developers Cookbook Notebook examples for building with OpenAI models Learn Docs, videos, and demo apps for building with OpenAI Community Programs, meetups, and support for builders Start searching API Dashboard Search the API docs Search docs Suggested responses create reasoning_effort realtime prompt caching Primary navigation API API Reference Codex ChatGPT Resources Search docs Suggested responses create reasoning_effort realtime prompt caching Get started Overview Quickstart Models Pricing SDKs and CLI OpenAI SDK Agents SDK OpenAI CLI Latest: GPT-5.5 Prompt guidance Core concepts Text generation Code generation Images and vision Audio and speech Structured output Function calling Responses API Using tools Agents SDK Overview Quickstart Agent definitions Models and providers Running agents Sandbox agents Orchestration Guardrails Results and state Integrations and observability Evaluate agent workflows Voice agents ChatKit Overview Customize Widgets Actions Adva
  • Supports: ASI02, ASI05, ASI08, ASI10
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI02-004, MCP-ASI08-008, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for anthropic

  • Tool: anthropic
  • Update ID: c03e04d0f5b12be1ea523f55
  • Source: https://www.anthropic.com/news
  • Proposed claim: Newsroom \ Anthropic Skip to main content Skip to footer Research Policy Commitments Learn News Try Claude Newsroom Press inquires press@anthropic.com Non-media inquiries How to get support Media assets Download press kit Redeploying Fable 5 Announcements Jun 30, 2026 Fable 5 returns globally July 1. We're also proposing an industry-wide framework for scoring jailbreak severity, together with Amazon, Microsoft, Google, and other Glasswing partners. Product Jun 30, 2026 Introducing Claude Sonnet 5 Sonnet 5 delivers frontier performance across coding, agents, and professional work at scale. Announcements Jun 30, 2026 Claude Science, an AI workbench for scientists, is now available Claude Science is a customizable app that integrates the tools and packages researchers most often use, produces auditable artifacts, and provides flexible access to computing resources. Product Jun 23, 2026 Introducing Claude Tag Claude Tag is a new way for teams to work with Claude. Policy Jun 10, 2026 Policy on the AI Exponential AI is advancing at exponential speed, and the policymaking process was built for a slower world. We’re sharing policy proposals to prepare our institutions for AI progress. News Search Date Category Title Jul 2, 2026 Announcements More details on Fable 5’s cyber safeguards and our jailbreak framework Jun 30, 2026 Product Introducing Claude Sonnet 5 Jun 30, 2026 Announcements Redeploying Fable 5 Jun 30, 2026 Announcements Claude Science, an AI workbench for scientists, is n
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI08-008, MCP-ASI01-005, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for anthropic

  • Tool: anthropic
  • Update ID: e7319e154baf5c78224b22a0
  • Source: https://docs.anthropic.com/
  • Proposed claim: Documentation - Claude Platform Docs Claude Platform Docs Messages Managed Agents Admin Resources   API reference English   Console Log in   Browse  Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Loading... Claude Platform Docs Solutions AI agents Code modernization Coding Customer support Education Financial services Government Life sciences Partners Claude on AWS Claude on Google Cloud Learn Blog Courses Use cases Connectors Customer stories Engineering at Anthropic Events Powered by Claude Service partners Startups program Company Anthropic Careers Economic Futures Research News Responsible Scaling Policy Security and compliance Transparency Learn Blog Courses Use cases Connectors Customer stories Engineering at Anthropic Events Powered by Claude Service partners Startups program Help and security Availability Status Support Discord Terms and policies Privacy policy Responsible disclosure policy Terms of service: Commercial Terms of service: Consumer Usage policy Claude Platform Start building with Claude Everything you need to integrate Claude into your applications. From first API call to production.  What do you want to build? ⌘K  Quickstart  Get API key  API reference Python TypeScript Go Java Ruby PHP C# cURL CLI  import anthropic client = anthropic.Anthropic() message = client.messages.create( model = "claude-opus-4-8" , max_tokens = 1024
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for azure-content-safety

  • Tool: azure-content-safety
  • Update ID: 69356c1a654f3e05623aed52
  • Source: https://learn.microsoft.com/en-us/azure/ai-services/content-safety/
  • Proposed claim: Azure AI Content Safety documentation - Quickstarts, Tutorials, API Reference - Foundry Tools | Microsoft Learn Skip to main content This browser is no longer supported. Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support. Download Microsoft Edge More info about Internet Explorer and Microsoft Edge Table of contents Read in English Edit Azure AI Content Safety documentation The cloud-based Azure AI Content Safety API provides developers with access to advanced algorithms for processing images and text and flagging content that is potentially offensive, risky, or otherwise undesirable. About Azure AI Content Safety Overview What is Azure AI Content Safety? Get started Content Safety Studio Concept Harm categories What's new What's new in Azure AI Content Safety Image moderation Concept Harm categories Custom categories (preview) Quickstart Using Content Safety Studio Using the REST API or client SDKs multimodal (/azure/machine-learning/concept-retrieval-augmented-generation How-To Guide Use custom categories (standard) (preview) Use custom categories (rapid) (preview) Text moderation Concept Harm categories Custom categories (preview) Groundedness detection Quickstart Using Content Safety Studio Using the REST API or client SDKs Detect groundedness in LLM responses How-To Guide Use custom categories (standard) (preview) Use custom categories (rapid) (preview) Use a blocklist User input risk detection Concept Prompt Shield
  • Supports: ASI02
  • Recommended benchmarks: RAG-ASI06-007, HITL-ASI09-010, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for aws-bedrock-guardrails

  • Tool: aws-bedrock-guardrails
  • Update ID: 247228a09d6bbe4b70d6ba99
  • Source: https://aws.amazon.com/bedrock/guardrails/
  • Proposed claim: Generative AI Data Governance – Amazon Bedrock Guardrails – AWS Skip to main content Filter: All English Contact us AWS Marketplace Support My account Search Filter: All Sign in to console Create account Amazon Bedrock Overview Getting Started Capabilities Agents Pricing More Generative AI Amazon Bedrock Guardrails Amazon Bedrock Guardrails Implement safeguards customized to your application requirements and responsible AI policies Get started with Guardrails Try free demo Build configurable safeguards Amazon Bedrock Guardrails helps you safely build and deploy responsible generative AI applications with confidence. With industry-leading safety protections that block up to 88% of harmful content and deliver auditable, mathematically verifiable explanations for validation decisions with 99% accuracy, Guardrails provides configurable safeguards to help detect and filter harmful text and image content, redact sensitive information, detect model hallucinations, and more. Guardrails work consistently across any foundation model whether you're using models in Amazon Bedrock or self-hosted models including third-party models such as OpenAI and Google Gemini — giving you the same safety, privacy, and responsible AI controls across all your generative AI applications. Play Explore cross-account safeguards You can now centrally manage AI safety controls across all AWS accounts in your organization while keeping your AI applications aligned with your responsible AI policies. Read the bl
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for aws-bedrock-guardrails

  • Tool: aws-bedrock-guardrails
  • Update ID: d03f151a7d22c491bf6f8a37
  • Source: https://aws.amazon.com/blogs/machine-learning/
  • Proposed claim: Artificial Intelligence Skip to Main Content Filter: All English Contact us AWS Marketplace Support My account Search Filter: All Sign in to console Create account AWS Blogs Home Blogs Editions Artificial Intelligence Safely Releasing Frontier Models to Customers by Amy Herzog on 30 JUN 2026 in Amazon Bedrock , Amazon Machine Learning , Announcements , Featured , Generative AI , Launch , News , Security & Governance , Thought Leadership Permalink Comments Share It’s our goal for AWS to be the most secure place to run any workload, and in support of that we’ve been deeply investing in security across our services since AWS’s inception more than two decades ago. Our AI services like Amazon Bedrock are built on this foundation and with the same focus. How frontier teams are reinventing AI-native development by Swami Sivasubramanian on 10 JUN 2026 in Generative AI , Thought Leadership Permalink Comments Share Frontier teams are not just using AI to code faster. They’re redesigning how software gets built. The result is 4.5x productivity gains, in some cases more than 10x. How Amazon Bedrock catches AI-generated phishing by Radha Panchap and Emilio Herrera on 02 JUL 2026 in Amazon Bedrock , Best Practices , Foundational (100) Permalink Share Social engineering through phishing remains one of the most common tactics for launching cyberattacks. AI-generated phishing email messages now pose a new challenge for security teams managing email systems, significantly raising the risk beca
  • Supports:
  • Recommended benchmarks: MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: a62ad3c1047966f12a22fa53
  • Source: https://cloud.google.com/security/products/model-armor
  • Proposed claim: Model Armor | Google Cloud Page Contents New: Model Armor now offers in-line protection for Gemini Enterprise Agent Platform, Langchain, and more. Learn more. Model Armor Protect AI prompts, responses, and agent interactions Runtime security for generative and agentic AI, Model Armor provides comprehensive protections against prompt injection, sensitive data leaks, and harmful content. Try Model Armor free Read documentation Model Armor offers a free tier; try it today . Product highlights Screens AI prompts, responses, and agent interactions Detects and prevents sensitive data leaks Blocks prompt injection, jailbreaking, and malware Get an executive summary on Model Armor Features AI model and agent threat detection Proactively identifies and blocks prompt injection and jailbreaking techniques designed to manipulate or compromise LLMs and agents. It also detects malicious URLs embedded in prompts or responses before they can cause harm. Granular content safety Provides fine-grained control of harmful, unethical, or undesirable content, such as hate speech, harassment, sexually explicit material, and dangerous topics. Use adjustable confidence thresholds to allow organizations to precisely tune enforcement based on specific application context, user base, and risk tolerance. Integrated sensitive data protection Integrated with Google Cloud's Sensitive Data Protection service and specifically adapted for the unpredictability of AI-generated text. Model Armor helps prevent the
  • Supports: ASI01, ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, MCP-ASI08-008, MCP-ASI01-005, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: b98b6a47921d3827a745f029
  • Source: https://cloud.google.com/vertex-ai/generative-ai/docs/release-notes
  • Proposed claim: Vertex AI release notes | Generative AI on Vertex AI | Google Cloud Documentation Skip to main content Technology areas close AI and ML Application development Application hosting Compute Data analytics and pipelines Databases Distributed, hybrid, and multicloud Industry solutions Migration Networking Observability and monitoring Security Storage Cross-product tools close Access and resources management Costs and usage management Infrastructure as code SDK, languages, frameworks, and tools / Console English Deutsch Español – América Latina Français Português – Brasil 中文 – 简体 日本語 한국어 Sign in Vertex AI Generative AI on Vertex AI Start free Guides API reference Vertex AI Cookbook Prompt gallery Resources FAQ Pricing Technology areas More Guides API reference Vertex AI Cookbook Prompt gallery Resources FAQ Pricing Cross-product tools More Console Resources Getting help Quotas and limits Throughput quota Release notes Current Archive Security bulletins Locations Deployments and endpoints Data residency Supported capabilities Vertex AI and zero data retention Service Level Agreement Deprecations and migrations Deprecations Generative AI module in Vertex AI SDK AI and ML Application development Application hosting Compute Data analytics and pipelines Databases Distributed, hybrid, and multicloud Industry solutions Migration Networking Observability and monitoring Security Storage Access and resources management Costs and usage management Infrastructure as code SDK, languages, framew
  • Supports: ASI02, ASI05, ASI06, ASI08, ASI10
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, MCP-ASI08-008, MCP-ASI02-004, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: a59ba35a99a110db1ece22f2
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.23.0
  • Proposed claim: ## What's Changed This release expands tool calling and observability in IORails. Tool calling now works for streaming and non-streaming requests, including local rails that validate model-emitted tool calls and application-returned tool results. The OpenAI-compatible server also supports tool calling and adds a new /v1/checks endpoint for running input or output rails without generating a new model response. NeMo Guardrails 0.23.0 also adds lightweight Hugging Face classifier rails, context bloat detection, and a Polygraf integration for PII detection and masking. Exact NumPy search replaces Annoy as the default embedding index, removing the native C++ dependency while preserving existing similarity-threshold semantics. Distribution wheels are now approximately ten times smaller. IORails OpenTelemetry support now includes opt-in content capture and richer request, response, and token-usage attributes. LangChain integrations add support for the OpenAI Responses API and Harmony response format models. This release requires Pydantic >=2.5, =2.5,<3.0 and migrate validators and model APIs to Pydantic 2 (#967) ## New Contri
  • Supports: ASI02, ASI08, ASI10
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, RAG-ASI06-007, MCP-ASI08-008, A2A-ASI07-006

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: bc756b11f43e657c4c2d2980
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.22.0
  • Proposed claim: ## What's changed The three major features in this release are: * Anonymous usage reporting: Basic usage reporting is documented with clear privacy boundaries and opt-out controls. The telemetry reference explains what fields are collected, what data is excluded, how local audit files work, and how to opt out with NEMO_GUARDRAILS_NO_USAGE_STATS=1, DO_NOT_TRACK=1, or the ~/.config/nemoguardrails/do_not_track file. * LangChain decoupling: We have had many customers ask us to be less dependent on LangChain. Now, LangChain is optional. For OpenAI-compatible LLMs, we now ship with a built-in client over httpx that offers better direct support. For others, you can bring LangChain back with NEMOGUARDRAILS_LLM_FRAMEWORK=langchain, and install the matching provider package. * IORails milestone 2: The second phase of our new, more efficient orchestration engine. This brings us closer to feature parity to LLMRails by including streaming, OpenTelemetry, and reasoning-model support. We also added speculative generation in non-streaming mode, which runs input rails in parallel with the main application LLM response generation. This hides the latency of input rails while still guara
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: c978da0e3fade029802b96ed
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.21.0
  • Proposed claim: ## What's Changed This release introduces IORails, a new optimized Input/Output rail engine that supports parallel execution of NemoGuard rails (content-safety, topic-safety, and jailbreak detection) with logging and unique request IDs. A new check_async method in LLMRails enables standalone input/output rails validation without requiring a full conversation flow. The guardrails server is now fully OpenAI-compatible (including a new v1/models endpoint), and a new GuardrailsMiddleware enables seamless integration with LangChain agents. New community integrations include PolicyAI for content moderation, CrowdStrike AIDR, and regex-based detection rails. Embedding index initialization is now lazy, improving startup performance. Streaming internals have been cleaned up along with a major documentation revamp. ### 🚀 Features - (library) Update Trend Micro Vision One AI Guard official endpoint (#1546) - (llmrails) Add check_async method for input/output rails validation (#1605) - (server) Make guardrails server OpenAI compatible ([#1340](https://github.com/NVIDI
  • Supports: ASI01
  • Recommended benchmarks: MCP-ASI01-005

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: 44cc7370ed14748551408f78
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.20.0
  • Proposed claim: ## What's Changed This release adds support for reasoning-capable content safety models like Nemotron-Content-Safety-Reasoning-4B (with configurable /think mode for explainable moderation), GLiNER for open-source PII detection, and multilingual refusal messages in content safety rails, along with a major documentation restructuring. Streaming configuration has been simplified (note: the streaming field has been removed from the config). ### 🚀 Features - (llm) Propagate model and base URL in LLMCallException; improve error handling (#1502) - (content_safety) Add support to auto select multilingual refusal bot messages (#1530) - (library) Adding GLiNER for PII detection (open alternative to PrivateAI) (#1545) - (benchmark) Implement Mock LLM streaming (#1564) - (library) Add reasoning guardrail connector (#1565) ### 🐛 Bug Fixes - (models) Surface relevant exception when initializing langchain
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for garak

  • Tool: garak
  • Update ID: 24a621b456556f8617b84273
  • Source: https://github.com/NVIDIA-NeMo/Guardrails/releases/tag/v0.19.0
  • Proposed claim: ## What's Changed LangChain v1.x Support Is Here! This release brings the highly requested LangChain 1.x compatibility to NeMo Guardrails. Many users in the community have been asking for this upgrade, and we're excited to deliver it. NeMo Guardrails now works with the latest LangChain ecosystem, including support for the new content blocks API that enables better handling of reasoning traces and tool calls. This release also introduces several important bug fixes including an important fix related to async streaming for the nim or nvidia_ai_endpoints provider. ### 🚀 Features - Support langchain v1 (#1472) - (llm) Add LangChain 1.x content blocks support for reasoning and tool calls (#1496) - (benchmark) Add Procfile to run Guardrails and mock LLMs (#1490) - (benchmark): Add AIPerf run script ((#1501)) ### 🐛 Bug Fixes - (llm) Add async streaming support to ChatNVIDIA provider patch ([#1504]([Question] caching issues? NVIDIA-NeMo/Guardrails#150
  • Supports: ASI02
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for cortex-cloud-ai-security

  • Tool: cortex-cloud-ai-security
  • Update ID: f951e937bd3fe32738d896e7
  • Source: https://www.paloaltonetworks.com/blog/
  • Proposed claim: Palo Alto Networks Blog | Stay protected in an AI world You are using an outdated browser. Please upgrade your browser to improve your experience. Protect Against Russia-Ukraine Cyber Activity --> Idira Identity Security Platform Reaches FedRAMP High Milestone Executive Summary: Palo Alto Networks has achieved FedRAMP High Authorization for its Identity Security Platform, providing federal agencies with a precertified, SaaS-delivered path to Zero Trust. This milestone enables agencies to protect their most sensitive unclassified data while accelerating Authorization to Operate (ATO) through a unified, platform-based approach to privileged access and identity governance.... Identity Security Products and Services Public Sector Vertical Jun 03, 2026 By Rahul Dubey Latest Blogs Partners A Defining Moment in Identity Security Jun 30, 2026 By Michael Khoury Announcement , Cybersecurity , Points of View New Executive Order Accelerates Post-Quantum Readiness Amid the Cryptograph... Jun 23, 2026 By Anand Oswal AI Security , Cybersecurity , Points of View Built to Last: What Stonehenge Teaches us About IT Architecture & Cybe... Jun 23, 2026 By Helmut Reisinger Announcement Expanding Our Footprint: Local Cloud Availability for Prisma AIRS in Japan Jun 18, 2026 By Hiroshi Alley and Hoseb Dermanillian Cybersecurity , Points of View The Invisible CEO of Crisis: Breaking the Cycle of CISO Burnout Jun 18, 2026 By Andy Schneider AI Security , Announcement Securing the Agentic AI Frontier: Pa
  • Supports: ASI03
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, HITL-ASI09-010, MCP-ASI08-008

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for wiz-ai-spm

  • Tool: wiz-ai-spm
  • Update ID: 633d6487c5b26dcf3f6be032
  • Source: https://www.wiz.io/blog
  • Proposed claim: Wiz Blog | Latest stories about Cloud Security Sign in Experiencing an incident? Wiz Pricing Get a demo Platform Solutions Pricing Resources Customers Company Get a demo All Research AI Product & Company News Security Product Public Sector Data Security CIRT Wiz Agents Blog Featured Build AI Security Agents with Wiz MCP Snegha Ramnarayanan , Shani Gafni , Hen Perez July 2, 2026 Power AI-driven security with trusted security context, Wiz AI Agents, and Wiz AI Skills. Breaking Down the White House’s Actions on Post-Quantum Cryptography Readiness Mitch Herckis July 2, 2026 The U.S. Federal Migration to PQC Just Got Real Start Secure in the AI Era: Accelerating AI Threat Readiness with WizOS Allison Jackson , Daniel Velikanski , Roee Kanari , Ofir Cohen June 30, 2026 As the time from vulnerability discovery to exploitation shrinks, building with minimal, secured components is more important than ever. Here is how WizOS helps. Bridging the Visibility Gap: A Unified Security Operating Model for Hybrid Cloud Teams Shashank Golla , Itay Gershon , Tomer Lev , Shahar Liberman June 29, 2026 Move beyond chasing vulnerabilities to a unified hybrid risk strategy. The Sensor Workload Scanner is now GA and extends our risk prioritization engine to on-premise environments to identify the critical attack paths across your hybrid cloud. The Borderless Attack Surface: Securing Public Sector Hybrid Environments Annam Iyer , Shaked Rotlevi , Bryan Rosensteel June 29, 2026 Aligning Modern CNAPP Tel
  • Supports: ASI02, ASI04, ASI05, ASI07
  • Recommended benchmarks: MCP-ASI04-001, MCP-ASI02-004, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, A2A-ASI07-006, HITL-ASI09-010, MCP-ASI08-008, MCP-ASI07-003

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for activefence-ai-security

  • Tool: activefence-ai-security
  • Update ID: 6a39da2223d845f1e4395049
  • Source: https://www.crowdstrike.com/en-us/blog/
  • Proposed claim: Cybersecurity Blog | CrowdStrike Blog Featured Browser Security: Zero-Days Are Only Part of the Problem Jun 30, 2026 Falcon Cloud Security June 2026 Release: Updates for Azure and Google Cloud Jun 29, 2026 The Identity Problem Hiding in AI Agent Deployments Jun 24, 2026 94% of Organizations Report Cloud Breaches: CrowdStrike State of CDR Survey Jun 22, 2026 Recent Browser Security: Zero-Days Are Only Part of the Problem Jun 30, 2026 Falcon Cloud Security June 2026 Release: Updates for Azure and Google Cloud Jun 29, 2026 The Identity Problem Hiding in AI Agent Deployments Jun 24, 2026 94% of Organizations Report Cloud Breaches: CrowdStrike State of CDR Survey Jun 22, 2026 Video Video Highlights the 4 Key Steps to Successful Incident Response Dec 02, 2019 Helping Non-Security Stakeholders Understand ATT&CK in 10 Minutes or Less [VIDEO] Feb 21, 2019 Analyzing Targeted Intrusions Through the ATT&CK Framework Lens [VIDEO] Jan 22, 2019 Qatar’s Commercial Bank Chooses CrowdStrike Falcon®: A Partnership Based on Trust [VIDEO] Aug 20, 2018 Category Agentic SOC Agentic SOC New Claude Integration Brings Audit Data into the Falcon Platform 05/21/26 How Charlotte AI AgentWorks Fuels Security's Agentic Ecosystem 03/25/26 CrowdStrike Services and Agentic MDR Put the Agentic SOC in Reach 03/24/26 4 Ways Businesses Use CrowdStrike Charlotte AI to Transform Security Operations 03/12/26 Cloud & Application Security Cloud & Application Security Falcon Cloud Security June 2026 Release: Updates fo
  • Supports: ASI03
  • Recommended benchmarks: MCP-ASI07-003, MCP-ASI02-004, HITL-ASI09-010

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for promptfoo

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for promptfoo

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for promptfoo

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

  • Tool: langchain-langgraph
  • Update ID: 0dbe5650469619a881165dd6
  • Source: https://github.com/langchain-ai/langchain/releases/tag/langchain-mistralai%3D%3D1.1.6
  • Proposed claim: Changes since langchain-mistralai==1.1.5 release(mistralai): 1.1.6 (#38684) feat(mistralai): surface citation metadata from chat responses (#37008) chore(model-profiles): refresh model profile data (#38663) chore: bump vcrpy from 8.1.1 to 8.2.1 in /libs/partners/mistralai (#38302) chore: bump langsmith from 0.8.5 to 0.8.18 in /libs/partners/mistralai (#38304) chore(model-profiles): refresh model profile data (#38210) docs: refresh README installation and resources (#38119) release(core): 1.4.7 (#38111) fix(core,partners): rename package version trace metadata (#38110) style(core,langchain,langchain-classic,partners): replace double backticks in docstrings (#38095) release(core): 1.4.6 (#38061) feat(core,partners): add package version tracking to tracing metadata (#35295) chore(infra): bump mypy to 2.1 and unify type-check config across the monorepo (#36470) feat(mistralai): support stop sequences (#38047)
  • Supports: ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

  • Tool: langchain-langgraph
  • Update ID: 405e27a9a97730a7a8fc8c16
  • Source: https://github.com/langchain-ai/langchain/releases/tag/langchain-fireworks%3D%3D1.4.3
  • Proposed claim: Changes since langchain-fireworks==1.4.2 release(fireworks): 1.4.3 chore: bump vcrpy from 8.1.1 to 8.2.1 in /libs/partners/fireworks (#38314) chore: bump langsmith from 0.8.16 to 0.8.18 in /libs/partners/fireworks (#38313) chore: bump langsmith from 0.8.14 to 0.8.16 in /libs/partners/fireworks (#38235) chore: bump pytest from 9.0.3 to 9.1.0 in /libs/partners/fireworks (#38233) chore(model-profiles): refresh model profile data (#38210) chore(model-profiles): refresh model profile data (#38191) chore(model-profiles): refresh model profile data (#38133) docs: refresh README installation and resources (#38119) release(core): 1.4.7 (#38111) fix(core,partners): rename package version trace metadata (#38110) style(core,langchain,langchain-classic,partners): replace double backticks in docstrings (#38095) chore: bump langsmith from 0.8.9 to 0.8.14 in /libs/partners/fireworks (#38093) release(core): 1.4.6 (#38061) feat(core,partners): add package version tracking to tracing metadata (#35295) chore(infra): bump mypy to 2.1 and unify type-check config across the monorepo (#36470) feat(standard-tests): validate tool call chunks during streaming (#34707) chore(partners): bump locks (#38052) h
  • Supports: ASI02, ASI05
  • Recommended benchmarks: MCP-ASI05-002, MCP-ASI04-009, MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for langchain-langgraph

  • Tool: langchain-langgraph
  • Update ID: 8947047e54ca4f85aad1c0ff
  • Source: https://github.com/langchain-ai/langchain/releases/tag/langchain-anthropic%3D%3D1.4.8
  • Proposed claim: Changes since langchain-anthropic==1.4.7 release(anthropic): 1.4.8 (#38490) fix(anthropic): keep initial text on content_block_start (#38442) chore: bump langgraph-checkpoint from 4.1.0 to 4.1.1 in /libs/partners/anthropic (#38479) fix(core): add messages to bare raise ValueError calls (#38158)
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: 1e0daeba326f4df5259a41e7
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.22
  • Proposed claim: # Release Notes ## [2026-05-14] ### llama-index-agent-agentmesh [0.2.0] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-agentops [0.5.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) - mass uv lock --upgrade (#21638) ### llama-index-callbacks-aim [0.4.1] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) ### llama-index-callbacks-arize-phoenix [0.7.0] - chore(deps): bump the pip group across 55 directories with 3 updates (#21435) - mass uv lock --upgrade (#21638) ### llama-index-callbacks-honeyhive [0.5.0] - mass uv lock --upgrade (#21638) ### llama-index-callbacks-langfuse [0.5.0] - chore(deps): bump
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: f8bbfc0a2d559be721845ddc
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.20
  • Proposed claim: # Release Notes ## [2026-04-03] ### llama-index-agent-agentmesh [0.2.0] - fix vulnerability with nltk (#21275) ### llama-index-callbacks-agentops [0.5.0] - chore(deps): bump the uv group across 50 directories with 2 updates (#21164) - chore(deps): bump the uv group across 24 directories with 1 update (#21219) - chore(deps): bump the uv group across 21 directories with 2 updates (#21221) - fix vulnerability with nltk (#21275) ### llama-index-callbacks-aim [0.4.1] - fix vulnerability with nltk (#21275) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the uv group across 58 directories with 1 update (#21166) - chore(deps): bump the uv group across 24 directories with 1 update (#21219) - chore(deps): bump the uv group across 21 directories w
  • Supports:
  • Recommended benchmarks: MCP-ASI02-004

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for llamaindex

  • Tool: llamaindex
  • Update ID: 8dab09487e0d30ef0cb6badf
  • Source: https://github.com/run-llama/llama_index/releases/tag/v0.14.19
  • Proposed claim: # Release Notes ## [2026-03-25] ### llama-index-agent-agentmesh [0.2.0] - chore(deps): bump the uv group across 49 directories with 1 update (#21083) ### llama-index-callbacks-argilla [0.5.0] - chore(deps): bump the uv group across 3 directories with 1 update (#21069) ### llama-index-core [0.14.19] - fix: pass delete_from_docstore parameter in BaseIndex.delete_ref_doc (#20990) - fix(core): preserve CTE names during schema prefixing in SQLDatabase.run_sql (#21028) - fix(core): align sync retrieval dedup key with async (hash + ref_doc_id) (#21034) - fix(core): raise ValueError instead of returning string from structured_predict (#21036) - fix(core): remove incorrect per-node delete calls in index helpers (#21050) - chore(deps): bump the uv group across 49 directories with 1 update ([#21083](https://git
  • Supports:
  • Recommended benchmarks: RAG-ASI06-007

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: 79f70c72fa4dd006b5118b4c
  • Source: https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices
  • Proposed claim: Security Best Practices - Model Context Protocol Documentation Index Fetch the complete documentation index at: /llms.txt Use this file to discover all available pages before exploring further. Skip to main content Model Context Protocol home page Search... ⌘ K Ask Assistant Blog GitHub Search... Navigation Security Security Best Practices Documentation Extensions Specification Registry SEPs Community Get started What is MCP? About MCP Architecture Servers Clients Versioning Develop with MCP Connect to local MCP servers Connect to remote MCP Servers Build with Agent Skills Build an MCP server Clients SDKs Security Understanding Authorization in MCP Security Best Practices Developer tools MCP Inspector Debugging Examples Example Servers On this page Introduction Purpose and Scope Attacks and Mitigations Confused Deputy Problem Terminology Vulnerable Conditions Architecture and Attack Flows Attack Description Mitigation Token Passthrough Risks Mitigation Server-Side Request Forgery (SSRF) Attack Description Risks Mitigation Resources and Tools Session Hijacking Session Hijack Prompt Injection Session Hijack Impersonation Attack Description Mitigation Local MCP Server Compromise Attack Description Risks Mitigation OAuth Authorization URL Validation Attack Description Risks Mitigation stdio Transport Security in Proxy Scenarios Attack Description Risks Mitigation Scope Minimization Attack Description Risks Mitigation Common Mistakes Security Security Best Practices Copy page Secu
  • Supports: ASI01, ASI02, ASI03, ASI04, ASI05, ASI07
  • Recommended benchmarks: MCP-ASI04-001, MCP-ASI02-004, MCP-ASI07-003, MCP-ASI05-002, MCP-ASI04-009, RAG-ASI06-007, A2A-ASI07-006, MCP-ASI01-005

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Review evidence update for google-model-armor

  • Tool: google-model-armor
  • Update ID: 7ee7f3a20c48374b6e5ce1af
  • Source: https://github.com/modelcontextprotocol/modelcontextprotocol/releases/tag/2026-07-28-RC
  • Proposed claim: This release marks the release candidate (RC) 2026-07-28 revision of the Model Context Protocol. The specification is available in draft form. For a detailed overview of changes, see 2026-07-28 draft changelog. >[!NOTE] >To users and implementers: this specification is not final. Changes may be introduced between the RC and the final release. SDKs will adopt this version at their own pace, and the prior version of the spec may remain in use for an undetermined amount of time. > >Refer to the Version Negotiation documentation to learn about the process for clients and servers to determine the version of the protocol being used.
  • Supports: ASI02, ASI04, ASI07
  • Recommended benchmarks: MCP-ASI04-001, MCP-ASI02-004, RAG-ASI06-007, MCP-ASI04-009, A2A-ASI07-006, MCP-ASI07-003

Human Review Checklist

  • Source is official or independently verified
  • Claim is not marketing-only
  • ASI mapping is supported by text evidence
  • No score increase unless benchmark-backed or implementation-verified
  • Relevant benchmark recommendation exists for runtime/security claims

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

automated-update needs-review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@icecreamyoubest