diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Microsoft.Extensions.AI.Evaluation.Console.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Microsoft.Extensions.AI.Evaluation.Console.csproj index 5980faefe72..91b398ce281 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Microsoft.Extensions.AI.Evaluation.Console.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/Microsoft.Extensions.AI.Evaluation.Console.csproj @@ -1,7 +1,7 @@  - A dotnet tool for managing the evaluation data and generating reports. + A command line dotnet tool for generating reports and managing evaluation data. Exe diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/README.md index ba6f61c446a..c21e2a299ad 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Console/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/Microsoft.Extensions.AI.Evaluation.Quality.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/Microsoft.Extensions.AI.Evaluation.Quality.csproj index ef591150f95..07771569ba0 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/Microsoft.Extensions.AI.Evaluation.Quality.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/Microsoft.Extensions.AI.Evaluation.Quality.csproj @@ -1,7 +1,7 @@  - A library containing a set of evaluators for evaluating the quality (coherence, relevance, truth, completeness, groundedness, fluency, equivalence etc.) of responses received from an LLM. + A library containing evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. $(TargetFrameworks);netstandard2.0 Microsoft.Extensions.AI.Evaluation.Quality diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/README.md index ba6f61c446a..c21e2a299ad 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Quality/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/Microsoft.Extensions.AI.Evaluation.Reporting.Azure.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/Microsoft.Extensions.AI.Evaluation.Reporting.Azure.csproj index f705add750e..669bbab7556 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/Microsoft.Extensions.AI.Evaluation.Reporting.Azure.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/Microsoft.Extensions.AI.Evaluation.Reporting.Azure.csproj @@ -1,7 +1,7 @@  - A library that provides additional an additional storage provider based on Azure Storage containers. + A library that supports the Microsoft.Extensions.AI.Evaluation.Reporting library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. $(TargetFrameworks);netstandard2.0 Microsoft.Extensions.AI.Evaluation.Reporting diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/README.md index ba6f61c446a..c21e2a299ad 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting.Azure/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Microsoft.Extensions.AI.Evaluation.Reporting.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Microsoft.Extensions.AI.Evaluation.Reporting.csproj index 1a920c47e8d..a06db14fffd 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Microsoft.Extensions.AI.Evaluation.Reporting.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/Microsoft.Extensions.AI.Evaluation.Reporting.csproj @@ -8,7 +8,7 @@ --> - A library for aggregating and reporting evaluation data. This library also includes support for caching LLM responses. + A library that contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. $(TargetFrameworks);netstandard2.0 Microsoft.Extensions.AI.Evaluation.Reporting diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/README.md index ba6f61c446a..c21e2a299ad 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/CSharp/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/CodeVulnerabilityEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/CodeVulnerabilityEvaluator.cs index c708b6c2cd3..9ab4508b407 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/CodeVulnerabilityEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/CodeVulnerabilityEvaluator.cs @@ -9,8 +9,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate code completion responses -/// produced by an AI model for the presence of vulnerable code. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate code completion +/// responses produced by an AI model for the presence of vulnerable code. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentHarmEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentHarmEvaluator.cs index 5c6c77e6791..e4788bcdc81 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentHarmEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentHarmEvaluator.cs @@ -9,8 +9,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of a variety of harmful content such as violence, hate speech, etc. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of a variety of harmful content such as violence, hate speech, etc. /// /// /// can be used to evaluate responses for all supported content harm metrics in one @@ -22,10 +22,10 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// /// /// A optional dictionary containing the mapping from the names of the metrics that are used when communicating -/// with the Azure AI Content Safety to the s of the +/// with the Azure AI Foundry Evaluation service, to the s of the /// s returned by this . /// -/// If omitted, includes mappings for all content harm metrics that are supported by the Azure AI Content Safety +/// If omitted, includes mappings for all content harm metrics that are supported by the Azure AI Foundry Evaluation /// service. This includes , /// , and /// . diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyChatClient.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyChatClient.cs index f6c2f028b81..69b47670935 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyChatClient.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyChatClient.cs @@ -19,7 +19,7 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; internal sealed class ContentSafetyChatClient : IChatClient { - private const string Moniker = "Azure AI Content Safety"; + private const string Moniker = "Azure AI Foundry Evaluation"; private readonly ContentSafetyService _service; private readonly IChatClient? _originalChatClient; diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyEvaluator.cs index b42631c3c06..5ad25917370 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyEvaluator.cs @@ -18,17 +18,17 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// /// An base class that can be used to implement s that utilize the -/// Azure AI Content Safety service to evaluate responses produced by an AI model for the presence of a variety of +/// Azure AI Foundry Evaluation service to evaluate responses produced by an AI model for the presence of a variety of /// unsafe content such as protected material, vulnerable code, harmful content etc. /// /// -/// The name of the annotation task that should be used when communicating with the Azure AI Content Safety service to -/// perform evaluations. +/// The name of the annotation task that should be used when communicating with the Azure AI Foundry Evaluation service +/// to perform evaluations. /// /// /// A dictionary containing the mapping from the names of the metrics that are used when communicating with the Azure -/// AI Content Safety to the s of the s returned by -/// this . +/// AI Foundry Evaluation service, to the s of the s +/// returned by this . /// #pragma warning disable S1694 // An abstract class should have both abstract and concrete methods public abstract class ContentSafetyEvaluator( @@ -58,12 +58,12 @@ public virtual ValueTask EvaluateAsync( } /// - /// Evaluates the supplied using the Azure AI Content Safety Service and returns - /// an containing one or more s. + /// Evaluates the supplied using the Azure AI Foundry Evaluation Service and + /// returns an containing one or more s. /// /// - /// The that should be used to communicate with the Azure AI Content Safety Service when - /// performing evaluations. + /// The that should be used to communicate with the Azure AI Foundry Evaluation Service + /// when performing evaluations. /// /// /// The conversation history including the request that produced the supplied . @@ -75,11 +75,11 @@ public virtual ValueTask EvaluateAsync( /// /// /// An identifier that specifies the format of the payload that should be used when communicating with the Azure AI - /// Content Safety service to perform evaluations. + /// Foundry Evaluation service to perform evaluations. /// /// /// A flag that indicates whether the names of the metrics should be included in the payload - /// that is sent to the Azure AI Content Safety service when performing evaluations. + /// that is sent to the Azure AI Foundry Evaluation service when performing evaluations. /// /// /// A that can cancel the evaluation operation. diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyService.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyService.cs index 291952d761b..3dce00b7624 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyService.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyService.cs @@ -208,7 +208,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to retrieve discovery URL for Azure AI Content Safety service. + {evaluatorName} failed to retrieve discovery URL for Azure AI Foundry Evaluation service. {response.StatusCode} ({(int)response.StatusCode}): {response.ReasonPhrase}. To troubleshoot, see https://aka.ms/azsdk/python/evaluation/safetyevaluator/troubleshoot. """); @@ -227,7 +227,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to retrieve discovery URL from the Azure AI Content Safety service's response below. + {evaluatorName} failed to retrieve discovery URL from the Azure AI Foundry Evaluation service's response below. To troubleshoot, see https://aka.ms/azsdk/python/evaluation/safetyevaluator/troubleshoot. {responseContent} @@ -256,7 +256,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to check service availability for the Azure AI Content Safety service. + {evaluatorName} failed to check service availability for the Azure AI Foundry Evaluation service. The service is either unavailable in this region, or you lack the necessary permissions to access the AI project. {response.StatusCode} ({(int)response.StatusCode}): {response.ReasonPhrase}. To troubleshoot, see https://aka.ms/azsdk/python/evaluation/safetyevaluator/troubleshoot. @@ -283,7 +283,7 @@ await GetResponseAsync( throw new InvalidOperationException( $""" - The required {nameof(capability)} '{capability}' required for {evaluatorName} is not supported by the Azure AI Content Safety service in this region. + The required {nameof(capability)} '{capability}' required for {evaluatorName} is not supported by the Azure AI Foundry Evaluation service in this region. To troubleshoot, see https://aka.ms/azsdk/python/evaluation/safetyevaluator/troubleshoot. The following response identifies the capabilities that are supported: @@ -311,7 +311,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to submit annotation request to the Azure AI Content Safety service. + {evaluatorName} failed to submit annotation request to the Azure AI Foundry Evaluation service. {response.StatusCode} ({(int)response.StatusCode}): {response.ReasonPhrase}. To troubleshoot, see https://aka.ms/azsdk/python/evaluation/safetyevaluator/troubleshoot. """); @@ -331,7 +331,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to retrieve the result location from the following response for the annotation request submitted to The Azure AI Content Safety service. + {evaluatorName} failed to retrieve the result location from the following response for the annotation request submitted to The Azure AI Foundry Evaluation service. {responseContent} """); @@ -369,7 +369,7 @@ await GetResponseAsync( { throw new InvalidOperationException( $""" - {evaluatorName} failed to retrieve annotation result from the Azure AI Content Safety service. + {evaluatorName} failed to retrieve annotation result from the Azure AI Foundry Evaluation service. The evaluation was timed out after {elapsedDuration} seconds (and {attempts} attempts). {response.StatusCode} ({(int)response.StatusCode}): {response.ReasonPhrase}. """); diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfiguration.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfiguration.cs index 615485cad5c..ec721fa59c7 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfiguration.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfiguration.cs @@ -13,8 +13,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// /// Specifies configuration parameters such as the Azure AI project that should be used, and the credentials that -/// should be used, when a communicates with the Azure AI Content Safety service -/// to perform evaluations. +/// should be used, when a communicates with the Azure AI Foundry Evaluation +/// service to perform evaluations. /// /// /// The Azure that should be used when authenticating requests. @@ -29,13 +29,13 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// The name of the Azure AI project. /// /// -/// The that should be used when communicating with the Azure AI Content -/// Safety service. While the parameter is optional, it is recommended to supply an -/// that is configured with robust resilience and retry policies. +/// The that should be used when communicating with the Azure AI Foundry Evaluation service. +/// While the parameter is optional, it is recommended to supply an that is configured with +/// robust resilience and retry policies. /// /// /// The timeout (in seconds) after which a should stop retrying failed attempts -/// to communicate with the Azure AI Content Safety service when performing evaluations. +/// to communicate with the Azure AI Foundry Evaluation service when performing evaluations. /// public sealed class ContentSafetyServiceConfiguration( TokenCredential credential, @@ -66,18 +66,18 @@ public sealed class ContentSafetyServiceConfiguration( public string ProjectName { get; } = projectName; /// - /// Gets the that should be used when communicating with the Azure AI - /// Content Safety service. + /// Gets the that should be used when communicating with the Azure AI Foundry Evaluation + /// service. /// /// - /// While supplying an is optional, it is recommended to supply one that - /// is configured with robust resilience and retry policies. + /// While supplying an is optional, it is recommended to supply one that is configured + /// with robust resilience and retry policies. /// public HttpClient? HttpClient { get; } = httpClient; /// /// Gets the timeout (in seconds) after which a should stop retrying failed - /// attempts to communicate with the Azure AI Content Safety service when performing evaluations. + /// attempts to communicate with the Azure AI Foundry Evaluation service when performing evaluations. /// public int TimeoutInSecondsForRetries { get; } = timeoutInSecondsForRetries; } diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfigurationExtensions.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfigurationExtensions.cs index c3c316cac2e..eded31ec0f8 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfigurationExtensions.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ContentSafetyServiceConfigurationExtensions.cs @@ -11,12 +11,12 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; public static class ContentSafetyServiceConfigurationExtensions { /// - /// Returns a that can be used to communicate with the Azure AI Content Safety + /// Returns a that can be used to communicate with the Azure AI Foundry Evaluation /// service for performing content safety evaluations. /// /// /// An object that specifies configuration parameters such as the Azure AI project that should be used, and the - /// credentials that should be used, when communicating with the Azure AI Content Safety service to perform + /// credentials that should be used, when communicating with the Azure AI Foundry Evaluation service to perform /// content safety evaluations. /// /// @@ -25,11 +25,11 @@ public static class ContentSafetyServiceConfigurationExtensions /// in being replaced with /// a new that can be used both to communicate with the AI model that /// is configured to communicate with, as well as to communicate with - /// the Azure AI Content Safety service. + /// the Azure AI Foundry Evaluation service. /// /// - /// A that can be used to communicate with the Azure AI Content Safety service for - /// performing content safety evaluations. + /// A that can be used to communicate with the Azure AI Foundry Evaluation service + /// for performing content safety evaluations. /// public static ChatConfiguration ToChatConfiguration( this ContentSafetyServiceConfiguration contentSafetyServiceConfiguration, @@ -47,23 +47,23 @@ public static ChatConfiguration ToChatConfiguration( } /// - /// Returns an that can be used to communicate with the Azure AI Content Safety service - /// for performing content safety evaluations. + /// Returns an that can be used to communicate with the Azure AI Foundry Evaluation + /// service for performing content safety evaluations. /// /// /// An object that specifies configuration parameters such as the Azure AI project that should be used, and the - /// credentials that should be used, when communicating with the Azure AI Content Safety service to perform + /// credentials that should be used, when communicating with the Azure AI Foundry Evaluation service to perform /// content safety evaluations. /// /// /// The original , if any. If specified, the returned /// will be a wrapper around that can be used both /// to communicate with the AI model that is configured to communicate with, - /// as well as to communicate with the Azure AI Content Safety service. + /// as well as to communicate with the Azure AI Foundry Evaluation service. /// /// - /// A that can be used to communicate with the Azure AI Content Safety service for - /// performing content safety evaluations. + /// A that can be used to communicate with the Azure AI Foundry Evaluation service + /// for performing content safety evaluations. /// public static IChatClient ToIChatClient( this ContentSafetyServiceConfiguration contentSafetyServiceConfiguration, diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/GroundednessProEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/GroundednessProEvaluator.cs index 3d05f3e7056..f65ddae4662 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/GroundednessProEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/GroundednessProEvaluator.cs @@ -11,7 +11,7 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate the groundedness of +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate the groundedness of /// responses produced by an AI model. /// /// @@ -28,7 +28,7 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// evaluate the contents of the last conversation turn. The contents of previous conversation turns will be ignored. /// /// -/// The Azure AI Content Safety service uses a finetuned model to perform this evaluation which is expected to +/// The Azure AI Foundry Evaluation service uses a finetuned model to perform this evaluation which is expected to /// produce more accurate results than similar evaluations performed using a regular (non-finetuned) model. /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/HateAndUnfairnessEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/HateAndUnfairnessEvaluator.cs index 718b742b29a..e4f57075e34 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/HateAndUnfairnessEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/HateAndUnfairnessEvaluator.cs @@ -6,8 +6,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of content that is hateful or unfair. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of content that is hateful or unfair. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/IndirectAttackEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/IndirectAttackEvaluator.cs index f65a5ee82f6..77e7afd6e5e 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/IndirectAttackEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/IndirectAttackEvaluator.cs @@ -6,8 +6,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of indirect attacks such as manipulated content, intrusion and information gathering. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of indirect attacks such as manipulated content, intrusion and information gathering. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/Microsoft.Extensions.AI.Evaluation.Safety.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/Microsoft.Extensions.AI.Evaluation.Safety.csproj index 0394056245f..12512e6884c 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/Microsoft.Extensions.AI.Evaluation.Safety.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/Microsoft.Extensions.AI.Evaluation.Safety.csproj @@ -1,7 +1,7 @@  - A library containing a set of evaluators for evaluating the content safety (hate and unfairness, self-harm, violence etc.) of responses received from an LLM. + A library that contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. $(TargetFrameworks);netstandard2.0 Microsoft.Extensions.AI.Evaluation.Safety diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ProtectedMaterialEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ProtectedMaterialEvaluator.cs index 25c99306c32..5b8c21b4a7f 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ProtectedMaterialEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ProtectedMaterialEvaluator.cs @@ -9,8 +9,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for presence of protected material. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for presence of protected material. /// /// /// @@ -85,8 +85,9 @@ await EvaluateContentSafetyAsync( includeMetricNamesInContentSafetyServicePayload: false, cancellationToken: cancellationToken).ConfigureAwait(false); - // If images are present in the conversation, do a second evaluation for protected material in images. - // The content safety service does not support evaluating both text and images in the same request currently. + // If images are present in the conversation, do a second evaluation for protected material in images. The + // Azure AI Foundry Evaluation service does not support evaluating both text and images as part of the same + // request currently. if (messages.ContainsImageWithSupportedFormat() || modelResponse.ContainsImageWithSupportedFormat()) { EvaluationResult imageResult = diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/README.md index 64d3930c106..c042da70deb 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SelfHarmEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SelfHarmEvaluator.cs index 5946bbf0a7b..deb1d81834a 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SelfHarmEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SelfHarmEvaluator.cs @@ -6,8 +6,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of content that indicates self harm. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of content that indicates self harm. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SexualEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SexualEvaluator.cs index bd5445ddd86..ed8093f5310 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SexualEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/SexualEvaluator.cs @@ -6,8 +6,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of sexual content. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of sexual content. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/UngroundedAttributesEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/UngroundedAttributesEvaluator.cs index a08acb45a0f..06019969345 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/UngroundedAttributesEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/UngroundedAttributesEvaluator.cs @@ -11,8 +11,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for presence of content that indicates ungrounded inference of human attributes. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for presence of content that indicates ungrounded inference of human attributes. /// /// /// @@ -31,7 +31,7 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// ignored. /// /// -/// The Azure AI Content Safety service uses a finetuned model to perform this evaluation which is expected to +/// The Azure AI Foundry Evaluation service uses a finetuned model to perform this evaluation which is expected to /// produce more accurate results than similar evaluations performed using a regular (non-finetuned) model. /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ViolenceEvaluator.cs b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ViolenceEvaluator.cs index 99928ff8184..3c7c5494fdf 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ViolenceEvaluator.cs +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation.Safety/ViolenceEvaluator.cs @@ -6,8 +6,8 @@ namespace Microsoft.Extensions.AI.Evaluation.Safety; /// -/// An that utilizes the Azure AI Content Safety service to evaluate responses produced by an -/// AI model for the presence of violent content. +/// An that utilizes the Azure AI Foundry Evaluation service to evaluate responses produced by +/// an AI model for the presence of violent content. /// /// /// diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation/Microsoft.Extensions.AI.Evaluation.csproj b/src/Libraries/Microsoft.Extensions.AI.Evaluation/Microsoft.Extensions.AI.Evaluation.csproj index 703c83cb390..3f098cf3026 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation/Microsoft.Extensions.AI.Evaluation.csproj +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation/Microsoft.Extensions.AI.Evaluation.csproj @@ -1,7 +1,7 @@ - A library containing core abstractions for evaluating responses received from an LLM. + A library that defines core abstractions and types for supporting evaluation. $(TargetFrameworks);netstandard2.0 Microsoft.Extensions.AI.Evaluation diff --git a/src/Libraries/Microsoft.Extensions.AI.Evaluation/README.md b/src/Libraries/Microsoft.Extensions.AI.Evaluation/README.md index ba6f61c446a..c21e2a299ad 100644 --- a/src/Libraries/Microsoft.Extensions.AI.Evaluation/README.md +++ b/src/Libraries/Microsoft.Extensions.AI.Evaluation/README.md @@ -4,7 +4,7 @@ * [`Microsoft.Extensions.AI.Evaluation`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation) - Defines core abstractions and types for supporting evaluation. * [`Microsoft.Extensions.AI.Evaluation.Quality`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Quality) - Contains evaluators that can be used to evaluate the quality of AI responses in your projects including Relevance, Truth, Completeness, Fluency, Coherence, Retrieval, Equivalence and Groundedness. -* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Content Safety service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. +* [`Microsoft.Extensions.AI.Evaluation.Safety`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Safety) - Contains a set of evaluators that are built atop the Azure AI Foundry Evaluation service that can be used to evaluate the content safety of AI responses in your projects including Protected Material, Groundedness Pro, Ungrounded Attributes, Hate and Unfairness, Self Harm, Violence, Sexual, Code Vulnerability and Indirect Attack. * [`Microsoft.Extensions.AI.Evaluation.Reporting`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting) - Contains support for caching LLM responses, storing the results of evaluations and generating reports from that data. * [`Microsoft.Extensions.AI.Evaluation.Reporting.Azure`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Reporting.Azure) - Supports the `Microsoft.Extensions.AI.Evaluation.Reporting` library with an implementation for caching LLM responses and storing the evaluation results in an Azure Storage container. * [`Microsoft.Extensions.AI.Evaluation.Console`](https://www.nuget.org/packages/Microsoft.Extensions.AI.Evaluation.Console) - A command line dotnet tool for generating reports and managing evaluation data. @@ -16,6 +16,7 @@ From the command-line: ```console dotnet add package Microsoft.Extensions.AI.Evaluation dotnet add package Microsoft.Extensions.AI.Evaluation.Quality +dotnet add package Microsoft.Extensions.AI.Evaluation.Safety dotnet add package Microsoft.Extensions.AI.Evaluation.Reporting ``` @@ -25,6 +26,7 @@ Or directly in the C# project file: + ``` diff --git a/test/Libraries/Microsoft.Extensions.AI.Evaluation.Integration.Tests/README.md b/test/Libraries/Microsoft.Extensions.AI.Evaluation.Integration.Tests/README.md index bae6542da6a..e8ffbcc18e1 100644 --- a/test/Libraries/Microsoft.Extensions.AI.Evaluation.Integration.Tests/README.md +++ b/test/Libraries/Microsoft.Extensions.AI.Evaluation.Integration.Tests/README.md @@ -1,7 +1,7 @@ ### Instructions -Some of the tests in this project (such as the tests in `EvaluatorTests.cs`) require special configuration to run. -These tests will be skipped by default if they have not been configured. +Some of the tests in this project (such as the tests in `QualityEvaluatorTests.cs` and `SafetyEvaluatorTests.cs`) +require special configuration to run. These tests will be skipped by default if they have not been configured. To configure the tests when running them locally on your machine, copy the `appsettings.json` file present in the current folder to a new file named `appsettings.local.json`, and fill in the values for the following properties: @@ -13,6 +13,9 @@ current folder to a new file named `appsettings.local.json`, and fill in the val "ModelName": "", "Endpoint": "", "StorageRootPath": "" + "AzureSubscriptionId": "", + "AzureResourceGroupName": "", + "AzureAIProjectName": "" } ```