Integration testing enhancements

[rodrigobr-msft]

This pull request introduces several improvements and new features to the microsoft-agents-testing documentation and CLI, focusing on transcript formatting and enhanced CLI usability. The changes clarify and expand documentation on transcript formatting, add a new JSON transcript formatter, and introduce a robust CLI (agt) with commands for environment management and scenario-based agent interaction.

Key changes include:

Transcript Formatting Enhancements

• Added JsonTranscriptFormatter for outputting transcripts as JSON arrays of Exchange objects, documented its usage, and updated all references and sample code to include it alongside the other formatters. Also, clarified the differences and use-cases for each formatter and removed references to the obsolete DetailLevel and TimeFormat enums. [1] [2] [3] [4] [5]
• Updated import paths and the public API to expose JsonTranscriptFormatter and convenience print functions (print_conversation, print_activities, print_json). [1] [2]

CLI Improvements

• Introduced a new CLI command group structure for env (environment) and scenario commands, and registered them for CLI use.
• Added the agt env show command to display environment details, including Python version, platform, working directory, number of registered scenarios, and the keys of loaded .env variables (not their values). [1] [2]
• Added the agt env help command to print the required .env variable names for authentication, making environment setup easier.
• Refactored CLI code to use new group modules and improved modularity for future CLI extension. [1] [2]

Documentation Updates

• Expanded and clarified documentation in API.md, README.md, and SAMPLES.md to cover the new CLI commands, transcript formatting options, and their usage, including concrete examples and output samples. [1] [2] [3]

These changes make the library easier to use from both code and the command line, provide more flexible transcript export options, and improve onboarding for new users.

[Copilot]

Pull request overview

This PR refactors microsoft-agents-testing transcript formatting into a new microsoft_agents.testing.formatting package (including a new JSON exchange formatter) and expands the agt CLI with new env and scenario command groups, while updating docs and tests to match the new API and behaviors.

Changes:

• Replaced the legacy transcript formatter module with a new formatter package, adding JsonTranscriptFormatter and convenience print_* helpers.
• Added cross-SDK “source scenario” support that launches an agent via PowerShell and ensures child processes are terminated.
• Reworked the CLI into modular command groups (agt env …, agt scenario …) and expanded docs/examples accordingly.

Reviewed changes

Copilot reviewed 43 out of 45 changed files in this pull request and generated 23 comments.

Show a summary per file

File	Description
dev/testing/microsoft-agents-testing/tests/test_transcript_formatter.py	Updates formatter tests to the new formatting module and JSON-based outputs.
dev/testing/microsoft-agents-testing/tests/cross_sdk/test_source_scenario.py	Adds tests for cross-SDK PowerShell-launched scenarios and process lifecycle behavior.
dev/testing/microsoft-agents-testing/tests/cross_sdk/init.py	Initializes the new cross-SDK test package (empty marker module).
dev/testing/microsoft-agents-testing/tests/core/test_external_scenario.py	Adjusts endpoint trailing-slash expectation to match new normalization logic.
dev/testing/microsoft-agents-testing/pyproject.toml	Adds psutil dependency to support process tree termination.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/transcript_formatter.py	Removes the legacy formatter implementation (replaced by formatting/).
dev/testing/microsoft-agents-testing/microsoft_agents/testing/pytest_plugin.py	Tweaks marker scenario resolution logic for Scenario instances.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/utils.py	Adds shared exchange sorting and timestamp formatting helpers.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/transcript_formatter.py	Introduces TranscriptFormatter protocol + BaseTranscriptFormatter callable base.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/print.py	Adds print_conversation, print_activities, print_json convenience functions.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/json_transcript_formatter.py	Adds JSON exchange-array transcript formatter.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/conversation_transcript_formatter.py	Adds conversation-style transcript formatter with timestamped lines.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/activity_transcript_formatter.py	Adds flat JSON activity-array transcript formatter.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/formatting/init.py	Exposes formatting API surface via package exports.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cross_sdk/utils.py	Adds helpers to create agent paths and scenarios for cross-SDK layout.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cross_sdk/types.py	Adds SDKVersion enum for cross-SDK path selection.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cross_sdk/source_scenario.py	Adds SourceScenario that runs a .ps1 and kills descendant processes on exit.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cross_sdk/constants.py	Adds constants for cross-SDK entrypoint name and default endpoint.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cross_sdk/init.py	Exposes cross-SDK helpers and scenarios as a public package.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/core/transport/transcript/exchange.py	Adds is_error, fixes latency typing, improves error/status handling, and sanitizes activities.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/core/external_scenario.py	Normalizes endpoints by stripping a trailing slash for consistency.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/core/utils.py	Clarifies URL detection semantics (HTTP/HTTPS).
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/core/output.py	Adds optional “exit” behavior to output error reporting.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/core/decorators.py	Refactors scenario resolution decorator to --url vs --agent options.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/core/cli_config.py	Updates environment loading documentation to match new behavior/return values.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/scenario_group.py	Adds scenario click group for modular scenario commands.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_utils.py	Adds shared scenario CLI helpers (activity loading, histogram output).
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_run.py	Adds agt scenario run command for long-running in-process scenarios.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_post.py	Adds agt scenario post command to send one activity/message and print transcript.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_load.py	Adds agt scenario load concurrent load test + latency stats reporting.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_list.py	Adds agt scenario list to discover registered scenarios.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/_chat.py	Adds interactive REPL chat command for scenarios.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario/init.py	Forces scenario submodules to load for CLI registration.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/scenario.py	Removes the monolithic scenario command module (split into group + subcommands).
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/environment/env_group.py	Adds env click group for environment commands.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/environment/_show.py	Adds agt env show with environment details and .env key listing.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/environment/_help.py	Adds agt env help to print required .env variables for auth.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/environment/init.py	Forces environment submodules to load for CLI registration.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/env.py	Stops printing .env values (prints keys only).
dev/testing/microsoft-agents-testing/microsoft_agents/testing/cli/commands/init.py	Registers new env_group/scenario_group instead of old commands.
dev/testing/microsoft-agents-testing/microsoft_agents/testing/init.py	Re-exports new formatting classes/functions at package top-level.
dev/testing/microsoft-agents-testing/docs/SAMPLES.md	Updates samples and adds CLI usage guidance.
dev/testing/microsoft-agents-testing/docs/README.md	Updates transcript formatting documentation + adds CLI section.
dev/testing/microsoft-agents-testing/docs/API.md	Updates public API docs for new formatters, print helpers, and CLI commands.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 53 changed files in this pull request and generated 9 comments.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 53 changed files in this pull request and generated 5 comments.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 53 changed files in this pull request and generated 11 comments.

[Copilot]

Pull request overview

Copilot reviewed 49 out of 53 changed files in this pull request and generated 11 comments.