Skip to content

feat: Add tools to update test point outcomes and record test run results #1177

@rabbyn

Description

@rabbyn

Feature Request — Update Test Point Outcomes / Record Test Run Results

Summary

Currently the Azure DevOps MCP exposes tools to create test plans, suites, and test cases, but provides no way to record the outcome of test execution (Passed / Failed / Blocked / Not Applicable / Not Run).

This makes it impossible for an AI agent to run a test plan end-to-end — it can build the plan and execute the validation logic, but cannot write results back into Azure DevOps.


Requested Tools

Tool Name Underlying ADO REST API Purpose
update_test_point_outcome PATCH /testplan/Plans/{planId}/Suites/{suiteId}/TestPoint Set outcome (Passed/Failed/Blocked/NotApplicable) on a test point
create_test_run POST /test/runs Create a test run for a given plan
update_test_run_results PATCH /test/runs/{runId}/results Record step-level and overall results for a test run
complete_test_run PATCH /test/runs/{runId} Mark a run as completed

At minimum, update_test_point_outcome would be sufficient to unblock the most common agentic scenario.


Motivation / Use Case

I recently used an AI agent (GitHub Copilot) to execute a full test plan via the Azure DevOps MCP:

  • The agent built 17 test cases across 7 suites from a User Story's acceptance criteria
  • It scanned Terraform Git repos, analyzed pipeline run outputs, and queried live infrastructure (Microsoft Fabric Remote MCP+ Microsoft Entra MCP) to validate each test case
  • It produced a full execution report with evidence (screenshots, API responses)

The only step that could not be automated was writing the results back into ADO — forcing a manual round-trip in the Execute tab just to mark outcomes that the agent had already determined.

Completing this loop would make the ADO MCP a fully autonomous testing toolchain for agentic workflows.

Image

Suggested Minimal Implementation

// update_test_point_outcome
{
  "project": "MyProject",
  "planId": 12723,
  "suiteId": 12729,
  "testPointId": 42,
  "outcome": "Passed" // Passed | Failed | Blocked | NotApplicable | NotRun
}

Metadata

Metadata

Assignees

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions