Improve test framework migrations

[thomhurst]

Summary

• Add expression replacements to the two-phase migration pipeline for framework context properties.
• Add TestContext.TestDirectory as the TUnit test assembly directory API.
• Migrate NUnit Assert.That(bool), TestContext output, directory, and attachment APIs.
• Migrate MSTest TestContext output, result-file, and directory APIs; keep xUnit output-helper migration on the shared invocation replacement path.

Tests

• dotnet test TUnit.Analyzers.Tests\TUnit.Analyzers.Tests.csproj --treenode-filter "/*/*/NUnitMigrationAnalyzerTests/*|/*/*/MSTestMigrationAnalyzerTests/*|/*/*/XUnitMigrationAnalyzerTests/*"
• dotnet test TUnit.PublicAPI\TUnit.PublicAPI.csproj --treenode-filter "/*/*/Tests/Core_Library_Has_No_API_Changes"

Closes #6338

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 7 complexity

Metric	Results
Complexity	7

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[claude]

Review: Improve test framework migrations (#6381)

Overall this is a solid, well-scoped change. The extraction of AddInvocationReplacement/AddExpressionReplacement into MigrationAnalyzer (and refactoring XUnitTwoPhaseAnalyzer.AnalyzeTestOutputHelperCalls to use the shared helper) is a good DRY cleanup — it removes a duplicated try/catch block and gives NUnit/MSTest a consistent way to register expression-level replacements. The is { Length: > 0 } x pattern-matching swaps for !string.IsNullOrEmpty(x) are a nice, low-risk modernization that also narrows nullability for the compiler.

Findings

1. ExpressionReplacement has no way to attach a warning/TODO, unlike AssertionConversion
AssertionConversion supports a TodoComment (used elsewhere in MigrationTransformer to leave a note in the generated code when a conversion may need manual review). ExpressionReplacement/AddExpressionReplacement has no equivalent — every expression rewrite is applied silently with full confidence.

That matters here because MSTestTwoPhaseAnalyzer maps both TestContext.TestDir and TestContext.DeploymentDirectory to TUnit.Core.TestContext.TestDirectory (which is just the test assembly's output directory). DeploymentDirectory is not always identical to the assembly directory — it's the deployment-item output directory (e.g. [DeploymentItem]-copied resources), which historically could diverge from the plain bin directory. For most modern MSTest-on-.NET projects the two coincide, but for anyone still relying on deployment items the migrated code will silently point somewhere different at runtime, with no compiler warning and no generated TODO to flag it.

Suggest reusing the TodoComment-style mechanism for ExpressionReplacement (or at minimum emitting a TODO comment specifically for the DeploymentDirectory → TestDirectory case), so users get a "verify this still points where you expect" nudge instead of a clean, silent build.

2. (Minor/low-confidence) Single-arg Assert.That(x) skips the boolean-type check that the 2-arg path applies

if (args.Count == 1)
    return ConvertBooleanAssertThat(node, args[0], null);

if (args.Count >= 2 && IsMessageArgument(args[1]) && IsBooleanExpression(args[0].Expression))
    return ConvertBooleanAssertThat(node, args[0], GetMessageArgument(args[1]));

The two-argument branch verifies IsBooleanExpression(args[0]) before converting, but the single-argument branch does not. In practice Assert.That(x) with one argument is only valid in NUnit for bool/Func<bool>, so this is likely fine — but the Func<bool> case would still get rewritten to await Assert.That(() => x).IsTrue(), which needs a TestDelegate/deferred-invocation shape that may not match TUnit's fluent API the same way. Worth a quick test case if Func<bool> migration is in scope; otherwise not blocking.

Not blocking

• TestContext.TestDirectory as a pure alias for TestContext.OutputDirectory (public static string? TestDirectory => OutputDirectory;) does introduce two public names for the same value. Given issue #6338 explicitly asks for this API name to match NUnit/MSTest vocabulary for migration purposes, this seems intentional rather than an oversight.
• Span-based node re-matching in AddExpressionReplacement/AddInvocationReplacement (searching currentRoot by Span) is the same pattern already used elsewhere in this file; since annotation-only replacements preserve node text/spans, this remains sound across the phases as far as I can tell.

Tests for the new behaviors (NUnit boolean overload, NUnit/MSTest TestContext shortcuts) are included and look reasonable. Didn't find correctness issues that would block merge — flagging (1) as worth addressing or at least discussing before merge, (2) as optional.

[claude]

Code review (follow-up)

Checked this against my previous review from 2026-07-09 (commit 816f5ff7). Since then, three commits (5ed8e86, 7aabdbf, bf4bcd0) were pushed that directly address both prior findings:

1. DeploymentDirectory → TestDirectory silent divergence — resolved. ExpressionReplacement now carries an optional TodoComment (mirroring AssertionConversion's existing mechanism), and MSTestTwoPhaseAnalyzer attaches a TODO specifically for the DeploymentDirectory case: // TODO: TUnit migration - MSTest DeploymentDirectory can differ from TUnit TestDirectory. Verify the migrated path. The TestDir case (a true alias) correctly gets no TODO. The PrependTodoComment helper extraction is also a nice dedup of the trivia-handling logic that used to be duplicated in TransformAssertions.

2. Single-arg Assert.That(x) / Func<bool> case — resolved (was already correct, now covered by a test). New test confirms Assert.That(deferredCondition) where deferredCondition is Func<bool> migrates to await Assert.That(deferredCondition).IsTrue(). Verified this is valid against TUnit.Assertions: Assert.That<TValue>(Func<TValue>) returns a FuncAssertion<TValue>, and the [GenerateAssertion]-generated IsTrue() extension applies to any bool-valued assertion source — so no lambda-wrapping is needed and the migrated code is correct.

No new issues in the delta. MigrationTransformer.TransformExpressionReplacements's new TODO-attachment branch (statement.ReplaceNode(expression, newExpression) → currentRoot.ReplaceNode(statement, newStatement)) correctly operates on nodes from the current tree, consistent with the existing pattern used for assertions.

No issues found. Checked for bugs and CLAUDE.md compliance.

[claude]

Code review

Reviewed the two-phase migration pipeline changes (expression replacements, TestContext.TestDirectory, NUnit Assert.That(bool) and TestContext migration, MSTest TestContext migration). CLAUDE.md compliance looks clean (snapshot files updated consistently across all 4 TFMs, no .received.txt committed, no VSTest/hot-path/AOT concerns apply to this analyzer-only diff). One correctness bug found via independent review + adversarial validation:

🐛 Invocation replacement silently reverts a nested expression replacement

File: TUnit.Analyzers.CodeFixers/Base/TwoPhase/MigrationTransformer.cs#L231-L250

Transform() runs TransformExpressionReplacements (step 2, L36) before TransformInvocationReplacements (step 3, L39). But TransformInvocationReplacements replaces the whole invocation node with SyntaxFactory.ParseExpression(replacement.ReplacementCode), where ReplacementCode was captured at analysis time from the original, pre-transform argument-list text (e.g. GetNUnitTestContextInvocationReplacement builds arguments = invocation.ArgumentList.ToString() from OriginalRoot).

AnalyzeNUnitTestContextDirectoryProperties (L2088) independently scans OriginalRoot for TestContext.CurrentContext.TestDirectory/WorkDirectory member accesses regardless of nesting, so a directory property nested inside a migrated invocation's arguments — e.g.:

TestContext.WriteLine($"Dir: {TestContext.CurrentContext.TestDirectory}");

gets an ExpressionReplacement for the inner property AND an InvocationReplacement for the outer call. The expression edit is applied first, but the invocation transform then re-parses the stale original argument text and replaces the entire node, reverting the inner edit. Since the annotation on the outer invocation survives the inner ReplaceNode (standard Roslyn behavior), the invocation replacement is found and applied — it isn't skipped. Result: Console.WriteLine($"Dir: {TestContext.CurrentContext.TestDirectory}"), leaving the NUnit-only TestContext.CurrentContext unmigrated inside an otherwise-converted call. Once NUnit usings are stripped later in the pipeline, this reference no longer resolves and the migrated file fails to compile. The same interaction applies to the MSTest path (TestContext.WriteLine(TestContext.TestDir) → Console.WriteLine(TestContext.TestDir), with TestDir left unmigrated).

This is new in this PR (expression replacements didn't exist before), and none of the added tests cover a directory property nested inside a converted invocation's arguments — both NUnit_TestContext_Shortcuts_Converted and MSTest_TestContext_Output_And_Directory_Converted only place these on separate statements, so the gap isn't caught.

Suggested fix: derive ReplacementCode for invocation replacements from the current (possibly-mutated) argument list at transform time instead of freezing the original text at analysis time — e.g. re-locate the invocation's argument list in currentRoot via annotations before building the replacement string, or process expression and invocation replacements in a single combined pass so nested edits aren't clobbered.

[claude]

Code review (status check)

Re-checked this PR at the current tip (ed5e3e19, matching HEAD of this checkout). No new commits have landed since my last review (submitted 11s after that commit), so the status of previously raised findings is unchanged:

1. ✅ DeploymentDirectory → TestDirectory silent divergence — resolved (TODO-comment mechanism added to ExpressionReplacement).
2. ✅ Single-arg Assert.That(x) / Func<bool> — resolved, covered by a test.
3. ❌ Still open: invocation replacement reverts a nested expression replacement — re-confirmed by reading the current code directly. In MigrationTransformer.Transform (MigrationTransformer.cs#L34-L39), TransformExpressionReplacements runs before TransformInvocationReplacements. The invocation replacement's ReplacementCode is captured from OriginalRoot-derived text at analysis time (e.g. NUnitTwoPhaseAnalyzer.cs#L2067 — arguments = invocation.ArgumentList.ToString()), so when TransformInvocationReplacements later runs currentRoot.ReplaceNode(invocation, newInvocation) using that stale string, it overwrites any nested TestContext directory-property rewrite already applied inside the invocation's own arguments. Example: TestContext.WriteLine($"Dir: {TestContext.CurrentContext.TestDirectory}") migrates to Console.WriteLine($"Dir: {TestContext.CurrentContext.TestDirectory}"), leaving the NUnit-only TestContext.CurrentContext reference unmigrated — once NUnit usings are stripped later in the pipeline, this fails to compile. No test currently covers a directory property nested inside a migrated invocation's arguments.

CLAUDE.md compliance still looks clean: verified-API snapshots present for all 4 TFMs, no .received.txt committed, and no VSTest/AOT/hot-path concerns apply here since this is analyzer/codefixer tooling rather than the runtime engine.

Since finding 3 is a genuine compile-breaking correctness bug in newly introduced code, I'd suggest resolving it before merge — e.g. re-derive invocation ReplacementCode from the current (possibly-mutated) tree at transform time, or merge the expression- and invocation-replacement passes so nested edits aren't clobbered.

[claude]

Review: Improve test framework migrations (#6381)

Overview

Solid extension of the two-phase migration pipeline: adds a generic ExpressionReplacement mechanism alongside the existing InvocationReplacement, wires up NUnit Assert.That(bool) conversion, and migrates NUnit/MSTest TestContext output/directory/attachment APIs. Good cleanup too — extracting AddInvocationReplacement/AddExpressionReplacement onto the shared MigrationAnalyzer base removes duplicated try/catch boilerplate that used to live inline in XUnitTwoPhaseAnalyzer, and PrependTodoComment is nicely factored out of TransformAssertions. Test coverage for the new paths (NUnit boolean overload, NUnit/MSTest TestContext shortcuts) looks appropriate and the TUnit.PublicAPI snapshots are updated correctly.

One genuinely nice fix buried in here: TransformInvocationReplacements now re-splices arguments from the current (already-transformed) invocation node instead of blindly re-parsing the analysis-time replacement string. That fixes a real ordering bug — without it, a nested ExpressionReplacement inside an invocation's arguments (e.g. TestContext.WriteLine($"...{TestContext.TestDir}...")) would have its inner replacement silently discarded once the outer WriteLine → Console.WriteLine replacement ran. Good catch.

Suggestions

1. TestContext.TestDirectory duplicates OutputDirectory with no different behavior
TUnit.Core/TestContext.cs:276 — TestDirectory is a pure passthrough (=> OutputDirectory). This grows the public API surface with two permanently-supported names for the same value (confirmed by the TUnit.PublicAPI snapshot diffs across all 4 TFMs). Since public API can't be removed without a breaking change, consider instead having the NUnit/MSTest codefixers target the existing OutputDirectory property directly rather than introducing an alias. If the intent is that TestDirectory becomes the preferred/idiomatic name going forward (matching NUnit's own naming), it'd be worth being explicit about that — e.g. [Obsolete] on OutputDirectory pointing at the new name — rather than leaving both as first-class equals indefinitely.

2. Migrated TestContext field/property declarations are left dangling
TUnit.Analyzers.Tests/MSTestMigrationAnalyzerTests.cs:363,381 — the new test's expected output keeps public TestContext TestContext { get; set; } completely unchanged after migration, even though the MSTest using gets stripped elsewhere (AnalyzeUsings removes the Microsoft.VisualStudio.TestTools prefix). In real migrated code this leaves either an unresolved type or, if TUnit.Core is implicitly usinged, a property that silently never gets populated (TUnit doesn't inject a TestContext member the way MSTest does) — a behavior difference that fails silently at runtime rather than at compile time. This looks like a pre-existing gap in the tool (the same pattern shows up for ClassSetup(TestContext context) parameters elsewhere in the test file) rather than something newly introduced here, but since this PR is actively improving TestContext migration fidelity, it may be worth a TODO-comment insertion for these declarations too, or a follow-up issue.

3. Positional argument mapping in BuildAttachArtifactCall (NUnit)
TUnit.Analyzers.CodeFixers/TwoPhase/NUnitTwoPhaseAnalyzer.cs — the 2-arg case reads args[0]/args[1] positionally for filePath/description. If a caller used named arguments out of order (AddTestAttachment(description: "d", filePath: "f")), the values would be silently swapped in the generated AttachArtifact call. Likely rare in practice, but worth a quick NameColon-aware lookup if you want to be defensive.

4. Minor: format args on the new boolean+message path
NUnitTwoPhaseAnalyzer.cs:242-245 — Assert.That(condition, "msg {0}", value) (3-arg, matching NUnit's Assert.That(bool, string, params object[])) only carries args[1] through as the .Because(...) message and silently drops any params format arguments. This mirrors an existing limitation in the constraint-based path just below it (args.Count >= 3 message handling also ignores further format args), so it's consistent with current tool behavior rather than a regression — just flagging in case it's not already tracked.

None of these block the PR — items 3/4 are edge cases, item 2 is pre-existing scope, and item 1 is a design call the maintainer may already have reasons for. Nice, well-tested improvement overall.