Skip to content

test: extract shared mocked MCP and Nock harness - #625

Closed
charliecreates[bot] wants to merge 4 commits into
mainfrom
test/shared-mocked-mcp-harness
Closed

test: extract shared mocked MCP and Nock harness#625
charliecreates[bot] wants to merge 4 commits into
mainfrom
test/shared-mocked-mcp-harness

Conversation

@charliecreates

@charliecreates charliecreates Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a typed, test-only MCP/Nock harness and named Hevy fixtures for shared client/server lifecycle and API mocking.
  • Migrate all 16 existing mocked integration tests and add focused failure-mode coverage for network, interceptor, lifecycle, cache, response, and fixture isolation.
  • Keep deterministic pull-request CI secret-free, with coverage uploaded through tokenless Codecov OIDC.

Safety and behavior

  • Tracks only helper-owned Nock scopes/interceptors, preserving unrelated mocks.
  • Explicitly restores the caller's prior network policy, including nested and matcher-restricted policies.
  • Aggregates harness-close, leaked-resource, and unused-interceptor failures while still forcing cleanup.
  • Resets the exercise-template catalog cache across harness creation and teardown.
  • Keeps all support under tests/; no production package exports or generated files change.

Validation

Validated with Node 24 and HEVY_API_KEY unset:

  • npx vitest run tests/support tests/integration/mocked — 4 files, 36 tests passed
  • npx vitest run tests/integration/mocked — 2 files, 16 tests passed
  • npx vitest run --exclude 'tests/integration/**' — 34 files, 538 tests passed
  • npm run check
  • npm run check:types
  • npm run build
  • npm run check:changeset
  • Parse .github/workflows/*.yml — 9/9 workflows passed
  • Credential scans for GitHub secret syntax and live credential identifiers
  • npm pack --dry-run --ignore-scripts — no test support included
  • git diff --check origin/main...HEAD

Gemini feedback was attempted as required, but the CLI stopped before review because no authentication method was configured; no Gemini review is claimed.

Changeset

Empty changeset: internal test/CI refactoring only; no package version bump.

Resolves #606

Refs #601

Strategy: #604

✨ PR Description

Purpose: Extract shared mocked MCP and Nock test harnesses into reusable support modules while removing live credential dependencies from CI workflow.

Main changes:

  • Created hevy-fixtures.ts with typed fixture factories providing immutable test data for Hevy API responses
  • Created mocked-mcp.ts with MCP harness lifecycle management, Nock scope tracking, and tool invocation utilities
  • Refactored integration tests to use shared harness instead of inline setup; removed GitHub secrets from CI workflow

Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Description using Guidelines Learn how

@charliecreates charliecreates Bot added testing refactoring Code refactoring or cleanup labels Jul 11, 2026
@mergify

mergify Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.56%. Comparing base (0f03660) to head (c421ae0).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #625      +/-   ##
==========================================
+ Coverage   92.94%   93.56%   +0.61%     
==========================================
  Files          35       37       +2     
  Lines        1559     1709     +150     
  Branches      392      432      +40     
==========================================
+ Hits         1449     1599     +150     
  Misses         48       48              
  Partials       62       62              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Unit Test Results

  1 files   34 suites   3s ⏱️
539 tests 539 ✅ 0 💤 0 ❌
543 runs  543 ✅ 0 💤 0 ❌

Results for commit c421ae0.

♻️ This comment has been updated with latest results.

@gitstream-cm gitstream-cm Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✨ PR Review

The PR is a well-structured test infrastructure refactor that centralises lifecycle management, fixture factories, and network isolation into typed, reusable helpers. The migration is clean and the new support files are clearly tested. One non-obvious state management concern and one workflow reliability gap warrant attention.

3 issues detected:

🐞 Bug - The `restorePolicy` argument passed by a second caller at depth > 0 is silently discarded; if two callers intend different post-run network states, only the first caller's intent is satisfied.

Details: disableMockedMcpExternalNetworking only stores restorePolicy when networkIsolationDepth === 0. If a second test suite (different describe block in the same worker, or a re-entrant call) calls this function while depth is already > 0, its restore policy is silently dropped. When the outermost caller eventually releases, it runs the first suite's restore callback, not the most-recently-provided one. If test suites are ordered such that the outer caller passes nock.enableNetConnect but the inner caller passed nock.disableNetConnect, the final state will be wrong and undetected by the harness.

File: tests/support/mocked-mcp.ts (107-127)

🐞 Bug - The `upload-coverage` job has no guard condition and will fail with a misleading artifact-not-found error whenever the Node 24 build leg is cancelled or its coverage steps do not produce output files.

Details: The upload-coverage job unconditionally downloads the coverage-reports-node-24 artifact and uses if-no-files-found: error on the upload step inside build. If the Node 24 matrix leg is cancelled or if either coverage run (tests/integration/mocked or the unit suite) fails before writing lcov output, the artifact is never created. The subsequent upload-coverage job will then fail at actions/download-artifact, blocking the entire workflow result and masking the original test failure.

File: .github/workflows/build-and-test.yml (119-143)

🐞 Bug - Pending-mocks are snapshotted before leaked harnesses are force-closed, so interceptors that would be consumed by those harnesses are incorrectly flagged as unused.

Details: In cleanupMockedMcpTestState, scope.pendingMocks() is evaluated before leaked harnesses are force-closed. If a leaked harness owns interceptors that it would have consumed on close, those interceptors are counted as unused and reported as failures—even though the subsequent harness.close() call a few lines later might have legitimately consumed them. This produces a spurious "Unused Nock interceptors" error alongside the legitimate "Unclosed mocked MCP harnesses" error, inflating the AggregateError with a misleading extra entry.

File: tests/support/mocked-mcp.ts (247-286)

Generated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using Guidelines Learn how

@charliecreates

Copy link
Copy Markdown
Contributor Author

Covered the missed fixture-default and mocked MCP lifecycle paths reported by codecov/project and codecov/patch in a5c1e10535fad90dadee2ecc7a3aba1ae44d4fb6. The push will trigger the checks automatically.

— charlied/pr-check-repair

@mergify

mergify Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-07-11 10:50 UTC · Rule: default · triggered by rule Queue pull requests labeled 'queue'
  • 🚫 Left the queue2026-07-11 10:54 UTC · at c421ae0bf8f0a41ea71b5ae50d54d8d3dc5492aa

This pull request spent 3 minutes 30 seconds in the queue, with no time running CI.

Reason

The pull request conflicts with pull requests ahead in queue: #626, #623, #624

Hint

There is nothing you can do for now. If the pull request ahead in the queue is merged, this pull request will become conflicting and you'll have to update it.
If the pull request ahead is not merged, you can requeue this pull request with a @mergifyio queue comment.

Tick the box to put this pull request back in the merge queue (same as @mergifyio queue).

  • Requeue this pull request

@mergify mergify Bot added dequeued and removed queued labels Jul 11, 2026
@charliecreates

Copy link
Copy Markdown
Contributor Author

I stopped the conflict repair because .github/workflows/build-and-test.yml now has two incompatible Codecov execution models: this PR adds an artifact-backed upload-coverage job, while main runs Codecov directly from the build matrix with OIDC. I verified the PR head and current main, and GitHub’s branch-update endpoint could not merge them. Please choose the intended coverage-upload topology, then resolve or direct the merge.

— charlied/pr-merge-conflict-repair

@charliecreates

Copy link
Copy Markdown
Contributor Author

I stopped automatic conflict repair.

  • GitHub's branch-update operation reported conflicts, and the current PR branch has no common Git ancestor with main; a merge-based repair would require merging unrelated repository histories.
  • I did not use --allow-unrelated-histories because it could import unrelated files and overwrite the PR's intended changes.

Please recreate or rebase this branch from current main, resolve the intended changes, and push the result.

— charlied/pr-merge-conflict-repair

@chrisdoc chrisdoc closed this Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dequeued queue Add to Mergify merge queue refactoring Code refactoring or cleanup testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: extract shared mocked MCP and Nock harness

2 participants