Skip to content

Commit fe75826

Browse files
authored
chore(skill): Improve test skill to include nested playwright tests (#20610)
chore(skill): Improve test skill to include nested playwright tests
1 parent 866958e commit fe75826

1 file changed

Lines changed: 95 additions & 9 deletions

File tree

.agents/skills/write-tests/SKILL.md

Lines changed: 95 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,17 @@ Follow these steps in order before writing any test code.
2222
1. **Decide the framework.** Testing a function's return value, side effects, or module interactions
2323
→ Vitest (lives under `packages/<name>/test/`). Testing that a real HTTP request to a running app
2424
produces the correct Sentry envelope → Playwright (lives under
25-
`dev-packages/e2e-tests/test-applications/<app>/tests/`).
25+
`dev-packages/e2e-tests/test-applications/<app>/tests/`). Testing Node SDK instrumentation
26+
against real envelope output → node-integration-tests (lives under
27+
`dev-packages/node-integration-tests/suites/`).
28+
29+
**Parameterization differs by framework — pick the right one:**
30+
31+
| Framework | How to parameterize |
32+
| ---------------------- | ------------------------------------------------------------- |
33+
| Vitest | `it.each` / `it.for` (runner-integrated, one test each) |
34+
| Playwright E2E | `.forEach()` outside `test()` (registers separate tests) |
35+
| Node integration tests | Loops **inside** a single `test()` body (one Node.js process) |
2636

2737
2. **Read 2–3 existing test files** in the target `test/` directory. Specifically note:
2838
- Which `vi.mock` style they use (string path or import form)
@@ -299,6 +309,64 @@ describe('patchRoute', () => {
299309

300310
---
301311

312+
## Writing node-integration-tests
313+
314+
Node integration tests (`dev-packages/node-integration-tests/`) use `createEsmAndCjsTests` to
315+
run a real Node scenario file and assert on captured Sentry envelopes.
316+
317+
### Minimize `test()` calls — each one spawns a separate Node process
318+
319+
**This is the opposite of the Playwright rule.** In Playwright, each `test()` is cheap — use
320+
`.forEach()` to register many tests. In node-integration-tests, each `test()` forks a fresh Node
321+
process with full startup cost. A `describe.each` matrix that looks reasonable in a unit test
322+
context balloons into dozens of cold starts and slows CI by a large factor.
323+
324+
**Rule: loop inside the test body, not around `test()` calls.**
325+
326+
```typescript
327+
// Bad: 2 routes × 5 methods = 10 separate Node processes
328+
createEsmAndCjsTests(__dirname, 'scenario.mjs', 'instrument.mjs', (createRunner, test) => {
329+
describe.each(['/sync', '/async'])('when using %s route', route => {
330+
describe.each(['get', 'post', 'put', 'delete', 'patch'])('when using %s method', method => {
331+
test('handles transaction', async () => {
332+
// ...
333+
});
334+
});
335+
});
336+
});
337+
```
338+
339+
```typescript
340+
// Good: one Node process, all combinations asserted in a single test run
341+
createEsmAndCjsTests(__dirname, 'scenario.mjs', 'instrument.mjs', (createRunner, test) => {
342+
test('handles transactions for all route/method/path combinations', async () => {
343+
const runner = createRunner();
344+
const requests: Array<{ method: string; url: string }> = [];
345+
346+
for (const route of ['/sync', '/async']) {
347+
for (const method of ['get', 'post', 'put', 'delete', 'patch']) {
348+
const fullPath = `${route}${path}`;
349+
runner.expect({
350+
transaction: { transaction: `${method.toUpperCase()} ${fullPath}` },
351+
});
352+
requests.push({ method, url: fullPath });
353+
}
354+
}
355+
356+
const started = runner.start();
357+
for (const req of requests) {
358+
await started.makeRequest(req.method, req.url);
359+
}
360+
await started.completed();
361+
}, 60_000);
362+
});
363+
```
364+
365+
If a subset of cases has meaningfully different expectations (e.g., error vs. success), split
366+
into two tests — not thirty.
367+
368+
---
369+
302370
## Writing Playwright E2E tests
303371

304372
### When to write E2E tests
@@ -366,17 +434,35 @@ expect(mechanism?.type).toBe('auto.http.hono.context_error');
366434

367435
### Parameterized E2E tests
368436

369-
For Playwright tests (unlike Vitest), `for...of` loops are the established codebase convention.
370-
Use `for...of` (not `.forEach()`) so Playwright's test registration works correctly:
437+
For Playwright tests (unlike Vitest), use standard JS `.forEach()` as this is recommended by Playwright,
438+
**not** `it.each` or `it.for`, which are Vitest-only APIs. The `.forEach()` runs at discovery time, registering
439+
each case as its own independent test. All cases then run separately at execution time.
371440

372441
```typescript
373-
for (const { name, prefix } of SCENARIOS) {
374-
test.describe(name, () => {
375-
test('captures named middleware span', async ({ baseURL }) => {
376-
// ...
377-
});
442+
[
443+
{ a: 1, b: 1, expected: 2 },
444+
{ a: 1, b: 2, expected: 3 },
445+
{ a: 2, b: 1, expected: 3 },
446+
].forEach(({ a, b, expected }) => {
447+
test(`given ${a} and ${b} as arguments, returns ${expected}`, ({ page }) => {
448+
expect(a + b).toEqual(expected);
378449
});
379-
}
450+
});
451+
```
452+
453+
**Don't put the loop inside a single test.** That collapses all cases into one test body — a
454+
failure in one iteration aborts the rest, and the runner reports a single failure with no
455+
per-case visibility:
456+
457+
```typescript
458+
// Bad: all routes tested in one test — a failure on /users skips /posts entirely
459+
test('captures transactions for all routes', async ({ baseURL }) => {
460+
for (const route of ['/users', '/posts', '/comments']) {
461+
const txn = await waitForTransaction(APP_NAME, e => e.transaction === `GET ${route}`);
462+
await fetch(`${baseURL}${route}`);
463+
expect(txn.contexts?.trace?.op).toBe('http.server');
464+
}
465+
});
380466
```
381467

382468
### Common pitfalls

0 commit comments

Comments
 (0)