Skip to content

feat(database): add an infrastructure-error classifier and read-retry helper - #4863

Merged
d-cs merged 1 commit into
mainfrom
tri-13553-infra-retry-util
Sep 2, 2026
Merged

feat(database): add an infrastructure-error classifier and read-retry helper#4863
d-cs merged 1 commit into
mainfrom
tri-13553-infra-retry-util

Conversation

@d-cs

@d-cs d-cs commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds two shared primitives to @internal/database for surviving brief database connection blips: a classifier that recognises connectivity/infrastructure failures (as opposed to real query errors), and withInfraRetry, a helper that retries an operation on those failures.

Details

isInfrastructureError / looksLikeConnectivityError recognise connection-level failures (unreachable server, closed connection, connection reset, connect timeouts) and deliberately do not match query or validation errors, so retries never mask a real bug.

withInfraRetry(run, config) retries run on a classified infrastructure error with jittered backoff, gated by a shared token-bucket budget so a mass freeze cannot amplify into a retry storm. It has an enabled kill-switch (off by default) and reuses the existing retry-budget primitive.

It must only wrap operations that are safe to run more than once (reads, or writes made idempotent); it deliberately provides no path that silently retries a non-idempotent write.

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: d85831a

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@d-cs d-cs self-assigned this Sep 1, 2026
@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 693c45de-66e3-488b-b78e-987280d429d9

📥 Commits

Reviewing files that changed from the base of the PR and between e77d850 and d85831a.

📒 Files selected for processing (4)
  • internal-packages/database/src/infraError.test.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraRetry.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: All PR Checks
🧰 Additional context used
📓 Path-based instructions (9)
We use vitest exclusively.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
**Prefer static imports over dynamic imports.**

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Add crumbs as you write code — not just when debugging.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Use `$replica` from `~/db.server` for read-heavy queries in the webapp.

📄 CodeRabbit inference engine (internal-packages/database/CLAUDE.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Use Prisma for database interactions in internal-packages/database with PostgreSQL

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Use vitest for all tests in the Trigger.dev repository

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Use function declarations instead of default exports

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
Use types over interfaces for TypeScript Avoid using enums; prefer string unions or const objects instead

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs Do not use high-cardinality attributes in OTEL metr...

📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)

Files:

  • internal-packages/database/src/infraRetry.ts
  • internal-packages/database/src/infraError.ts
  • internal-packages/database/src/infraRetry.test.ts
  • internal-packages/database/src/infraError.test.ts
🧠 Learnings (4)
📚 Learning: 2026-06-04T18:16:35.386Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 3836
File: apps/supervisor/src/backpressure/backpressureMonitor.ts:3-5
Timestamp: 2026-06-04T18:16:35.386Z
Learning: When reviewing TypeScript in this repo, apply the rule “prefer type aliases over interfaces” only to data/object shapes and union/intersection type modeling. If an interface is being used as a behavioral contract for collaborators to implement (e.g., method-shape interfaces that define required behavior, such as `BackpressureLogger` / `BackpressureSignalSource` in `apps/supervisor/src/backpressure/backpressureMonitor.ts`), keep it as an `interface` and do not flag it as a type-alias-vs-interface violation.

Applied to files:

  • internal-packages/database/src/infraRetry.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma errors for P1001 ("Can't reach database server"), do not assume it only appears under a single property name. Prisma may surface P1001 via either `PrismaClientKnownRequestError` (`err.code === "P1001"`, e.g., mid-query connection drops) or `PrismaClientInitializationError` (`err.errorCode === "P1001"`, e.g., client startup connection failure). To reliably detect the condition, check `err.code === "P1001" || err.errorCode === "P1001"`, and avoid review rules that would incorrectly flag `err.code === "P1001"` as unreachable/never-matching.

Applied to files:

  • internal-packages/database/src/infraError.ts
📚 Learning: 2026-05-18T08:21:27.694Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3632
File: apps/webapp/sentry.server.ts:4-21
Timestamp: 2026-05-18T08:21:27.694Z
Learning: When handling Prisma error P1001 ("Can't reach database server") in TypeScript, don’t assume a single error shape. Prisma can surface P1001 via two different error classes/fields: `PrismaClientKnownRequestError` exposes it as `err.code === "P1001"` (common during mid-query connection drops), while `PrismaClientInitializationError` exposes it as `err.errorCode === "P1001"` (common on client startup failure). Therefore, predicates should use `err.code === "P1001" || err.errorCode === "P1001"`. Do not flag `err.code === "P1001"` as “unreachable/never matches,” as it is expected in production.

Applied to files:

  • internal-packages/database/src/infraError.ts
📚 Learning: 2026-06-16T09:19:47.637Z
Learnt from: d-cs
Repo: triggerdotdev/trigger.dev PR: 3960
File: apps/webapp/test/prismaInfrastructureErrorCapture.test.ts:0-0
Timestamp: 2026-06-16T09:19:47.637Z
Learning: In this repo’s Vitest setup, `vitest.config.ts` uses `globals: true`, so identifiers like `vi`, `describe`, `it`, and `expect` are available as globals in Vitest test files. During code review, do not flag missing `vi`/`describe`/`it`/`expect` imports as a runtime error or correctness issue when they’re used in `*.test.ts/tsx` or `*.spec.ts/tsx` files. Explicit imports are still preferred for consistency, but they’re not required for runtime behavior.

Applied to files:

  • internal-packages/database/src/infraRetry.test.ts

Walkthrough

The database package adds Prisma and connectivity-error classification. It adds withInfraRetry with configurable attempts, jittered backoff, retry budgets, injected timing, and callbacks. Tests cover classification, retry behavior, budget limits, default classification, attempt limits, and delay calculation. The package index re-exports both new modules.

Merge Risk: ⚪ Minimal · up to d8583

This localized change adds database error classification and an opt-in retry helper; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 6 functions across 5 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the two primary changes: infrastructure-error classification and a database retry helper.
Description check ✅ Passed The description is on-topic and clearly explains the classifier, retry behavior, safety constraints, kill-switch, backoff, and shared retry budget. It omits the template sections for issue closure, ch…
Full details: Description check

Explanation

The description is on-topic and clearly explains the classifier, retry behavior, safety constraints, kill-switch, backoff, and shared retry budget. It omits the template sections for issue closure, checklist, explicit testing steps, changelog, and screenshots, but the core change description is complete.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch tri-13553-infra-retry-util

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch 2 times, most recently from e35afcf to f4f22da Compare September 2, 2026 08:34
coderabbitai[bot]

This comment was marked as resolved.

@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch from f4f22da to 0436e22 Compare September 2, 2026 11:10
coderabbitai[bot]

This comment was marked as resolved.

@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch from 0436e22 to e77d850 Compare September 2, 2026 12:21
coderabbitai[bot]

This comment was marked as resolved.

@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch from e77d850 to 8dbc68b Compare September 2, 2026 12:49
@d-cs
d-cs marked this pull request as ready for review September 2, 2026 13:01
devin-ai-integration[bot]

This comment was marked as resolved.

@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch from 8dbc68b to 0e36f7c Compare September 2, 2026 15:17
devin-ai-integration[bot]

This comment was marked as resolved.

Adds a shared classifier (isInfrastructureError / looksLikeConnectivityError)
recognising connection-blip failures (P1001/P1002/P1008/P1017, ECONNRESET,
"connection terminated", "server has closed the connection"), and
withInfraRetry — a retry helper gated by an enabled kill-switch (default off)
and the existing TokenBucketRetryBudget.

Only for operations safe to run more than once (reads, or writes made
idempotent); it never authorises retrying a bare non-idempotent write.

TRI-13553
@d-cs
d-cs force-pushed the tri-13553-infra-retry-util branch from 0e36f7c to d85831a Compare September 2, 2026 15:34
@d-cs
d-cs added this pull request to the merge queue Sep 2, 2026
Merged via the queue into main with commit 7317fe1 Sep 2, 2026
54 checks passed
@d-cs
d-cs deleted the tri-13553-infra-retry-util branch September 2, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants