feat(queue): add helix-shared-queue base package - #1272
Merged
Conversation
Introduces a pluggable queue abstraction (QueueService -> Queue, backed by an injectable QueueBackend) mirroring helix-shared-storage's Storage/Bucket/backendFactory pattern, so a concrete provider package (e.g. a future helix-shared-queue-sqs) can be swapped in without forking BatchedQueueClient (@adobe/helix-admin-support). This base package ships no cloud SDK; sendBatch/receiveBatch/deleteBatch are fully backend-owned since batching limits, long-poll shape, and ack-token shape are all provider-specific. Addresses #1269. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Sep 10, 2026
|
This PR will trigger a minor release when merged. |
… spillover receive() no longer promises transparent dereferencing of spilled messages -- that was forcing an I/O fetch for every swapped message regardless of whether the caller actually needed the full body, defeating an optimization existing consumers rely on (e.g. reading just owner/repo off a message without ever touching blob storage). Add isSwapped(message)/deserialize(message) to AbstractQueueBackend (generic defaults: always false / return unchanged, since spillover support is optional) and Queue (thin delegation), so callers opt into the blob fetch only when they need the real body, and this works identically regardless of which backend is plugged in. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ntion Queue#backend/#log and QueueService#log/#backendFactory/#closed become real private class fields -- neither is subclassed at the instance level (QueueServiceSqs/QueueServiceServiceBus only override the static fromContext), so nothing needs prototype-chain access to them. AbstractQueueBackend._wrapError stays _-prefixed: it's called via this._wrapError(...) from concrete backend subclasses' own method bodies, and true #private members aren't reachable across that inheritance boundary -- documented inline. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sugar over send([message]) unwrapping the single resulting messageId, for callers that only ever send one message at a time (e.g. helix-api-service's Job.js, which currently does new SQSClient().send(new SendMessageCommand(...)) directly and reads result.MessageId).
stefan-guggisberg
approved these changes
Sep 11, 2026
Contributor
Author
|
🎉 This PR is included in version 9.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
Author
|
🎉 This PR is included in version @adobe/helix-shared-queue-v1.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@adobe/helix-shared-queue, a pluggable queue abstraction (QueueService→.queue(name)→Queue, backed by an injectableQueueBackend/backendFactory), mirroring@adobe/helix-shared-storage'sStorage/Bucketpattern so a concrete provider package (e.g. a futurehelix-shared-queue-sqs) can be swapped in without forkingBatchedQueueClient(@adobe/helix-admin-support).Queue#send/receive/deletegeneralizeBatchedQueueClient's send/long-poll-receive/batch-delete semantics;groupId/dedupIdare named generically (not SQS'sMessageGroupId/MessageDeduplicationId) so a future Azure Service Bus backend can map them onto session id + native dedup.sendBatch/receiveBatch/deleteBatchare fully mandatory onAbstractQueueBackendwith no generic defaults, since batching limits, long-poll call shape, and ack-token shape are all provider-specific. This base package has zero cloud SDK dependencies.isSwapped(message)/deserialize(message)(added after the initial version of this PR, once two concrete backends surfaced the need): a backend that spills oversized messages to blob storage should not havereceive()transparently fetch every spilled message's real content — that forces an I/O round-trip even when the caller only needed a couple of cheap fields off the message (e.g.helix-indexer'snotify()reads justowner/repowithout ever touching storage). These two methods let a caller check cheaply (no I/O) and fetch only when actually needed, identically regardless of which backend is plugged in. Both have generic defaults onAbstractQueueBackend(isSwappedalwaysfalse,deserializereturns the message unchanged), since spillover support is optional.CLAUDE.mdwith a package entry.Addresses #1269. Companion issues #1270 (SQS backend) and #1271 (Azure Service Bus backend) are now implemented in #1274 and #1275 respectively, both depending on this PR.
Test plan
npm run lint(workspace-wide, 19 projects) passesnpm test(workspace-wide, 19 projects) passes.nycrc.jsongate), includingisSwapped()/deserialize()'s delegation and generic-default behaviorbackendFactory(new QueueService({ backendFactory }).queue('name').send([...]))🤖 Generated with Claude Code