Skip to content

Commit ab4c607

Browse files
committed
docs: use-case-first structure for queue concurrency
A Use cases index links each goal to its section, the multi-queue section names the home queue and gate concepts once and gives each pattern its own worked example (per-tenant cap across tasks, global cap for a shared resource via a combined-only queue, pinned-key shared pool), and the per-key-except-combined rule gets a warning callout. Folds in the simplified wording and removes self-hosting notes.
1 parent f31efe8 commit ab4c607

1 file changed

Lines changed: 56 additions & 31 deletions

File tree

docs/queue-concurrency.mdx

Lines changed: 56 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,16 @@ Controlling concurrency is useful when you have a task that can't be run concurr
1111

1212
It's important to note that only actively executing runs count towards concurrency limits. Runs that are delayed or waiting in a queue do not consume concurrency slots until they begin execution.
1313

14+
## Use cases
15+
16+
- **Limit how many runs of a task execute at once**: [Setting task concurrency](#setting-task-concurrency)
17+
- **Share one limit across several tasks**: [Sharing concurrency between tasks](#sharing-concurrency-between-tasks)
18+
- **Give each tenant its own separate concurrency**: [Concurrency keys and per-tenant queuing](#concurrency-keys-and-per-tenant-queuing)
19+
- **Per-tenant limits with a ceiling on the whole queue**: [Combined concurrency across keys](#combined-concurrency-across-keys)
20+
- **Cap a tenant across every task they run**: [A per-tenant cap across multiple tasks](#a-per-tenant-cap-across-multiple-tasks)
21+
- **Cap a shared resource, like an external API, across tasks and tenants**: [A global cap for a shared resource](#a-global-cap-for-a-shared-resource)
22+
- **Funnel every run into one shared pool**: [One shared pool ignoring keys](#one-shared-pool-ignoring-keys)
23+
1424
## Default concurrency
1525

1626
By default, all tasks have an unbounded concurrency limit, limited only by the overall concurrency limits of your environment.
@@ -168,26 +178,37 @@ export const perUserQueue = queue({
168178
name: "per-user-queue",
169179
//each user runs at most 1 at a time...
170180
concurrencyLimit: 1,
171-
//...and at most 10 users can be running at once
181+
//...and at most 10 total runs across all users
172182
combinedConcurrencyLimit: 10,
173183
});
174184
```
175185

176186
The combined limit only applies to runs triggered with a `concurrencyKey`; runs without a key are governed by `concurrencyLimit` alone. On the Queues page in the dashboard, a queue with a combined limit shows it in brackets next to the per-key limit, e.g. `1 (10)`.
177187

178-
<Note>
179-
If you self-host, combined limits are enforced by default and can be disabled with
180-
`RUN_ENGINE_TOTAL_CONCURRENCY_LIMITS_ENABLED=0`. When enforcement is disabled the limit is
181-
still accepted, stored, and shown, but runs are not held back by it.
182-
</Note>
188+
<Warning>
189+
On a queue used with `concurrencyKey`, every limit applies per key value except
190+
`combinedConcurrencyLimit`, which is the only cap that spans the whole queue.
191+
</Warning>
183192

184-
## Holding slots in more than one queue (queue gates)
193+
## Using multiple queues at once
185194

186-
Sometimes one limit isn't enough: each tenant's webhook processing should be capped, but the tenant should also have a global cap across every task they run. Queue gates let a run hold a concurrency slot in more than one queue at once.
195+
Sometimes one limit isn't enough. A run always waits in one queue, its **home queue**, but it can also hold a concurrency slot in up to two more queues, called **gates**. A run starts only when its home queue and every gate all have capacity, occupies a slot in each while it executes, and releases them together when it finishes or suspends.
187196

188-
Pass an array as `queue`: the first entry is the run's home queue (where it waits), and up to two more entries name gates — other queues the run must also have capacity in before it starts, and occupies while it executes:
197+
Pass an array as `queue`: the first entry is the home queue, the rest name gates. The same array form works when you trigger, replacing the task's gates for that run:
198+
199+
```ts
200+
await processWebhook.trigger(payload, {
201+
queue: ["webhooks", "tenant"],
202+
concurrencyKey: tenantId,
203+
});
204+
```
205+
206+
### A per-tenant cap across multiple tasks
207+
208+
A gate without a `concurrencyKey` uses the run's own key. Declare a shared queue and gate every relevant task on it, and each tenant gets one cap spanning all of those tasks:
189209

190210
```ts /trigger/webhooks.ts
211+
//each tenant runs at most 10 at once across every task that gates on this queue
191212
export const tenantQueue = queue({ name: "tenant", concurrencyLimit: 10 });
192213

193214
export const processWebhook = task({
@@ -204,43 +225,47 @@ export const processWebhook = task({
204225
await processWebhook.trigger(payload, { concurrencyKey: tenantId });
205226
```
206227

207-
Because the trigger passes a `concurrencyKey`, the home queue splits per key as usual: `concurrencyLimit: 2` caps each tenant's webhook runs, not the task overall (add a `combinedConcurrencyLimit` to the home queue to bound it across all tenants).
228+
Because the trigger passes a `concurrencyKey`, the home queue splits per key as usual: `concurrencyLimit: 2` caps each tenant's webhook runs, not the task overall.
208229

209-
A gate without a `concurrencyKey` uses the run's own key, so the shared `tenant` queue above caps each tenant across every task that names it as a gate. Give the gate a literal key to pin it to a single slot pool instead, for example capping all traffic to one external provider across your whole environment:
230+
### A global cap for a shared resource
231+
232+
To cap something global, like total traffic to an external API, across many tasks and all tenants: declare a queue with only a `combinedConcurrencyLimit` and gate on it. Tenant keys still split the gate into per-key pools, but with no per-key limit the combined cap is the only constraint:
210233

211234
```ts /trigger/sync.ts
212-
//the gate's capacity comes from the queue's own declaration
213-
export const providerApiQueue = queue({ name: "provider-api", concurrencyLimit: 5 });
235+
//at most 10 concurrent provider calls across every task and every tenant
236+
export const providerApiQueue = queue({
237+
name: "provider-api",
238+
combinedConcurrencyLimit: 10,
239+
});
214240

215241
export const syncToProvider = task({
216242
id: "sync-to-provider",
217-
queue: [
218-
{ name: "sync-home", concurrencyLimit: 20 },
219-
//every run shares one "provider-api" pool regardless of its own key
220-
{ name: "provider-api", concurrencyKey: "shared" },
221-
],
243+
queue: [{ name: "sync-home", concurrencyLimit: 20 }, "provider-api"],
222244
run: async (payload) => {
223245
//...
224246
},
225247
});
226248
```
227249

228-
The same array form works when you trigger, replacing the task's gates for that run:
250+
### One shared pool ignoring keys
229251

230-
```ts
231-
await processWebhook.trigger(payload, {
232-
queue: ["webhooks", "tenant"],
233-
concurrencyKey: tenantId,
234-
});
235-
```
252+
Give a gate a literal `concurrencyKey` to pin every run into a single first-come-first-served pool, regardless of each run's own key:
236253

237-
A run starts only when its home queue and every gate all have capacity, and it releases all of its slots together when it finishes or suspends.
254+
```ts /trigger/print.ts
255+
export const printerQueue = queue({ name: "printer", concurrencyLimit: 1 });
238256

239-
<Note>
240-
Queue gates are enforced when the server has them enabled. If you self-host, set
241-
`RUN_ENGINE_QUEUE_GATES_ENABLED=1`; servers without gates enabled accept the option but run
242-
without it.
243-
</Note>
257+
export const printLabel = task({
258+
id: "print-label",
259+
queue: [
260+
{ name: "print-home", concurrencyLimit: 5 },
261+
//every run shares the single "printer" slot no matter its own key
262+
{ name: "printer", concurrencyKey: "shared" },
263+
],
264+
run: async (payload) => {
265+
//...
266+
},
267+
});
268+
```
244269

245270
## Concurrency and subtasks
246271

0 commit comments

Comments
 (0)