feat(serenity): opt-in async provisioning for createMarket, createBrandForOrg, activate (LLMO-7352/LLMO-7418)

Converts the three remaining synchronous Semrush sub-workspace/project
provisioning endpoints to an opt-in async model, gated by a per-request
`async: true` flag (mirroring the existing `createPrompts` precedent) so
default behavior is unchanged for every current consumer.

- createMarket, createBrandForOrg's initial market, and activate's
  project-activation branch now mint a provisioning attempt and enqueue
  provision-workspace-job -> {create-market,activate-markets}-job when
  `async: true` is supplied, returning 202 with a pollable jobId.
- Business logic shared between the sync controller path and the async
  job handler is extracted into create-market-orchestration.js and
  activate-markets-orchestration.js.
- guardAgainstConcurrentProvisioning protects every synchronous branch
  against racing a live async attempt for the same brand.
- Adds a generic /serenity/jobs/:jobId polling alias.
- Adds recordFreshBrandProvisioningStartFailure so a brand row created in
  the same request is marked failed (not left silently inert) if minting
  the provisioning attempt itself throws.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Author: Iris Alexandrescu

docs/openapi/brands-v2-api.yaml

@@ -79,6 +79,19 @@ v2-brands-for-org:
           application/json:
             schema:
               $ref: './schemas.yaml#/V2Brand'
+      '202':
+        description: |
+          PR-C (LLMO-7352/LLMO-7418): the request set `async: true` alongside
+          `semrushMarket`. The brand row is persisted immediately (visible,
+          non-active — "Setting up", no workspace pointer yet); the
+          sub-workspace ready-check and the initial market's project create/
+          publish are enqueued to a background job chain instead of running
+          inline. Poll the returned `jobId` at `GET .../serenity/jobs/{jobId}`
+          for status + result.
+        content:
+          application/json:
+            schema:
+              $ref: './schemas.yaml#/SerenityPromptsJobAccepted'
       '400':
         $ref: './responses.yaml#/400'
       '403':
@@ -104,6 +117,8 @@ v2-brands-for-org:
             organization (`code: brand_site_org_mismatch`).
           - The brand's primary URL is already the primary URL (base site) of
             another brand in this organization.
+          - (PR-C, `async: true` only) a provisioning attempt is already in flight
+            for this brand (`error: semrushProvisioningInProgress`).
         headers:
           X-Error:
             $ref: './headers.yaml#/xError'

docs/openapi/schemas.yaml

@@ -8068,6 +8068,50 @@ V2BrandInput:
         onboarded inline. Written to `brands.site_id` as the brand's primary-URL
         anchor. Sent on every create (a Serenity brand is anchored by BOTH its
         sub-workspace and this primary Site).
+    semrushMarket:
+      type: object
+      description: >-
+        Provisions an initial Semrush market (project) for the brand at create
+        time, in a serenity-active organization. Requires `market` and
+        `languageCode`; omit entirely for a sub-workspace-only create (markets
+        added afterwards from the Markets tab).
+      required: [market, languageCode]
+      properties:
+        market:
+          type: string
+          pattern: '^[A-Za-z]{2}$'
+          description: ISO 3166-1 alpha-2 country code; resolved server-side to a Geo Target ID.
+        languageCode:
+          type: string
+          pattern: '^[a-z]{2,3}(-[a-z]{2,4})?$'
+    semrushModelIds:
+      type: array
+      items: { type: string }
+      description: >-
+        AI model (LLM) ids to attach to the initial market's project. Required
+        (non-empty) when `generatePrompts` is true; optional otherwise (the
+        project is created model-less until models are added later).
+    generatePrompts:
+      type: boolean
+      default: false
+      description: >-
+        Whether to generate topics/prompts for the initial `semrushMarket`
+        project. Requires a primary URL and `semrushModelIds`.
+    async:
+      type: boolean
+      default: false
+      description: >
+        PR-C (LLMO-7352/LLMO-7418). Only meaningful when `semrushMarket` is
+        supplied. When true, the brand row is persisted first (visible,
+        non-active — "Setting up"), then the sub-workspace ready-check and the
+        initial market's project create/publish are enqueued to a background
+        job chain instead of running inline, and the endpoint returns 202 with
+        a job id to poll via `GET .../serenity/jobs/{jobId}`. Omitted/false
+        processes synchronously as today (still 201) — this is opt-in only and
+        not yet the default. Unlike `SerenityCreatePromptsRequest`'s `async`
+        flag, this one is not intended to be a permanent dual-mode feature: the
+        synchronous path is the LLMO-7352 bug pattern itself, and is expected
+        to be retired once known callers have migrated to `async: true`.
 
 V2BrandUpdateInput:
   type: object
@@ -12178,6 +12222,37 @@ SerenityPromptsJobStatus:
             message: { type: string }
         - { type: 'null' }
 
+SerenityJobStatus:
+  type: object
+  description: |
+    Poll response for ANY async Serenity job — serenity-classify-prompts,
+    serenity-provision-workspace, serenity-create-market, or
+    serenity-activate-markets (PR-C, LLMO-7352/LLMO-7418). Secret-free by
+    design — the job's internal metadata (promise token etc.) is never exposed.
+  required: [jobId, status]
+  properties:
+    jobId:
+      type: string
+      format: uuid
+    status:
+      type: string
+      enum: [IN_PROGRESS, COMPLETED, FAILED]
+    result:
+      description: >
+        The job result once COMPLETED — the same body the equivalent
+        synchronous endpoint would have returned (shape depends on which
+        `async: true` call created this job: SerenityCreatePromptsResponse,
+        SerenityMarket, or SerenityActivateResponse). Null while IN_PROGRESS or
+        on FAILED.
+    error:
+      description: The failure envelope once FAILED; null otherwise.
+      oneOf:
+        - type: object
+          properties:
+            code: { type: string }
+            message: { type: string }
+        - { type: 'null' }
+
 SerenityUpdatePromptRequest:
   type: object
   description: |
@@ -12430,6 +12505,18 @@ SerenityActivateRequest:
             description: >-
               Optional AI model (LLM) ids to attach to this market's project
               (already-active brand reactivation).
+    async:
+      type: boolean
+      default: false
+      description: >
+        PR-C (LLMO-7352/LLMO-7418). Only meaningful on the already-active-brand,
+        markets-supplied path — a pending or bare-reactivation activation always
+        stays synchronous regardless of this flag. When true, the sub-workspace
+        ready-check and the per-market project create/publish + site-link + active
+        flip are enqueued to a background job chain instead of running inline, and
+        the endpoint returns 202 with a job id to poll via
+        `GET .../serenity/jobs/{jobId}`. Omitted/false processes synchronously as
+        today — this is opt-in only and not yet the default.
 
 SerenityActivateResponse:
   type: object
@@ -12560,6 +12647,20 @@ SerenityCreateMarketRequest:
         e.g. www.nba.com/kings). Written to Semrush `settings.ai.primary_url` via
         a best-effort PATCH before publish (serenity-docs#348). Optional — falls
         back to `brandDomain` (host-only) when omitted.
+    async:
+      type: boolean
+      default: false
+      description: >
+        PR-C (LLMO-7352/LLMO-7418). When true (sub-workspace-mode brands only),
+        the sub-workspace-ready check and the project create/publish are
+        enqueued to a background job chain instead of running inline, and the
+        endpoint returns 202 with a job id to poll via
+        `GET .../serenity/jobs/{jobId}`. Omitted/false processes synchronously
+        as today — this is opt-in only and not yet the default. Unlike
+        `SerenityCreatePromptsRequest`'s `async` flag, this one is not intended
+        to be a permanent dual-mode feature: the synchronous path is the
+        LLMO-7352 bug pattern itself, and is expected to be retired once known
+        callers have migrated to `async: true`.
 
 SerenityTag:
   type: object

docs/openapi/serenity-api.yaml

@@ -327,6 +327,58 @@ v2-serenity-prompts-job-by-id:
         description: Organization, brand, or job not found, serenity is not active for the brand, or the job belongs to a different brand.
       '500': { $ref: './responses.yaml#/500' }
 
+v2-serenity-job-by-id:
+  parameters:
+    - name: spaceCatId
+      in: path
+      required: true
+      description: SpaceCat Organization ID (UUID)
+      schema:
+        type: string
+        format: uuid
+    - name: brandId
+      in: path
+      required: true
+      description: Brand ID (UUID-only on the /serenity/* surface)
+      schema:
+        type: string
+        format: uuid
+    - name: jobId
+      in: path
+      required: true
+      description: 'Async job id returned by any `async: true` 202 response.'
+      schema:
+        type: string
+        format: uuid
+  get:
+    tags: [serenity]
+    summary: Poll any Serenity async job (PR-C, LLMO-7352/LLMO-7418)
+    description: |
+      Job-type-agnostic alias of `GET .../serenity/prompts/jobs/{jobId}` (same
+      handler — it never inspects the job's `type`, only its `brandId`). Use this
+      path, not the prompts-named one, to poll a job returned by `async: true` on
+      `POST .../serenity/markets`, `POST .../serenity/activate`, or
+      `POST /v2/orgs/{spaceCatId}/brands` — the prompts path works too, but this
+      one doesn't say "prompts" for a workspace-provisioning or market/activation
+      job. Access is gated by the same org/brand access + serenity-active checks
+      as every other serenity endpoint; a job that does not belong to the
+      addressed brand 404s exactly like a missing job (jobs are never leaked
+      across brands). The response is secret-free — only
+      `{ jobId, status, result, error }`.
+    operationId: getSerenityJobStatus
+    security:
+      - session_token: []
+    responses:
+      '200':
+        description: Job status (and result when COMPLETED / error when FAILED).
+        content:
+          application/json:
+            schema: { $ref: './schemas.yaml#/SerenityJobStatus' }
+      '400': { $ref: './responses.yaml#/400' }
+      '404':
+        description: Organization, brand, or job not found, serenity is not active for the brand, or the job belongs to a different brand.
+      '500': { $ref: './responses.yaml#/500' }
+
 # ─── Markets ──────────────────────────────────────────────────────────────────
 
 v2-serenity-markets:
@@ -393,12 +445,25 @@ v2-serenity-markets:
         content:
           application/json:
             schema: { $ref: './schemas.yaml#/SerenityMarket' }
+      '202':
+        description: |
+          PR-C (LLMO-7352/LLMO-7418): the request set `async: true` (a
+          sub-workspace-mode brand only). The sub-workspace-ready check and the
+          project create/publish are enqueued to a background job chain instead
+          of running inline. Poll the returned `jobId` at
+          `GET .../serenity/jobs/{jobId}` for status + result — on COMPLETED,
+          `result` is the same body a synchronous `201` would have returned.
+          Absent/false runs synchronously as today; this is opt-in only and not
+          yet the default.
+        content:
+          application/json:
+            schema: { $ref: './schemas.yaml#/SerenityPromptsJobAccepted' }
       '400':
         description: Bad request (unknown market, malformed languageCode, missing field).
       '404':
         description: Organization has no workspace, or serenity is not active for the organization.
       '409':
-        description: A market already exists for this (brandId, geoTargetId, languageCode) slice.
+        description: A market already exists for this (brandId, geoTargetId, languageCode) slice, or (async only) a provisioning attempt is already in flight for this brand.
       '502':
         description: Upstream returned a non-2xx response.
         content:
@@ -915,6 +980,14 @@ v2-serenity-activate:
       pending and the response is HTTP 502 (its stash + workspace pointer are kept
       intact; retry converges idempotently). An already-active brand re-supplying
       markets is never downgraded (a partial failure is reported as 207).
+
+      PR-C (LLMO-7352/LLMO-7418): when a primary URL/domain and markets are
+      present (the branch described above — NOT the sub-workspace-only pending
+      or bare-reactivation paths, which always stay synchronous), the caller may
+      set `async: true` to enqueue the whole batch to a background job chain
+      instead of running it inline; see the `202` response. Absent/false runs
+      synchronously as described above — this is opt-in only and not yet the
+      default.
     operationId: activateSerenityBrand
     security:
       - session_token: []
@@ -929,6 +1002,19 @@ v2-serenity-activate:
         content:
           application/json:
             schema: { $ref: './schemas.yaml#/SerenityActivateResponse' }
+      '202':
+        description: |
+          PR-C (LLMO-7352/LLMO-7418): the request set `async: true` on the
+          project-activation branch (primary URL + markets present). The whole
+          batch (sub-workspace ready-check, every market's project create/
+          publish, the site link, and the active flip) is enqueued to a
+          background job chain instead of running inline. Poll the returned
+          `jobId` at `GET .../serenity/jobs/{jobId}` for status + result — on
+          COMPLETED, `result` is the same body a synchronous `200`/`207` would
+          have returned.
+        content:
+          application/json:
+            schema: { $ref: './schemas.yaml#/SerenityPromptsJobAccepted' }
       '207':
         description: >-
           An already-active brand re-supplied markets and at least one failed; the
@@ -944,9 +1030,11 @@ v2-serenity-activate:
           Either (a) the primary domain is already another active brand's primary
           site (`brands_base_site_unique`): the brand STAYS `pending` and the body
           carries `error: serenityActivationSiteConflict` plus the per-market
-          outcomes — the operator must choose a different primary URL; or (b)
+          outcomes — the operator must choose a different primary URL; (b)
           misconfiguration: the brand's sub-workspace equals the organization parent
-          workspace (refused to avoid acting on the shared parent pool).
+          workspace (refused to avoid acting on the shared parent pool); or (c)
+          (PR-C, `async: true` only) a provisioning attempt is already in flight for
+          this brand (`error: semrush_provisioning_in_progress`) — retry shortly.
         content:
           application/json:
             schema: