feat(o11y): stamp testhubBuildUuid + buildProductMap on cypress build caps [SDK-7285]

[kamal-kaur04]

What is this about?

Two commits. ed3a58a adds the two capabilities; 5c9a9e0 makes buildProductMap.observability report actual state rather than requested state (product-owner requirement, 2026-09-10). Detail + the before/after toggle in this comment.

Cypress is the only SDK line that stamps no TestHub capability at all — measured, 0 of 16,629 sessions carry one, across all 32 CLI versions present including the newest. bin/ has zero references to testhubBuildUuid, buildProductMap or a browserstackSDK cap; the build id goes to the in-browser reporter instead.

That leaves cypress with a single route into Test Reporting & Analytics — the BTCER session_hashed_id fallback — where every other SDK line has two. That fallback misses ~32.5% of cypress sessions, and those sessions are then counted against ingestion stability with no build to attach to.

Population: 2,917 orphaned sessions in the measured cohort · 0.5882 pp of instability · 76 accounts / 56 groups. Cypress is the only diffuse line in that bucket — top group 10.6%, 24 groups to reach 90%, where every other line is 83–100% a single customer. So this is a product gap affecting many customers a little, not a support case.

This PR adds the missing surface. It does not on its own move the metric — see Dependent PRs / release order.

Related Jira task/s

• SDK-7285 — item A1.1 of the approved orphan-bucket action-item ledger

Dependent PRs / release order

⚠️ This PR is the last of three legs to become useful, and it is inert until the second one ships. Please do not expect a metric movement from merging it alone.

Leg	Where	State
A1.1 — send the two keys on the build-creation payload	this PR	✅ here
A1.1b — map both keys into the session capabilities hash	automate rails	❌ not started, no owner
A1.2 — extract instead of CAST(NULL AS STRING)	zombies l4_metrics.yml:2081-2087	❌ open

Why A1.1b exists. The capabilities column the metric reads on automate.cypress_test_sessions_partitioned is not the payload the CLI sends — it is rebuilt server-side from an explicit key list. Measured over 3,782 cypress sessions (BQ job_5VlbncWZuByJPM9vetoSvqNYMkmK):

CLI-sent key	Appears in session capabilities
run_settings (sent on every build)	0 / 3,782 — dropped
cypress_cli_user_agent (sent on every build)	0 / 3,782 — dropped
parallels	0 / 3,782 — dropped
test_suite	3,782 — renamed to testSuite
local_mode_inferred	3,782 — verbatim

Three keys sent on every build never arrive; one is renamed while another is preserved verbatim. Mixed casing plus outright drops rules out a blanket transform.

Automation cases to add

None. The change adds two keys to an outbound payload; there is no user-visible behaviour to assert from a wrapper test, and the value is only observable in BigQuery. Covered by unit tests here + the end-to-end wire capture below.

Code changes to check

• bin/helpers/capabilityHelper.js — the two new keys, and the new require of ../testhub/utils
• test/unit/bin/helpers/capabilityHelper.js — new context("testhub build attribution")

The change

obj.testhubBuildUuid = process.env.BROWSERSTACK_TESTHUB_UUID || "";
obj.buildProductMap  = testhubUtils.getProductMap(bsConfig);

Four things worth a reviewer's attention, all deliberate:

1. No sequencing work was needed. runs.js:137 already awaits TestHubHandler.launchBuild, which sets BROWSERSTACK_TESTHUB_UUID at testhub/utils.js:236, before capabilityHelper.validate → build.createBuild → caps(). Confirmed on the wire below.
2. getProductMap() is reused, not reimplemented (bin/testhub/utils.js:38) — same five-product shape the other SDK lines stamp. Calling it at caps-build time is deliberate: it reports the products still enabled after the accessibility/observability gates at runs.js:118-137, rather than the ones originally requested.
3. No new require cycle. bin/helpers/utils.js:30 already top-level-requires ../testhub/utils, and capabilityHelper already requires ./utils, so the module was in the graph. getProductMap is reached through the module object (late-bound) rather than destructured at load time, so a partially-initialised export cannot bite.
4. Both keys are written unconditionally (|| ""). The metric treats an absent key and '' identically, so the empty string costs nothing — and it preserves the distinction between a cap writer ran and had nothing to name and no cap writer ran at all, which for cypress is currently unobservable anywhere. Key names deliberately match the other five SDK lines so the A1.2 branch can reuse their existing REGEXP_EXTRACT verbatim.

Second commit — the map must not lie about observability

buildProductMap is stamped twice with two different meanings: on the build-start request it is intent, on the session caps it is outcome. Only the first was honest.

extractDataFromResponse already clears BROWSERSTACK_TEST_OBSERVABILITY for a 2xx carrying observability.success = false, but launchBuild's catch — every non-2xx and every network error — only logged, leaving the flag true. The session then advertised observability with no build behind it. The collector rejects 7,442 × 400 + 442 × 403 build-starts per week, so this was the common path, not an edge case.

Three coupled changes: handleErrorForObservability also clears the flag (one funnel, all failure paths agree); launchBuild's catch routes through it, before the error.success === false branch so network errors are covered too; and caps() no longer stamps the "null" sentinel as a uuid — which is required because of the second change, since that is what now causes the sentinel to be written on this path.

An explicit testObservability: false in config already resolved correctly and was left alone; it is covered by a test purely as a regression guard.

Verification

Unit — 2 new tests; whole-suite comparison against a stashed baseline, so "no regressions" is measured rather than asserted:

	passing	failing
full unit suite with this change	691	16
full unit suite, change stashed (8731a23)	689	16

Same 16 failures either way — all pre-existing on master (runs ×7, fileHelpers ×2, readCypressConfigUtil ×3, syncSpecsLogs ×1, utils › getVideoConfig ×3). Heads-up that master's unit suite does not currently pass clean.

End-to-end on production infra — patched CLI served from source, --cli-debug --sync, 1 spec / 2 tests, Chrome 136 / Windows 11. Total tests: 1, passed: 1, failed: 0, exit 0.

TestHub build	xcsouiygyettrxqdxyxmi18qnkyb6alfxqnj8x4z
Automate build	b1caa6889d0ce4122dd9c601374d9462e01f389b
Session	ecbded54b9f90a9a4dc9ebffdac03db3ae1d6e21

CLI debug, showing the value was in hand before caps were built:

debug: TestHub build uuid stamped on caps: xcsouiygyettrxqdxyxmi18qnkyb6alfxqnj8x4z

On the wire — captured by wrapping axios.post from outside the repo, so this is the request as sent rather than the CLI's own account of it:

POST https://api.browserstack.com/automate/cypress/v1/builds/   → 201

keys sent: [buildProductMap, cache_dependencies, customBuildName, cypress_cli_user_agent,
            devices, local, parallels, project, run_settings, test_suite, testhubBuildUuid]

"testhubBuildUuid": "xcsouiygyettrxqdxyxmi18qnkyb6alfxqnj8x4z"
"buildProductMap":  {"observability": true, "accessibility": false,
                     "percy": false, "automate": true, "app_automate": false}

response: {"message":"Success","build_id":"b1caa6889d0ce4122dd9c601374d9462e01f389b", ...}

The uuid on the caps payload is byte-identical to the TestHub build the same run created, and the hub accepted the payload with both previously-unknown keys present (201) — which settles the open "confirm the hub tolerates an unknown cap key before rollout" question in the affirmative.

What is NOT verified here: whether the keys survive into automate.cypress_test_sessions_partitioned.capabilities. That is leg A1.1b and is a property of the automate rails layer, not of this diff. The probe session's row had not landed at the time of writing; I will post the result as a comment either way. Note the metric excludes group_id IN (2, 1224686) and this run is group 2, so the probe is plumbing-only and never enters the stability numbers.

Release

Version bump:

• minor (backwards-compatible feature)
• patch (bug fix or other small change)

Release notes type:

• Improvement

Release notes (customer-facing):

• Cypress builds now report their Test Reporting & Analytics build id and enabled-product map with the session, so Cypress sessions can be attributed to the build that created them. When Test Observability is disabled or its build could not be started, the reported product map now correctly shows it as off.

Release notes (internal):

• bin/helpers/capabilityHelper.js: caps() now stamps testhubBuildUuid (from BROWSERSTACK_TESTHUB_UUID, set by TestHubHandler.launchBuild before caps() runs) and buildProductMap (from the existing testhub/utils.js getProductMap()) on the build-creation payload. Cypress previously stamped no TestHub capability on any session (0 of 16,629 measured), leaving the BTCER session_hashed_id fallback as its only route into TRA; that fallback misses ~32.5%, worth 2,917 orphaned sessions / 0.5882 pp across 76 accounts. Both keys are written unconditionally so an empty uuid is distinguishable from an absent writer.
• Added test/unit/bin/helpers/capabilityHelper.js → context("testhub build attribution").
• bin/testhub/utils.js + bin/testhub/testhubHandler.js: handleErrorForObservability now also clears BROWSERSTACK_TEST_OBSERVABILITY, and launchBuild's catch routes through it rather than only logging — so a non-2xx or network failure on build start no longer leaves buildProductMap.observability reporting true with no build behind it (verified against an injected collector 400: true before, false after). caps() additionally guards against stamping the "null" uuid sentinel that funnel writes. Also logBuildError treated only undefined as "no error object", so a null error logged the string "null" — this affected the accessibility no-response path as well.
• Inert until the automate rails layer maps both keys into the session capabilities hash (that column is rebuilt server-side from an explicit key list — three keys the CLI sends on every build appear 0/3,782 times), and until the zombies cypress branch extracts testhubBuildUuid instead of hardcoding CAST(NULL AS STRING).

Checklist

• Ready to review
• Has it been approved by a member of your team?
• Verified end-to-end on BrowserStack infra (build links above)
• Payload verified on the wire, separately from the pass/fail result
• No-regression measured against a stashed baseline, not asserted
• Fix demonstrated by toggle (4 new tests fail on unfixed source, pass on fixed; plus a before/after production run)
• Downstream dependencies and release order stated

Reviewer notes

The one judgement call is point 4 above — writing "" rather than omitting the key when build start produced no uuid. The metric cannot tell the two apart today, so this is free on the measurement side; the reason to prefer the empty string is diagnostic, and it is the only thing that would let us separate build start failed from TestHub was never requested on a cypress session. Happy to guard it instead if you would rather the key never appear empty.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited), Workspace UI (inherited)

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 4c4df2bb-2ca8-4298-8e35-e01638d49351

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[kamal-kaur04]

A1.1b verdict: CONFIRMED. This PR is inert until the automate rails layer maps the two keys through.

The probe session's row landed 34 minutes after the run (hashed_id = ecbded54b9f90a9a4dc9ebffdac03db3ae1d6e21). It is the direct test the description said was pending, and it came back negative:

has_testhub_uuid : false
has_product_map  : false

The stored capabilities for that exact session, in full — this is the column the ingestion-stability metric reads:

{"devices": ["Windows 11-chrome136"], "testSuite": "dd42ed9e0572e7f79410495f3a01fefa513808a8",
 "npmPackageSuite": null, "cacheDependencies": 0,
 "customBuildName": "SDK-7285-A1-e2e-20260909-133652", "local": 0, "localIdentifier": null,
 "localVersion": null, "callbackURL": null, "projectNotifyURL": null,
 "local_mode_inferred": "", "local_inferred": 0, "sync_inferred": 0, "geolocation": null,
 "build_artifacts": 0, "video_config": {"video": false}, "spec_timeout": 1800000,
 "interactiveDebugging": 1, "project": "TRA-A1-cypress-caps-instrumentation",
 "node_version": "20.16.0", "network_logs": "false", "framework": "cypress", "headless": false}

Same run, same build, minutes apart:

	testhubBuildUuid	buildProductMap
On the wire — POST /automate/cypress/v1/builds/ → 201	✅ xcsouiygyettrxqdxyxmi18qnkyb6alfxqnj8x4z	✅ present
Stored session capabilities	❌ absent	❌ absent

So this is no longer an inference from the caps corpus — it is demonstrated on a single controlled run. The CLI side is correct and complete; the keys are dropped downstream.

Note what the stored hash does contain, which shows the mechanism rather than just the outcome: testSuite (the CLI sent test_suite — renamed), local_mode_inferred (verbatim), and spec_timeout / network_logs / headless / node_version / video_config / geolocation / build_artifacts / interactiveDebugging — all of which the CLI sends inside the run_settings JSON string and which are hoisted out server-side. An explicit key-by-key construction, exactly as the description predicted.

What I'd suggest for this PR

The diff is correct and I'd argue it should still land — it is a prerequisite, it is inert rather than risky, and shipping it early means the rails change has something to receive. But it should not be released with a customer-facing note claiming attribution works, because it will not until the rails leg is in. Two options, reviewer's call:

1. Merge now, hold the release note. Keep the internal note, drop the customer-facing line until A1.1b lands.
2. Keep as draft until the automate side is committed to, then merge the two together.

I've left it as draft and have not marked it ready.

Incidental finding, worth knowing if you ever time-bound a query on this table

created_at on automate.cypress_test_sessions_partitioned is not the session start. For this session the Automate API reports 2026-09-09T13:45:30Z, while the BQ row carries 14:17:20Z — ~32 minutes later — and the row became visible ~2 minutes after its own created_at. That reconciles with unrelated rows appearing at age_min ≈ 2: the row is written when the cypress orchestration finalizes the session record, not when the session began. Anything that windows cypress sessions on created_at inherits that offset.

[kamal-kaur04]

Second commit — buildProductMap.observability now reports reality, not intent

Product-owner requirement: the map must say observability: false (a) when the user explicitly disabled it in config, and (b) when build start failed for any reason. Commit 5c9a9e0.

Rebased onto the branch, not force-pushed — master had been merged in remotely (7ff0971, bringing #1177 and others) while I was working. My commit replayed cleanly on top; the merge touched none of the files this change depends on (git diff ed3a58a 7ff0971 -- bin/testhub/ bin/helpers/capabilityHelper.js is empty).

Case (a) — already correct. Verified, not changed.

setTestObservabilityFlags (testObservability/helper/helper.js:631) already resolves an explicit testObservability: false into BROWSERSTACK_TEST_OBSERVABILITY = "false", and getProductMap() reads exactly that. No code change was needed. Confirmed end-to-end (below) and now covered by a test so it cannot regress silently.

Case (b) — was broken, in the common path

The map is stamped twice, with two different meanings, and only the first was honest:

• on the build-start request (generateBuildUpstreamData) it is intent — what the run is asking for;
• on the session caps it is outcome — what is actually live.

extractDataFromResponse already clears the flag when a 2xx comes back carrying observability.success = false. But launchBuild's catch — a non-2xx or a network error, i.e. the entire build-start-failed population — only logged. The flag stayed true, so the session advertised observability with no build behind it.

That is not an edge case. Per the orphan-bucket analysis the collector rejects 7,442 × 400 + 442 × 403 build-starts per week, and every one of them landed here.

The change

1. handleErrorForObservability also clears BROWSERSTACK_TEST_OBSERVABILITY. One funnel, so all observability-failure paths agree — it already nulls the id family, and everything downstream reads isTestObservabilitySession() rather than those ids to decide whether observability is live.
2. launchBuild's catch routes through that funnel instead of only logging. Placed before the error.success === false branch so a plain network error is covered too, not just a non-2xx.
3. caps() no longer stamps the "null" sentinel as a uuid.

(3) is load-bearing precisely because of (2). Routing the catch through the funnel is what now causes BROWSERSTACK_TESTHUB_UUID to be set to the string "null" on this path — where before it was simply unset. Without (3) the run would stamp the literal "null" as a build uuid, which is worse than empty because a downstream REGEXP_EXTRACT would treat it as a real value. The three land together.

Incidental, one line: logBuildError treated only undefined as "no error object", so a null error logged the string "null" instead of "OBSERVABILITY Build creation failed". The accessibility no-response path hit this too.

Verification — the toggle, on production

Same config, same injected fault (a collector 400 on POST api/v2/builds, resolved at the transport with the real error shape {success:false, errors:[…]} so nodeRequest produces the exact rejection the catch sees):

	BEFORE (unfixed)	AFTER
caps buildProductMap.observability	true ❌	false ✅
BROWSERSTACK_TEST_OBSERVABILITY	"true"	"false"
BROWSERSTACK_TESTHUB_UUID	never set	"null" (sentinel, scrubbed from caps)
BS_TESTOPS_BUILD_COMPLETED	never set	"false"
caps testhubBuildUuid	""	""
user's tests	1 passed, exit 0	1 passed, exit 0

Run	Automate build	Session
Case (a), testObservability: false	0a5a64bd53c8abaec11fd24d0bf922a73380041f	— (see note)
Case (b), after	916337360d759acd17cf7066d965d9463cf22ceb	e435ffd0c34aea77ee2aeebe6650f1d2b3bca2c8
Case (b), before (baseline)	066f1b9fce74a5d2d69f4c985a22d392594511a2	8a1ecf09b0bbfb6ca3631833a8be9a9e8b69747f

Case (a) on the wire: buildProductMap.observability = false, testhubBuildUuid = "", build create 201.

Graceful degradation held — both failed-build-start runs finished exit 0 with the customer's test passing. The injected error message still surfaced to the user through logBuildError.

Unit tests — and they demonstrably detect the defect

12 new tests (test/unit/bin/testhub/utils.js is new). Full suite on the rebased tree: 723 passing / 16 failing, those 16 being the same pre-existing master failures.

More usefully, 4 of the new tests fail against the unfixed source and pass against the fixed one — so they are defect detectors, not decoration:

1) turns observability off in the product map when build start fails
2) does not report observability as still enabled to shouldProcessEventForTesthub
3) treats a null error the same as a missing one
4) flips observability to false and drops the null sentinel when build start failed

The case-(a) test passes both before and after — deliberately, since that behaviour already worked and the test exists purely as a regression guard.

One thing for the reviewer to weigh

Clearing BROWSERSTACK_TEST_OBSERVABILITY on a failed build start also affects other readers of isTestObservabilitySession(), all of which I believe become more correct: printBuildLink already suppresses the link when BS_TESTOPS_BUILD_HASHED_ID == "null", stopBuildUpstream already gates on BS_TESTOPS_BUILD_COMPLETED === "true", and getLocalSessionReporter additionally requires BS_TESTOPS_JWT. The one visible difference is that utils.js:1281 now takes its else branch (process.exit(exitCode)) instead of calling printBuildLink — same terminal behaviour, since that function's only remaining act on this path was to exit with the same code. Worth a second pair of eyes.

Also noticed, not fixed here

Disabling observability can break a project whose cypress.config.js imports the BrowserStack plugin: browserstack-cypress-cli is added to npm_dependencies only as a side effect of observability succeeding (setTestObservabilityVariables → setBrowserstackCypressCliDependency). Case (a) hit this — 204 No specs ran — until the dependency was declared explicitly. Pre-existing, out of scope, deserves its own ticket.