Two pieces: instrumenting the onboarding path so we can see where people fall out of it, and splitting new signups into the new path and the old one so the comparison means something.
Worth knowing
The before-number is contested, and that's worth knowing up front rather than discovering it in a review. There are at least three figures in circulation and none of them is verified: roughly 17% of signups opening an instance editor within 14 days, product#557's ~22.3% activation to first instance, and product#558's ~5% building a first application. Those two issues exist precisely because nobody trusts the numbers. So whatever we measure here, expect the baseline to be argued about, and it's worth agreeing which number we're moving before we claim to have moved it.
The cohort split has a constraint that shapes it. The decision about whether to auto-provision used to happen at email verification, server-side, when no browser is running. And forge/product/index.js only decorates capture, so there's no server-side PostHog flag evaluation available even though the library supports it. That's why the split works by turning auto-provisioning off for everyone on the flagged deployment and having the control cohort call the on-demand provisioning endpoint on arrival instead. Assignment stays entirely in the browser and forge needs no new capability.
posthogFlags already exists on the account-settings store, so flags are surfaced to the frontend today.
Two pieces: instrumenting the onboarding path so we can see where people fall out of it, and splitting new signups into the new path and the old one so the comparison means something.
Worth knowing
The before-number is contested, and that's worth knowing up front rather than discovering it in a review. There are at least three figures in circulation and none of them is verified: roughly 17% of signups opening an instance editor within 14 days, product#557's ~22.3% activation to first instance, and product#558's ~5% building a first application. Those two issues exist precisely because nobody trusts the numbers. So whatever we measure here, expect the baseline to be argued about, and it's worth agreeing which number we're moving before we claim to have moved it.
The cohort split has a constraint that shapes it. The decision about whether to auto-provision used to happen at email verification, server-side, when no browser is running. And
forge/product/index.jsonly decoratescapture, so there's no server-side PostHog flag evaluation available even though the library supports it. That's why the split works by turning auto-provisioning off for everyone on the flagged deployment and having the control cohort call the on-demand provisioning endpoint on arrival instead. Assignment stays entirely in the browser and forge needs no new capability.posthogFlagsalready exists on theaccount-settingsstore, so flags are surfaced to the frontend today.