Skip to content

Fix AppKit showcase app templates (route 404s, lockfile, SP seeding, serving, analytics)#237

Merged
pkosiec merged 20 commits into
databricks:mainfrom
pkosiec:pkosiec/update-app-templates
Jun 10, 2026
Merged

Fix AppKit showcase app templates (route 404s, lockfile, SP seeding, serving, analytics)#237
pkosiec merged 20 commits into
databricks:mainfrom
pkosiec:pkosiec/update-app-templates

Conversation

@pkosiec

@pkosiec pkosiec commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

Fixes for the Node.js / AppKit showcase app templates so they build, deploy, and serve data. Companion to universe PR databricks-eng/universe#2053887 (adds the four showcase apps — RAG Chat, SaaS Tracker, Content Moderator, Vacation Rentals — to the App Builder catalog).

Changes

  • appkit → 0.40.0 across all 6 showcase templates + current server API (onPluginsReady, no manual server.start()): fixes the /api/* 404-on-deploy race. Five templates jump from 0.21.0; vacation-rentals from 0.23.0. Pinned to 0.40.0 (not 0.41.x) because the Apps build container's npm proxy has a ~1-day date cutoff and 0.41.x (published 2026-06-09) isn't installable there yet; 0.40.0 already has the non-blocking typegen we need.
  • Drop the package-lock.json bundle-exclude; all lockfiles pinned to the public npm registry (no internal-proxy hosts) so the Apps build container installs cleanly.
  • SP self-seed on startup (saas-tracker, content-moderator): demo data is created and owned by the app service principal; removes the human seed/ scripts.
  • content-moderator serving: app.yaml uses the FM-API literal databricks-gpt-5-4-mini + runtime SP token via the SDK (forcing an https:// scheme on the bare DATABRICKS_HOST); the redundant serving-endpoint bundle resource is removed (mirrors rag-chat — FM pay-per-token needs no per-endpoint grant).
  • rag-chat gateway URL: derive the AI Gateway domain from DATABRICKS_HOST instead of hardcoding the prod suffix, so chat resolves on non-prod workspaces (the hardcoded ai-gateway.cloud.databricks.com is NXDOMAIN on staging).
  • Analytics templates adopt the appkit- family typegen layout* (all 5 with analytics): prebuild runs generate-types --wait, tsconfig.client includes shared/appkit-types, generated types are gitignored, and the stale committed client/src/appKitTypes.d.ts is removed (committed types are the pre-0.40 pattern). Analytics queries use <catalog>.<schema>.<table> (dropped the nonexistent app_data segment); the catalog name is chosen by the operator when registering the Lakebase DB in UC (documented in the READMEs).
  • READMEs: databricks apps init URLs → databricks/app-templates; self-seed + analytics catalog-registration/SP-grant runbook.

Verification

Deploy-verified on e2-dogfood (all 4 App Builder apps):

  • rag-chat/api/chats → 200; RAG retrieval returns seeded Wikipedia context and the chat streams a model response (gateway fix confirmed).
  • vacation-rentals — build-time typegen --wait resolves against samples.wanderbricks; /api/bookings/* → 200; full 0.23→0.40 upgrade.
  • saas-tracker — build typegen resolves against the registered Lakebase UC catalog; /api/subscriptions → 200 with seeded rows.
  • content-moderator/api/submissions → 200; submitting content returns an AI compliance score from databricks-gpt-5-4-mini (confirmed working with the serving resource removed).

Note: with committed types removed, accurate analytics types are generated by typegen during install/build (the appkit-* family model), so building requires a reachable warehouse + the registered catalog rather than shipping stale types.

agentic-support-console + inventory-intelligence are not in the App Builder catalog (they require Lakehouse Sync + medallion pipelines) and are not deploy-verified. They received the same appkit-0.40 + typegen-layout changes; inventory-intelligence has no analytics queries, while agentic-support-console builds once its catalog is set and its pipelines have produced the gold views (consistent with its other REPLACE_ME placeholders).

This pull request and its description were written by Isaac.

pkosiec added 7 commits June 9, 2026 17:34
The macOS-generated package-lock.json pins host-specific/prerelease native
binaries and its `resolved` URLs point to the internal npm proxy
(npm-proxy.cloud.databricks.com), which the Databricks Apps Linux build
container cannot reach — so the deployed `npm install` crashes
("Exit handler never called!" / ENOTEMPTY). Excluding the lockfile from the
bundle (it stays in-repo for local `npm ci`) lets the build container do a
clean public-registry install. Mirrors rag-chat / inventory-intelligence,
which already do this.

Co-authored-by: Isaac
On @databricks/appkit@0.21.0 the showcase templates start the server with
`await createApp({ plugins:[server()], onPluginsReady })`, which races the
listener / SPA 404 catch-all so `appkit.server.extend(...)` API routes are
never mounted — every `/api/*` returns 404 on deploy.

Upgrade @databricks/appkit + @databricks/appkit-ui to 0.38.1 (and
@databricks/sdk-experimental to 0.17.0) to match the appkit-* template
family, and adopt the 0.38.1 server API used by appkit-all-in-one: no
top-level await, no manual server.start() — the runtime awaits
onPluginsReady (where routes register) before it listens.

rag-chat additionally fires its slow Wikipedia seed without awaiting so it
can't delay route registration or the deploy health-check.

Templates: rag-chat, saas-tracker, content-moderator, agentic-support-console,
inventory-intelligence. (agentic-support-console and inventory-intelligence
share the same 0.21.0 pattern but were not deploy-verified.)

Also:
- appkit.plugins.json regenerated to schema v2.0 by the 0.38.1 toolchain
  (adds available, non-required plugins to the catalog; requiredByTemplate
  set unchanged).
- package-lock.json files regenerated and pinned to the public npm registry
  (registry.npmjs.org), not the internal proxy.
- content-moderator: remove an unused `Badge` import that failed the strict
  (noUnusedLocals) client build.

All five build clean (client + server) on 0.38.1.

Co-authored-by: Isaac
…tor)

The README had a human run `npm run seed` with their own DATABASE_URL, which
created the Postgres schema (saas_tracker / content_moderation) owned by that
human. The deployed app service principal (CAN_CONNECT_AND_CREATE) cannot
operate on a schema it doesn't own, so its startup CREATE TABLE / queries
failed and the OLTP/CRUD path broke.

Move the demo data into server-side lib/seed-data.ts and seed it (idempotently,
only when the table doesn't yet exist) inside the existing SP-driven startup
setup via appkit.lakebase.query — so the SP creates and owns the schema,
tables, and rows from first boot. Delete the standalone human seed/ scripts.

For these small SQL seeds the insert runs inside onPluginsReady (before the
server listens), so data is present on the first request.

Also remove an unused SUBMISSION_STATUSES constant in content-moderator that
failed `npm run typecheck` (the server build uses tsdown, which doesn't
type-check, so it was latent).

Co-authored-by: Isaac
Two fixes so AI compliance scoring actually runs:

1. app.yaml: SERVING_ENDPOINT was wired via `valueFrom: serving-endpoint`, but
   appkit.plugins.json declares no serving plugin, so the App Builder (which
   deletes databricks.yml and resolves only app.yaml valueFrom entries that map
   to declared plugins) left SERVING_ENDPOINT undefined and scoring silently
   no-op'd. Use a Foundation Model API literal (databricks-gpt-5-4-mini), like
   rag-chat, instead of adding a serving plugin (which would force the universe
   App Builder catalog to declare a serving-endpoint resource).

2. moderation-routes.ts: the analysis call read only DATABRICKS_TOKEN /
   DATABRICKS_API_TOKEN, which a deployed App never receives (it gets app-SP
   OAuth via DATABRICKS_CLIENT_ID/SECRET). Resolve a bearer token at runtime
   through the SDK Config (mirrors rag-chat's getDatabricksToken), keeping the
   app-SP flow with no OBO.

Co-authored-by: Isaac
The analytics .sql files referenced `<schema>.app_data.<table>`
(saas_tracker.app_data.subscriptions; content_moderation.app_data.*), but the
app/seed only ever creates the `<schema>` (saas_tracker / content_moderation);
no `app_data` schema exists, and `databricks apps init` does not template the
name, so the warehouse query failed to resolve.

When the Lakebase database is registered in Unity Catalog
(`databricks postgres create-catalog`), its data is reachable from the SQL
Warehouse as `<catalog>.<pg_schema>.<table>`. Use that form with the documented
convention of naming the catalog after the schema:
  saas_tracker.saas_tracker.subscriptions
  content_moderation.content_moderation.{submissions,ai_analyses}

The READMEs document the matching `create-catalog` name and the app-SP grants
that build-time typegen (DESCRIBE) needs.

Co-authored-by: Isaac
- Point `databricks apps init --template` at github.com/databricks/app-templates
  (not the old devhub URLs) in agentic-support-console, content-moderator,
  rag-chat, and saas-tracker — that's where these templates now live and how the
  App Builder catalog consumes them.
- content-moderator / saas-tracker: drop the manual `npm run seed` step (the app
  now self-seeds on startup as the service principal) and document the optional
  analytics path: register the Lakebase DB as a UC catalog (named after the
  schema) and grant the app SP USE CATALOG / USE SCHEMA / SELECT so build-time
  type generation and dashboard queries resolve.
- content-moderator: note AI scoring uses the built-in Foundation Model API
  endpoint via the app SP, no extra setup.

Co-authored-by: Isaac
Every template's committed package-lock.json uses the public npm registry
(registry.npmjs.org) with no internal-proxy (npm-proxy.cloud.databricks.com)
references, so the Databricks Apps build container can `npm ci` from it
directly — the bundle-exclude is unnecessary. Remove it from all app templates:
the appkit-* family plus the showcase templates, including the pre-existing
upstream exclude on rag-chat and inventory-intelligence (whose lockfiles are now
de-proxied to the public registry).

Caveat: package-lock.json is not gitignored, so `databricks bundle deploy` syncs
the working-tree copy. Running `npm install` on a machine with the internal npm
proxy enforced rewrites resolved URLs to the proxy; deploy from a clean working
tree (or a non-proxy environment / CI) to keep the synced lockfile clean.

Co-authored-by: Isaac
@pkosiec
pkosiec force-pushed the pkosiec/update-app-templates branch from 0ce1f76 to 952d7b9 Compare June 9, 2026 16:33
pkosiec added 13 commits June 9, 2026 19:28
An earlier pass only rewrote npm-proxy.cloud.databricks.com; these lockfiles
also pinned a second internal host, npm-proxy.dev.databricks.com (rag-chat 924,
inventory-intelligence 647, and the agentic-support-console / inventory seed
lockfiles 48 each). Rewrite all of them to registry.npmjs.org so every
resolved URL across the repo uses the public registry — the Apps build
container can install from a clean lockfile, and no internal-proxy host remains.

Co-authored-by: Isaac
…plates)

Build-time `appkit generate-types` (postinstall/prebuild) DESCRIBEs the
analytics tables against the warehouse and, on 0.38.1, hard-fails the build if
a table is missing. Combined with SP-driven self-seeding (tables created at app
startup, not before deploy), the first deploy of content-moderator / saas-tracker
failed at build with TABLE_OR_VIEW_NOT_FOUND.

appkit 0.40.0 made typegen non-blocking (databricks/appkit#406): it reuses
committed/cached types and falls back to `result: unknown` instead of failing
the build on warehouse/missing-table errors. Bump to 0.41.1 (latest;
no breaking changes from 0.38.1 per changelog — the manifest-format break was
0.38.0, already adopted). The committed appKitTypes.d.ts stays authoritative.

Templates: rag-chat, saas-tracker, content-moderator, agentic-support-console,
inventory-intelligence (appkit + appkit-ui -> 0.41.1; sdk-experimental stays
0.17.0, which 0.41.1 depends on). All five build + typecheck clean locally.

- appkit.plugins.json: 0.41.1 `plugin sync` adds one guidance line (schema v2.0
  unchanged; requiredByTemplate set unchanged).
- package-lock.json (rag-chat, agentic-support-console, inventory-intelligence):
  regenerated at 0.41.1 and pinned to the public npm registry (no proxy hosts).

Note: deploy validation on e2-dogfood is pending (Apps Workspace snapshot-import
infra was failing intermittently at the time); to be validated separately.

Co-authored-by: Isaac
Drop multi-line rationale comments added in this branch that are inconsistent
with these otherwise comment-free files (seed-data headers, getDatabricksToken,
the self-seed "first boot" notes, and the rag-chat background-seed note). The
code (`void ...catch`, the SP-owned setup, the SDK token resolution) is
self-explanatory and matches the reference patterns, which carry no such
comments.

Co-authored-by: Isaac
Fourth App Builder catalog app (universe #2053887). Bumps appkit and
appkit-ui 0.23.0 -> 0.41.1 and sdk-experimental 0.14.2 -> 0.17.0, and
adopts the onPluginsReady server API (drops the manual
server({autoStart:false}) + appkit.server.start() pattern that races
route registration). Regenerates the lockfile (de-proxied) and the
appkit.plugins.json manifest (v1.0 -> v2.0).

Co-authored-by: Isaac
Fixes prettier drift introduced earlier in the branch (line wrapping in
rag-chat seed .catch and the content-moderator seedModerationData
signature). Formatting only.

Co-authored-by: Isaac
The Databricks Apps build container's npm proxy applies a ~1-day date
cutoff, and appkit 0.41.0/0.41.1 (published 2026-06-09) are too new to
install there yet -- in-container `npm install` fails with
ETARGET "No matching version ... with a date before 6/9/2026".

0.40.0 (2026-06-05) is the newest proxy-available release and already
contains the non-blocking typegen fix (the reason for going past
0.38.1), plus the onPluginsReady server API. Re-pins all 6 showcase
templates 0.41.1 -> 0.40.0, regenerates + de-proxies the lockfiles, and
regenerates the plugin manifests.

Co-authored-by: Isaac
…erator

Align the two Lakebase-analytics showcase templates with the appkit-* family
and appkit 0.40's typegen model:
- prebuild runs `generate-types --wait` (blocking) so the Apps build container
  generates real analytics types instead of degrading to empty `{}` (which
  fails the client `tsc` build under 0.40's non-blocking default).
- tsconfig.client includes `shared/appkit-types` (the current generated-types
  location); generated files are gitignored.
- Remove the stale committed client/src/appKitTypes.d.ts (the family commits
  none; types are generated at build).

Requires the Lakebase DB to be registered as a UC catalog and the app SP
granted USE CATALOG/SCHEMA + SELECT (documented Bug 5 analytics setup).

Co-authored-by: Isaac
In the Apps runtime DATABRICKS_HOST is the bare hostname (no scheme), so
`${host}/serving-endpoints/.../invocations` failed with "Failed to parse URL"
and AI compliance scoring silently errored. Prepend https:// when the host
lacks a scheme.

Co-authored-by: Isaac
The chat completion call hardcoded the production gateway domain
(ai-gateway.cloud.databricks.com), which doesn't resolve on non-prod
workspaces (e.g. e2-dogfood staging -> NXDOMAIN), so chat responses failed
with getaddrinfo ENOTFOUND even though the API routes served fine. Derive
the gateway domain from the workspace host (drop its first label) so it
resolves on staging and prod alike; falls back to the prod suffix when
DATABRICKS_HOST is unset. (Embeddings already use the SDK workspace client,
so they were unaffected.)

Co-authored-by: Isaac
app.yaml uses the FM-API literal databricks-gpt-5-4-mini and the analysis
call mints the app-SP token at runtime via the SDK -- the same pattern as
rag-chat, which declares no serving resource. The bundle's serving-endpoint
resource (defaulting to a different endpoint than the app actually calls)
was redundant and inconsistent, so remove it along with its variable.

Co-authored-by: Isaac
…lates

Align vacation-rentals, agentic-support-console, and inventory-intelligence
with appkit 0.40's typegen model (matching saas-tracker/content-moderator):
prebuild runs `generate-types --wait`, tsconfig.client includes
`shared/appkit-types`, generated types are gitignored, and the stale committed
client/src/appKitTypes.d.ts is removed (committed types are the pre-0.40
pattern).

- vacation-rentals queries samples.wanderbricks (always present) -> builds.
- inventory-intelligence has no analytics queries -> empty registry, builds.
- agentic-support-console queries REPLACE_ME.gold.* (pipeline views); like the
  rest of its REPLACE_ME placeholders, it builds once the operator sets the
  catalog and runs the medallion pipelines.

Co-authored-by: Isaac
The analytics .sql files hardcode <catalog>.<schema>.<table>, and build-time
typegen (--wait) DESCRIBEs them as the app SP. Make explicit that the operator
must register the catalog under the exact name the queries use (saas_tracker /
content_moderation) and grant the SP before the next build, or edit the .sql --
otherwise typegen degrades to empty types and the client build fails.

Co-authored-by: Isaac
Code-correctness and security fixes surfaced by a multi-agent review of the
showcase app templates (validated with Context7 for Spark/Vercel AI SDK/SDK
usage). Scoped to showcase-app code only; agent-template and repo-tooling
findings intentionally excluded.

agentic-support-console:
- support-routes: stop bypassing the admin_decisions SERIAL PK with a
  MAX(id)+1 subquery (race / PK-collision); let the sequence assign the id.
- support-routes: commit the decision + reply-message + case-status writes
  atomically via a single data-modifying CTE (appkit.lakebase exposes only
  query(), no transaction client).
- support-routes: PATCH /api/cases/:id/status now returns 404 when the case
  does not exist (RETURNING id) instead of a silent 200.
- generate_responses: parameterize the user_id Spark SQL (spark.sql args=)
  to remove the f-string SQL-injection vector.

content-moderator:
- moderation-routes: stamp the audit columns (created_by / updated_by /
  reviewer_email) from the gateway-trusted x-forwarded-email header, falling
  back to the request body only for local dev (forgeable audit trail).
- moderation-routes: commit the review insert + submission-status update
  atomically via a single data-modifying CTE.
- moderation-routes: fence user-submitted title/body with explicit
  untrusted-content markers to harden the moderation prompt against injection.
- moderation-routes: getDatabricksToken now calls config.ensureResolved()
  before authenticate(), matching rag-chat (fixes silent null token locally).

rag-chat:
- chat-routes: gatewayBaseUrl throws an explicit error when
  DATABRICKS_WORKSPACE_ID is unset instead of building an https://undefined.…
  gateway URL that fails as a generic ENOTFOUND.

Note: the data-modifying CTE rewrites preserve response shapes and typecheck
cleanly, but were not executed against a live Lakebase — verify the decision
and review flows end-to-end before relying on them.

Co-authored-by: Isaac
@pkosiec
pkosiec marked this pull request as ready for review June 10, 2026 12:48
@pkosiec
pkosiec merged commit 2302cd3 into databricks:main Jun 10, 2026
@pkosiec
pkosiec deleted the pkosiec/update-app-templates branch June 10, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants