Skip to content

Eliminate Node Renderer cold-start latency by pre-seeding bundle cache in Docker builds #3122

@justin808

Description

@justin808

Problem

When a new deployment starts, the Node Renderer has an empty bundle cache. The first SSR request triggers a costly round-trip:

  1. Rails sends render request without bundle
  2. Renderer returns 410 "No bundle uploaded"
  3. Rails retries with the full bundle attached via multipart POST
  4. Renderer receives, caches, and builds VM context
  5. Renderer finally returns the rendered HTML

This 410→retry flow adds 200ms–1s+ of latency to the first SSR request per deployment, depending on bundle size and network conditions. In high-traffic environments, this is a significant problem:

  • During rolling deploys, new pods/containers serve their first request slowly
  • Load balancers may route hundreds of concurrent requests to a new instance before the bundle is cached, causing a thundering herd of 410→retry cycles
  • The per-bundle lock in the renderer serializes these uploads, meaning only one request uploads while others queue behind the lock
  • Users experience a visible latency spike on every deployment

Current bundle lifecycle

Webpack Build                    Node Renderer
─────────────                    ─────────────
server-bundle.js                 (empty cache)
  written to                           │
  public/packs/                        │
       │                               │
       ▼                               │
Rails App (runtime)                    │
  1st SSR request ────────────────►  410 "No bundle"
  retry with bundle ──────────────►  cache + build VM
  subsequent requests ────────────►  200 (fast)

Proposed solution: Pre-seed the renderer cache during Docker build

Core idea

During the Docker image build, copy the compiled server bundle directly into the renderer's serverBundleCachePath directory in the correct structure (<bundleHash>/<bundleHash>.js). When the container starts, the renderer finds the bundle already cached — zero cold-start latency.

Implementation: two-layer cache seeding

Layer 1: Pre-seed the NEW bundle (eliminates 410 on current deployment)

After webpack compiles the server bundle, compute its hash and copy it into the renderer cache:

# After webpack build step
ARG RENDERER_CACHE_DIR=/app/.node-renderer-bundles

# Copy the compiled bundle into the renderer's cache structure
RUN BUNDLE_HASH=$(node -e "
  const crypto = require('crypto');
  const fs = require('fs');
  const content = fs.readFileSync('public/packs/server-bundle.js');
  console.log(crypto.createHash('md5').update(content).digest('hex'));
") && \
  mkdir -p ${RENDERER_CACHE_DIR}/${BUNDLE_HASH} && \
  cp public/packs/server-bundle.js ${RENDERER_CACHE_DIR}/${BUNDLE_HASH}/${BUNDLE_HASH}.js

Layer 2: Pre-seed the PREVIOUS bundle (eliminates 410 during rolling deploys)

During a rolling deploy, old Rails instances still reference the previous bundle hash. If the new renderer doesn't have the old bundle cached, those requests also hit the 410→retry flow. To prevent this:

# Fetch the previous deployment's bundle hash from the running deployment
# (e.g., from a known config endpoint, an artifact store, or a build arg)
ARG PREV_BUNDLE_HASH=""
ARG PREV_BUNDLE_URL=""

RUN if [ -n "${PREV_BUNDLE_HASH}" ] && [ -n "${PREV_BUNDLE_URL}" ]; then \
  mkdir -p ${RENDERER_CACHE_DIR}/${PREV_BUNDLE_HASH} && \
  curl -sS -o ${RENDERER_CACHE_DIR}/${PREV_BUNDLE_HASH}/${PREV_BUNDLE_HASH}.js \
    "${PREV_BUNDLE_URL}"; \
fi

Where to get the previous bundle

Several approaches, depending on infrastructure:

  1. Artifact store (recommended): After each successful deploy, upload the server bundle to S3/GCS/etc. keyed by its hash. During the next build, fetch the current production hash from the running deployment and pull the artifact. This is the most reliable approach.

  2. Docker layer caching: If the renderer cache directory is on a persistent volume, the previous bundle survives container restarts. But this doesn't help with fresh node replacements.

  3. Control Plane image registry: If using Control Plane (cpln), the previous deployment's image is still available. A multi-stage build can COPY --from=<previous-image> the cache directory.

  4. Git-tracked manifest: Store the current production bundle hash in a known file (e.g., .bundle-manifest.json). The CI pipeline reads this to determine which previous bundle to fetch.

Integration with existing rake tasks

The existing pre_stage_bundle_for_node_renderer rake task already handles the webpack→cache symlink for local/CI use. For Docker builds, we need a similar mechanism that:

  1. Computes the bundle hash the same way Rails does (via ReactOnRailsPro::ServerRenderingPool::NodeRenderingPool.renderer_bundle_file_name)
  2. Copies (not symlinks) the bundle into the cache directory
  3. Optionally fetches and copies the previous bundle

A new rake task or enhancement to the existing one could handle this:

# Proposed: rake react_on_rails_pro:pre_seed_renderer_cache
namespace :react_on_rails_pro do
  desc "Pre-seed renderer cache for Docker builds (copy, not symlink)"
  task pre_seed_renderer_cache: :environment do
    src = ReactOnRails::Utils.server_bundle_js_file_path
    bundle_name = ReactOnRailsPro::ServerRenderingPool::NodeRenderingPool.renderer_bundle_file_name
    cache_dir = ENV["RENDERER_SERVER_BUNDLE_CACHE_PATH"] || Rails.root.join(".node-renderer-bundles").to_s

    dest_dir = File.join(cache_dir, bundle_name.to_s)
    FileUtils.mkdir_p(dest_dir)
    FileUtils.cp(src, File.join(dest_dir, "#{bundle_name}.js"))
    puts "[ReactOnRailsPro] Pre-seeded renderer cache: #{dest_dir}"

    # Copy assets
    ReactOnRailsPro.configuration.assets_to_copy&.each do |asset_path|
      next unless File.exist?(asset_path)
      FileUtils.cp(asset_path, File.join(dest_dir, asset_path.basename.to_s))
    end
  end
end

Assets must also be pre-seeded

The renderer cache stores not just the bundle but also associated assets (e.g., loadable-stats.json, manifest.json). These must be copied alongside the bundle in the same <hash>/ directory. The rake task above handles this via assets_to_copy.

Impact

Scenario Before After
First request on fresh deploy 410→retry: 200ms–1s+ Direct render: <50ms
Rolling deploy (old Rails → new renderer) 410→retry per old instance Direct render (previous bundle pre-seeded)
Thundering herd on new pod N requests queue behind per-bundle lock All requests served immediately
Bundle size: 5MB ~500ms upload per 410 retry 0ms (already cached)
Bundle size: 20MB ~2s upload per 410 retry 0ms (already cached)

Deprecation: RENDERER_BUNDLE_PATH env var

Related: RENDERER_BUNDLE_PATH is a deprecated alias for serverBundleCachePath (the cache directory). It does NOT point to the webpack output directory. The env var name is misleading and should be formally deprecated with a clear migration path to RENDERER_SERVER_BUNDLE_CACHE_PATH. Updated docs guidance in PR #3087.

Related

  • PR docs: RSC integration pitfalls from tutorial app #3087 — docs: RSC integration pitfalls (corrected CI snippet to remove RENDERER_BUNDLE_PATH and use rake pre-staging)
  • configBuilder.ts:188-189serverBundleCachePath resolution chain
  • prepare_node_renderer_bundles.rb — existing symlink-based pre-staging for local/CI
  • node_rendering_pool.rb:73-76 — 410 retry logic
  • worker.ts — multipart upload handling and per-bundle locking

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions