Skip to content

fix(lambda-nodejs): run beforeInstall hook after workspace files are written - #37899

Open
Zelys-DFKH wants to merge 6 commits into
aws:mainfrom
Zelys-DFKH:fix-37898
Open

fix(lambda-nodejs): run beforeInstall hook after workspace files are written#37899
Zelys-DFKH wants to merge 6 commits into
aws:mainfrom
Zelys-DFKH:fix-37898

Conversation

@Zelys-DFKH

@Zelys-DFKH Zelys-DFKH commented May 17, 2026

Copy link
Copy Markdown
Contributor

Fixes #37898.

Credit to @cmodijk: the issue report traces the execution order exactly, identifies the root cause, and spells out what the fix should look like. This PR implements what you described.


The problem

On pnpm v11, strictDepBuilds is the effective default. Any NodejsFunction that bundles nodeModules with a native-build dependency (e.g. ssh2-sftp-clientcpu-features) hits this at deploy time:

ERR_PNPM_INSTALL_SCRIPTS_NOT_ALLOWED  cpu-features@1.x.x is not allowed to run install scripts

The beforeInstall hook was designed to handle this: append allowBuilds entries to pnpm-workspace.yaml before the install runs. It silently doesn't work, which is worse than not existing.

Why

CDK writes an empty pnpm-workspace.yaml to prevent pnpm from walking up to a monorepo root (introduced in #21910, still the right call). The bug was ordering. Before this fix:

beforeBundling → esbuild → [beforeInstall] → write pnpm-workspace.yaml + install → afterBundling
                                ↑ hook fires here        ↑ empty yaml written here

Whatever the user wrote in beforeInstall got overwritten a step later. The hook's name is a lie.

Fix

Splits node-modules file-ops into two phases via a new NodeModuleFileOps interface:

  • prepareSteps: writes pnpm-workspace.yaml, package.json, and the lockfile
  • installSteps: runs the package manager and post-install cleanup

createBundlingSteps puts beforeInstall between them, for both Docker and local bundling.

beforeBundling → esbuild → write pnpm-workspace.yaml → [beforeInstall] → install → afterBundling
                                        ↑ files written here    ↑ hook fires here

No behavior change for callers where beforeInstall returns [] (the default).

The refactor makes createBundlingSteps slightly more involved. dockerFileOps and localFileOps are both private, so there's no API surface change.

Validation

Unit test (bundling.test.ts): The beforeInstall hook fires after pnpm workspace files are written (Docker) test asserts the Docker shell command orders pnpm-workspace.yaml write → beforeInstallpnpm install. This is the test that would have caught the bug.

E2E behavioral test (bundling-package-managers-e2e.test.ts): The pnpm-specific / beforeInstall writes to pnpm-workspace.yaml survive CDK workspace setup test runs CDK synthesis with a minimal pnpm project (local and Docker where available), writes allowBuilds: to pnpm-workspace.yaml in beforeInstall, reads the output file, and checks the content survived. Skipped when pnpm is unavailable locally and Docker bundling is off.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions Bot added bug This issue is a bug. p2 beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK labels May 17, 2026

@aws-cdk-automation aws-cdk-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@aws-cdk-automation

Copy link
Copy Markdown
Collaborator

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

@aws-cdk-automation aws-cdk-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter fails with the following errors:

❌ Fixes must contain a change to an integration test file and the resulting snapshot.

If you believe this pull request should receive an exemption, please comment and provide a justification. A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed, add Clarification Request to a comment.

✅ A exemption request has been requested. Please wait for a maintainer's review.

@Zelys-DFKH

Copy link
Copy Markdown
Contributor Author

Exemption Request

This fix reorders internal bundling steps: it splits fileOps() into prepareSteps / installSteps so beforeInstall fires after workspace files are written. Synthesis produces an identical CloudFormation template — no L1/L2 resource changes. Snapshots capture the output template, not hook execution order; they wouldn't show whether the hook ran before or after the package.json write. The existing unit tests (test_bundling.ts) cover this.

@aws-cdk-automation aws-cdk-automation added the pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. label Jun 4, 2026
Zelys-DFKH and others added 3 commits July 16, 2026 12:19
…written

Splits the node-modules file-ops step into two phases — prepareSteps (writes
pnpm-workspace.yaml, package.json, and the lockfile) and installSteps (runs
the package manager and cleanup) — so that the beforeInstall command hook fires
between them. Previously, beforeInstall ran before depsCommand, which meant CDK's
empty pnpm-workspace.yaml write always silently overrode any allowBuilds entries a
user appended there. This made the hook useless for pnpm v11 workspaces that need
allowBuilds to permit native build scripts (e.g. cpu-features via ssh2-sftp-client).

Fixes aws#37898.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rdering

Adds two layers of test coverage for the ordering fix from the previous commit:

1. E2e behavioral test in bundling-e2e.test.ts — asserts that allowBuilds
   written to pnpm-workspace.yaml by beforeInstall survives CDK's own workspace
   file setup. Runs under both local (skipped if pnpm absent) and Docker
   bundling. Closes the test gap that allowed the bug to exist.

2. CDK integration test (integ.dependencies-pnpm-before-install.ts) — deploys
   a Lambda that uses delay@5.0.0 with a beforeInstall hook writing to
   pnpm-workspace.yaml, exercising the full bundling pipeline. Snapshot hashes
   are placeholder (all-zeros); run `yarn integ` in framework-integ to
   regenerate from the Docker bundled output.

Fixes aws#37898

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The integ test required Docker + AWS credentials to generate real asset
hashes, blocking CI. The unit test in bundling.test.ts directly asserts
the ordering fix (pnpm-workspace.yaml write → beforeInstall → pnpm install)
and the e2e test in bundling-package-managers-e2e.test.ts validates the
behavioral fix end-to-end without a live deploy.
…ing test

Docker bundling always builds commands with osPlatform 'linux', so
OsCommand.write escapes the output path via posixShellEscape and emits
single quotes. The assertion matched the win32 double-quoted form, which
that code path never produces, so it failed regardless of hook ordering.

The behaviour under test is unchanged: the workspace file is written,
then the beforeInstall hook runs, then pnpm install.
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 7, 2026
@Zelys-DFKH

Copy link
Copy Markdown
Contributor Author

Thanks for writing this up, @marco2216. The trustLockfile detail is a useful one.

Your case is the same write this PR reorders, just surfacing as slowness rather than a failure. The blank pnpm-workspace.yaml drops your settings, so pnpm re-verifies the whole copied lockfile with live registry lookups every time the bundle cache misses. Nothing errors. You just wait.

It also sharpens the pending exemption question, since a template snapshot can't see your symptom either. Synthesis output is byte-identical either way. The only thing that changes is when the hook runs relative to the file write.

Three reports now (@cmodijk, @pivstone, @marco2216), three symptoms, one ordering bug. Branch is updated against main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK bug This issue is a bug. effort/medium Medium work item – several days of effort p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

(lambda-nodejs): nodeModules + pnpm v11 empty pnpm-workspace.yaml blocks native module builds, beforeInstall cannot override it

4 participants