Skip to content

Commit 5049679

Browse files
Adam DomuradAdam Domurad
authored andcommitted
feat(labs): one command, one page — scripts/labs, REPO.md and the labs-workflow skill
The labs workflow read as a sequence of scripts to remember; this collapses it: - `scripts/labs <verb>`: apply, status, export, check, bump <ref> [--commit|--pr], upstream <n>.. [--push], port <pr> [--push], run <cmd>, test. - `apply` marks the labs gitlink skip-worktree, so git status, `git add -u` and `git commit -a` leave the permanently-ahead submodule alone and a bare `git add labs` is a no-op; the pin moves only with bump. The pre-commit guard reads the index with --ignore-submodules=none so a deliberately staged patch commit is still refused. - `apply` installs a post-commit hook in labs/ that re-exports the series, so the .patch files cannot lag the checkout (the marker commit and the tooling's own commits opt out). - `--push` on upstream/port runs the printed push and gh pr create; `bump --commit` records gitlink, foundry locks and series in one commit, `--pr` pushes it as labs-bump/<short> and opens the PR, listing the patches upstream absorbed. - REPO.md: the cross-repository workflow (foundation, private, labs) on one page; the `labs-workflow` skill points at it; CLAUDE.md and the README point at both. Lifecycle test covers the hidden gitlink, the auto-export hook and bump --commit.
1 parent 2909670 commit 5049679

8 files changed

Lines changed: 196 additions & 22 deletions

File tree

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
name: labs-workflow
3+
description: Work across the foundation (aztec-packages), aztec-packages-private and the labs repo (aztec-node) — changing labs from the foundation as patches, upstreaming or porting to aztec-node, bumping the labs pin, private/public prototyping and release testing. Use when a task touches labs/, labs-patches/, the submodule pin, an aztec-node PR, or the private repo.
4+
---
5+
6+
Read `REPO.md` at the repo root first; it is the whole workflow on one page. Everything goes through `scripts/labs <verb>`:
7+
8+
| Verb | Use |
9+
|---|---|
10+
| `apply` / `status` / `export` / `check` | series on the checkout; `status` lists unexported commits |
11+
| `upstream <n>... [--push]` | one aztec-node PR from the named patches (batch what belongs together) |
12+
| `port <pr> [--push]` | replay a labs-only aztec-packages PR onto aztec-node main |
13+
| `bump <ref> [--commit\|--pr]` | move the pin, re-apply, commit gitlink + foundry locks + patches, open the PR |
14+
| `run <cmd>` | a command inside labs/ with the right environment |
15+
16+
Rules that prevent real mistakes:
17+
- Never stage the `labs` gitlink by hand; the pin moves only with `bump`. The pre-commit hook refuses a gitlink that points at a patch or marker commit.
18+
- Commits inside `labs/` re-export the series; commit the regenerated `labs-patches/*.patch` in the foundation PR.
19+
- A PR that mixes foundation and labs files is not ported; split it.
20+
- Private prototypes with no private content go into the private repo's `public-next`, never `next`.

CLAUDE.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,11 +53,11 @@ Follow Conventional Commits: `fix:`, `feat:`, `chore:`, `refactor:`, `docs:`, `t
5353
</commits_and_prs>
5454

5555
<git_staging>
56-
When staging files, prefer `git add -u` or name specific files rather than `git add -A` or `git add .`. The aggregate flags will pick up unrelated untracked working directories (e.g. personal scratch projects at the repo root) and quietly stage them. `git add -u` and `git commit -a` also stage the `labs` gitlink whenever the patch series is applied (`git status` shows ` M labs`); unstage it with `git restore --staged labs` (see `<labs_submodule_patches>`).
56+
When staging files, prefer `git add -u` or name specific files rather than `git add -A` or `git add .`. The aggregate flags will pick up unrelated untracked working directories (e.g. personal scratch projects at the repo root) and quietly stage them. The `labs` gitlink is skip-worktree while the patch series is applied, so `git add -u`/`git commit -a` leave it alone (see `<labs_submodule_patches>`).
5757
</git_staging>
5858

5959
<labs_submodule_patches>
60-
`labs/` (the aztec-node submodule) carries the foundation's patch series from `labs-patches/*.patch`, applied with `git am` by `labs-patches/bootstrap.sh apply` (run by the root bootstrap, the git hooks and `make labs-patched`), so `labs/` HEAD normally sits ahead of the recorded gitlink. Never `git add labs` by hand — that records a patch commit that does not exist upstream; move the pin with `labs-patches/bootstrap.sh bump <ref>`. To change a patch, commit inside `labs/` on top of the applied series and run `labs-patches/bootstrap.sh export`, then commit the regenerated `.patch` files. To send patches to aztec-node, `labs-patches/bootstrap.sh upstream <n>...` prepares one branch with the named patches and prints the push/PR commands. To port a whole aztec-packages PR to aztec-node (a v5 PR in the in-tree layout, or a `next` PR that only changes `labs-patches/*.patch`), `scripts/labs_port_pr.sh <pr>` checks the PR touches nothing but the labs half, replays it 3-way onto aztec-node `main` in a `port/pr-<n>` branch of `labs/`'s repository, and prints the push/PR commands; mixed PRs are refused, not split. See `labs-patches/README.md`.
60+
The cross-repository workflow (foundation ↔ labs ↔ private) is on one page in `REPO.md`; `scripts/labs <verb>` drives it. `labs/` (the aztec-node submodule) carries the foundation's patch series from `labs-patches/*.patch`, applied with `git am` by `labs-patches/bootstrap.sh apply` (run by the root bootstrap, the git hooks and `make labs-patched`), so `labs/` HEAD normally sits ahead of the recorded gitlink. The gitlink is marked skip-worktree by `apply`, so `git status`/`git add -u` ignore it and a bare `git add labs` is a no-op; the pin moves only with `scripts/labs bump <ref>`. To change a patch, commit inside `labs/` on top of the applied series — the series re-exports itself on commit — then commit the regenerated `.patch` files. To send patches to aztec-node, `labs-patches/bootstrap.sh upstream <n>...` prepares one branch with the named patches and prints the push/PR commands. To port a whole aztec-packages PR to aztec-node (a v5 PR in the in-tree layout, or a `next` PR that only changes `labs-patches/*.patch`), `scripts/labs_port_pr.sh <pr>` checks the PR touches nothing but the labs half, replays it 3-way onto aztec-node `main` in a `port/pr-<n>` branch of `labs/`'s repository, and prints the push/PR commands; mixed PRs are refused, not split. See `labs-patches/README.md`.
6161

6262
The tooling, all in `labs-patches/bootstrap.sh` (foundation-owned; only the `.patch` contents ever go upstream):
6363

@@ -66,8 +66,10 @@ The tooling, all in `labs-patches/bootstrap.sh` (foundation-owned; only the `.pa
6666
| `apply` | Checks out the gitlink (`update = none` submodule) and `git am`s the series with a fixed committer identity, so the applied SHAs are identical everywhere. Idempotent; refuses to drop `labs/` commits that are not in the series (`LABS_PATCHES_FORCE=1` overrides); stashes uncommitted edits. |
6767
| `export` | Regenerates `*.patch` from the commits above the gitlink, skipping marker commits. Run it after committing inside `labs/`. |
6868
| `check` | Applies the committed series to the gitlink in a temporary worktree; CI runs it. |
69-
| `bump <ref>` | Fetches an aztec-node ref, stages the new gitlink and re-applies; refuses on unexported work. |
70-
| `upstream <n>... [--branch <name>]` | Creates a branch in `labs/`'s repository with the named patches (in series order) applied under your own identity, and prints the push and `gh pr create` commands. Batch the patches that belong together; one PR per patch is not required. |
69+
| `bump <ref> [--commit\|--pr]` | Fetches an aztec-node ref, stages the new gitlink, keeps the foundry locks in step and re-applies; refuses on unexported work; `--commit`/`--pr` record and open the bump. |
70+
| `upstream <n>... [--branch <name>] [--push]` | Creates a branch in `labs/`'s repository with the named patches (in series order) applied under your own identity; prints or (`--push`) runs the push and `gh pr create`. Batch the patches that belong together. |
71+
| `port <pr> [--push]` | `scripts/labs_port_pr.sh`: replays a labs-only aztec-packages PR onto aztec-node `main`. |
72+
| `run <cmd>` | `scripts/labs_env.sh`: a command inside `labs/` with the right environment. |
7173
| `status` | Base, checkout, series, whether it is applied, and unexported commits. |
7274
| `commit-use-local` | Commits the build's manifest rewrite as the marker commit (`labs-patches: use-local rewrite (never exported)`), staging only `package.json`/`yarn.lock`/`Nargo.toml`/`fnd-hashes`. Called by the build, not by hand. |
7375
| `check_staged` | The `pre-commit` hook: rejects a staged gitlink that is a series or marker commit. |

REPO.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Working across aztec-packages, aztec-packages-private and aztec-node
2+
3+
Three repositories: **public** `AztecProtocol/aztec-packages` (the foundation; releases are cut here), **private** `aztec-packages-private` (same tree plus embargoed work; `public-next` mirrors public `next`; releases go only to the internal Artifact Registry), **labs** `aztec-labs-eng/aztec-node` (node, PXE/wallets, aztec-nr, docs, playground, spartan; consumes the foundation through published `@aztec-foundation/*` packages and bb/nargo release binaries).
4+
5+
The foundation builds labs from the `labs/` submodule against its own tree, so a foundation change is tested against labs before it ships. Foundation-side changes to labs are **patches** (`labs-patches/*.patch`) applied on top of the pinned submodule commit. One command drives all of it: `scripts/labs`.
6+
7+
## Build and test
8+
9+
```
10+
./bootstrap.sh # applies the labs series; labs/ HEAD sits above the gitlink and git ignores that
11+
make fast # foundation + labs against this tree
12+
make fast-labs # labs only: yarn-project (+tests, e2e), aztec-nr, noir-contracts, snapshots
13+
make labs-full # + docs, spartan, playground, claude tests, benches
14+
scripts/labs run <cmd> # any command inside labs/ with the right environment
15+
```
16+
17+
Labs cache keys follow the foundation tree (`labs/labs-aztec-toolchain/fnd-hashes`, committed by the build as a marker commit); tests that only run from a labs checkout (compose/web3signer/ha e2e, playground browser tests, docs examples) are skipped here and belong to labs CI.
18+
19+
## Change labs from the foundation
20+
21+
```
22+
scripts/labs apply # (already done by bootstrap) patched labs/ checkout
23+
cd labs && <edit> && git commit # the series re-exports itself on commit
24+
git add labs-patches && git commit # the .patch files are what your PR carries
25+
scripts/labs upstream 4 5 --push # one aztec-node PR for the patches that belong together
26+
```
27+
28+
Batch patches that are independent of a foundation version bump; a patch that needs one waits for that release. When the aztec-node PR lands, the next `bump` drops the patch from the series. `scripts/labs status` shows unexported commits; `apply` never discards them (`LABS_PATCHES_FORCE=1` is the last resort).
29+
30+
## Port a whole PR to labs
31+
32+
```
33+
scripts/labs port 24800 --push # a v5 PR in the in-tree layout, or a next PR that only changes labs-patches/*.patch
34+
```
35+
36+
The diff must touch only the labs half (in-tree `yarn-project/`, `noir-projects/labs/`, `docs/`, `playground/`, `spartan/`, `aztec-up/`, `release-image/`, `labs-aztec-toolchain/`, or `labs-patches/*.patch`); anything else is refused with the files listed. It is replayed 3-way onto aztec-node `main` (already-landed hunks become no-ops); remaining conflicts are committed with markers and listed, foundation-path references flagged. v5 PRs get the `port-to-aztec-node` label automatically; a person runs this.
37+
38+
## Take labs changes in
39+
40+
```
41+
scripts/labs bump main --pr # move the pin, re-apply the series, commit gitlink + foundry locks + patches, open the PR
42+
```
43+
44+
`bump` refuses on unexported work and lists the patches upstream absorbed. If a patch no longer applies: fix it in `labs/` (`git am --3way` conflicts), commit, and the series re-exports. Labs takes foundation changes through releases (`labs-aztec-toolchain` pins) and, immediately, through foundation CI building the submodule.
45+
46+
## Public ↔ private
47+
48+
- `public-next` (private repo) is a force-mirror of public `next`; a merge into private `next` follows. When that merge conflicts, AztecBot opens a raw-merge PR plus a resolution PR on top: land resolution into raw, raw into `next`.
49+
- Prototype a private-only mechanism without private content: branch from `public-next`, cherry-pick the public PRs, open the PR **into `public-next`**, label `ci-release-pr`.
50+
- Port private work to public by cherry-picking onto a public branch with `--author`; public → private is the mirror.
51+
- `ci-release-pr` on any PR tags its head and runs the full release flow: public → real npm (trusted publishing, no tokens) / GitHub / crates prereleases at `0.0.1-commit.<sha>`; private → Artifact Registry only (`GCP_PRIVATE_NPM_DEPLOY`). The build log is attached to the run as the `ci-logs` artifact.
52+
53+
## If this happens
54+
55+
| Situation | Do |
56+
|---|---|
57+
| a bb/noir/circuit change must reach labs code | build, `make fast-labs` |
58+
| `apply`: a patch does not apply | fix in `labs/`, commit (auto-export) |
59+
| `apply`: commits would be lost | `scripts/labs export`, or `git -C labs reset`, or `LABS_PATCHES_FORCE=1` |
60+
| CI red in a labs test | run the `/tmp/test_cmds` line from the root, or `scripts/labs run <cmd>` |
61+
| labs tests say `disabled-cache` locally | by design; per-package hashes resolve in CI only |
62+
| labs-only PR on a v5 line | `scripts/labs port <pr> --push` |
63+
| the private `public-next → next` merge is stuck | land the AztecBot pair |

labs-patches/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# labs-patches
22

3+
The one-page workflow is `REPO.md` at the repo root; `scripts/labs <verb>` is the entry point for everything below.
4+
35
The foundation's changes to the labs repo, kept as a `git format-patch` series that is applied on top of the `labs/` submodule (aztec-node) every time it is checked out.
46

57
`labs/` is pinned to an upstream commit (the gitlink). `bootstrap.sh apply` checks that commit out and runs `git am` over `*.patch` in name order, so the patches become real commits in the submodule and `labs/` HEAD sits ahead of the gitlink. It is idempotent and runs from the root `bootstrap.sh`, the `post-merge`/`post-checkout` hooks and `make labs-patched`, so a fresh clone, a pull and a build all land on the same patched tree. It never discards commits that are not in the series: if the base or the series changed under work in progress, it stops and asks you to `export` or drop it (`LABS_PATCHES_FORCE=1` discards them — a last resort, not a workflow). Uncommitted tracked edits in `labs/` are stashed (`git -C labs stash list`) rather than reset when a re-apply is needed.

0 commit comments

Comments
 (0)