chore: formalize release workflow

This commit is contained in:
Dotta
2026-03-09 08:49:42 -05:00
parent ccd501ea02
commit a7cfd9f24b
9 changed files with 1431 additions and 1091 deletions

View File

@@ -1,196 +1,119 @@
# Publishing to npm
This document covers how to build and publish the `paperclipai` CLI package to npm.
Low-level reference for how Paperclip packages are built for npm.
## Prerequisites
For the maintainer release workflow, use [doc/RELEASING.md](RELEASING.md). This document is only about packaging internals and the scripts that produce publishable artifacts.
- Node.js 20+
- pnpm 9.15+
- An npm account with publish access to the `paperclipai` package
- Logged in to npm: `npm login`
## Current Release Entry Points
## One-Command Publish
Use these scripts instead of older one-off publish commands:
The fastest way to publish — bumps version, builds, publishes, restores, commits, and tags in one shot:
- [`scripts/release.sh`](../scripts/release.sh) for canary and stable npm publishes
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) to repoint `latest` during rollback
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) after a stable push
```bash
./scripts/bump-and-publish.sh patch # 0.1.1 → 0.1.2
./scripts/bump-and-publish.sh minor # 0.1.1 → 0.2.0
./scripts/bump-and-publish.sh major # 0.1.1 → 1.0.0
./scripts/bump-and-publish.sh 2.0.0 # set explicit version
./scripts/bump-and-publish.sh patch --dry-run # everything except npm publish
```
## Why the CLI needs special packaging
The script runs all 6 steps below in order. It requires a clean working tree and an active `npm login` session (unless `--dry-run`). After it finishes, push:
The CLI package, `paperclipai`, imports code from workspace packages such as:
```bash
git push && git push origin v<version>
```
- `@paperclipai/server`
- `@paperclipai/db`
- `@paperclipai/shared`
- adapter packages under `packages/adapters/`
## Manual Step-by-Step
Those workspace references use `workspace:*` during development. npm cannot install those references directly for end users, so the release build has to transform the CLI into a publishable standalone package.
If you prefer to run each step individually:
## `build-npm.sh`
### Quick Reference
```bash
# Bump version
./scripts/version-bump.sh patch # 0.1.0 → 0.1.1
# Build
./scripts/build-npm.sh
# Preview what will be published
cd cli && npm pack --dry-run
# Publish
cd cli && npm publish --access public
# Restore dev package.json
mv cli/package.dev.json cli/package.json
```
## Step-by-Step
### 1. Bump the version
```bash
./scripts/version-bump.sh <patch|minor|major|X.Y.Z>
```
This updates the version in two places:
- `cli/package.json` — the source of truth
- `cli/src/index.ts` — the Commander `.version()` call
Examples:
```bash
./scripts/version-bump.sh patch # 0.1.0 → 0.1.1
./scripts/version-bump.sh minor # 0.1.0 → 0.2.0
./scripts/version-bump.sh major # 0.1.0 → 1.0.0
./scripts/version-bump.sh 1.2.3 # set explicit version
```
### 2. Build
Run:
```bash
./scripts/build-npm.sh
```
The build script runs five steps:
This script does six things:
1. **Forbidden token check** — scans tracked files for tokens listed in `.git/hooks/forbidden-tokens.txt`. If the file is missing (e.g. on a contributor's machine), the check passes silently. The script never prints which tokens it's searching for.
2. **TypeScript type-check** — runs `pnpm -r typecheck` across all workspace packages.
3. **esbuild bundle** — bundles the CLI entry point (`cli/src/index.ts`) and all workspace package code (`@paperclipai/*`) into a single file at `cli/dist/index.js`. External npm dependencies (express, postgres, etc.) are kept as regular imports.
4. **Generate publishable package.json** — replaces `cli/package.json` with a version that has real npm dependency ranges instead of `workspace:*` references (see [package.dev.json](#packagedevjson) below).
5. **Summary** — prints the bundle size and next steps.
1. Runs the forbidden token check unless `--skip-checks` is supplied
2. Runs `pnpm -r typecheck`
3. Bundles the CLI entrypoint with esbuild into `cli/dist/index.js`
4. Verifies the bundled entrypoint with `node --check`
5. Rewrites `cli/package.json` into a publishable npm manifest and stores the dev copy as `cli/package.dev.json`
6. Copies the repo `README.md` into `cli/README.md` for npm package metadata
To skip the forbidden token check (e.g. in CI without the token list):
`build-npm.sh` is used by the release script so that npm users install a real package rather than unresolved workspace dependencies.
## Publishable CLI layout
During development, [`cli/package.json`](../cli/package.json) contains workspace references.
During release preparation:
- `cli/package.json` becomes a publishable manifest with external npm dependency ranges
- `cli/package.dev.json` stores the development manifest temporarily
- `cli/dist/index.js` contains the bundled CLI entrypoint
- `cli/README.md` is copied in for npm metadata
After release finalization, the release script restores the development manifest and removes the temporary README copy.
## Package discovery
The release tooling scans the workspace for public packages under:
- `packages/`
- `server/`
- `cli/`
`ui/` remains ignored for npm publishing because it is private.
This matters because all public packages are versioned and published together as one release unit.
## Canary packaging model
Canaries are published as semver prereleases such as:
- `1.2.3-canary.0`
- `1.2.3-canary.1`
They are published under the npm dist-tag `canary`.
This means:
- `npx paperclipai@canary onboard` can install them explicitly
- `npx paperclipai onboard` continues to resolve `latest`
- the stable changelog can stay at `releases/v1.2.3.md`
## Stable packaging model
Stable releases publish normal semver versions such as `1.2.3` under the npm dist-tag `latest`.
The stable publish flow also creates the local release commit and git tag. Pushing the commit/tag and creating the GitHub Release happen afterward as separate maintainer steps.
## Rollback model
Rollback does not unpublish packages.
Instead, the maintainer should move the `latest` dist-tag back to the previous good stable version with:
```bash
./scripts/build-npm.sh --skip-checks
./scripts/rollback-latest.sh <stable-version>
```
### 3. Preview (optional)
That keeps history intact while restoring the default install path quickly.
See what npm will publish:
## Notes for CI
```bash
cd cli && npm pack --dry-run
```
The repo includes a manual GitHub Actions release workflow at [`.github/workflows/release.yml`](../.github/workflows/release.yml).
### 4. Publish
Recommended CI release setup:
```bash
cd cli && npm publish --access public
```
- use npm trusted publishing via GitHub OIDC
- require approval through the `npm-release` environment
- run releases from `master`
- use canary first, then stable
### 5. Restore dev package.json
## Related Files
After publishing, restore the workspace-aware `package.json`:
```bash
mv cli/package.dev.json cli/package.json
```
### 6. Commit and tag
```bash
git add cli/package.json cli/src/index.ts
git commit -m "chore: bump version to X.Y.Z"
git tag vX.Y.Z
```
## package.dev.json
During development, `cli/package.json` contains `workspace:*` references like:
```json
{
"dependencies": {
"@paperclipai/server": "workspace:*",
"@paperclipai/db": "workspace:*"
}
}
```
These tell pnpm to resolve those packages from the local monorepo. This is great for development but **npm doesn't understand `workspace:*`** — publishing with these references would cause install failures for users.
The build script solves this with a two-file swap:
1. **Before building:** `cli/package.json` has `workspace:*` refs (the dev version).
2. **During build (`build-npm.sh` step 4):**
- The dev `package.json` is copied to `package.dev.json` as a backup.
- `generate-npm-package-json.mjs` reads every workspace package's `package.json`, collects all their external npm dependencies, and writes a new `cli/package.json` with those real dependency ranges — no `workspace:*` refs.
3. **After publishing:** you restore the dev version with `mv package.dev.json package.json`.
The generated publishable `package.json` looks like:
```json
{
"name": "paperclipai",
"version": "0.1.0",
"bin": { "paperclipai": "./dist/index.js" },
"dependencies": {
"express": "^5.1.0",
"postgres": "^3.4.5",
"commander": "^13.1.0"
}
}
```
`package.dev.json` is listed in `.gitignore` — it only exists temporarily on disk during the build/publish cycle.
## How the bundle works
The CLI is a monorepo package that imports code from `@paperclipai/server`, `@paperclipai/db`, `@paperclipai/shared`, and several adapter packages. These workspace packages don't exist on npm.
**esbuild** bundles all workspace TypeScript code into a single `dist/index.js` file (~250kb). External npm packages (express, postgres, zod, etc.) are left as normal `import` statements — they get installed by npm when a user runs `npx paperclipai onboard`.
The esbuild configuration lives at `cli/esbuild.config.mjs`. It automatically reads every workspace package's `package.json` to determine which dependencies are external (real npm packages) vs. internal (workspace code to bundle).
## Forbidden token enforcement
The build process includes the same forbidden-token check used by the git pre-commit hook. This catches any accidentally committed tokens before they reach npm.
- Token list: `.git/hooks/forbidden-tokens.txt` (one token per line, `#` comments supported)
- The file lives inside `.git/` and is never committed
- If the file is missing, the check passes — contributors without the list can still build
- The script never prints which tokens are being searched for
- Matches are printed so you know which files to fix, but not which token triggered it
Run the check standalone:
```bash
pnpm check:tokens
```
## npm scripts reference
| Script | Command | Description |
|---|---|---|
| `bump-and-publish` | `pnpm bump-and-publish <type>` | One-command bump + build + publish + commit + tag |
| `build:npm` | `pnpm build:npm` | Full build (check + typecheck + bundle + package.json) |
| `version:bump` | `pnpm version:bump <type>` | Bump CLI version |
| `check:tokens` | `pnpm check:tokens` | Run forbidden token check only |
- [`scripts/build-npm.sh`](../scripts/build-npm.sh)
- [`scripts/generate-npm-package-json.mjs`](../scripts/generate-npm-package-json.mjs)
- [`cli/esbuild.config.mjs`](../cli/esbuild.config.mjs)
- [`doc/RELEASING.md`](RELEASING.md)

437
doc/RELEASING.md Normal file
View File

@@ -0,0 +1,437 @@
# Releasing Paperclip
Maintainer runbook for shipping a full Paperclip release across npm, GitHub, and the website-facing changelog surface.
This document is intentionally practical:
- TL;DR command sequences are at the top.
- Detailed checklists come next.
- Motivation, failure handling, and rollback playbooks follow after that.
## Release Surfaces
Every Paperclip release has four separate surfaces:
1. **Verification** — the exact git SHA must pass typecheck, tests, and build.
2. **npm**`paperclipai` and the public workspace packages are published.
3. **GitHub** — the stable release gets a git tag and a GitHub Release.
4. **Website / announcements** — the stable changelog is published externally and announced.
Treat those as related but separate. npm can succeed while the GitHub Release is still pending. GitHub can be correct while the website changelog is stale. A maintainer release is done only when all four surfaces are handled.
## TL;DR
### Canary release
Use this when you want an installable prerelease without changing `latest`.
```bash
# 0. Start clean
git status --short
# 1. Verify the candidate SHA
pnpm -r typecheck
pnpm test:run
pnpm build
# 2. Draft or update the stable changelog
# releases/vX.Y.Z.md
# 3. Preview the canary release
./scripts/release.sh patch --canary --dry-run
# 4. Publish the canary
./scripts/release.sh patch --canary
# 5. Smoke test what users will actually install
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
# Users install with:
npx paperclipai@canary onboard
```
Result:
- npm gets a prerelease such as `1.2.3-canary.0` under dist-tag `canary`
- `latest` is unchanged
- no git tag is created
- no GitHub Release is created
- the working tree returns to clean after the script finishes
### Stable release
Use this only after the canary SHA is good enough to become the public default.
```bash
# 0. Start from the vetted commit
git checkout master
git pull
git status --short
# 1. Verify again on the exact release SHA
pnpm -r typecheck
pnpm test:run
pnpm build
# 2. Confirm the stable changelog exists
ls releases/v*.md
# 3. Preview the stable publish
./scripts/release.sh patch --dry-run
# 4. Publish the stable release to npm and create the local release commit + tag
./scripts/release.sh patch
# 5. Push the release commit and tag
git push origin HEAD:master --follow-tags
# 6. Create or update the GitHub Release from the pushed tag
./scripts/create-github-release.sh X.Y.Z
```
Result:
- npm gets stable `X.Y.Z` under dist-tag `latest`
- a local git commit and tag `vX.Y.Z` are created
- after push, GitHub gets the matching Release
- the website and announcement steps still need to be handled manually
### Emergency rollback
If `latest` is broken after publish, repoint it to the last known good stable version first, then work on the fix.
```bash
# Preview
./scripts/rollback-latest.sh X.Y.Z --dry-run
# Roll back latest for every public package
./scripts/rollback-latest.sh X.Y.Z
```
This does **not** unpublish anything. It only moves the `latest` dist-tag back to the last good stable release.
### GitHub Actions release
There is also a manual workflow at [`.github/workflows/release.yml`](../.github/workflows/release.yml). It is designed for npm trusted publishing via GitHub OIDC instead of long-lived npm tokens.
Use it from the Actions tab:
1. Choose `Release`
2. Choose `channel`: `canary` or `stable`
3. Choose `bump`: `patch`, `minor`, or `major`
4. Choose whether this is a `dry_run`
5. Run it from `master`
The workflow:
- reruns `typecheck`, `test:run`, and `build`
- gates publish behind the `npm-release` environment
- can publish canaries without touching `latest`
- can publish stable, push the release commit and tag, and create the GitHub Release
## Release Checklist
### Before any publish
- [ ] The working tree is clean, including untracked files
- [ ] The target branch and SHA are the ones you actually want to release
- [ ] The required verification gate passed on that exact SHA
- [ ] The bump type is correct for the user-visible impact
- [ ] The stable changelog file exists or is ready to be written at `releases/vX.Y.Z.md`
- [ ] You know which previous stable version you would roll back to if needed
### Before a canary
- [ ] You are intentionally testing something that should be installable before it becomes default
- [ ] You are comfortable with users installing it via `npx paperclipai@canary onboard`
- [ ] You understand that each canary is a new immutable npm version such as `1.2.3-canary.1`
### Before a stable
- [ ] The candidate has already passed smoke testing
- [ ] The changelog should be the stable version only, for example `v1.2.3`
- [ ] You are ready to push the release commit and tag immediately after npm publish
- [ ] You are ready to create the GitHub Release immediately after the push
- [ ] You have a post-release website / announcement plan
### After a stable
- [ ] `npm view paperclipai@latest version` matches the new stable version
- [ ] The git tag exists on GitHub
- [ ] The GitHub Release exists and uses `releases/vX.Y.Z.md`
- [ ] The website changelog is updated
- [ ] Any announcement copy matches the shipped release, not the canary
## Verification Gate
The repository standard is:
```bash
pnpm -r typecheck
pnpm test:run
pnpm build
```
This matches [`.github/workflows/pr-verify.yml`](../.github/workflows/pr-verify.yml). Run it before claiming a release candidate is ready.
## Versioning Policy
### Stable versions
Stable releases use normal semver:
- `patch` for bug fixes
- `minor` for additive features, endpoints, and additive migrations
- `major` for destructive migrations, removed APIs, or other breaking behavior
### Canary versions
Canaries are semver prereleases of the **intended stable version**:
- `1.2.3-canary.0`
- `1.2.3-canary.1`
- `1.2.3-canary.2`
That gives you three useful properties:
1. Users can install the prerelease explicitly with `@canary`
2. `latest` stays safe
3. The stable changelog can remain just `v1.2.3`
We do **not** create separate changelog files for canary versions.
## Changelog Policy
The maintainer changelog source of truth is:
- `releases/vX.Y.Z.md`
That file is for the eventual stable release. It should not include `-canary` in the filename or heading.
Recommended structure:
- `Breaking Changes` when needed
- `Highlights`
- `Improvements`
- `Fixes`
- `Upgrade Guide` when needed
Package-level `CHANGELOG.md` files are generated as part of the release mechanics. They are not the main release narrative.
## Detailed Workflow
### 1. Decide the bump
Review the range since the last stable tag:
```bash
LAST_TAG=$(git tag --list 'v*' --sort=-version:refname | head -1)
git log "${LAST_TAG}..HEAD" --oneline --no-merges
git diff --name-only "${LAST_TAG}..HEAD" -- packages/db/src/migrations/
git diff "${LAST_TAG}..HEAD" -- packages/db/src/schema/
git log "${LAST_TAG}..HEAD" --format="%s" | grep -E 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true
```
Use the higher bump if there is any doubt.
### 2. Write the stable changelog first
Create or update:
```bash
releases/vX.Y.Z.md
```
This is deliberate. The release notes should describe the stable story, not the canary mechanics.
### 3. Publish one or more canaries
Run:
```bash
./scripts/release.sh <patch|minor|major> --canary
```
What the script does:
1. Verifies the working tree is clean
2. Computes the intended stable version from the last stable tag
3. Computes the next canary ordinal from npm
4. Versions the public packages to `X.Y.Z-canary.N`
5. Builds the workspace and publishable CLI
6. Publishes to npm under dist-tag `canary`
7. Cleans up the temporary versioning state so your branch returns to clean
This means the script is safe to repeat as many times as needed while iterating:
- `1.2.3-canary.0`
- `1.2.3-canary.1`
- `1.2.3-canary.2`
The target stable release can still remain `1.2.3`.
### 4. Smoke test the canary
Run the actual install path in Docker:
```bash
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
```
Minimum checks:
- [ ] `npx paperclipai@canary onboard` installs
- [ ] onboarding completes without crashes
- [ ] the server boots
- [ ] the UI loads
- [ ] basic company creation and dashboard load work
### 5. Publish stable from the vetted commit
Once the candidate SHA is good, run the stable flow on that exact commit:
```bash
./scripts/release.sh <patch|minor|major>
```
What the script does:
1. Verifies the working tree is clean
2. Versions the public packages to the stable semver
3. Builds the workspace and CLI publish bundle
4. Publishes to npm under `latest`
5. Restores temporary publish artifacts
6. Creates the local release commit and git tag
What it does **not** do:
- it does not push for you
- it does not update the website
- it does not announce the release for you
### 6. Push the release and create the GitHub Release
After a stable publish succeeds:
```bash
git push origin HEAD:master --follow-tags
./scripts/create-github-release.sh X.Y.Z
```
The GitHub release notes come from:
- `releases/vX.Y.Z.md`
### 7. Complete the external surfaces
After GitHub is correct:
- publish the changelog on the website
- write the announcement copy
- ensure public docs and install guidance point to the stable version
## GitHub Actions and npm Trusted Publishing
If you want GitHub to own the actual npm publish, use [`.github/workflows/release.yml`](../.github/workflows/release.yml) together with npm trusted publishing.
Recommended setup:
1. Configure the GitHub Actions workflow as a trusted publisher for **every public package** on npm
2. Use the `npm-release` GitHub environment with required reviewers
3. Run stable publishes from `master` only
4. Keep the workflow manual via `workflow_dispatch`
Why this is the right shape:
- no long-lived npm token needs to live in GitHub secrets
- reviewers can approve the publish step at the environment gate
- the workflow reruns verification on the release SHA before publish
- stable and canary use the same mechanics
## Failure Playbooks
### If the canary fails before publish
Nothing shipped. Fix the code and rerun the canary workflow.
### If the canary publishes but the smoke test fails
Do **not** publish stable.
Instead:
1. Fix the issue
2. Publish another canary
3. Re-run smoke testing
The canary version number will increase, but the stable target version can remain the same.
### If the stable npm publish succeeds but push fails
This is a partial release. npm is already live.
Do this immediately:
1. Fix the git issue
2. Push the release commit and tag from the same checkout
3. Create the GitHub Release
Do **not** publish the same version again.
### If the stable release is bad after `latest` moves
Use the rollback script first:
```bash
./scripts/rollback-latest.sh <last-good-version>
```
Then:
1. open an incident note or maintainer comment
2. fix forward on a new patch release
3. update the changelog / release notes if the user-facing guidance changed
### If the GitHub Release is wrong
Edit it by re-running:
```bash
./scripts/create-github-release.sh X.Y.Z
```
This updates the release notes if the GitHub Release already exists.
### If the website changelog is wrong
Fix the website independently. Do not republish npm just to repair the website surface.
## Rollback Strategy
The default rollback strategy is **dist-tag rollback, then fix forward**.
Why:
- npm versions are immutable
- users need `npx paperclipai onboard` to recover quickly
- moving `latest` back is faster and safer than trying to delete history
Rollback procedure:
1. identify the last known good stable version
2. run `./scripts/rollback-latest.sh <version>`
3. verify `npm view paperclipai@latest version`
4. fix forward with a new stable release
## Scripts Reference
- [`scripts/release.sh`](../scripts/release.sh) — stable and canary npm publish flow
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) — create or update the GitHub Release after push
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) — repoint `latest` to the last good stable release
- [`scripts/docker-onboard-smoke.sh`](../scripts/docker-onboard-smoke.sh) — Docker smoke test for the installed CLI
## Related Docs
- [doc/PUBLISHING.md](PUBLISHING.md) — low-level npm build and packaging internals
- [skills/release/SKILL.md](../skills/release/SKILL.md) — agent release coordination workflow
- [skills/release-changelog/SKILL.md](../skills/release-changelog/SKILL.md) — stable changelog drafting workflow