aevgarik/paperclip
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: automate canary and stable releases

Browse Source
This commit is contained in:
Dotta
2026-03-17 14:08:55 -05:00
parent 7b9718cbaa
commit 21c1235277
18 changed files with 1536 additions and 1260 deletions
Download Patch File Download Diff File Expand all files Collapse all files

141
doc/PUBLISHING.md
View File

@@ -1,18 +1,19 @@
# Publishing to npm
Low-level reference for how Paperclip packages are built for npm.
Low-level reference for how Paperclip packages are prepared and published to npm.
For the maintainer release workflow, use [doc/RELEASING.md](RELEASING.md). This document is only about packaging internals and the scripts that produce publishable artifacts.
For the maintainer workflow, use [doc/RELEASING.md](RELEASING.md). This document focuses on packaging internals.
## Current Release Entry Points
Use these scripts instead of older one-off publish commands:
Use these scripts:
- [`scripts/release-start.sh`](../scripts/release-start.sh) to create or resume `release/X.Y.Z`
- [`scripts/release-preflight.sh`](../scripts/release-preflight.sh) before any canary or stable release
- [`scripts/release.sh`](../scripts/release.sh) for canary and stable npm publishes
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) to repoint `latest` during rollback
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) after pushing the stable branch tag
- [`scripts/release.sh`](../scripts/release.sh) for canary and stable publish flows
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) after pushing a stable tag
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) to repoint `latest`
- [`scripts/build-npm.sh`](../scripts/build-npm.sh) for the CLI packaging build
Paperclip no longer uses release branches or Changesets for publishing.
## Why the CLI needs special packaging
@@ -23,7 +24,7 @@ The CLI package, `paperclipai`, imports code from workspace packages such as:
- `@paperclipai/shared`
- adapter packages under `packages/adapters/`
Those workspace references use `workspace:*` during development. npm cannot install those references directly for end users, so the release build has to transform the CLI into a publishable standalone package.
Those workspace references are valid in development but not in a publishable npm package. The release flow rewrites versions temporarily, then builds a publishable CLI bundle.
## `build-npm.sh`
@@ -33,89 +34,107 @@ Run:
./scripts/build-npm.sh
```
This script does six things:
This script:
1. Runs the forbidden token check unless `--skip-checks` is supplied
2. Runs `pnpm -r typecheck`
3. Bundles the CLI entrypoint with esbuild into `cli/dist/index.js`
4. Verifies the bundled entrypoint with `node --check`
5. Rewrites `cli/package.json` into a publishable npm manifest and stores the dev copy as `cli/package.dev.json`
6. Copies the repo `README.md` into `cli/README.md` for npm package metadata
1. runs the forbidden token check unless `--skip-checks` is supplied
2. runs `pnpm -r typecheck`
3. bundles the CLI entrypoint with esbuild into `cli/dist/index.js`
4. verifies the bundled entrypoint with `node --check`
5. rewrites `cli/package.json` into a publishable npm manifest and stores the dev copy as `cli/package.dev.json`
6. copies the repo `README.md` into `cli/README.md` for npm metadata
`build-npm.sh` is used by the release script so that npm users install a real package rather than unresolved workspace dependencies.
After the release script exits, the dev manifest and temporary files are restored automatically.
## Publishable CLI layout
## Package discovery and versioning
During development, [`cli/package.json`](../cli/package.json) contains workspace references.
During release preparation:
- `cli/package.json` becomes a publishable manifest with external npm dependency ranges
- `cli/package.dev.json` stores the development manifest temporarily
- `cli/dist/index.js` contains the bundled CLI entrypoint
- `cli/README.md` is copied in for npm metadata
After release finalization, the release script restores the development manifest and removes the temporary README copy.
## Package discovery
The release tooling scans the workspace for public packages under:
Public packages are discovered from:
- `packages/`
- `server/`
- `cli/`
`ui/` remains ignored for npm publishing because it is private.
`ui/` is ignored because it is private.
This matters because all public packages are versioned and published together as one release unit.
The version rewrite step now uses [`scripts/release-package-map.mjs`](../scripts/release-package-map.mjs), which:
## Canary packaging model
- finds all public packages
- sorts them topologically by internal dependencies
- rewrites each package version to the target release version
- rewrites internal `workspace:*` dependency references to the exact target version
- updates the CLI's displayed version string
Canaries are published as semver prereleases such as:
Those rewrites are temporary. The working tree is restored after publish or dry-run.
- `1.2.3-canary.0`
- `1.2.3-canary.1`
## Version formats
They are published under the npm dist-tag `canary`.
Paperclip uses calendar versions:
This means:
- stable: `YYYY.M.D`
- canary: `YYYY.M.D-canary.N`
- `npx paperclipai@canary onboard` can install them explicitly
- `npx paperclipai onboard` continues to resolve `latest`
- the stable changelog can stay at `releases/v1.2.3.md`
Examples:
## Stable packaging model
- stable: `2026.3.17`
- canary: `2026.3.17-canary.2`
Stable releases publish normal semver versions such as `1.2.3` under the npm dist-tag `latest`.
## Publish model
The stable publish flow also creates the local release commit and git tag on `release/X.Y.Z`. Pushing that branch commit/tag, creating the GitHub Release, and merging the release branch back to `master` happen afterward as separate maintainer steps.
### Canary
Canaries publish under the npm dist-tag `canary`.
Example:
- `paperclipai@2026.3.17-canary.2`
This keeps the default install path unchanged while allowing explicit installs with:
```bash
npx paperclipai@canary onboard
```
### Stable
Stable publishes use the npm dist-tag `latest`.
Example:
- `paperclipai@2026.3.17`
Stable publishes do not create a release commit. Instead:
- package versions are rewritten temporarily
- packages are published from the chosen source commit
- git tag `vYYYY.M.D` points at that original commit
## Trusted publishing
The intended CI model is npm trusted publishing through GitHub OIDC.
That means:
- no long-lived `NPM_TOKEN` in repository secrets
- GitHub Actions obtains short-lived publish credentials
- trusted publisher rules are configured per workflow file
See [doc/RELEASE-AUTOMATION-SETUP.md](RELEASE-AUTOMATION-SETUP.md) for the GitHub/npm setup steps.
## Rollback model
Rollback does not unpublish packages.
Rollback does not unpublish anything.
Instead, the maintainer should move the `latest` dist-tag back to the previous good stable version with:
It repoints the `latest` dist-tag to a prior stable version:
```bash
./scripts/rollback-latest.sh <stable-version>
./scripts/rollback-latest.sh 2026.3.16
```
That keeps history intact while restoring the default install path quickly.
## Notes for CI
The repo includes a manual GitHub Actions release workflow at [`.github/workflows/release.yml`](../.github/workflows/release.yml).
Recommended CI release setup:
- use npm trusted publishing via GitHub OIDC
- require approval through the `npm-release` environment
- run releases from `release/X.Y.Z`
- use canary first, then stable
This is the fastest way to restore the default install path if a stable release is bad.
## Related Files
- [`scripts/build-npm.sh`](../scripts/build-npm.sh)
- [`scripts/generate-npm-package-json.mjs`](../scripts/generate-npm-package-json.mjs)
- [`scripts/release-package-map.mjs`](../scripts/release-package-map.mjs)
- [`cli/esbuild.config.mjs`](../cli/esbuild.config.mjs)
- [`doc/RELEASING.md`](RELEASING.md)

271
doc/RELEASE-AUTOMATION-SETUP.md Normal file
View File

@@ -0,0 +1,271 @@
# Release Automation Setup
This document covers the GitHub and npm setup required for the current Paperclip release model:
- automatic canaries from `master`
- manual stable promotion from a chosen source ref
- npm trusted publishing via GitHub OIDC
- protected release infrastructure in a public repository
Repo-side files that depend on this setup:
- `.github/workflows/release-canary.yml`
- `.github/workflows/release-stable.yml`
- `.github/CODEOWNERS`
Note:
- the release workflows intentionally use `pnpm install --no-frozen-lockfile`
- this matches the repo's current policy where `pnpm-lock.yaml` is refreshed by GitHub automation after manifest changes land on `master`
## 1. Merge the Repo Changes First
Before touching GitHub or npm settings, merge the release automation code so the referenced workflow filenames already exist on the default branch.
Required files:
- `.github/workflows/release-canary.yml`
- `.github/workflows/release-stable.yml`
- `.github/CODEOWNERS`
## 2. Configure npm Trusted Publishing
Do this for every public package that Paperclip publishes.
At minimum that includes:
- `paperclipai`
- `@paperclipai/server`
- public packages under `packages/`
### 2.1. In npm, open each package settings page
For each package:
1. open npm as an owner of the package
2. go to the package settings / publishing access area
3. add a trusted publisher for the GitHub repository `paperclipai/paperclip`
### 2.2. Add two trusted publisher entries per package
Because npm trusted publishing is tied to the workflow filename, configure both:
- workflow: `.github/workflows/release-canary.yml`
- workflow: `.github/workflows/release-stable.yml`
Repository:
- `paperclipai/paperclip`
Branch expectations:
- canary workflow should only ever run from `master`
- stable workflow is manual but should also be restricted to `master` by GitHub environment policy
### 2.3. Verify trusted publishing before removing old auth
After the workflows are live:
1. run a canary publish
2. confirm npm publish succeeds without any `NPM_TOKEN`
3. run a stable dry-run
4. run one real stable publish
Only after that should you remove old token-based access.
## 3. Remove Legacy npm Tokens
After trusted publishing works:
1. revoke any repository or organization `NPM_TOKEN` secrets used for publish
2. revoke any personal automation token that used to publish Paperclip
3. if npm offers a package-level setting to restrict publishing to trusted publishers, enable it
Goal:
- no long-lived npm publishing token should remain in GitHub Actions
## 4. Create GitHub Environments
Create two environments in the GitHub repository:
- `npm-canary`
- `npm-stable`
Path:
1. GitHub repository
2. `Settings`
3. `Environments`
4. `New environment`
## 5. Configure `npm-canary`
Recommended settings for `npm-canary`:
- environment name: `npm-canary`
- required reviewers: none
- wait timer: none
- deployment branches and tags:
- selected branches only
- allow `master`
Reasoning:
- every push to `master` should be able to publish a canary automatically
- no human approval should be required for canaries
## 6. Configure `npm-stable`
Recommended settings for `npm-stable`:
- environment name: `npm-stable`
- required reviewers: at least one maintainer other than the person triggering the workflow when possible
- prevent self-review: enabled
- admin bypass: disabled if your team can tolerate it
- wait timer: optional
- deployment branches and tags:
- selected branches only
- allow `master`
Reasoning:
- stable publishes should require an explicit human approval gate
- the workflow is manual, but the environment should still be the real control point
## 7. Protect `master`
Open the branch protection settings for `master`.
Recommended rules:
1. require pull requests before merging
2. require status checks to pass before merging
3. require review from code owners
4. dismiss stale approvals when new commits are pushed
5. restrict who can push directly to `master`
At minimum, make sure workflow and release script changes cannot land without review.
## 8. Enforce CODEOWNERS Review
This repo now includes `.github/CODEOWNERS`, but GitHub only enforces it if branch protection requires code owner reviews.
In branch protection for `master`, enable:
- `Require review from Code Owners`
Then verify the owner entries are correct for your actual maintainer set.
Current file:
- `.github/CODEOWNERS`
If `@dotta` is not the right reviewer identity in the public repo, change it before enabling enforcement.
## 9. Protect Release Infrastructure Specifically
These files should always trigger code owner review:
- `.github/workflows/release-canary.yml`
- `.github/workflows/release-stable.yml`
- `scripts/release.sh`
- `scripts/release-lib.sh`
- `scripts/release-package-map.mjs`
- `scripts/create-github-release.sh`
- `scripts/rollback-latest.sh`
- `doc/RELEASING.md`
- `doc/PUBLISHING.md`
If you want stronger controls, add a repository ruleset that explicitly blocks direct pushes to:
- `.github/workflows/**`
- `scripts/release*`
## 10. Do Not Store a Claude Token in GitHub Actions
Do not add a personal Claude or Anthropic token for automatic changelog generation.
Recommended policy:
- stable changelog generation happens locally from a trusted maintainer machine
- canaries never generate changelogs
This keeps LLM spending intentional and avoids a high-value token sitting in Actions.
## 11. Verify the Canary Workflow
After setup:
1. merge a harmless commit to `master`
2. open the `Release Canary` workflow run
3. confirm it passes verification
4. confirm publish succeeds under the `npm-canary` environment
5. confirm npm now shows a new `canary` release
6. confirm a git tag named `canary/vYYYY.M.D-canary.N` was pushed
Install-path check:
```bash
npx paperclipai@canary onboard
```
## 12. Verify the Stable Workflow
After at least one good canary exists:
1. prepare `releases/vYYYY.M.D.md` on the source commit you want to promote
2. open `Actions` -> `Release Stable`
3. run it with:
- `source_ref`: the tested commit SHA or canary tag source commit
- `stable_date`: leave blank or set the intended UTC date
- `dry_run`: `true`
4. confirm the dry-run succeeds
5. rerun with `dry_run: false`
6. approve the `npm-stable` environment when prompted
7. confirm npm `latest` points to the new stable version
8. confirm git tag `vYYYY.M.D` exists
9. confirm the GitHub Release was created
## 13. Suggested Maintainer Policy
Use this policy going forward:
- canaries are automatic and cheap
- stables are manual and approved
- only stables get public notes and announcements
- release notes are committed before stable publish
- rollback uses `npm dist-tag`, not unpublish
## 14. Troubleshooting
### Trusted publishing fails with an auth error
Check:
1. the workflow filename on GitHub exactly matches the filename configured in npm
2. the package has the trusted publisher entry for the correct repository
3. the job has `id-token: write`
4. the job is running from the expected repository, not a fork
### Stable workflow runs but never asks for approval
Check:
1. the `publish` job uses environment `npm-stable`
2. the environment actually has required reviewers configured
3. the workflow is running in the canonical repository, not a fork
### CODEOWNERS does not trigger
Check:
1. `.github/CODEOWNERS` is on the default branch
2. branch protection on `master` requires code owner review
3. the owner identities in the file are valid reviewers with repository access
## Related Docs
- [doc/RELEASING.md](RELEASING.md)
- [doc/PUBLISHING.md](PUBLISHING.md)
- [doc/plans/2026-03-17-release-automation-and-versioning.md](plans/2026-03-17-release-automation-and-versioning.md)

444
doc/RELEASING.md
View File

@@ -1,74 +1,66 @@
# Releasing Paperclip
Maintainer runbook for shipping a full Paperclip release across npm, GitHub, and the website-facing changelog surface.
Maintainer runbook for shipping Paperclip across npm, GitHub, and the website-facing changelog surface.
The release model is branch-driven:
The release model is now commit-driven:
1. Start a release train on `release/X.Y.Z`
2. Draft the stable changelog on that branch
3. Publish one or more canaries from that branch
4. Publish stable from that same branch head
5. Push the branch commit and tag
6. Create the GitHub Release
7. Merge `release/X.Y.Z` back to `master` without squash or rebase
1. Every push to `master` publishes a canary automatically.
2. Stable releases are manually promoted from a chosen tested commit or canary tag.
3. Stable release notes live in `releases/vYYYY.M.D.md`.
4. Only stable releases get GitHub Releases.
## Versioning Model
Paperclip uses calendar versions that still fit semver syntax:
- stable: `YYYY.M.D`
- canary: `YYYY.M.D-canary.N`
Examples:
- stable on March 17, 2026: `2026.3.17`
- fourth canary on March 17, 2026: `2026.3.17-canary.3`
Important constraints:
- do not use leading zeroes such as `2026.03.17`
- do not use four numeric segments such as `2026.03.17.1`
- the semver-safe canary form is `2026.3.17-canary.1`
## Release Surfaces
Every release has four separate surfaces:
Every stable release has four separate surfaces:
1. **Verification** — the exact git SHA passes typecheck, tests, and build
2. **npm** — `paperclipai` and public workspace packages are published
3. **GitHub** — the stable release gets a git tag and GitHub Release
4. **Website / announcements** — the stable changelog is published externally and announced
A release is done only when all four surfaces are handled.
A stable release is done only when all four surfaces are handled.
Canaries only cover the first two surfaces plus an internal traceability tag.
## Core Invariants
- Canary and stable for `X.Y.Z` must come from the same `release/X.Y.Z` branch.
- The release scripts must run from the matching `release/X.Y.Z` branch.
- Once `vX.Y.Z` exists locally, on GitHub, or on npm, that release train is frozen.
- Do not squash-merge or rebase-merge a release branch PR back to `master`.
- The stable changelog is always `releases/vX.Y.Z.md`. Never create canary changelog files.
The reason for the merge rule is simple: the tag must keep pointing at the exact published commit. Squash or rebase breaks that property.
- canaries publish from `master`
- stables publish from an explicitly chosen source ref
- tags point at the original source commit, not a generated release commit
- stable notes are always `releases/vYYYY.M.D.md`
- canaries never create GitHub Releases
- canaries never require changelog generation
## TL;DR
### 1. Start the release train
### Canary
Use this to compute the next version, create or resume the branch, create or resume a dedicated worktree, and push the branch to GitHub.
Every push to `master` runs [`.github/workflows/release-canary.yml`](../.github/workflows/release-canary.yml).
```bash
./scripts/release-start.sh patch
```
It:
That script:
- fetches the release remote and tags
- computes the next stable version from the latest `v*` tag
- creates or resumes `release/X.Y.Z`
- creates or resumes a dedicated worktree
- pushes the branch to the remote by default
- refuses to reuse a frozen release train
### 2. Draft the stable changelog
From the release worktree:
```bash
VERSION=X.Y.Z
claude --print --output-format stream-json --verbose --dangerously-skip-permissions --model claude-opus-4-6 "Use the release-changelog skill to draft or update releases/v${VERSION}.md for Paperclip. Read doc/RELEASING.md and .agents/skills/release-changelog/SKILL.md, then generate the stable changelog for v${VERSION} from commits since the last stable tag. Do not create a canary changelog."
```
### 3. Verify and publish a canary
```bash
./scripts/release-preflight.sh canary patch
./scripts/release.sh patch --canary --dry-run
./scripts/release.sh patch --canary
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
```
- verifies the pushed commit
- computes the canary version for the current UTC date
- publishes under npm dist-tag `canary`
- creates a git tag `canary/vYYYY.M.D-canary.N`
Users install canaries with:
@@ -76,145 +68,91 @@ Users install canaries with:
npx paperclipai@canary onboard
```
### 4. Publish stable
### Stable
Use [`.github/workflows/release-stable.yml`](../.github/workflows/release-stable.yml) from the Actions tab.
Inputs:
- `source_ref`
- commit SHA, branch, or tag
- `stable_date`
- optional UTC date override in `YYYY-MM-DD`
- `dry_run`
- preview only when true
Before running stable:
1. pick the canary commit or tag you trust
2. create or update `releases/vYYYY.M.D.md` on that source ref
3. run the stable workflow from that source ref
The workflow:
- re-verifies the exact source ref
- publishes `YYYY.M.D` under npm dist-tag `latest`
- creates git tag `vYYYY.M.D`
- creates or updates the GitHub Release from `releases/vYYYY.M.D.md`
## Local Commands
### Preview a canary locally
```bash
./scripts/release-preflight.sh stable patch
./scripts/release.sh patch --dry-run
./scripts/release.sh patch
git push public-gh HEAD --follow-tags
./scripts/create-github-release.sh X.Y.Z
./scripts/release.sh canary --dry-run
```
Then open a PR from `release/X.Y.Z` to `master` and merge without squash or rebase.
## Release Branches
Paperclip uses one release branch per target stable version:
- `release/0.3.0`
- `release/0.3.1`
- `release/1.0.0`
Do not create separate per-canary branches like `canary/0.3.0-1`. A canary is just a prerelease snapshot of the same stable train.
## Script Entry Points
- [`scripts/release-start.sh`](../scripts/release-start.sh) — create or resume the release train branch/worktree
- [`scripts/release-preflight.sh`](../scripts/release-preflight.sh) — validate branch, version plan, git/npm state, and verification gate
- [`scripts/release.sh`](../scripts/release.sh) — publish canary or stable from the release branch
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh) — create or update the GitHub Release after pushing the tag
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh) — repoint `latest` to the last good stable version
## Detailed Workflow
### 1. Start or resume the release train
Run:
### Preview a stable locally
```bash
./scripts/release-start.sh <patch|minor|major>
./scripts/release.sh stable --dry-run
```
Useful options:
### Publish a stable locally
This is mainly for emergency/manual use. The normal path is the GitHub workflow.
```bash
./scripts/release-start.sh patch --dry-run
./scripts/release-start.sh minor --worktree-dir ../paperclip-release-0.4.0
./scripts/release-start.sh patch --no-push
./scripts/release.sh stable
git push public-gh refs/tags/vYYYY.M.D
./scripts/create-github-release.sh YYYY.M.D
```
The script is intentionally idempotent:
## Stable Changelog Workflow
- if `release/X.Y.Z` already exists locally, it reuses it
- if the branch already exists on the remote, it resumes it locally
- if the branch is already checked out in another worktree, it points you there
- if `vX.Y.Z` already exists locally, remotely, or on npm, it refuses to reuse that train
Stable changelog files live at:
### 2. Write the stable changelog early
- `releases/vYYYY.M.D.md`
Create or update:
Canaries do not get changelog files.
- `releases/vX.Y.Z.md`
That file is for the eventual stable release. It should not include `-canary` in the filename or heading.
Recommended structure:
- `Breaking Changes` when needed
- `Highlights`
- `Improvements`
- `Fixes`
- `Upgrade Guide` when needed
- `Contributors` — @-mention every contributor by GitHub username (no emails)
Package-level `CHANGELOG.md` files are generated as part of the release mechanics. They are not the main release narrative.
### 3. Run release preflight
From the `release/X.Y.Z` worktree:
Recommended local generation flow:
```bash
./scripts/release-preflight.sh canary <patch|minor|major>
# or
./scripts/release-preflight.sh stable <patch|minor|major>
VERSION=2026.3.17
claude --print --output-format stream-json --verbose --dangerously-skip-permissions --model claude-opus-4-6 "Use the release-changelog skill to draft or update releases/v${VERSION}.md for Paperclip. Read doc/RELEASING.md and .agents/skills/release-changelog/SKILL.md, then generate the stable changelog for v${VERSION} from commits since the last stable tag. Do not create a canary changelog."
```
The preflight script now checks all of the following before it runs the verification gate:
The repo intentionally does not run this through GitHub Actions because:
- the worktree is clean, including untracked files
- the current branch matches the computed `release/X.Y.Z`
- the release train is not frozen
- the target version is still free on npm
- the target tag does not already exist locally or remotely
- whether the remote release branch already exists
- whether `releases/vX.Y.Z.md` is present
- canaries are too frequent
- stable notes are the only public narrative surface that needs LLM help
- maintainer LLM tokens should not live in Actions
Then it runs:
## Smoke Testing
```bash
pnpm -r typecheck
pnpm test:run
pnpm build
```
### 4. Publish one or more canaries
Run:
```bash
./scripts/release.sh <patch|minor|major> --canary --dry-run
./scripts/release.sh <patch|minor|major> --canary
```
Result:
- npm gets a prerelease such as `1.2.3-canary.0` under dist-tag `canary`
- `latest` is unchanged
- no git tag is created
- no GitHub Release is created
- the worktree returns to clean after the script finishes
Guardrails:
- the script refuses to run from the wrong branch
- the script refuses to publish from a frozen train
- the canary is always derived from the next stable version
- if the stable notes file is missing, the script warns before you forget it
Concrete example:
- if the latest stable is `0.2.7`, a patch canary targets `0.2.8-canary.0`
- `0.2.7-canary.N` is invalid because `0.2.7` is already stable
### 5. Smoke test the canary
Run the actual install path in Docker:
For a canary:
```bash
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
```
For the current stable:
```bash
PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
```
Useful isolated variants:
```bash
@@ -222,14 +160,6 @@ HOST_PORT=3232 DATA_DIR=./data/release-smoke-canary PAPERCLIPAI_VERSION=canary .
HOST_PORT=3233 DATA_DIR=./data/release-smoke-stable PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
```
If you want to exercise onboarding from the current committed ref instead of npm, use:
```bash
./scripts/clean-onboard-ref.sh
PAPERCLIP_PORT=3234 ./scripts/clean-onboard-ref.sh
./scripts/clean-onboard-ref.sh HEAD
```
Minimum checks:
- `npx paperclipai@canary onboard` installs
@@ -238,185 +168,59 @@ Minimum checks:
- the UI loads
- basic company creation and dashboard load work
If smoke testing fails:
## Rollback
1. stop the stable release
2. fix the issue on the same `release/X.Y.Z` branch
3. publish another canary
4. rerun smoke testing
Rollback does not unpublish versions.
### 6. Publish stable from the same release branch
Once the branch head is vetted, run:
It only moves the `latest` dist-tag back to a previous stable:
```bash
./scripts/release.sh <patch|minor|major> --dry-run
./scripts/release.sh <patch|minor|major>
./scripts/rollback-latest.sh 2026.3.16 --dry-run
./scripts/rollback-latest.sh 2026.3.16
```
Stable publish:
- publishes `X.Y.Z` to npm under `latest`
- creates the local release commit
- creates the local tag `vX.Y.Z`
Stable publish refuses to proceed if:
- the current branch is not `release/X.Y.Z`
- the remote release branch does not exist yet
- the stable notes file is missing
- the target tag already exists locally or remotely
- the stable version already exists on npm
Those checks intentionally freeze the train after stable publish.
### 7. Push the stable branch commit and tag
After stable publish succeeds:
```bash
git push public-gh HEAD --follow-tags
./scripts/create-github-release.sh X.Y.Z
```
The GitHub Release notes come from:
- `releases/vX.Y.Z.md`
### 8. Merge the release branch back to `master`
Open a PR:
- base: `master`
- head: `release/X.Y.Z`
Merge rule:
- allowed: merge commit or fast-forward
- forbidden: squash merge
- forbidden: rebase merge
Post-merge verification:
```bash
git fetch public-gh --tags
git merge-base --is-ancestor "vX.Y.Z" "public-gh/master"
```
That command must succeed. If it fails, the published tagged commit is not reachable from `master`, which means the merge strategy was wrong.
### 9. Finish the external surfaces
After GitHub is correct:
- publish the changelog on the website
- write and send the announcement copy
- ensure public docs and install guidance point to the stable version
## GitHub Actions Release
There is also a manual workflow at [`.github/workflows/release.yml`](../.github/workflows/release.yml).
Use it from the Actions tab on the relevant `release/X.Y.Z` branch:
1. Choose `Release`
2. Choose `channel`: `canary` or `stable`
3. Choose `bump`: `patch`, `minor`, or `major`
4. Choose whether this is a `dry_run`
5. Run it from the release branch, not from `master`
The workflow:
- reruns `typecheck`, `test:run`, and `build`
- gates publish behind the `npm-release` environment
- can publish canaries without touching `latest`
- can publish stable, push the stable branch commit and tag, and create the GitHub Release
It does not merge the release branch back to `master` for you.
## Release Checklist
### Before any publish
- [ ] The release train exists on `release/X.Y.Z`
- [ ] The working tree is clean, including untracked files
- [ ] If package manifests changed, the CI-owned `pnpm-lock.yaml` refresh is already merged on `master` before the train is cut
- [ ] The required verification gate passed on the exact branch head you want to publish
- [ ] The bump type is correct for the user-visible impact
- [ ] The stable changelog file exists or is ready at `releases/vX.Y.Z.md`
- [ ] You know which previous stable version you would roll back to if needed
### Before a stable
- [ ] The candidate has already passed smoke testing
- [ ] The remote `release/X.Y.Z` branch exists
- [ ] You are ready to push the stable branch commit and tag immediately after npm publish
- [ ] You are ready to create the GitHub Release immediately after the push
- [ ] You are ready to open the PR back to `master`
### After a stable
- [ ] `npm view paperclipai@latest version` matches the new stable version
- [ ] The git tag exists on GitHub
- [ ] The GitHub Release exists and uses `releases/vX.Y.Z.md`
- [ ] `vX.Y.Z` is reachable from `master`
- [ ] The website changelog is updated
- [ ] Announcement copy matches the stable release, not the canary
Then fix forward with a new stable release date.
## Failure Playbooks
### If the canary publishes but the smoke test fails
### If the canary publishes but smoke testing fails
Do not publish stable.
Do not run stable.
Instead:
1. fix the issue on `release/X.Y.Z`
2. publish another canary
3. rerun smoke testing
1. fix the issue on `master`
2. merge the fix
3. wait for the next automatic canary
4. rerun smoke testing
### If stable npm publish succeeds but push or GitHub release creation fails
### If stable npm publish succeeds but tag push or GitHub release creation fails
This is a partial release. npm is already live.
Do this immediately:
1. fix the git or GitHub issue from the same checkout
2. push the stable branch commit and tag
3. create the GitHub Release
1. push the missing tag
2. rerun `./scripts/create-github-release.sh YYYY.M.D`
3. verify the GitHub Release notes point at `releases/vYYYY.M.D.md`
Do not republish the same version.
### If `latest` is broken after stable publish
Preview:
Roll back the dist-tag:
```bash
./scripts/rollback-latest.sh X.Y.Z --dry-run
./scripts/rollback-latest.sh YYYY.M.D
```
Roll back:
Then fix forward with a new stable release.
```bash
./scripts/rollback-latest.sh X.Y.Z
```
## Related Files
This does not unpublish anything. It only moves the `latest` dist-tag back to the last good stable release.
Then fix forward with a new patch release.
### If the GitHub Release notes are wrong
Re-run:
```bash
./scripts/create-github-release.sh X.Y.Z
```
If the release already exists, the script updates it.
## Related Docs
- [doc/PUBLISHING.md](PUBLISHING.md) — low-level npm build and packaging internals
- [.agents/skills/release/SKILL.md](../.agents/skills/release/SKILL.md) — maintainer release coordination workflow
- [.agents/skills/release-changelog/SKILL.md](../.agents/skills/release-changelog/SKILL.md) — stable changelog drafting workflow
- [`scripts/release.sh`](../scripts/release.sh)
- [`scripts/release-package-map.mjs`](../scripts/release-package-map.mjs)
- [`scripts/create-github-release.sh`](../scripts/create-github-release.sh)
- [`scripts/rollback-latest.sh`](../scripts/rollback-latest.sh)
- [`doc/PUBLISHING.md`](PUBLISHING.md)
- [`doc/RELEASE-AUTOMATION-SETUP.md`](RELEASE-AUTOMATION-SETUP.md)

494
doc/plans/2026-03-17-release-automation-and-versioning.md Normal file
View File

@@ -0,0 +1,494 @@
# Release Automation and Versioning Simplification Plan
## Context
Paperclip's current release flow is documented in `doc/RELEASING.md` and implemented through:
- `.github/workflows/release.yml`
- `scripts/release-lib.sh`
- `scripts/release-start.sh`
- `scripts/release-preflight.sh`
- `scripts/release.sh`
- `scripts/create-github-release.sh`
Today the model is:
1. pick `patch`, `minor`, or `major`
2. create `release/X.Y.Z`
3. draft `releases/vX.Y.Z.md`
4. publish one or more canaries from that release branch
5. publish stable from that same branch
6. push tag + create GitHub Release
7. merge the release branch back to `master`
That is workable, but it creates friction in exactly the places that should be cheap:
- deciding `patch` vs `minor` vs `major`
- cutting and carrying release branches
- manually publishing canaries
- thinking about changelog generation for canaries
- handling npm credentials safely in a public repo
The target state from this discussion is simpler:
- every push to `master` publishes a canary automatically
- stable releases are promoted deliberately from a vetted commit
- versioning is date-driven instead of semantics-driven
- stable publishing is secure even in a public open-source repository
- changelog generation happens only for real stable releases
## Recommendation In One Sentence
Move Paperclip to semver-compatible calendar versioning, auto-publish canaries from `master`, promote stable from a chosen tested commit, and use npm trusted publishing plus GitHub environments so no long-lived npm or LLM token needs to live in Actions.
## Core Decisions
### 1. Use calendar versions, but keep semver syntax
The repo and npm tooling still assume semver-shaped version strings in many places. That does not mean Paperclip must keep semver as a product policy. It does mean the version format should remain semver-valid.
Recommended format:
- stable: `YYYY.M.D`
- canary: `YYYY.M.D-canary.N`
Examples:
- stable on March 17, 2026: `2026.3.17`
- third canary on March 17, 2026: `2026.3.17-canary.2`
Why this shape:
- it removes `patch/minor/major` decisions
- it is valid semver syntax
- it stays compatible with npm, dist-tags, and existing semver validators
- it is close to the format you actually want
Important constraints:
- `2026.03.17` is not the format to use
- numeric semver identifiers do not allow leading zeroes
- `2026.03.16.8` is not the format to use
- semver has three numeric components, not four
- the practical semver-safe equivalent of your example is `2026.3.16-canary.8`
This is effectively CalVer on semver rails.
### 2. Accept that CalVer changes the compatibility contract
This is not semver in spirit anymore. It is semver in syntax only.
That tradeoff is probably acceptable for Paperclip, but it should be explicit:
- consumers no longer infer compatibility from `major/minor/patch`
- release notes become the compatibility signal
- downstream users should prefer exact pins or deliberate upgrades
This is especially relevant for public library packages like `@paperclipai/shared`, `@paperclipai/db`, and the adapter packages.
### 3. Drop release branches for normal publishing
If every merge to `master` publishes a canary, the current `release/X.Y.Z` train model becomes more ceremony than value.
Recommended replacement:
- `master` is the only canary train
- every push to `master` can publish a canary
- stable is published from a chosen commit or canary tag on `master`
This matches the workflow you actually want:
- merge continuously
- let npm always have a fresh canary
- choose a known-good canary later and promote that commit to stable
### 4. Promote by source ref, not by "renaming" a canary
This is the most important mechanical constraint.
npm can move dist-tags, but it does not let you rename an already-published version. That means:
- you can move `latest` to `paperclipai@1.2.3`
- you cannot turn `paperclipai@2026.3.16-canary.8` into `paperclipai@2026.3.17`
So "promote canary to stable" really means:
1. choose the commit or canary tag you trust
2. rebuild from that exact commit
3. publish it again with the stable version string
Because of that, the stable workflow should take a source ref, not just a bump type.
Recommended stable input:
- `source_ref`
- commit SHA, or
- a canary git tag such as `canary/v2026.3.16-canary.8`
### 5. Only stable releases get release notes, tags, and GitHub Releases
Canaries should stay lightweight:
- publish to npm under `canary`
- optionally create a lightweight or annotated git tag
- do not create GitHub Releases
- do not require `releases/v*.md`
- do not spend LLM tokens
Stable releases should remain the public narrative surface:
- git tag `v2026.3.17`
- GitHub Release `v2026.3.17`
- stable changelog file `releases/v2026.3.17.md`
## Security Model
### Recommendation
Use npm trusted publishing with GitHub Actions OIDC, then disable token-based publishing access for the packages.
Why:
- no long-lived `NPM_TOKEN` in repo or org secrets
- no personal npm token in Actions
- short-lived credentials minted only for the authorized workflow
- automatic npm provenance for public packages in public repos
This is the cleanest answer to the open-repo security concern.
### Concrete controls
#### 1. Split canary and stable into separate workflow files
Do not use one workflow file for both.
Recommended:
- `.github/workflows/release-canary.yml`
- `.github/workflows/release-stable.yml`
Why:
- npm trusted publishing is configured per workflow filename
- canary and stable need different blast radii
- stable should have stronger GitHub environment rules than canary
#### 2. Use separate GitHub environments
Recommended environments:
- `npm-canary`
- `npm-stable`
Recommended policy:
- `npm-canary`
- allowed branch: `master`
- no human reviewer required
- `npm-stable`
- allowed branch: `master`
- required reviewer enabled
- prevent self-review enabled
- admin bypass disabled
Stable should require an explicit second human gate even if the workflow is manually dispatched.
#### 3. Lock down workflow edits
Add or tighten `CODEOWNERS` coverage for:
- `.github/workflows/*`
- `scripts/release*`
- `doc/RELEASING.md`
This matters because trusted publishing authorizes a workflow file. The biggest remaining risk is not secret exfiltration from forks. It is a maintainer-approved change to the release workflow itself.
#### 4. Remove traditional npm token access after OIDC works
After trusted publishing is verified:
- set package publishing access to require 2FA and disallow tokens
- revoke any legacy automation tokens
That eliminates the "someone stole the npm token" class of failure.
### What not to do
- do not put your personal Claude or npm token in GitHub Actions
- do not run release logic from `pull_request_target`
- do not make stable publishing depend on a repo secret if OIDC can handle it
- do not create canary GitHub Releases
## Changelog Strategy
### Recommendation
Generate stable changelogs only, and keep LLM-assisted changelog generation out of CI for now.
Reasoning:
- canaries happen too often
- canaries do not need polished public notes
- putting a personal Claude token into Actions is not worth the risk
- stable release cadence is low enough that a human-in-the-loop step is acceptable
Recommended stable path:
1. pick a canary commit or tag
2. run changelog generation locally from a trusted machine
3. commit `releases/vYYYY.M.D.md`
4. run stable promotion
If the notes are not ready yet, a fallback is acceptable:
- publish stable
- create a minimal GitHub Release
- update `releases/vYYYY.M.D.md` immediately afterward
But the better steady-state is to have the stable notes committed before stable publish.
### Future option
If you later want CI-assisted changelog drafting, do it with:
- a dedicated service account
- a token scoped only for changelog generation
- a manual workflow
- a dedicated environment with required reviewers
That is phase-two hardening work, not a phase-one requirement.
## Proposed Future Workflow
### Canary workflow
Trigger:
- `push` on `master`
Steps:
1. checkout the merged `master` commit
2. run verification on that exact commit
3. compute canary version for current UTC date
4. version public packages to `YYYY.M.D-canary.N`
5. publish to npm with dist-tag `canary`
6. create a canary git tag for traceability
Recommended canary tag format:
- `canary/v2026.3.17-canary.4`
Outputs:
- npm canary published
- git tag created
- no GitHub Release
- no changelog file required
### Stable workflow
Trigger:
- `workflow_dispatch`
Inputs:
- `source_ref`
- optional `stable_date`
- `dry_run`
Steps:
1. checkout `source_ref`
2. run verification on that exact commit
3. compute stable version from UTC date or provided override
4. fail if `vYYYY.M.D` already exists
5. require `releases/vYYYY.M.D.md`
6. version public packages to `YYYY.M.D`
7. publish to npm under `latest`
8. create git tag `vYYYY.M.D`
9. push tag
10. create GitHub Release from `releases/vYYYY.M.D.md`
Outputs:
- stable npm release
- stable git tag
- GitHub Release
- clean public changelog surface
## Implementation Guidance
### 1. Replace bump-type version math with explicit version computation
The current release scripts depend on:
- `patch`
- `minor`
- `major`
That logic should be replaced with:
- `compute_canary_version_for_date`
- `compute_stable_version_for_date`
For example:
- `stable_version_for_utc_date(2026-03-17) -> 2026.3.17`
- `next_canary_for_utc_date(2026-03-17) -> 2026.3.17-canary.0`
### 2. Stop requiring `release/X.Y.Z`
These current invariants should be removed from the happy path:
- "must run from branch `release/X.Y.Z`"
- "stable and canary for `X.Y.Z` come from the same release branch"
- `release-start.sh`
Replace them with:
- canary must run from `master`
- stable may run from a pinned `source_ref`
### 3. Keep Changesets only if it stays helpful
The current system uses Changesets to:
- rewrite package versions
- maintain package-level `CHANGELOG.md` files
- publish packages
With CalVer, Changesets may still be useful for publish orchestration, but it should no longer own version selection.
Recommended implementation order:
1. keep `changeset publish` if it works with explicitly-set versions
2. replace version computation with a small explicit versioning script
3. if Changesets keeps fighting the model, remove it from release publishing entirely
Paperclip's release problem is now "publish the whole fixed package set at one explicit version", not "derive the next semantic bump from human intent".
### 4. Add a dedicated versioning script
Recommended new script:
- `scripts/set-release-version.mjs`
Responsibilities:
- set the version in all public publishable packages
- update any internal exact-version references needed for publishing
- update CLI version strings
- avoid broad string replacement across unrelated files
This is safer than keeping a bump-oriented changeset flow and then forcing it into a date-based scheme.
### 5. Keep rollback based on dist-tags
`rollback-latest.sh` should stay, but it should stop assuming a semver meaning beyond syntax.
It should continue to:
- repoint `latest` to a prior stable version
- never unpublish
## Tradeoffs and Risks
### 1. One stable per UTC day
With plain `YYYY.M.D`, you get one stable release per UTC day.
That is probably fine, but it is a real product rule.
If you need multiple same-day stables later, you have three options:
1. accept a less pretty stable format
2. go back to a serial patch component
3. keep daily stable cadence and use canaries for same-day fixes
My recommendation is to accept one stable per UTC day unless reality proves otherwise.
### 2. Public package consumers lose semver intent signaling
This is the main downside of CalVer.
If that becomes a problem, one alternative is:
- use CalVer for the CLI package only
- keep semver for library packages
That is more complex operationally, so I would not start there unless package consumers actually need it.
### 3. Auto-canary means more publish traffic
Publishing on every `master` merge means:
- more npm versions
- more git tags
- more registry noise
That is acceptable if canaries stay clearly separate:
- npm dist-tag `canary`
- no GitHub Release
- no external announcement
## Rollout Plan
### Phase 1: Security foundation
1. Create `release-canary.yml` and `release-stable.yml`
2. Configure npm trusted publishers for all public packages
3. Create `npm-canary` and `npm-stable` environments
4. Add `CODEOWNERS` protection for release files
5. Verify OIDC publishing works
6. Disable token-based publishing access and revoke old tokens
### Phase 2: Canary automation
1. Add canary workflow on `push` to `master`
2. Add explicit calendar-version computation
3. Add canary git tagging
4. Remove changelog requirement from canaries
5. Update `doc/RELEASING.md`
### Phase 3: Stable promotion
1. Add manual stable workflow with `source_ref`
2. Require stable notes file
3. Publish stable + tag + GitHub Release
4. Update rollback docs and scripts
5. Retire release-branch assumptions
### Phase 4: Cleanup
1. Remove `release-start.sh` from the primary path
2. Remove `patch/minor/major` from maintainer docs
3. Decide whether to keep or remove Changesets from publishing
4. Document the CalVer compatibility contract publicly
## Concrete Recommendation
Paperclip should adopt this model:
- stable versions: `YYYY.M.D`
- canary versions: `YYYY.M.D-canary.N`
- canaries auto-published on every push to `master`
- stables manually promoted from a chosen tested commit or canary tag
- no release branches in the default path
- no canary changelog files
- no canary GitHub Releases
- no Claude token in GitHub Actions
- no npm automation token in GitHub Actions
- npm trusted publishing plus GitHub environments for release security
That gets rid of the annoying part of semver without fighting npm, makes canaries cheap, keeps stables deliberate, and materially improves the security posture of the public repository.
## External References
- npm trusted publishing: https://docs.npmjs.com/trusted-publishers/
- npm dist-tags: https://docs.npmjs.com/adding-dist-tags-to-packages/
- npm semantic versioning guidance: https://docs.npmjs.com/about-semantic-versioning/
- GitHub environments and deployment protection rules: https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments
- GitHub secrets behavior for forks: https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-secrets