Merge pull request #1216 from paperclipai/split/release-smoke-calver
Release smoke workflow and CalVer patch-slot updates
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: release-changelog
|
||||
description: >
|
||||
Generate the stable Paperclip release changelog at releases/v{version}.md by
|
||||
Generate the stable Paperclip release changelog at releases/vYYYY.MDD.P.md by
|
||||
reading commits, changesets, and merged PR context since the last stable tag.
|
||||
---
|
||||
|
||||
@@ -9,20 +9,33 @@ description: >
|
||||
|
||||
Generate the user-facing changelog for the **stable** Paperclip release.
|
||||
|
||||
## Versioning Model
|
||||
|
||||
Paperclip uses **calendar versioning (calver)**:
|
||||
|
||||
- Stable releases: `YYYY.MDD.P` (e.g. `2026.318.0`)
|
||||
- Canary releases: `YYYY.MDD.P-canary.N` (e.g. `2026.318.1-canary.0`)
|
||||
- Git tags: `vYYYY.MDD.P` for stable, `canary/vYYYY.MDD.P-canary.N` for canary
|
||||
|
||||
There are no major/minor/patch bumps. The stable version is derived from the
|
||||
intended release date (UTC) plus the next same-day stable patch slot.
|
||||
|
||||
Output:
|
||||
|
||||
- `releases/v{version}.md`
|
||||
- `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Important rule:
|
||||
Important rules:
|
||||
|
||||
- even if there are canary releases such as `1.2.3-canary.0`, the changelog file stays `releases/v1.2.3.md`
|
||||
- even if there are canary releases such as `2026.318.1-canary.0`, the changelog file stays `releases/v2026.318.1.md`
|
||||
- do not derive versions from semver bump types
|
||||
- do not create canary changelog files
|
||||
|
||||
## Step 0 — Idempotency Check
|
||||
|
||||
Before generating anything, check whether the file already exists:
|
||||
|
||||
```bash
|
||||
ls releases/v{version}.md 2>/dev/null
|
||||
ls releases/vYYYY.MDD.P.md 2>/dev/null
|
||||
```
|
||||
|
||||
If it exists:
|
||||
@@ -41,13 +54,14 @@ git tag --list 'v*' --sort=-version:refname | head -1
|
||||
git log v{last}..HEAD --oneline --no-merges
|
||||
```
|
||||
|
||||
The planned stable version comes from one of:
|
||||
The stable version comes from one of:
|
||||
|
||||
- an explicit maintainer request
|
||||
- the chosen bump type applied to the last stable tag
|
||||
- `./scripts/release.sh stable --date YYYY-MM-DD --print-version`
|
||||
- the release plan already agreed in `doc/RELEASING.md`
|
||||
|
||||
Do not derive the changelog version from a canary tag or prerelease suffix.
|
||||
Do not derive major/minor/patch bumps from API intent — calver uses the date and same-day stable slot.
|
||||
|
||||
## Step 2 — Gather the Raw Inputs
|
||||
|
||||
@@ -73,7 +87,6 @@ Look for:
|
||||
- destructive migrations
|
||||
- removed or changed API fields/endpoints
|
||||
- renamed or removed config keys
|
||||
- `major` changesets
|
||||
- `BREAKING:` or `BREAKING CHANGE:` commit signals
|
||||
|
||||
Key commands:
|
||||
@@ -85,7 +98,8 @@ git diff v{last}..HEAD -- server/src/routes/ server/src/api/
|
||||
git log v{last}..HEAD --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true
|
||||
```
|
||||
|
||||
If the requested bump is lower than the minimum required bump, flag that before the release proceeds.
|
||||
If breaking changes are detected, flag them prominently — they must appear in the
|
||||
Breaking Changes section with an upgrade path.
|
||||
|
||||
## Step 4 — Categorize for Users
|
||||
|
||||
@@ -130,9 +144,9 @@ Rules:
|
||||
Template:
|
||||
|
||||
```markdown
|
||||
# v{version}
|
||||
# vYYYY.MDD.P
|
||||
|
||||
> Released: {YYYY-MM-DD}
|
||||
> Released: YYYY-MM-DD
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
|
||||
@@ -2,23 +2,21 @@
|
||||
name: release
|
||||
description: >
|
||||
Coordinate a full Paperclip release across engineering verification, npm,
|
||||
GitHub, website publishing, and announcement follow-up. Use when leadership
|
||||
asks to ship a release, not merely to discuss version bumps.
|
||||
GitHub, smoke testing, and announcement follow-up. Use when leadership asks
|
||||
to ship a release, not merely to discuss versioning.
|
||||
---
|
||||
|
||||
# Release Coordination Skill
|
||||
|
||||
Run the full Paperclip release as a maintainer workflow, not just an npm publish.
|
||||
Run the full Paperclip maintainer release workflow, not just an npm publish.
|
||||
|
||||
This skill coordinates:
|
||||
|
||||
- stable changelog drafting via `release-changelog`
|
||||
- release-train setup via `scripts/release-start.sh`
|
||||
- prerelease canary publishing via `scripts/release.sh --canary`
|
||||
- canary verification and publish status from `master`
|
||||
- Docker smoke testing via `scripts/docker-onboard-smoke.sh`
|
||||
- stable publishing via `scripts/release.sh`
|
||||
- pushing the stable branch commit and tag
|
||||
- GitHub Release creation via `scripts/create-github-release.sh`
|
||||
- manual stable promotion from a chosen source ref
|
||||
- GitHub Release creation
|
||||
- website / announcement follow-up tasks
|
||||
|
||||
## Trigger
|
||||
@@ -26,8 +24,9 @@ This skill coordinates:
|
||||
Use this skill when leadership asks for:
|
||||
|
||||
- "do a release"
|
||||
- "ship the next patch/minor/major"
|
||||
- "release vX.Y.Z"
|
||||
- "ship the release"
|
||||
- "promote this canary to stable"
|
||||
- "cut the stable release"
|
||||
|
||||
## Preconditions
|
||||
|
||||
@@ -35,10 +34,10 @@ Before proceeding, verify all of the following:
|
||||
|
||||
1. `.agents/skills/release-changelog/SKILL.md` exists and is usable.
|
||||
2. The repo working tree is clean, including untracked files.
|
||||
3. There are commits since the last stable tag.
|
||||
4. The release SHA has passed the verification gate or is about to.
|
||||
5. If package manifests changed, the CI-owned `pnpm-lock.yaml` refresh is already merged on `master` before the release branch is cut.
|
||||
6. npm publish rights are available locally, or the GitHub release workflow is being used with trusted publishing.
|
||||
3. There is at least one canary or candidate commit since the last stable tag.
|
||||
4. The candidate SHA has passed the verification gate or is about to.
|
||||
5. If manifests changed, the CI-owned `pnpm-lock.yaml` refresh is already merged on `master`.
|
||||
6. npm publish rights are available through GitHub trusted publishing, or through local npm auth for emergency/manual use.
|
||||
7. If running through Paperclip, you have issue context for status updates and follow-up task creation.
|
||||
|
||||
If any precondition fails, stop and report the blocker.
|
||||
@@ -47,78 +46,67 @@ If any precondition fails, stop and report the blocker.
|
||||
|
||||
Collect these inputs up front:
|
||||
|
||||
- requested bump: `patch`, `minor`, or `major`
|
||||
- whether this run is a dry run or live release
|
||||
- whether the release is being run locally or from GitHub Actions
|
||||
- whether the target is a canary check or a stable promotion
|
||||
- the candidate `source_ref` for stable
|
||||
- whether the stable run is dry-run or live
|
||||
- release issue / company context for website and announcement follow-up
|
||||
|
||||
## Step 0 — Release Model
|
||||
|
||||
Paperclip now uses this release model:
|
||||
Paperclip now uses a commit-driven release model:
|
||||
|
||||
1. Start or resume `release/X.Y.Z`
|
||||
2. Draft the **stable** changelog as `releases/vX.Y.Z.md`
|
||||
3. Publish one or more **prerelease canaries** such as `X.Y.Z-canary.0`
|
||||
4. Smoke test the canary via Docker
|
||||
5. Publish the stable version `X.Y.Z`
|
||||
6. Push the stable branch commit and tag
|
||||
7. Create the GitHub Release
|
||||
8. Merge `release/X.Y.Z` back to `master` without squash or rebase
|
||||
9. Complete website and announcement surfaces
|
||||
1. every push to `master` publishes a canary automatically
|
||||
2. canaries use `YYYY.MDD.P-canary.N`
|
||||
3. stable releases use `YYYY.MDD.P`
|
||||
4. the middle slot is `MDD`, where `M` is the UTC month and `DD` is the zero-padded UTC day
|
||||
5. the stable patch slot increments when more than one stable ships on the same UTC date
|
||||
6. stable releases are manually promoted from a chosen tested commit or canary source commit
|
||||
7. only stable releases get `releases/vYYYY.MDD.P.md`, git tag `vYYYY.MDD.P`, and a GitHub Release
|
||||
|
||||
Critical consequence:
|
||||
Critical consequences:
|
||||
|
||||
- Canaries do **not** use promote-by-dist-tag anymore.
|
||||
- The changelog remains stable-only. Do not create `releases/vX.Y.Z-canary.N.md`.
|
||||
- do not use release branches as the default path
|
||||
- do not derive major/minor/patch bumps
|
||||
- do not create canary changelog files
|
||||
- do not create canary GitHub Releases
|
||||
|
||||
## Step 1 — Decide the Stable Version
|
||||
## Step 1 — Choose the Candidate
|
||||
|
||||
Start the release train first:
|
||||
For canary validation:
|
||||
|
||||
- inspect the latest successful canary run on `master`
|
||||
- record the canary version and source SHA
|
||||
|
||||
For stable promotion:
|
||||
|
||||
1. choose the tested source ref
|
||||
2. confirm it is the exact SHA you want to promote
|
||||
3. resolve the target stable version with `./scripts/release.sh stable --date YYYY-MM-DD --print-version`
|
||||
|
||||
Useful commands:
|
||||
|
||||
```bash
|
||||
./scripts/release-start.sh {patch|minor|major}
|
||||
git tag --list 'v*' --sort=-version:refname | head -1
|
||||
git log --oneline --no-merges
|
||||
npm view paperclipai@canary version
|
||||
```
|
||||
|
||||
Then run release preflight:
|
||||
|
||||
```bash
|
||||
./scripts/release-preflight.sh canary {patch|minor|major}
|
||||
# or
|
||||
./scripts/release-preflight.sh stable {patch|minor|major}
|
||||
```
|
||||
|
||||
Then use the last stable tag as the base:
|
||||
|
||||
```bash
|
||||
LAST_TAG=$(git tag --list 'v*' --sort=-version:refname | head -1)
|
||||
git log "${LAST_TAG}..HEAD" --oneline --no-merges
|
||||
git diff --name-only "${LAST_TAG}..HEAD" -- packages/db/src/migrations/
|
||||
git diff "${LAST_TAG}..HEAD" -- packages/db/src/schema/
|
||||
git log "${LAST_TAG}..HEAD" --format="%s" | rg -n 'BREAKING CHANGE|BREAKING:|^[a-z]+!:' || true
|
||||
```
|
||||
|
||||
Bump policy:
|
||||
|
||||
- destructive migrations, removed APIs, breaking config changes -> `major`
|
||||
- additive migrations or clearly user-visible features -> at least `minor`
|
||||
- fixes only -> `patch`
|
||||
|
||||
If the requested bump is too low, escalate it and explain why.
|
||||
|
||||
## Step 2 — Draft the Stable Changelog
|
||||
|
||||
Invoke `release-changelog` and generate:
|
||||
Stable changelog files live at:
|
||||
|
||||
- `releases/vX.Y.Z.md`
|
||||
- `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Invoke `release-changelog` and generate or update the stable notes only.
|
||||
|
||||
Rules:
|
||||
|
||||
- review the draft with a human before publish
|
||||
- preserve manual edits if the file already exists
|
||||
- keep the heading and filename stable-only, for example `v1.2.3`
|
||||
- do not create a separate canary changelog file
|
||||
- keep the filename stable-only
|
||||
- do not create a canary changelog file
|
||||
|
||||
## Step 3 — Verify the Release SHA
|
||||
## Step 3 — Verify the Candidate SHA
|
||||
|
||||
Run the standard gate:
|
||||
|
||||
@@ -128,41 +116,27 @@ pnpm test:run
|
||||
pnpm build
|
||||
```
|
||||
|
||||
If the release will be run through GitHub Actions, the workflow can rerun this gate. Still report whether the local tree currently passes.
|
||||
If the GitHub release workflow will run the publish, it can rerun this gate. Still report local status if you checked it.
|
||||
|
||||
The GitHub Actions release workflow installs with `pnpm install --frozen-lockfile`. Treat that as a release invariant, not a nuisance: if manifests changed and the lockfile refresh PR has not landed yet, stop and wait for `master` to contain the committed lockfile before shipping.
|
||||
For PRs that touch release logic, the repo also runs a canary release dry-run in CI. That is a release-specific guard, not a substitute for the standard gate.
|
||||
|
||||
## Step 4 — Publish a Canary
|
||||
## Step 4 — Validate the Canary
|
||||
|
||||
Run from the `release/X.Y.Z` branch:
|
||||
The normal canary path is automatic from `master` via:
|
||||
|
||||
```bash
|
||||
./scripts/release.sh {patch|minor|major} --canary --dry-run
|
||||
./scripts/release.sh {patch|minor|major} --canary
|
||||
```
|
||||
- `.github/workflows/release.yml`
|
||||
|
||||
What this means:
|
||||
Confirm:
|
||||
|
||||
- npm receives `X.Y.Z-canary.N` under dist-tag `canary`
|
||||
- `latest` remains unchanged
|
||||
- no git tag is created
|
||||
- the script cleans the working tree afterward
|
||||
1. verification passed
|
||||
2. npm canary publish succeeded
|
||||
3. git tag `canary/vYYYY.MDD.P-canary.N` exists
|
||||
|
||||
Guard:
|
||||
|
||||
- if the current stable is `0.2.7`, the next patch canary is `0.2.8-canary.0`
|
||||
- the tooling must never publish `0.2.7-canary.N` after `0.2.7` is already stable
|
||||
|
||||
After publish, verify:
|
||||
Useful checks:
|
||||
|
||||
```bash
|
||||
npm view paperclipai@canary version
|
||||
```
|
||||
|
||||
The user install path is:
|
||||
|
||||
```bash
|
||||
npx paperclipai@canary onboard
|
||||
git tag --list 'canary/v*' --sort=-version:refname | head -5
|
||||
```
|
||||
|
||||
## Step 5 — Smoke Test the Canary
|
||||
@@ -173,60 +147,70 @@ Run:
|
||||
PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
|
||||
```
|
||||
|
||||
Useful isolated variant:
|
||||
|
||||
```bash
|
||||
HOST_PORT=3232 DATA_DIR=./data/release-smoke-canary PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
|
||||
```
|
||||
|
||||
Confirm:
|
||||
|
||||
1. install succeeds
|
||||
2. onboarding completes
|
||||
3. server boots
|
||||
4. UI loads
|
||||
5. basic company/dashboard flow works
|
||||
2. onboarding completes without crashes
|
||||
3. the server boots
|
||||
4. the UI loads
|
||||
5. basic company creation and dashboard load work
|
||||
|
||||
If smoke testing fails:
|
||||
|
||||
- stop the stable release
|
||||
- fix the issue
|
||||
- publish another canary
|
||||
- repeat the smoke test
|
||||
- fix the issue on `master`
|
||||
- wait for the next automatic canary
|
||||
- rerun smoke testing
|
||||
|
||||
Each retry should create a higher canary ordinal, while the stable target version can stay the same.
|
||||
## Step 6 — Preview or Publish Stable
|
||||
|
||||
## Step 6 — Publish Stable
|
||||
The normal stable path is manual `workflow_dispatch` on:
|
||||
|
||||
Once the SHA is vetted, run:
|
||||
- `.github/workflows/release.yml`
|
||||
|
||||
Inputs:
|
||||
|
||||
- `source_ref`
|
||||
- `stable_date`
|
||||
- `dry_run`
|
||||
|
||||
Before live stable:
|
||||
|
||||
1. resolve the target stable version with `./scripts/release.sh stable --date YYYY-MM-DD --print-version`
|
||||
2. ensure `releases/vYYYY.MDD.P.md` exists on the source ref
|
||||
3. run the stable workflow in dry-run mode first when practical
|
||||
4. then run the real stable publish
|
||||
|
||||
The stable workflow:
|
||||
|
||||
- re-verifies the exact source ref
|
||||
- computes the next stable patch slot for the chosen UTC date
|
||||
- publishes `YYYY.MDD.P` under dist-tag `latest`
|
||||
- creates git tag `vYYYY.MDD.P`
|
||||
- creates or updates the GitHub Release from `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Local emergency/manual commands:
|
||||
|
||||
```bash
|
||||
./scripts/release.sh {patch|minor|major} --dry-run
|
||||
./scripts/release.sh {patch|minor|major}
|
||||
./scripts/release.sh stable --dry-run
|
||||
./scripts/release.sh stable
|
||||
git push public-gh refs/tags/vYYYY.MDD.P
|
||||
./scripts/create-github-release.sh YYYY.MDD.P
|
||||
```
|
||||
|
||||
Stable publish does this:
|
||||
|
||||
- publishes `X.Y.Z` to npm under `latest`
|
||||
- creates the local release commit
|
||||
- creates the local git tag `vX.Y.Z`
|
||||
|
||||
Stable publish does **not** push the release for you.
|
||||
|
||||
## Step 7 — Push and Create GitHub Release
|
||||
|
||||
After stable publish succeeds:
|
||||
|
||||
```bash
|
||||
git push public-gh HEAD --follow-tags
|
||||
./scripts/create-github-release.sh X.Y.Z
|
||||
```
|
||||
|
||||
Use the stable changelog file as the GitHub Release notes source.
|
||||
|
||||
Then open the PR from `release/X.Y.Z` back to `master` and merge without squash or rebase.
|
||||
|
||||
## Step 8 — Finish the Other Surfaces
|
||||
## Step 7 — Finish the Other Surfaces
|
||||
|
||||
Create or verify follow-up work for:
|
||||
|
||||
- website changelog publishing
|
||||
- launch post / social announcement
|
||||
- any release summary in Paperclip issue context
|
||||
- release summary in Paperclip issue context
|
||||
|
||||
These should reference the stable release, not the canary.
|
||||
|
||||
@@ -236,9 +220,9 @@ If the canary is bad:
|
||||
|
||||
- publish another canary, do not ship stable
|
||||
|
||||
If stable npm publish succeeds but push or GitHub release creation fails:
|
||||
If stable npm publish succeeds but tag push or GitHub release creation fails:
|
||||
|
||||
- fix the git/GitHub issue immediately from the same checkout
|
||||
- fix the git/GitHub issue immediately from the same release result
|
||||
- do not republish the same version
|
||||
|
||||
If `latest` is bad after stable publish:
|
||||
@@ -247,15 +231,17 @@ If `latest` is bad after stable publish:
|
||||
./scripts/rollback-latest.sh <last-good-version>
|
||||
```
|
||||
|
||||
Then fix forward with a new patch release.
|
||||
Then fix forward with a new stable release.
|
||||
|
||||
## Output
|
||||
|
||||
When the skill completes, provide:
|
||||
|
||||
- stable version and, if relevant, the final canary version tested
|
||||
- candidate SHA and tested canary version, if relevant
|
||||
- stable version, if promoted
|
||||
- verification status
|
||||
- npm status
|
||||
- smoke-test status
|
||||
- git tag / GitHub Release status
|
||||
- website / announcement follow-up status
|
||||
- rollback recommendation if anything is still partially complete
|
||||
|
||||
118
.github/workflows/release-smoke.yml
vendored
Normal file
118
.github/workflows/release-smoke.yml
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
name: Release Smoke
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
paperclip_version:
|
||||
description: Published Paperclip dist-tag to test
|
||||
required: true
|
||||
default: canary
|
||||
type: choice
|
||||
options:
|
||||
- canary
|
||||
- latest
|
||||
host_port:
|
||||
description: Host port for the Docker smoke container
|
||||
required: false
|
||||
default: "3232"
|
||||
type: string
|
||||
artifact_name:
|
||||
description: Artifact name for uploaded diagnostics
|
||||
required: false
|
||||
default: release-smoke
|
||||
type: string
|
||||
workflow_call:
|
||||
inputs:
|
||||
paperclip_version:
|
||||
required: true
|
||||
type: string
|
||||
host_port:
|
||||
required: false
|
||||
default: "3232"
|
||||
type: string
|
||||
artifact_name:
|
||||
required: false
|
||||
default: release-smoke
|
||||
type: string
|
||||
|
||||
jobs:
|
||||
smoke:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup pnpm
|
||||
uses: pnpm/action-setup@v4
|
||||
with:
|
||||
version: 9.15.4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 24
|
||||
cache: pnpm
|
||||
|
||||
- name: Install dependencies
|
||||
run: pnpm install --no-frozen-lockfile
|
||||
|
||||
- name: Install Playwright browser
|
||||
run: npx playwright install --with-deps chromium
|
||||
|
||||
- name: Launch Docker smoke harness
|
||||
run: |
|
||||
metadata_file="$RUNNER_TEMP/release-smoke.env"
|
||||
HOST_PORT="${{ inputs.host_port }}" \
|
||||
DATA_DIR="$RUNNER_TEMP/release-smoke-data" \
|
||||
PAPERCLIPAI_VERSION="${{ inputs.paperclip_version }}" \
|
||||
SMOKE_DETACH=true \
|
||||
SMOKE_METADATA_FILE="$metadata_file" \
|
||||
./scripts/docker-onboard-smoke.sh
|
||||
set -a
|
||||
source "$metadata_file"
|
||||
set +a
|
||||
{
|
||||
echo "SMOKE_BASE_URL=$SMOKE_BASE_URL"
|
||||
echo "SMOKE_ADMIN_EMAIL=$SMOKE_ADMIN_EMAIL"
|
||||
echo "SMOKE_ADMIN_PASSWORD=$SMOKE_ADMIN_PASSWORD"
|
||||
echo "SMOKE_CONTAINER_NAME=$SMOKE_CONTAINER_NAME"
|
||||
echo "SMOKE_DATA_DIR=$SMOKE_DATA_DIR"
|
||||
echo "SMOKE_IMAGE_NAME=$SMOKE_IMAGE_NAME"
|
||||
echo "SMOKE_PAPERCLIPAI_VERSION=$SMOKE_PAPERCLIPAI_VERSION"
|
||||
echo "SMOKE_METADATA_FILE=$metadata_file"
|
||||
} >> "$GITHUB_ENV"
|
||||
|
||||
- name: Run release smoke Playwright suite
|
||||
env:
|
||||
PAPERCLIP_RELEASE_SMOKE_BASE_URL: ${{ env.SMOKE_BASE_URL }}
|
||||
PAPERCLIP_RELEASE_SMOKE_EMAIL: ${{ env.SMOKE_ADMIN_EMAIL }}
|
||||
PAPERCLIP_RELEASE_SMOKE_PASSWORD: ${{ env.SMOKE_ADMIN_PASSWORD }}
|
||||
run: pnpm run test:release-smoke
|
||||
|
||||
- name: Capture Docker logs
|
||||
if: always()
|
||||
run: |
|
||||
if [[ -n "${SMOKE_CONTAINER_NAME:-}" ]]; then
|
||||
docker logs "$SMOKE_CONTAINER_NAME" >"$RUNNER_TEMP/docker-onboard-smoke.log" 2>&1 || true
|
||||
fi
|
||||
|
||||
- name: Upload diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: ${{ inputs.artifact_name }}
|
||||
path: |
|
||||
${{ runner.temp }}/docker-onboard-smoke.log
|
||||
${{ env.SMOKE_METADATA_FILE }}
|
||||
tests/release-smoke/playwright-report/
|
||||
tests/release-smoke/test-results/
|
||||
retention-days: 14
|
||||
|
||||
- name: Stop Docker smoke container
|
||||
if: always()
|
||||
run: |
|
||||
if [[ -n "${SMOKE_CONTAINER_NAME:-}" ]]; then
|
||||
docker rm -f "$SMOKE_CONTAINER_NAME" >/dev/null 2>&1 || true
|
||||
fi
|
||||
2
.github/workflows/release.yml
vendored
2
.github/workflows/release.yml
vendored
@@ -12,7 +12,7 @@ on:
|
||||
type: string
|
||||
default: master
|
||||
stable_date:
|
||||
description: Stable release date in UTC (YYYY-MM-DD). Defaults to today.
|
||||
description: Stable release date in UTC (YYYY-MM-DD). First stable that day is .0, then .1, and so on.
|
||||
required: false
|
||||
type: string
|
||||
dry_run:
|
||||
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -46,5 +46,7 @@ tmp/
|
||||
# Playwright
|
||||
tests/e2e/test-results/
|
||||
tests/e2e/playwright-report/
|
||||
tests/release-smoke/test-results/
|
||||
tests/release-smoke/playwright-report/
|
||||
.superset/
|
||||
.claude/worktrees/
|
||||
|
||||
@@ -120,6 +120,7 @@ Useful overrides:
|
||||
```sh
|
||||
HOST_PORT=3200 PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
|
||||
PAPERCLIP_DEPLOYMENT_MODE=authenticated PAPERCLIP_DEPLOYMENT_EXPOSURE=private ./scripts/docker-onboard-smoke.sh
|
||||
SMOKE_DETACH=true SMOKE_METADATA_FILE=/tmp/paperclip-smoke.env PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
|
||||
```
|
||||
|
||||
Notes:
|
||||
@@ -131,4 +132,5 @@ Notes:
|
||||
- Smoke script also defaults `PAPERCLIP_PUBLIC_URL` to `http://localhost:<HOST_PORT>` so bootstrap invite URLs and auth callbacks use the reachable host port instead of the container's internal `3100`.
|
||||
- In authenticated mode, the smoke script defaults `SMOKE_AUTO_BOOTSTRAP=true` and drives the real bootstrap path automatically: it signs up a real user, runs `paperclipai auth bootstrap-ceo` inside the container to mint a real bootstrap invite, accepts that invite over HTTP, and verifies board session access.
|
||||
- Run the script in the foreground to watch the onboarding flow; stop with `Ctrl+C` after validation.
|
||||
- Set `SMOKE_DETACH=true` to leave the container running for automation and optionally write shell-ready metadata to `SMOKE_METADATA_FILE`.
|
||||
- The image definition is in `Dockerfile.onboard-smoke`.
|
||||
|
||||
@@ -69,13 +69,13 @@ Those rewrites are temporary. The working tree is restored after publish or dry-
|
||||
|
||||
Paperclip uses calendar versions:
|
||||
|
||||
- stable: `YYYY.M.D`
|
||||
- canary: `YYYY.M.D-canary.N`
|
||||
- stable: `YYYY.MDD.P`
|
||||
- canary: `YYYY.MDD.P-canary.N`
|
||||
|
||||
Examples:
|
||||
|
||||
- stable: `2026.3.17`
|
||||
- canary: `2026.3.17-canary.2`
|
||||
- stable: `2026.318.0`
|
||||
- canary: `2026.318.1-canary.2`
|
||||
|
||||
## Publish model
|
||||
|
||||
@@ -85,7 +85,7 @@ Canaries publish under the npm dist-tag `canary`.
|
||||
|
||||
Example:
|
||||
|
||||
- `paperclipai@2026.3.17-canary.2`
|
||||
- `paperclipai@2026.318.1-canary.2`
|
||||
|
||||
This keeps the default install path unchanged while allowing explicit installs with:
|
||||
|
||||
@@ -99,13 +99,13 @@ Stable publishes use the npm dist-tag `latest`.
|
||||
|
||||
Example:
|
||||
|
||||
- `paperclipai@2026.3.17`
|
||||
- `paperclipai@2026.318.0`
|
||||
|
||||
Stable publishes do not create a release commit. Instead:
|
||||
|
||||
- package versions are rewritten temporarily
|
||||
- packages are published from the chosen source commit
|
||||
- git tag `vYYYY.M.D` points at that original commit
|
||||
- git tag `vYYYY.MDD.P` points at that original commit
|
||||
|
||||
## Trusted publishing
|
||||
|
||||
@@ -126,7 +126,7 @@ Rollback does not unpublish anything.
|
||||
It repoints the `latest` dist-tag to a prior stable version:
|
||||
|
||||
```bash
|
||||
./scripts/rollback-latest.sh 2026.3.16
|
||||
./scripts/rollback-latest.sh 2026.318.0
|
||||
```
|
||||
|
||||
This is the fastest way to restore the default install path if a stable release is bad.
|
||||
|
||||
@@ -205,7 +205,7 @@ After setup:
|
||||
3. confirm it passes verification
|
||||
4. confirm publish succeeds under the `npm-canary` environment
|
||||
5. confirm npm now shows a new `canary` release
|
||||
6. confirm a git tag named `canary/vYYYY.M.D-canary.N` was pushed
|
||||
6. confirm a git tag named `canary/vYYYY.MDD.P-canary.N` was pushed
|
||||
|
||||
Install-path check:
|
||||
|
||||
@@ -217,18 +217,19 @@ npx paperclipai@canary onboard
|
||||
|
||||
After at least one good canary exists:
|
||||
|
||||
1. prepare `releases/vYYYY.M.D.md` on the source commit you want to promote
|
||||
2. open `Actions` -> `Release`
|
||||
3. run it with:
|
||||
1. resolve the target stable version with `./scripts/release.sh stable --date YYYY-MM-DD --print-version`
|
||||
2. prepare `releases/vYYYY.MDD.P.md` on the source commit you want to promote
|
||||
3. open `Actions` -> `Release`
|
||||
4. run it with:
|
||||
- `source_ref`: the tested commit SHA or canary tag source commit
|
||||
- `stable_date`: leave blank or set the intended UTC date
|
||||
- `dry_run`: `true`
|
||||
4. confirm the dry-run succeeds
|
||||
5. rerun with `dry_run: false`
|
||||
6. approve the `npm-stable` environment when prompted
|
||||
7. confirm npm `latest` points to the new stable version
|
||||
8. confirm git tag `vYYYY.M.D` exists
|
||||
9. confirm the GitHub Release was created
|
||||
5. confirm the dry-run succeeds
|
||||
6. rerun with `dry_run: false`
|
||||
7. approve the `npm-stable` environment when prompted
|
||||
8. confirm npm `latest` points to the new stable version
|
||||
9. confirm git tag `vYYYY.MDD.P` exists
|
||||
10. confirm the GitHub Release was created
|
||||
|
||||
## 13. Suggested Maintainer Policy
|
||||
|
||||
|
||||
@@ -6,26 +6,29 @@ The release model is now commit-driven:
|
||||
|
||||
1. Every push to `master` publishes a canary automatically.
|
||||
2. Stable releases are manually promoted from a chosen tested commit or canary tag.
|
||||
3. Stable release notes live in `releases/vYYYY.M.D.md`.
|
||||
3. Stable release notes live in `releases/vYYYY.MDD.P.md`.
|
||||
4. Only stable releases get GitHub Releases.
|
||||
|
||||
## Versioning Model
|
||||
|
||||
Paperclip uses calendar versions that still fit semver syntax:
|
||||
|
||||
- stable: `YYYY.M.D`
|
||||
- canary: `YYYY.M.D-canary.N`
|
||||
- stable: `YYYY.MDD.P`
|
||||
- canary: `YYYY.MDD.P-canary.N`
|
||||
|
||||
Examples:
|
||||
|
||||
- stable on March 17, 2026: `2026.3.17`
|
||||
- fourth canary on March 17, 2026: `2026.3.17-canary.3`
|
||||
- first stable on March 18, 2026: `2026.318.0`
|
||||
- second stable on March 18, 2026: `2026.318.1`
|
||||
- fourth canary for the `2026.318.1` line: `2026.318.1-canary.3`
|
||||
|
||||
Important constraints:
|
||||
|
||||
- do not use leading zeroes such as `2026.03.17`
|
||||
- do not use four numeric segments such as `2026.03.17.1`
|
||||
- the semver-safe canary form is `2026.3.17-canary.1`
|
||||
- the middle numeric slot is `MDD`, where `M` is the UTC month and `DD` is the zero-padded UTC day
|
||||
- use `2026.303.0` for March 3, not `2026.33.0`
|
||||
- do not use leading zeroes such as `2026.0318.0`
|
||||
- do not use four numeric segments such as `2026.3.18.1`
|
||||
- the semver-safe canary form is `2026.318.0-canary.1`
|
||||
|
||||
## Release Surfaces
|
||||
|
||||
@@ -45,7 +48,7 @@ Canaries only cover the first two surfaces plus an internal traceability tag.
|
||||
- canaries publish from `master`
|
||||
- stables publish from an explicitly chosen source ref
|
||||
- tags point at the original source commit, not a generated release commit
|
||||
- stable notes are always `releases/vYYYY.M.D.md`
|
||||
- stable notes are always `releases/vYYYY.MDD.P.md`
|
||||
- canaries never create GitHub Releases
|
||||
- canaries never require changelog generation
|
||||
|
||||
@@ -60,12 +63,14 @@ It:
|
||||
- verifies the pushed commit
|
||||
- computes the canary version for the current UTC date
|
||||
- publishes under npm dist-tag `canary`
|
||||
- creates a git tag `canary/vYYYY.M.D-canary.N`
|
||||
- creates a git tag `canary/vYYYY.MDD.P-canary.N`
|
||||
|
||||
Users install canaries with:
|
||||
|
||||
```bash
|
||||
npx paperclipai@canary onboard
|
||||
# or
|
||||
npx paperclipai@canary onboard --data-dir "$(mktemp -d /tmp/paperclip-canary.XXXXXX)"
|
||||
```
|
||||
|
||||
### Stable
|
||||
@@ -84,15 +89,17 @@ Inputs:
|
||||
Before running stable:
|
||||
|
||||
1. pick the canary commit or tag you trust
|
||||
2. create or update `releases/vYYYY.M.D.md` on that source ref
|
||||
3. run the stable workflow from that source ref
|
||||
2. resolve the target stable version with `./scripts/release.sh stable --date YYYY-MM-DD --print-version`
|
||||
3. create or update `releases/vYYYY.MDD.P.md` on that source ref
|
||||
4. run the stable workflow from that source ref
|
||||
|
||||
The workflow:
|
||||
|
||||
- re-verifies the exact source ref
|
||||
- publishes `YYYY.M.D` under npm dist-tag `latest`
|
||||
- creates git tag `vYYYY.M.D`
|
||||
- creates or updates the GitHub Release from `releases/vYYYY.M.D.md`
|
||||
- computes the next stable patch slot for the chosen UTC date
|
||||
- publishes `YYYY.MDD.P` under npm dist-tag `latest`
|
||||
- creates git tag `vYYYY.MDD.P`
|
||||
- creates or updates the GitHub Release from `releases/vYYYY.MDD.P.md`
|
||||
|
||||
## Local Commands
|
||||
|
||||
@@ -114,22 +121,22 @@ This is mainly for emergency/manual use. The normal path is the GitHub workflow.
|
||||
|
||||
```bash
|
||||
./scripts/release.sh stable
|
||||
git push public-gh refs/tags/vYYYY.M.D
|
||||
./scripts/create-github-release.sh YYYY.M.D
|
||||
git push public-gh refs/tags/vYYYY.MDD.P
|
||||
./scripts/create-github-release.sh YYYY.MDD.P
|
||||
```
|
||||
|
||||
## Stable Changelog Workflow
|
||||
|
||||
Stable changelog files live at:
|
||||
|
||||
- `releases/vYYYY.M.D.md`
|
||||
- `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Canaries do not get changelog files.
|
||||
|
||||
Recommended local generation flow:
|
||||
|
||||
```bash
|
||||
VERSION=2026.3.17
|
||||
VERSION="$(./scripts/release.sh stable --date 2026-03-18 --print-version)"
|
||||
claude --print --output-format stream-json --verbose --dangerously-skip-permissions --model claude-opus-4-6 "Use the release-changelog skill to draft or update releases/v${VERSION}.md for Paperclip. Read doc/RELEASING.md and .agents/skills/release-changelog/SKILL.md, then generate the stable changelog for v${VERSION} from commits since the last stable tag. Do not create a canary changelog."
|
||||
```
|
||||
|
||||
@@ -160,13 +167,22 @@ HOST_PORT=3232 DATA_DIR=./data/release-smoke-canary PAPERCLIPAI_VERSION=canary .
|
||||
HOST_PORT=3233 DATA_DIR=./data/release-smoke-stable PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
|
||||
```
|
||||
|
||||
Automated browser smoke is also available:
|
||||
|
||||
```bash
|
||||
gh workflow run release-smoke.yml -f paperclip_version=canary
|
||||
gh workflow run release-smoke.yml -f paperclip_version=latest
|
||||
```
|
||||
|
||||
Minimum checks:
|
||||
|
||||
- `npx paperclipai@canary onboard` installs
|
||||
- onboarding completes without crashes
|
||||
- the server boots
|
||||
- the UI loads
|
||||
- basic company creation and dashboard load work
|
||||
- authenticated login works with the smoke credentials
|
||||
- the browser lands in onboarding on a fresh instance
|
||||
- company creation succeeds
|
||||
- the first CEO agent is created
|
||||
- the first CEO heartbeat run is triggered
|
||||
|
||||
## Rollback
|
||||
|
||||
@@ -175,11 +191,11 @@ Rollback does not unpublish versions.
|
||||
It only moves the `latest` dist-tag back to a previous stable:
|
||||
|
||||
```bash
|
||||
./scripts/rollback-latest.sh 2026.3.16 --dry-run
|
||||
./scripts/rollback-latest.sh 2026.3.16
|
||||
./scripts/rollback-latest.sh 2026.318.0 --dry-run
|
||||
./scripts/rollback-latest.sh 2026.318.0
|
||||
```
|
||||
|
||||
Then fix forward with a new stable release date.
|
||||
Then fix forward with a new stable patch slot or release date.
|
||||
|
||||
## Failure Playbooks
|
||||
|
||||
@@ -201,8 +217,8 @@ This is a partial release. npm is already live.
|
||||
Do this immediately:
|
||||
|
||||
1. push the missing tag
|
||||
2. rerun `./scripts/create-github-release.sh YYYY.M.D`
|
||||
3. verify the GitHub Release notes point at `releases/vYYYY.M.D.md`
|
||||
2. rerun `./scripts/create-github-release.sh YYYY.MDD.P`
|
||||
3. verify the GitHub Release notes point at `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Do not republish the same version.
|
||||
|
||||
@@ -211,7 +227,7 @@ Do not republish the same version.
|
||||
Roll back the dist-tag:
|
||||
|
||||
```bash
|
||||
./scripts/rollback-latest.sh YYYY.M.D
|
||||
./scripts/rollback-latest.sh YYYY.MDD.P
|
||||
```
|
||||
|
||||
Then fix forward with a new stable release.
|
||||
|
||||
424
doc/plans/2026-03-17-docker-release-browser-e2e.md
Normal file
424
doc/plans/2026-03-17-docker-release-browser-e2e.md
Normal file
@@ -0,0 +1,424 @@
|
||||
# Docker Release Browser E2E Plan
|
||||
|
||||
## Context
|
||||
|
||||
Today release smoke testing for published Paperclip packages is manual and shell-driven:
|
||||
|
||||
```sh
|
||||
HOST_PORT=3232 DATA_DIR=./data/release-smoke-canary PAPERCLIPAI_VERSION=canary ./scripts/docker-onboard-smoke.sh
|
||||
HOST_PORT=3233 DATA_DIR=./data/release-smoke-stable PAPERCLIPAI_VERSION=latest ./scripts/docker-onboard-smoke.sh
|
||||
```
|
||||
|
||||
That is useful because it exercises the same public install surface users hit:
|
||||
|
||||
- Docker
|
||||
- `npx paperclipai@canary`
|
||||
- `npx paperclipai@latest`
|
||||
- authenticated bootstrap flow
|
||||
|
||||
But it still leaves the most important release questions to a human with a browser:
|
||||
|
||||
- can I sign in with the smoke credentials?
|
||||
- do I land in onboarding?
|
||||
- can I complete onboarding?
|
||||
- does the initial CEO agent actually get created and run?
|
||||
|
||||
The repo already has two adjacent pieces:
|
||||
|
||||
- `tests/e2e/onboarding.spec.ts` covers the onboarding wizard against the local source tree
|
||||
- `scripts/docker-onboard-smoke.sh` boots a published Docker install and auto-bootstraps authenticated mode, but only verifies the API/session layer
|
||||
|
||||
What is missing is one deterministic browser test that joins those two paths.
|
||||
|
||||
## Goal
|
||||
|
||||
Add a release-grade Docker-backed browser E2E that validates the published `canary` and `latest` installs end to end:
|
||||
|
||||
1. boot the published package in Docker
|
||||
2. sign in with known smoke credentials
|
||||
3. verify the user is routed into onboarding
|
||||
4. complete onboarding in the browser
|
||||
5. verify the first CEO agent exists
|
||||
6. verify the initial CEO run was triggered and reached a terminal or active state
|
||||
|
||||
Then wire that test into GitHub Actions so release validation is no longer manual-only.
|
||||
|
||||
## Recommendation In One Sentence
|
||||
|
||||
Turn the current Docker smoke script into a machine-friendly test harness, add a dedicated Playwright release-smoke spec that drives the authenticated browser flow against published Docker installs, and run it in GitHub Actions for both `canary` and `latest`.
|
||||
|
||||
## What We Have Today
|
||||
|
||||
### Existing local browser coverage
|
||||
|
||||
`tests/e2e/onboarding.spec.ts` already proves the onboarding wizard can:
|
||||
|
||||
- create a company
|
||||
- create a CEO agent
|
||||
- create an initial issue
|
||||
- optionally observe task progress
|
||||
|
||||
That is a good base, but it does not validate the public npm package, Docker path, authenticated login flow, or release dist-tags.
|
||||
|
||||
### Existing Docker smoke coverage
|
||||
|
||||
`scripts/docker-onboard-smoke.sh` already does useful setup work:
|
||||
|
||||
- builds `Dockerfile.onboard-smoke`
|
||||
- runs `paperclipai@${PAPERCLIPAI_VERSION}` inside Docker
|
||||
- waits for health
|
||||
- signs up or signs in a smoke admin user
|
||||
- generates and accepts the bootstrap CEO invite in authenticated mode
|
||||
- verifies a board session and `/api/companies`
|
||||
|
||||
That means the hard bootstrap problem is mostly solved already. The main gap is that the script is human-oriented and never hands control to a browser test.
|
||||
|
||||
### Existing CI shape
|
||||
|
||||
The repo already has:
|
||||
|
||||
- `.github/workflows/e2e.yml` for manual Playwright runs against local source
|
||||
- `.github/workflows/release.yml` for canary publish on `master` and manual stable promotion
|
||||
|
||||
So the right move is to extend the current test/release system, not create a parallel one.
|
||||
|
||||
## Product Decision
|
||||
|
||||
### 1. The release smoke should stay deterministic and token-free
|
||||
|
||||
The first version should not require OpenAI, Anthropic, or external agent credentials.
|
||||
|
||||
Use the onboarding flow with a deterministic adapter that can run on a stock GitHub runner and inside the published Docker install. The existing `process` adapter with a trivial command is the right base path for this release gate.
|
||||
|
||||
That keeps this test focused on:
|
||||
|
||||
- release packaging
|
||||
- auth/bootstrap
|
||||
- UI routing
|
||||
- onboarding contract
|
||||
- agent creation
|
||||
- heartbeat invocation plumbing
|
||||
|
||||
Later we can add a second credentialed smoke lane for real model-backed agents.
|
||||
|
||||
### 2. Smoke credentials become an explicit test contract
|
||||
|
||||
The current defaults in `scripts/docker-onboard-smoke.sh` should be treated as stable test fixtures:
|
||||
|
||||
- email: `smoke-admin@paperclip.local`
|
||||
- password: `paperclip-smoke-password`
|
||||
|
||||
The browser test should log in with those exact values unless overridden by env vars.
|
||||
|
||||
### 3. Published-package smoke and source-tree E2E stay separate
|
||||
|
||||
Keep two lanes:
|
||||
|
||||
- source-tree E2E for feature development
|
||||
- published Docker release smoke for release confidence
|
||||
|
||||
They overlap on onboarding assertions, but they guard different failure classes.
|
||||
|
||||
## Proposed Design
|
||||
|
||||
## 1. Add a CI-friendly Docker smoke harness
|
||||
|
||||
Refactor `scripts/docker-onboard-smoke.sh` so it can run in two modes:
|
||||
|
||||
- interactive mode
|
||||
- current behavior
|
||||
- streams logs and waits in foreground for manual inspection
|
||||
- CI mode
|
||||
- starts the container
|
||||
- waits for health and authenticated bootstrap
|
||||
- prints machine-readable metadata
|
||||
- exits while leaving the container running for Playwright
|
||||
|
||||
Recommended shape:
|
||||
|
||||
- keep `scripts/docker-onboard-smoke.sh` as the public entry point
|
||||
- add a `SMOKE_DETACH=true` or `--detach` mode
|
||||
- emit a JSON blob or `.env` file containing:
|
||||
- `SMOKE_BASE_URL`
|
||||
- `SMOKE_ADMIN_EMAIL`
|
||||
- `SMOKE_ADMIN_PASSWORD`
|
||||
- `SMOKE_CONTAINER_NAME`
|
||||
- `SMOKE_DATA_DIR`
|
||||
|
||||
The workflow and Playwright tests can then consume the emitted metadata instead of scraping logs.
|
||||
|
||||
### Why this matters
|
||||
|
||||
The current script always tails logs and then blocks on `wait "$LOG_PID"`. That is convenient for manual smoke testing, but it is the wrong shape for CI orchestration.
|
||||
|
||||
## 2. Add a dedicated Playwright release-smoke spec
|
||||
|
||||
Create a second Playwright entry point specifically for published Docker installs, for example:
|
||||
|
||||
- `tests/release-smoke/playwright.config.ts`
|
||||
- `tests/release-smoke/docker-auth-onboarding.spec.ts`
|
||||
|
||||
This suite should not use Playwright `webServer`, because the app server will already be running inside Docker.
|
||||
|
||||
### Browser scenario
|
||||
|
||||
The first release-smoke scenario should validate:
|
||||
|
||||
1. open `/`
|
||||
2. unauthenticated user is redirected to `/auth`
|
||||
3. sign in using the smoke credentials
|
||||
4. authenticated user lands on onboarding when no companies exist
|
||||
5. onboarding wizard appears with the expected step labels
|
||||
6. create a company
|
||||
7. create the first agent using `process`
|
||||
8. create the initial issue
|
||||
9. finish onboarding and open the created issue
|
||||
10. verify via API:
|
||||
- company exists
|
||||
- CEO agent exists
|
||||
- issue exists and is assigned to the CEO
|
||||
11. verify the first heartbeat run was triggered:
|
||||
- either by checking issue status changed from initial state, or
|
||||
- by checking agent/runs API shows a run for the CEO, or
|
||||
- both
|
||||
|
||||
The test should tolerate the run completing quickly. For this reason, the assertion should accept:
|
||||
|
||||
- `queued`
|
||||
- `running`
|
||||
- `succeeded`
|
||||
|
||||
and similarly for issue progression if the issue status changes before the assertion runs.
|
||||
|
||||
### Why a separate spec instead of reusing `tests/e2e/onboarding.spec.ts`
|
||||
|
||||
The local-source test and release-smoke test have different assumptions:
|
||||
|
||||
- different server lifecycle
|
||||
- different auth path
|
||||
- different deployment mode
|
||||
- published npm package instead of local workspace code
|
||||
|
||||
Trying to force both through one spec will make both worse.
|
||||
|
||||
## 3. Add a release-smoke workflow in GitHub Actions
|
||||
|
||||
Add a workflow dedicated to this surface, ideally reusable:
|
||||
|
||||
- `.github/workflows/release-smoke.yml`
|
||||
|
||||
Recommended triggers:
|
||||
|
||||
- `workflow_dispatch`
|
||||
- `workflow_call`
|
||||
|
||||
Recommended inputs:
|
||||
|
||||
- `paperclip_version`
|
||||
- `canary` or `latest`
|
||||
- `host_port`
|
||||
- optional, default runner-safe port
|
||||
- `artifact_name`
|
||||
- optional for clearer uploads
|
||||
|
||||
### Job outline
|
||||
|
||||
1. checkout repo
|
||||
2. install Node/pnpm
|
||||
3. install Playwright browser dependencies
|
||||
4. launch Docker smoke harness in detached mode with the chosen dist-tag
|
||||
5. run the release-smoke Playwright suite against the returned base URL
|
||||
6. always collect diagnostics:
|
||||
- Playwright report
|
||||
- screenshots
|
||||
- trace
|
||||
- `docker logs`
|
||||
- harness metadata file
|
||||
7. stop and remove container
|
||||
|
||||
### Why a reusable workflow
|
||||
|
||||
This lets us:
|
||||
|
||||
- run the smoke manually on demand
|
||||
- call it from `release.yml`
|
||||
- reuse the same job for both `canary` and `latest`
|
||||
|
||||
## 4. Integrate it into release automation incrementally
|
||||
|
||||
### Phase A: Manual workflow only
|
||||
|
||||
First ship the workflow as manual-only so the harness and test can be stabilized without blocking releases.
|
||||
|
||||
### Phase B: Run automatically after canary publish
|
||||
|
||||
After `publish_canary` succeeds in `.github/workflows/release.yml`, call the reusable release-smoke workflow with:
|
||||
|
||||
- `paperclip_version=canary`
|
||||
|
||||
This proves the just-published public canary really boots and onboards.
|
||||
|
||||
### Phase C: Run automatically after stable publish
|
||||
|
||||
After `publish_stable` succeeds, call the same workflow with:
|
||||
|
||||
- `paperclip_version=latest`
|
||||
|
||||
This gives us post-publish confirmation that the stable dist-tag is healthy.
|
||||
|
||||
### Important nuance
|
||||
|
||||
Testing `latest` from npm cannot happen before stable publish, because the package under test does not exist under `latest` yet. So the `latest` smoke is a post-publish verification, not a pre-publish gate.
|
||||
|
||||
If we later want a true pre-publish stable gate, that should be a separate source-ref or locally built package smoke job.
|
||||
|
||||
## 5. Make diagnostics first-class
|
||||
|
||||
This workflow is only valuable if failures are fast to debug.
|
||||
|
||||
Always capture:
|
||||
|
||||
- Playwright HTML report
|
||||
- Playwright trace on failure
|
||||
- final screenshot on failure
|
||||
- full `docker logs` output
|
||||
- emitted smoke metadata
|
||||
- optional `curl /api/health` snapshot
|
||||
|
||||
Without that, the test will become a flaky black box and people will stop trusting it.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
## Phase 1: Harness refactor
|
||||
|
||||
Files:
|
||||
|
||||
- `scripts/docker-onboard-smoke.sh`
|
||||
- optionally `scripts/lib/docker-onboard-smoke.sh` or similar helper
|
||||
- `doc/DOCKER.md`
|
||||
- `doc/RELEASING.md`
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Add detached/CI mode to the Docker smoke script.
|
||||
2. Make the script emit machine-readable connection metadata.
|
||||
3. Keep the current interactive manual mode intact.
|
||||
4. Add reliable cleanup commands for CI.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- a script invocation can start the published Docker app, auto-bootstrap it, and return control to the caller with enough metadata for browser automation
|
||||
|
||||
## Phase 2: Browser release-smoke suite
|
||||
|
||||
Files:
|
||||
|
||||
- `tests/release-smoke/playwright.config.ts`
|
||||
- `tests/release-smoke/docker-auth-onboarding.spec.ts`
|
||||
- root `package.json`
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Add a dedicated Playwright config for external server testing.
|
||||
2. Implement login + onboarding + CEO creation flow.
|
||||
3. Assert a CEO run was created or completed.
|
||||
4. Add a root script such as:
|
||||
- `test:release-smoke`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- the suite passes locally against both:
|
||||
- `PAPERCLIPAI_VERSION=canary`
|
||||
- `PAPERCLIPAI_VERSION=latest`
|
||||
|
||||
## Phase 3: GitHub Actions workflow
|
||||
|
||||
Files:
|
||||
|
||||
- `.github/workflows/release-smoke.yml`
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Add manual and reusable workflow entry points.
|
||||
2. Install Chromium and runner dependencies.
|
||||
3. Start Docker smoke in detached mode.
|
||||
4. Run the release-smoke Playwright suite.
|
||||
5. Upload diagnostics artifacts.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- a maintainer can run the workflow manually for either `canary` or `latest`
|
||||
|
||||
## Phase 4: Release workflow integration
|
||||
|
||||
Files:
|
||||
|
||||
- `.github/workflows/release.yml`
|
||||
- `doc/RELEASING.md`
|
||||
|
||||
Tasks:
|
||||
|
||||
1. Trigger release smoke automatically after canary publish.
|
||||
2. Trigger release smoke automatically after stable publish.
|
||||
3. Document expected behavior and failure handling.
|
||||
|
||||
Acceptance:
|
||||
|
||||
- canary releases automatically produce a published-package browser smoke result
|
||||
- stable releases automatically produce a `latest` browser smoke result
|
||||
|
||||
## Phase 5: Future extension for real model-backed agent validation
|
||||
|
||||
Not part of the first implementation, but this should be the next layer after the deterministic lane is stable.
|
||||
|
||||
Possible additions:
|
||||
|
||||
- a second Playwright project gated on repo secrets
|
||||
- real `claude_local` or `codex_local` adapter validation in Docker-capable environments
|
||||
- assertion that the CEO posts a real task/comment artifact
|
||||
- stable release holdback until the credentialed lane passes
|
||||
|
||||
This should stay optional until the token-free lane is trustworthy.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
The plan is complete when the implemented system can demonstrate all of the following:
|
||||
|
||||
1. A published `paperclipai@canary` Docker install can be smoke-tested by Playwright in CI.
|
||||
2. A published `paperclipai@latest` Docker install can be smoke-tested by Playwright in CI.
|
||||
3. The test logs into authenticated mode with the smoke credentials.
|
||||
4. The test sees onboarding for a fresh instance.
|
||||
5. The test completes onboarding in the browser.
|
||||
6. The test verifies the initial CEO agent was created.
|
||||
7. The test verifies at least one CEO heartbeat run was triggered.
|
||||
8. Failures produce actionable artifacts rather than just a red job.
|
||||
|
||||
## Risks And Decisions To Make
|
||||
|
||||
### 1. Fast process runs may finish before the UI visibly updates
|
||||
|
||||
That is expected. The assertions should prefer API polling for run existence/status rather than only visual indicators.
|
||||
|
||||
### 2. `latest` smoke is post-publish, not preventive
|
||||
|
||||
This is a real limitation of testing the published dist-tag itself. It is still valuable, but it should not be confused with a pre-publish gate.
|
||||
|
||||
### 3. We should not overcouple the test to cosmetic onboarding text
|
||||
|
||||
The important contract is flow success, created entities, and run creation. Use visible labels sparingly and prefer stable semantic selectors where possible.
|
||||
|
||||
### 4. Keep the smoke adapter path boring
|
||||
|
||||
For release safety, the first test should use the most boring runnable adapter possible. This is not the place to validate every adapter.
|
||||
|
||||
## Recommended First Slice
|
||||
|
||||
If we want the fastest path to value, ship this in order:
|
||||
|
||||
1. add detached mode to `scripts/docker-onboard-smoke.sh`
|
||||
2. add one Playwright spec for authenticated login + onboarding + CEO run verification
|
||||
3. add manual `release-smoke.yml`
|
||||
4. once stable, wire canary into `release.yml`
|
||||
5. after that, wire stable `latest` smoke into `release.yml`
|
||||
|
||||
That gives release confidence quickly without turning the first version into a large CI redesign.
|
||||
@@ -49,13 +49,13 @@ The repo and npm tooling still assume semver-shaped version strings in many plac
|
||||
|
||||
Recommended format:
|
||||
|
||||
- stable: `YYYY.M.D`
|
||||
- canary: `YYYY.M.D-canary.N`
|
||||
- stable: `YYYY.MDD.P`
|
||||
- canary: `YYYY.MDD.P-canary.N`
|
||||
|
||||
Examples:
|
||||
|
||||
- stable on March 17, 2026: `2026.3.17`
|
||||
- third canary on March 17, 2026: `2026.3.17-canary.2`
|
||||
- first stable on March 17, 2026: `2026.317.0`
|
||||
- third canary on the `2026.317.0` line: `2026.317.0-canary.2`
|
||||
|
||||
Why this shape:
|
||||
|
||||
@@ -66,11 +66,12 @@ Why this shape:
|
||||
|
||||
Important constraints:
|
||||
|
||||
- the middle numeric slot should be `MDD`, where `M` is the month and `DD` is the zero-padded day
|
||||
- `2026.03.17` is not the format to use
|
||||
- numeric semver identifiers do not allow leading zeroes
|
||||
- `2026.03.16.8` is not the format to use
|
||||
- `2026.3.17.1` is not the format to use
|
||||
- semver has three numeric components, not four
|
||||
- the practical semver-safe equivalent of your example is `2026.3.16-canary.8`
|
||||
- the practical semver-safe equivalent is `2026.317.0-canary.8`
|
||||
|
||||
This is effectively CalVer on semver rails.
|
||||
|
||||
@@ -109,7 +110,7 @@ This is the most important mechanical constraint.
|
||||
npm can move dist-tags, but it does not let you rename an already-published version. That means:
|
||||
|
||||
- you can move `latest` to `paperclipai@1.2.3`
|
||||
- you cannot turn `paperclipai@2026.3.16-canary.8` into `paperclipai@2026.3.17`
|
||||
- you cannot turn `paperclipai@2026.317.0-canary.8` into `paperclipai@2026.317.0`
|
||||
|
||||
So "promote canary to stable" really means:
|
||||
|
||||
@@ -123,7 +124,7 @@ Recommended stable input:
|
||||
|
||||
- `source_ref`
|
||||
- commit SHA, or
|
||||
- a canary git tag such as `canary/v2026.3.16-canary.8`
|
||||
- a canary git tag such as `canary/v2026.317.1-canary.8`
|
||||
|
||||
### 5. Only stable releases get release notes, tags, and GitHub Releases
|
||||
|
||||
@@ -137,9 +138,9 @@ Canaries should stay lightweight:
|
||||
|
||||
Stable releases should remain the public narrative surface:
|
||||
|
||||
- git tag `v2026.3.17`
|
||||
- GitHub Release `v2026.3.17`
|
||||
- stable changelog file `releases/v2026.3.17.md`
|
||||
- git tag `v2026.317.0`
|
||||
- GitHub Release `v2026.317.0`
|
||||
- stable changelog file `releases/v2026.317.0.md`
|
||||
|
||||
## Security Model
|
||||
|
||||
@@ -233,14 +234,14 @@ Recommended stable path:
|
||||
|
||||
1. pick a canary commit or tag
|
||||
2. run changelog generation locally from a trusted machine
|
||||
3. commit `releases/vYYYY.M.D.md`
|
||||
3. commit `releases/vYYYY.MDD.P.md`
|
||||
4. run stable promotion
|
||||
|
||||
If the notes are not ready yet, a fallback is acceptable:
|
||||
|
||||
- publish stable
|
||||
- create a minimal GitHub Release
|
||||
- update `releases/vYYYY.M.D.md` immediately afterward
|
||||
- update `releases/vYYYY.MDD.P.md` immediately afterward
|
||||
|
||||
But the better steady-state is to have the stable notes committed before stable publish.
|
||||
|
||||
@@ -268,13 +269,13 @@ Steps:
|
||||
1. checkout the merged `master` commit
|
||||
2. run verification on that exact commit
|
||||
3. compute canary version for current UTC date
|
||||
4. version public packages to `YYYY.M.D-canary.N`
|
||||
4. version public packages to `YYYY.MDD.P-canary.N`
|
||||
5. publish to npm with dist-tag `canary`
|
||||
6. create a canary git tag for traceability
|
||||
|
||||
Recommended canary tag format:
|
||||
|
||||
- `canary/v2026.3.17-canary.4`
|
||||
- `canary/v2026.317.1-canary.4`
|
||||
|
||||
Outputs:
|
||||
|
||||
@@ -299,14 +300,14 @@ Steps:
|
||||
|
||||
1. checkout `source_ref`
|
||||
2. run verification on that exact commit
|
||||
3. compute stable version from UTC date or provided override
|
||||
4. fail if `vYYYY.M.D` already exists
|
||||
5. require `releases/vYYYY.M.D.md`
|
||||
6. version public packages to `YYYY.M.D`
|
||||
3. compute the next stable patch slot for the UTC date or provided override
|
||||
4. fail if `vYYYY.MDD.P` already exists
|
||||
5. require `releases/vYYYY.MDD.P.md`
|
||||
6. version public packages to `YYYY.MDD.P`
|
||||
7. publish to npm under `latest`
|
||||
8. create git tag `vYYYY.M.D`
|
||||
8. create git tag `vYYYY.MDD.P`
|
||||
9. push tag
|
||||
10. create GitHub Release from `releases/vYYYY.M.D.md`
|
||||
10. create GitHub Release from `releases/vYYYY.MDD.P.md`
|
||||
|
||||
Outputs:
|
||||
|
||||
@@ -332,8 +333,8 @@ That logic should be replaced with:
|
||||
|
||||
For example:
|
||||
|
||||
- `stable_version_for_utc_date(2026-03-17) -> 2026.3.17`
|
||||
- `next_canary_for_utc_date(2026-03-17) -> 2026.3.17-canary.0`
|
||||
- `next_stable_version(2026-03-17) -> 2026.317.0`
|
||||
- `next_canary_for_utc_date(2026-03-17) -> 2026.317.0-canary.0`
|
||||
|
||||
### 2. Stop requiring `release/X.Y.Z`
|
||||
|
||||
@@ -392,19 +393,15 @@ It should continue to:
|
||||
|
||||
## Tradeoffs and Risks
|
||||
|
||||
### 1. One stable per UTC day
|
||||
### 1. The stable patch slot is now part of the version contract
|
||||
|
||||
With plain `YYYY.M.D`, you get one stable release per UTC day.
|
||||
With `YYYY.MDD.P`, same-day hotfixes are supported, but the stable patch slot is now part of the visible version format.
|
||||
|
||||
That is probably fine, but it is a real product rule.
|
||||
That is the right tradeoff because:
|
||||
|
||||
If you need multiple same-day stables later, you have three options:
|
||||
|
||||
1. accept a less pretty stable format
|
||||
2. go back to a serial patch component
|
||||
3. keep daily stable cadence and use canaries for same-day fixes
|
||||
|
||||
My recommendation is to accept one stable per UTC day unless reality proves otherwise.
|
||||
1. npm still gets semver-valid versions
|
||||
2. same-day hotfixes stay possible
|
||||
3. chronological ordering still works as long as the day is zero-padded inside `MDD`
|
||||
|
||||
### 2. Public package consumers lose semver intent signaling
|
||||
|
||||
@@ -469,8 +466,8 @@ That is acceptable if canaries stay clearly separate:
|
||||
|
||||
Paperclip should adopt this model:
|
||||
|
||||
- stable versions: `YYYY.M.D`
|
||||
- canary versions: `YYYY.M.D-canary.N`
|
||||
- stable versions: `YYYY.MDD.P`
|
||||
- canary versions: `YYYY.MDD.P-canary.N`
|
||||
- canaries auto-published on every push to `master`
|
||||
- stables manually promoted from a chosen tested commit or canary tag
|
||||
- no release branches in the default path
|
||||
|
||||
@@ -29,7 +29,9 @@
|
||||
"smoke:openclaw-docker-ui": "./scripts/smoke/openclaw-docker-ui.sh",
|
||||
"smoke:openclaw-sse-standalone": "./scripts/smoke/openclaw-sse-standalone.sh",
|
||||
"test:e2e": "npx playwright test --config tests/e2e/playwright.config.ts",
|
||||
"test:e2e:headed": "npx playwright test --config tests/e2e/playwright.config.ts --headed"
|
||||
"test:e2e:headed": "npx playwright test --config tests/e2e/playwright.config.ts --headed",
|
||||
"test:release-smoke": "npx playwright test --config tests/release-smoke/playwright.config.ts",
|
||||
"test:release-smoke:headed": "npx playwright test --config tests/release-smoke/playwright.config.ts --headed"
|
||||
},
|
||||
"devDependencies": {
|
||||
"cross-env": "^10.1.0",
|
||||
|
||||
@@ -14,8 +14,8 @@ Usage:
|
||||
./scripts/create-github-release.sh <version> [--dry-run]
|
||||
|
||||
Examples:
|
||||
./scripts/create-github-release.sh 2026.3.17
|
||||
./scripts/create-github-release.sh 2026.3.17 --dry-run
|
||||
./scripts/create-github-release.sh 2026.318.0
|
||||
./scripts/create-github-release.sh 2026.318.0 --dry-run
|
||||
|
||||
Notes:
|
||||
- Run this after pushing the stable tag.
|
||||
@@ -48,7 +48,7 @@ if [ -z "$version" ]; then
|
||||
fi
|
||||
|
||||
if [[ ! "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo "Error: version must be a stable calendar version like 2026.3.17." >&2
|
||||
echo "Error: version must be a stable calendar version like 2026.318.0." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ HOST_PORT="${HOST_PORT:-3131}"
|
||||
PAPERCLIPAI_VERSION="${PAPERCLIPAI_VERSION:-latest}"
|
||||
DATA_DIR="${DATA_DIR:-$REPO_ROOT/data/docker-onboard-smoke}"
|
||||
HOST_UID="${HOST_UID:-$(id -u)}"
|
||||
SMOKE_DETACH="${SMOKE_DETACH:-false}"
|
||||
SMOKE_METADATA_FILE="${SMOKE_METADATA_FILE:-}"
|
||||
PAPERCLIP_DEPLOYMENT_MODE="${PAPERCLIP_DEPLOYMENT_MODE:-authenticated}"
|
||||
PAPERCLIP_DEPLOYMENT_EXPOSURE="${PAPERCLIP_DEPLOYMENT_EXPOSURE:-private}"
|
||||
PAPERCLIP_PUBLIC_URL="${PAPERCLIP_PUBLIC_URL:-http://localhost:${HOST_PORT}}"
|
||||
@@ -18,6 +20,7 @@ CONTAINER_NAME="${IMAGE_NAME//[^a-zA-Z0-9_.-]/-}"
|
||||
LOG_PID=""
|
||||
COOKIE_JAR=""
|
||||
TMP_DIR=""
|
||||
PRESERVE_CONTAINER_ON_EXIT="false"
|
||||
|
||||
mkdir -p "$DATA_DIR"
|
||||
|
||||
@@ -25,7 +28,9 @@ cleanup() {
|
||||
if [[ -n "$LOG_PID" ]]; then
|
||||
kill "$LOG_PID" >/dev/null 2>&1 || true
|
||||
fi
|
||||
docker stop "$CONTAINER_NAME" >/dev/null 2>&1 || true
|
||||
if [[ "$PRESERVE_CONTAINER_ON_EXIT" != "true" ]]; then
|
||||
docker stop "$CONTAINER_NAME" >/dev/null 2>&1 || true
|
||||
fi
|
||||
if [[ -n "$TMP_DIR" && -d "$TMP_DIR" ]]; then
|
||||
rm -rf "$TMP_DIR"
|
||||
fi
|
||||
@@ -33,6 +38,12 @@ cleanup() {
|
||||
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
container_is_running() {
|
||||
local running
|
||||
running="$(docker inspect -f '{{.State.Running}}' "$CONTAINER_NAME" 2>/dev/null || true)"
|
||||
[[ "$running" == "true" ]]
|
||||
}
|
||||
|
||||
wait_for_http() {
|
||||
local url="$1"
|
||||
local attempts="${2:-60}"
|
||||
@@ -42,11 +53,36 @@ wait_for_http() {
|
||||
if curl -fsS "$url" >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
if ! container_is_running; then
|
||||
echo "Smoke bootstrap failed: container $CONTAINER_NAME exited before $url became ready" >&2
|
||||
docker logs "$CONTAINER_NAME" >&2 || true
|
||||
return 1
|
||||
fi
|
||||
sleep "$sleep_seconds"
|
||||
done
|
||||
if ! container_is_running; then
|
||||
echo "Smoke bootstrap failed: container $CONTAINER_NAME exited before readiness check completed" >&2
|
||||
docker logs "$CONTAINER_NAME" >&2 || true
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
write_metadata_file() {
|
||||
if [[ -z "$SMOKE_METADATA_FILE" ]]; then
|
||||
return 0
|
||||
fi
|
||||
mkdir -p "$(dirname "$SMOKE_METADATA_FILE")"
|
||||
{
|
||||
printf 'SMOKE_BASE_URL=%q\n' "$PAPERCLIP_PUBLIC_URL"
|
||||
printf 'SMOKE_ADMIN_EMAIL=%q\n' "$SMOKE_ADMIN_EMAIL"
|
||||
printf 'SMOKE_ADMIN_PASSWORD=%q\n' "$SMOKE_ADMIN_PASSWORD"
|
||||
printf 'SMOKE_CONTAINER_NAME=%q\n' "$CONTAINER_NAME"
|
||||
printf 'SMOKE_DATA_DIR=%q\n' "$DATA_DIR"
|
||||
printf 'SMOKE_IMAGE_NAME=%q\n' "$IMAGE_NAME"
|
||||
printf 'SMOKE_PAPERCLIPAI_VERSION=%q\n' "$PAPERCLIPAI_VERSION"
|
||||
} >"$SMOKE_METADATA_FILE"
|
||||
}
|
||||
|
||||
generate_bootstrap_invite_url() {
|
||||
local bootstrap_output
|
||||
local bootstrap_status
|
||||
@@ -214,9 +250,12 @@ echo "==> Running onboard smoke container"
|
||||
echo " UI should be reachable at: http://localhost:$HOST_PORT"
|
||||
echo " Public URL: $PAPERCLIP_PUBLIC_URL"
|
||||
echo " Smoke auto-bootstrap: $SMOKE_AUTO_BOOTSTRAP"
|
||||
echo " Detached mode: $SMOKE_DETACH"
|
||||
echo " Data dir: $DATA_DIR"
|
||||
echo " Deployment: $PAPERCLIP_DEPLOYMENT_MODE/$PAPERCLIP_DEPLOYMENT_EXPOSURE"
|
||||
echo " Live output: onboard banner and server logs stream in this terminal (Ctrl+C to stop)"
|
||||
if [[ "$SMOKE_DETACH" != "true" ]]; then
|
||||
echo " Live output: onboard banner and server logs stream in this terminal (Ctrl+C to stop)"
|
||||
fi
|
||||
|
||||
docker rm -f "$CONTAINER_NAME" >/dev/null 2>&1 || true
|
||||
|
||||
@@ -231,8 +270,10 @@ docker run -d --rm \
|
||||
-v "$DATA_DIR:/paperclip" \
|
||||
"$IMAGE_NAME" >/dev/null
|
||||
|
||||
docker logs -f "$CONTAINER_NAME" &
|
||||
LOG_PID=$!
|
||||
if [[ "$SMOKE_DETACH" != "true" ]]; then
|
||||
docker logs -f "$CONTAINER_NAME" &
|
||||
LOG_PID=$!
|
||||
fi
|
||||
|
||||
TMP_DIR="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-onboard-smoke.XXXXXX")"
|
||||
COOKIE_JAR="$TMP_DIR/cookies.txt"
|
||||
@@ -246,4 +287,17 @@ if [[ "$SMOKE_AUTO_BOOTSTRAP" == "true" && "$PAPERCLIP_DEPLOYMENT_MODE" == "auth
|
||||
auto_bootstrap_authenticated_smoke
|
||||
fi
|
||||
|
||||
write_metadata_file
|
||||
|
||||
if [[ "$SMOKE_DETACH" == "true" ]]; then
|
||||
PRESERVE_CONTAINER_ON_EXIT="true"
|
||||
echo "==> Smoke container ready for automation"
|
||||
echo " Smoke base URL: $PAPERCLIP_PUBLIC_URL"
|
||||
echo " Smoke admin credentials: $SMOKE_ADMIN_EMAIL / $SMOKE_ADMIN_PASSWORD"
|
||||
if [[ -n "$SMOKE_METADATA_FILE" ]]; then
|
||||
echo " Smoke metadata file: $SMOKE_METADATA_FILE"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
wait "$LOG_PID"
|
||||
|
||||
@@ -107,7 +107,7 @@ get_current_stable_version() {
|
||||
fi
|
||||
}
|
||||
|
||||
stable_version_for_date() {
|
||||
stable_version_slot_for_date() {
|
||||
node - "${1:-}" <<'NODE'
|
||||
const input = process.argv[2];
|
||||
|
||||
@@ -117,7 +117,10 @@ if (Number.isNaN(date.getTime())) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
process.stdout.write(`${date.getUTCFullYear()}.${date.getUTCMonth() + 1}.${date.getUTCDate()}`);
|
||||
const month = String(date.getUTCMonth() + 1);
|
||||
const day = String(date.getUTCDate()).padStart(2, '0');
|
||||
|
||||
process.stdout.write(`${date.getUTCFullYear()}.${month}${day}`);
|
||||
NODE
|
||||
}
|
||||
|
||||
@@ -131,6 +134,53 @@ process.stdout.write(`${y}-${m}-${d}`);
|
||||
NODE
|
||||
}
|
||||
|
||||
next_stable_version() {
|
||||
local release_date="$1"
|
||||
shift
|
||||
|
||||
node - "$release_date" "$@" <<'NODE'
|
||||
const input = process.argv[2];
|
||||
const packageNames = process.argv.slice(3);
|
||||
const { execSync } = require("node:child_process");
|
||||
|
||||
const date = input ? new Date(`${input}T00:00:00Z`) : new Date();
|
||||
if (Number.isNaN(date.getTime())) {
|
||||
console.error(`invalid date: ${input}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const stableSlot = `${date.getUTCFullYear()}.${date.getUTCMonth() + 1}${String(date.getUTCDate()).padStart(2, "0")}`;
|
||||
const pattern = new RegExp(`^${stableSlot.replace(/\./g, '\\.')}\.(\\d+)$`);
|
||||
let max = -1;
|
||||
|
||||
for (const packageName of packageNames) {
|
||||
let versions = [];
|
||||
|
||||
try {
|
||||
const raw = execSync(`npm view ${JSON.stringify(packageName)} versions --json`, {
|
||||
encoding: "utf8",
|
||||
stdio: ["ignore", "pipe", "ignore"],
|
||||
}).trim();
|
||||
|
||||
if (raw) {
|
||||
const parsed = JSON.parse(raw);
|
||||
versions = Array.isArray(parsed) ? parsed : [parsed];
|
||||
}
|
||||
} catch {
|
||||
versions = [];
|
||||
}
|
||||
|
||||
for (const version of versions) {
|
||||
const match = version.match(pattern);
|
||||
if (!match) continue;
|
||||
max = Math.max(max, Number(match[1]));
|
||||
}
|
||||
}
|
||||
|
||||
process.stdout.write(`${stableSlot}.${max + 1}`);
|
||||
NODE
|
||||
}
|
||||
|
||||
next_canary_version() {
|
||||
local stable_version="$1"
|
||||
shift
|
||||
|
||||
@@ -10,6 +10,7 @@ channel=""
|
||||
release_date=""
|
||||
dry_run=false
|
||||
skip_verify=false
|
||||
print_version_only=false
|
||||
tag_name=""
|
||||
|
||||
cleanup_on_exit=false
|
||||
@@ -17,20 +18,23 @@ cleanup_on_exit=false
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/release.sh <canary|stable> [--date YYYY-MM-DD] [--dry-run] [--skip-verify]
|
||||
./scripts/release.sh <canary|stable> [--date YYYY-MM-DD] [--dry-run] [--skip-verify] [--print-version]
|
||||
|
||||
Examples:
|
||||
./scripts/release.sh canary
|
||||
./scripts/release.sh canary --date 2026-03-17 --dry-run
|
||||
./scripts/release.sh stable
|
||||
./scripts/release.sh stable --date 2026-03-17 --dry-run
|
||||
./scripts/release.sh stable --date 2026-03-18 --print-version
|
||||
|
||||
Notes:
|
||||
- Canary releases publish YYYY.M.D-canary.N under the npm dist-tag "canary"
|
||||
and create the git tag canary/vYYYY.M.D-canary.N.
|
||||
- Stable releases publish YYYY.M.D under the npm dist-tag "latest" and create
|
||||
the git tag vYYYY.M.D.
|
||||
- Stable release notes must already exist at releases/vYYYY.M.D.md.
|
||||
- Stable versions use YYYY.MDD.P, where M is the UTC month, DD is the
|
||||
zero-padded UTC day, and P is the same-day stable patch slot.
|
||||
- Canary releases publish YYYY.MDD.P-canary.N under the npm dist-tag
|
||||
"canary" and create the git tag canary/vYYYY.MDD.P-canary.N.
|
||||
- Stable releases publish YYYY.MDD.P under the npm dist-tag "latest" and
|
||||
create the git tag vYYYY.MDD.P.
|
||||
- Stable release notes must already exist at releases/vYYYY.MDD.P.md.
|
||||
- The script rewrites versions temporarily and restores the working tree on
|
||||
exit. Tags always point at the original source commit, not a generated
|
||||
release commit.
|
||||
@@ -94,6 +98,7 @@ while [ $# -gt 0 ]; do
|
||||
;;
|
||||
--dry-run) dry_run=true ;;
|
||||
--skip-verify) skip_verify=true ;;
|
||||
--print-version) print_version_only=true ;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
@@ -118,15 +123,20 @@ CURRENT_SHA="$(git -C "$REPO_ROOT" rev-parse HEAD)"
|
||||
LAST_STABLE_TAG="$(get_last_stable_tag)"
|
||||
CURRENT_STABLE_VERSION="$(get_current_stable_version)"
|
||||
RELEASE_DATE="${release_date:-$(utc_date_iso)}"
|
||||
TARGET_STABLE_VERSION="$(stable_version_for_date "$RELEASE_DATE")"
|
||||
TARGET_PUBLISH_VERSION="$TARGET_STABLE_VERSION"
|
||||
DIST_TAG="latest"
|
||||
|
||||
PUBLIC_PACKAGE_INFO="$(list_public_package_info)"
|
||||
mapfile -t PUBLIC_PACKAGE_NAMES < <(printf '%s\n' "$PUBLIC_PACKAGE_INFO" | cut -f2)
|
||||
PUBLIC_PACKAGE_NAMES=()
|
||||
while IFS= read -r package_name; do
|
||||
[ -n "$package_name" ] || continue
|
||||
PUBLIC_PACKAGE_NAMES+=("$package_name")
|
||||
done < <(printf '%s\n' "$PUBLIC_PACKAGE_INFO" | cut -f2)
|
||||
|
||||
[ -n "$PUBLIC_PACKAGE_INFO" ] || release_fail "no public packages were found in the workspace."
|
||||
|
||||
TARGET_STABLE_VERSION="$(next_stable_version "$RELEASE_DATE" "${PUBLIC_PACKAGE_NAMES[@]}")"
|
||||
TARGET_PUBLISH_VERSION="$TARGET_STABLE_VERSION"
|
||||
DIST_TAG="latest"
|
||||
|
||||
if [ "$channel" = "canary" ]; then
|
||||
require_on_master_branch
|
||||
TARGET_PUBLISH_VERSION="$(next_canary_version "$TARGET_STABLE_VERSION" "${PUBLIC_PACKAGE_NAMES[@]}")"
|
||||
@@ -136,6 +146,11 @@ else
|
||||
tag_name="$(stable_tag_name "$TARGET_STABLE_VERSION")"
|
||||
fi
|
||||
|
||||
if [ "$print_version_only" = true ]; then
|
||||
printf '%s\n' "$TARGET_PUBLISH_VERSION"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
NOTES_FILE="$(release_notes_file "$TARGET_STABLE_VERSION")"
|
||||
|
||||
require_clean_worktree
|
||||
|
||||
@@ -12,8 +12,8 @@ Usage:
|
||||
./scripts/rollback-latest.sh <stable-version> [--dry-run]
|
||||
|
||||
Examples:
|
||||
./scripts/rollback-latest.sh 2026.3.17
|
||||
./scripts/rollback-latest.sh 2026.3.17 --dry-run
|
||||
./scripts/rollback-latest.sh 2026.318.0
|
||||
./scripts/rollback-latest.sh 2026.318.0 --dry-run
|
||||
|
||||
Notes:
|
||||
- This repoints the npm dist-tag "latest" for every public package.
|
||||
@@ -45,7 +45,7 @@ if [ -z "$version" ]; then
|
||||
fi
|
||||
|
||||
if [[ ! "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo "Error: version must be a stable calendar version like 2026.3.17." >&2
|
||||
echo "Error: version must be a stable calendar version like 2026.318.0." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
146
tests/release-smoke/docker-auth-onboarding.spec.ts
Normal file
146
tests/release-smoke/docker-auth-onboarding.spec.ts
Normal file
@@ -0,0 +1,146 @@
|
||||
import { expect, test, type Page } from "@playwright/test";
|
||||
|
||||
const ADMIN_EMAIL =
|
||||
process.env.PAPERCLIP_RELEASE_SMOKE_EMAIL ??
|
||||
process.env.SMOKE_ADMIN_EMAIL ??
|
||||
"smoke-admin@paperclip.local";
|
||||
const ADMIN_PASSWORD =
|
||||
process.env.PAPERCLIP_RELEASE_SMOKE_PASSWORD ??
|
||||
process.env.SMOKE_ADMIN_PASSWORD ??
|
||||
"paperclip-smoke-password";
|
||||
|
||||
const COMPANY_NAME = `Release-Smoke-${Date.now()}`;
|
||||
const AGENT_NAME = "CEO";
|
||||
const TASK_TITLE = "Release smoke task";
|
||||
|
||||
async function signIn(page: Page) {
|
||||
await page.goto("/");
|
||||
await expect(page).toHaveURL(/\/auth/);
|
||||
|
||||
await page.locator('input[type="email"]').fill(ADMIN_EMAIL);
|
||||
await page.locator('input[type="password"]').fill(ADMIN_PASSWORD);
|
||||
await page.getByRole("button", { name: "Sign In" }).click();
|
||||
|
||||
await expect(page).not.toHaveURL(/\/auth/, { timeout: 20_000 });
|
||||
}
|
||||
|
||||
async function openOnboarding(page: Page) {
|
||||
const wizardHeading = page.locator("h3", { hasText: "Name your company" });
|
||||
const startButton = page.getByRole("button", { name: "Start Onboarding" });
|
||||
|
||||
await expect(wizardHeading.or(startButton)).toBeVisible({ timeout: 20_000 });
|
||||
|
||||
if (await startButton.isVisible()) {
|
||||
await startButton.click();
|
||||
}
|
||||
|
||||
await expect(wizardHeading).toBeVisible({ timeout: 10_000 });
|
||||
}
|
||||
|
||||
test.describe("Docker authenticated onboarding smoke", () => {
|
||||
test("logs in, completes onboarding, and triggers the first CEO run", async ({
|
||||
page,
|
||||
}) => {
|
||||
await signIn(page);
|
||||
await openOnboarding(page);
|
||||
|
||||
await page.locator('input[placeholder="Acme Corp"]').fill(COMPANY_NAME);
|
||||
await page.getByRole("button", { name: "Next" }).click();
|
||||
|
||||
await expect(
|
||||
page.locator("h3", { hasText: "Create your first agent" })
|
||||
).toBeVisible({ timeout: 10_000 });
|
||||
|
||||
await expect(page.locator('input[placeholder="CEO"]')).toHaveValue(AGENT_NAME);
|
||||
await page.getByRole("button", { name: "Process" }).click();
|
||||
await page.locator('input[placeholder="e.g. node, python"]').fill("echo");
|
||||
await page
|
||||
.locator('input[placeholder="e.g. script.js, --flag"]')
|
||||
.fill("release smoke");
|
||||
await page.getByRole("button", { name: "Next" }).click();
|
||||
|
||||
await expect(
|
||||
page.locator("h3", { hasText: "Give it something to do" })
|
||||
).toBeVisible({ timeout: 10_000 });
|
||||
await page
|
||||
.locator('input[placeholder="e.g. Research competitor pricing"]')
|
||||
.fill(TASK_TITLE);
|
||||
await page.getByRole("button", { name: "Next" }).click();
|
||||
|
||||
await expect(
|
||||
page.locator("h3", { hasText: "Ready to launch" })
|
||||
).toBeVisible({ timeout: 10_000 });
|
||||
await expect(page.getByText(COMPANY_NAME)).toBeVisible();
|
||||
await expect(page.getByText(AGENT_NAME)).toBeVisible();
|
||||
await expect(page.getByText(TASK_TITLE)).toBeVisible();
|
||||
|
||||
await page.getByRole("button", { name: "Create & Open Issue" }).click();
|
||||
await expect(page).toHaveURL(/\/issues\//, { timeout: 10_000 });
|
||||
|
||||
const baseUrl = new URL(page.url()).origin;
|
||||
|
||||
const companiesRes = await page.request.get(`${baseUrl}/api/companies`);
|
||||
expect(companiesRes.ok()).toBe(true);
|
||||
const companies = (await companiesRes.json()) as Array<{ id: string; name: string }>;
|
||||
const company = companies.find((entry) => entry.name === COMPANY_NAME);
|
||||
expect(company).toBeTruthy();
|
||||
|
||||
const agentsRes = await page.request.get(
|
||||
`${baseUrl}/api/companies/${company!.id}/agents`
|
||||
);
|
||||
expect(agentsRes.ok()).toBe(true);
|
||||
const agents = (await agentsRes.json()) as Array<{
|
||||
id: string;
|
||||
name: string;
|
||||
role: string;
|
||||
adapterType: string;
|
||||
}>;
|
||||
const ceoAgent = agents.find((entry) => entry.name === AGENT_NAME);
|
||||
expect(ceoAgent).toBeTruthy();
|
||||
expect(ceoAgent!.role).toBe("ceo");
|
||||
expect(ceoAgent!.adapterType).toBe("process");
|
||||
|
||||
const issuesRes = await page.request.get(
|
||||
`${baseUrl}/api/companies/${company!.id}/issues`
|
||||
);
|
||||
expect(issuesRes.ok()).toBe(true);
|
||||
const issues = (await issuesRes.json()) as Array<{
|
||||
id: string;
|
||||
title: string;
|
||||
assigneeAgentId: string | null;
|
||||
}>;
|
||||
const issue = issues.find((entry) => entry.title === TASK_TITLE);
|
||||
expect(issue).toBeTruthy();
|
||||
expect(issue!.assigneeAgentId).toBe(ceoAgent!.id);
|
||||
|
||||
await expect.poll(
|
||||
async () => {
|
||||
const runsRes = await page.request.get(
|
||||
`${baseUrl}/api/companies/${company!.id}/heartbeat-runs?agentId=${ceoAgent!.id}`
|
||||
);
|
||||
expect(runsRes.ok()).toBe(true);
|
||||
const runs = (await runsRes.json()) as Array<{
|
||||
agentId: string;
|
||||
invocationSource: string;
|
||||
status: string;
|
||||
}>;
|
||||
const latestRun = runs.find((entry) => entry.agentId === ceoAgent!.id);
|
||||
return latestRun
|
||||
? {
|
||||
invocationSource: latestRun.invocationSource,
|
||||
status: latestRun.status,
|
||||
}
|
||||
: null;
|
||||
},
|
||||
{
|
||||
timeout: 30_000,
|
||||
intervals: [1_000, 2_000, 5_000],
|
||||
}
|
||||
).toEqual(
|
||||
expect.objectContaining({
|
||||
invocationSource: "assignment",
|
||||
status: expect.stringMatching(/^(queued|running|succeeded)$/),
|
||||
})
|
||||
);
|
||||
});
|
||||
});
|
||||
28
tests/release-smoke/playwright.config.ts
Normal file
28
tests/release-smoke/playwright.config.ts
Normal file
@@ -0,0 +1,28 @@
|
||||
import { defineConfig } from "@playwright/test";
|
||||
|
||||
const BASE_URL =
|
||||
process.env.PAPERCLIP_RELEASE_SMOKE_BASE_URL ?? "http://127.0.0.1:3232";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: ".",
|
||||
testMatch: "**/*.spec.ts",
|
||||
timeout: 90_000,
|
||||
expect: {
|
||||
timeout: 15_000,
|
||||
},
|
||||
retries: process.env.CI ? 1 : 0,
|
||||
use: {
|
||||
baseURL: BASE_URL,
|
||||
headless: true,
|
||||
screenshot: "only-on-failure",
|
||||
trace: "retain-on-failure",
|
||||
},
|
||||
projects: [
|
||||
{
|
||||
name: "chromium",
|
||||
use: { browserName: "chromium" },
|
||||
},
|
||||
],
|
||||
outputDir: "./test-results",
|
||||
reporter: [["list"], ["html", { open: "never", outputFolder: "./playwright-report" }]],
|
||||
});
|
||||
Reference in New Issue
Block a user