diff --git a/doc/OPENCLAW_ONBOARDING.md b/doc/OPENCLAW_ONBOARDING.md index e31a6f8b..b55e755a 100644 --- a/doc/OPENCLAW_ONBOARDING.md +++ b/doc/OPENCLAW_ONBOARDING.md @@ -25,29 +25,53 @@ Open the printed `Dashboard URL` (includes `#token=...`) in your browser. 5. Approve the join request in Paperclip UI, then confirm the OpenClaw agent appears in CLA agents. -6. Case A (manual issue test). +6. Gateway preflight (required before task tests). +- Confirm the created agent uses `openclaw_gateway` (not `openclaw`). +- Confirm gateway URL is `ws://...` or `wss://...`. +- Confirm gateway token is non-trivial (not empty / not 1-char placeholder). +- Confirm pairing mode is explicit: + - recommended default: `adapterConfig.disableDeviceAuth` is false/absent and `adapterConfig.devicePrivateKeyPem` is present + - fallback only: `adapterConfig.disableDeviceAuth=true` when pairing cannot be supported in that environment +- If you can run API checks with board auth: +```bash +AGENT_ID="" +curl -sS -H "Cookie: $PAPERCLIP_COOKIE" "http://127.0.0.1:3100/api/agents/$AGENT_ID" | jq '{adapterType,adapterConfig:{url:.adapterConfig.url,tokenLen:(.adapterConfig.headers["x-openclaw-token"] // .adapterConfig.headers["x-openclaw-auth"] // "" | length),disableDeviceAuth:(.adapterConfig.disableDeviceAuth // false),hasDeviceKey:(.adapterConfig.devicePrivateKeyPem // "" | length > 0)}}' +``` +- Expected: `adapterType=openclaw_gateway`, `tokenLen >= 16`, `hasDeviceKey=true`, and `disableDeviceAuth=false`. + +Pairing handshake note: +- The first gateway run may return `pairing required` once for a new device key. +- Approve it in OpenClaw, then retry the task. +- For local docker smoke, you can approve from host: +```bash +docker exec openclaw-docker-openclaw-gateway-1 sh -lc 'openclaw devices approve --latest --json --url "ws://127.0.0.1:18789" --token "$(node -p \"require(process.env.HOME+\\\"/.openclaw/openclaw.json\\\").gateway.auth.token\")"' +``` + +7. Case A (manual issue test). - Create an issue assigned to the OpenClaw agent. - Put instructions: “post comment `OPENCLAW_CASE_A_OK_` and mark done.” - Verify in UI: issue status becomes `done` and comment exists. -7. Case B (message tool test). +8. Case B (message tool test). - Create another issue assigned to OpenClaw. - Instructions: “send `OPENCLAW_CASE_B_OK_` to main webchat via message tool, then comment same marker on issue, then mark done.” - Verify both: - marker comment on issue - marker text appears in OpenClaw main chat -8. Case C (new session memory/skills test). +9. Case C (new session memory/skills test). - In OpenClaw, start `/new` session. - Ask it to create a new CLA issue in Paperclip with unique title `OPENCLAW_CASE_C_CREATED_`. - Verify in Paperclip UI that new issue exists. -9. Watch logs during test (optional but helpful): +10. Watch logs during test (optional but helpful): ```bash docker compose -f /tmp/openclaw-docker/docker-compose.yml -f /tmp/openclaw-docker/.paperclip-openclaw.override.yml logs -f openclaw-gateway ``` -10. Expected pass criteria. +11. Expected pass criteria. +- Preflight: `openclaw_gateway` + non-placeholder token (`tokenLen >= 16`). +- Pairing mode: stable `devicePrivateKeyPem` configured with device auth enabled (default path). - Case A: `done` + marker comment. - Case B: `done` + marker comment + main-chat message visible. - Case C: original task done and new issue created from `/new` session. diff --git a/doc/plugins/PLUGIN_SPEC.md b/doc/plugins/PLUGIN_SPEC.md new file mode 100644 index 00000000..5c3d3993 --- /dev/null +++ b/doc/plugins/PLUGIN_SPEC.md @@ -0,0 +1,1140 @@ +# Paperclip Plugin System Specification + +Status: proposed complete spec for the post-V1 plugin system + +This document is the complete specification for Paperclip's plugin and extension architecture. +It expands the brief plugin notes in [doc/SPEC.md](../SPEC.md) and should be read alongside the comparative analysis in [doc/plugins/ideas-from-opencode.md](./ideas-from-opencode.md). + +This is not part of the V1 implementation contract in [doc/SPEC-implementation.md](../SPEC-implementation.md). +It is the full target architecture for the plugin system that should follow V1. + +## 1. Scope + +This spec covers: + +- plugin packaging and installation +- runtime model +- trust model +- capability system +- UI extension surfaces +- event, job, and webhook surfaces +- workspace-oriented extension surfaces +- Postgres persistence for extensions +- operator workflows +- compatibility and upgrade rules + +This spec does not cover: + +- a public marketplace +- cloud/SaaS multi-tenancy +- arbitrary third-party schema migrations in the first plugin version +- arbitrary frontend bundle injection in the first plugin version + +## 2. Core Assumptions + +Paperclip plugin design is based on the following assumptions: + +1. Paperclip is single-tenant and self-hosted. +2. Plugin installation is global to the instance. +3. "Companies" remain core Paperclip business objects, but they are not plugin trust boundaries. +4. Board governance, approval gates, budget hard-stops, and core task invariants remain owned by Paperclip core. +5. Projects already have a real workspace model via `project_workspaces`, and local/runtime plugins should build on that instead of inventing a separate workspace abstraction. + +## 3. Goals + +The plugin system must: + +1. Let operators install global instance-wide plugins. +2. Let plugins add major capabilities without editing Paperclip core. +3. Keep core governance and auditing intact. +4. Support both local/runtime plugins and external SaaS connectors. +5. Support future plugin categories such as: + - new agent adapters + - revenue tracking + - knowledge base + - issue tracker sync + - metrics/dashboards + - file/project tooling +6. Use simple, explicit, typed contracts. +7. Keep failures isolated so one plugin does not crash the entire instance. + +## 4. Non-Goals + +The first plugin system must not: + +1. Allow arbitrary plugins to override core routes or core invariants. +2. Allow arbitrary plugins to mutate approval, auth, issue checkout, or budget enforcement logic. +3. Allow arbitrary third-party plugins to run free-form DB migrations. +4. Depend on project-local plugin folders such as `.paperclip/plugins`. +5. Depend on automatic install-and-execute behavior at server startup from arbitrary config files. + +## 5. Terminology + +### 5.1 Instance + +The single Paperclip deployment an operator installs and controls. + +### 5.2 Company + +A first-class Paperclip business object inside the instance. + +### 5.3 Project Workspace + +A workspace attached to a project through `project_workspaces`. +This is the primary local runtime anchor for file, terminal, git, and process tooling. + +### 5.4 Platform Module + +A trusted in-process extension loaded directly by Paperclip core. + +Examples: + +- agent adapters +- storage providers +- secret providers +- run-log backends + +### 5.5 Plugin + +An installable instance-wide extension package loaded through the Paperclip plugin runtime. + +Examples: + +- Linear sync +- GitHub Issues sync +- Grafana widgets +- Stripe revenue sync +- file browser +- terminal +- git workflow + +### 5.6 Plugin Worker + +The runtime process used for a plugin. +In this spec, third-party plugins run out-of-process by default. + +### 5.7 Capability + +A named permission the host grants to a plugin. +Plugins may only call host APIs that are covered by granted capabilities. + +## 6. Extension Classes + +Paperclip has two extension classes. + +## 6.1 Platform Modules + +Platform modules are: + +- trusted +- in-process +- host-integrated +- low-level + +They use explicit registries, not the general plugin worker protocol. + +Platform module surfaces: + +- `registerAgentAdapter()` +- `registerStorageProvider()` +- `registerSecretProvider()` +- `registerRunLogStore()` +- future `registerWorkspaceRuntime()` if needed + +Platform modules are the right place for: + +- new agent adapter packages +- new storage backends +- new secret backends +- other host-internal systems that need direct process or DB integration + +## 6.2 Plugins + +Plugins are: + +- globally installed per instance +- loaded through the plugin runtime +- additive +- capability-gated +- isolated from core via a stable SDK and host protocol + +Plugin categories: + +- `connector` +- `workspace` +- `automation` +- `ui` + +A plugin may declare more than one category. + +## 7. Project Workspaces Are The Local Tooling Anchor + +Paperclip already has a concrete workspace model: + +- projects expose `workspaces` +- projects expose `primaryWorkspace` +- the database contains `project_workspaces` +- project routes already manage workspaces +- heartbeat resolution already prefers project workspaces before falling back to task-session or agent-home workspaces + +Therefore: + +1. File plugins should browse project workspaces first. +2. Terminal sessions should launch against project workspaces by default. +3. Git plugins should treat the selected project workspace as the repo root anchor. +4. Process/server tracking should attach to project workspaces whenever possible. +5. Issue and agent views may deep-link into project workspace context. + +Project workspaces may exist in two modes: + +- local directory mode: `cwd` is present +- repo-only mode: `repoUrl` and optional `repoRef` exist, but there is no local `cwd` + +Plugins must handle repo-only workspaces explicitly: + +- they may show metadata +- they may show sync state +- they may not assume local file/PTY/git access until a real `cwd` exists + +## 8. Installation Model + +Plugin installation is global and operator-driven. + +There is no per-company install table and no per-company enable/disable switch. + +If a plugin needs business-object-specific mappings, those are stored as plugin configuration or plugin state. + +Examples: + +- one global Linear plugin install +- mappings from company A to Linear team X and company B to Linear team Y +- one global git plugin install +- per-project workspace state stored under `project_workspace` + +## 8.1 On-Disk Layout + +Plugins live under the Paperclip instance directory. + +Suggested layout: + +- `~/.paperclip/instances/default/plugins/package.json` +- `~/.paperclip/instances/default/plugins/node_modules/` +- `~/.paperclip/instances/default/plugins/.cache/` +- `~/.paperclip/instances/default/data/plugins//` + +The package install directory and the plugin data directory are separate. + +## 8.2 Operator Commands + +Paperclip should add CLI commands: + +- `pnpm paperclipai plugin list` +- `pnpm paperclipai plugin install ` +- `pnpm paperclipai plugin uninstall ` +- `pnpm paperclipai plugin upgrade [version]` +- `pnpm paperclipai plugin doctor ` + +These commands are instance-level operations. + +## 8.3 Install Process + +The install process is: + +1. Resolve npm package and version. +2. Install into the instance plugin directory. +3. Read and validate plugin manifest. +4. Reject incompatible plugin API versions. +5. Display requested capabilities to the operator. +6. Persist install record in Postgres. +7. Start plugin worker and run health/validation. +8. Mark plugin `ready` or `error`. + +## 9. Load Order And Precedence + +Load order must be deterministic. + +1. core platform modules +2. built-in first-party plugins +3. installed plugins sorted by: + - explicit operator-configured order if present + - otherwise manifest `id` + +Rules: + +- plugin contributions are additive by default +- plugins may not override core routes or core actions by name collision +- if two plugins contribute the same route slug or UI slot id, the host must reject startup or force the operator to resolve the collision explicitly + +## 10. Package Contract + +Each plugin package must export a manifest and a worker entrypoint. + +Suggested package layout: + +- `dist/manifest.js` +- `dist/worker.js` + +Suggested `package.json` keys: + +```json +{ + "name": "@paperclip/plugin-linear", + "version": "0.1.0", + "paperclipPlugin": { + "manifest": "./dist/manifest.js", + "worker": "./dist/worker.js" + } +} +``` + +## 10.1 Manifest Shape + +Normative manifest shape: + +```ts +export interface PaperclipPluginManifestV1 { + id: string; + apiVersion: 1; + version: string; + displayName: string; + description: string; + categories: Array<"connector" | "workspace" | "automation" | "ui">; + minimumPaperclipVersion?: string; + capabilities: string[]; + entrypoints: { + worker: string; + }; + instanceConfigSchema?: JsonSchema; + jobs?: PluginJobDeclaration[]; + webhooks?: PluginWebhookDeclaration[]; + ui?: PluginUiDeclaration; +} +``` + +Rules: + +- `id` must be globally unique +- `id` should normally equal the npm package name +- `apiVersion` must match the host-supported plugin API version +- `capabilities` must be static and install-time visible +- config schema must be JSON Schema compatible + +## 11. Runtime Model + +## 11.1 Process Model + +Third-party plugins run out-of-process by default. + +Default runtime: + +- Paperclip server starts one worker process per installed plugin +- the worker process is a Node process +- host and worker communicate over JSON-RPC on stdio + +This design provides: + +- failure isolation +- clearer logging boundaries +- easier resource limits +- a cleaner trust boundary than arbitrary in-process execution + +## 11.2 Host Responsibilities + +The host is responsible for: + +- package install +- manifest validation +- capability enforcement +- process supervision +- job scheduling +- webhook routing +- activity log writes +- secret resolution +- workspace service enforcement +- UI route registration + +## 11.3 Worker Responsibilities + +The plugin worker is responsible for: + +- validating its own config +- handling domain events +- handling scheduled jobs +- handling webhooks +- producing UI view models +- invoking host services through the SDK +- reporting health information + +## 11.4 Failure Policy + +If a worker fails: + +- mark plugin status `error` +- surface error in plugin health UI +- keep the rest of the instance running +- retry start with bounded backoff +- do not drop other plugins or core services + +## 12. Host-Worker Protocol + +The host must support the following worker RPC methods. + +Required methods: + +- `initialize(input)` +- `health()` +- `shutdown()` + +Optional methods: + +- `validateConfig(input)` +- `onEvent(input)` +- `runJob(input)` +- `handleWebhook(input)` +- `getPageModel(input)` +- `getWidgetModel(input)` +- `getDetailTabModel(input)` +- `performAction(input)` + +### 12.1 `initialize` + +Called once on worker startup. + +Input includes: + +- plugin manifest +- resolved plugin config +- instance info +- host API version + +### 12.2 `health` + +Returns: + +- status +- current error if any +- optional plugin-reported diagnostics + +### 12.3 `validateConfig` + +Runs after config changes and startup. + +Returns: + +- `ok` +- warnings +- errors + +### 12.4 `onEvent` + +Receives one typed Paperclip domain event. + +Delivery semantics: + +- at least once +- plugin must be idempotent +- no global ordering guarantee across all event types +- per-entity ordering is best effort but not guaranteed after retries + +### 12.5 `runJob` + +Runs a declared scheduled job. + +The host provides: + +- job key +- trigger source +- run id +- schedule metadata + +### 12.6 `handleWebhook` + +Receives inbound webhook payload routed by the host. + +The host provides: + +- endpoint key +- headers +- raw body +- parsed body if applicable +- request id + +### 12.7 `getPageModel` + +Returns a schema-driven view model for the plugin's main page. + +### 12.8 `getWidgetModel` + +Returns a schema-driven view model for a dashboard widget. + +### 12.9 `getDetailTabModel` + +Returns a schema-driven view model for a project, issue, agent, goal, or run detail tab. + +### 12.10 `performAction` + +Runs an explicit plugin action initiated by the board UI. + +Examples: + +- "resync now" +- "link GitHub issue" +- "create branch from issue" +- "restart process" + +## 13. SDK Surface + +Plugins do not talk to the DB directly. +Plugins do not read raw secret material from persisted config. +Plugins do not touch the filesystem directly outside the host services. + +The SDK exposed to workers must provide typed host clients. + +Required SDK clients: + +- `ctx.config` +- `ctx.events` +- `ctx.jobs` +- `ctx.http` +- `ctx.secrets` +- `ctx.assets` +- `ctx.activity` +- `ctx.state` +- `ctx.entities` +- `ctx.projects` +- `ctx.issues` +- `ctx.agents` +- `ctx.goals` +- `ctx.workspace` +- `ctx.logger` + +## 13.1 Example SDK Shape + +```ts +export interface PluginContext { + manifest: PaperclipPluginManifestV1; + config: { + get(): Promise>; + }; + events: { + on(name: string, fn: (event: unknown) => Promise): void; + }; + jobs: { + register(key: string, input: { cron: string }, fn: (job: PluginJobContext) => Promise): void; + }; + state: { + get(input: ScopeKey): Promise; + set(input: ScopeKey, value: unknown): Promise; + delete(input: ScopeKey): Promise; + }; + entities: { + upsert(input: PluginEntityUpsert): Promise; + list(input: PluginEntityQuery): Promise; + }; + workspace: WorkspacePluginApi; +} +``` + +## 14. Capability Model + +Capabilities are mandatory and static. +Every plugin declares them up front. + +The host enforces capabilities in the SDK layer and refuses calls outside the granted set. + +## 14.1 Capability Categories + +### Data Read + +- `companies.read` +- `projects.read` +- `project.workspaces.read` +- `issues.read` +- `issue.comments.read` +- `agents.read` +- `goals.read` +- `activity.read` +- `costs.read` + +### Data Write + +- `issues.create` +- `issues.update` +- `issue.comments.create` +- `assets.write` +- `assets.read` +- `activity.log.write` +- `metrics.write` + +### Runtime / Integration + +- `events.subscribe` +- `jobs.schedule` +- `webhooks.receive` +- `http.outbound` +- `secrets.read-ref` + +### UI + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.detailTab.register` +- `ui.dashboardWidget.register` +- `ui.action.register` + +### Workspace + +- `workspace.fs.read` +- `workspace.fs.write` +- `workspace.fs.stat` +- `workspace.fs.search` +- `workspace.pty.open` +- `workspace.pty.input` +- `workspace.pty.resize` +- `workspace.pty.terminate` +- `workspace.pty.subscribe` +- `workspace.git.status` +- `workspace.git.diff` +- `workspace.git.log` +- `workspace.git.branch.create` +- `workspace.git.commit` +- `workspace.git.worktree.create` +- `workspace.git.push` +- `workspace.process.register` +- `workspace.process.list` +- `workspace.process.read` +- `workspace.process.terminate` +- `workspace.process.restart` +- `workspace.process.logs.read` +- `workspace.http.probe` + +## 14.2 Forbidden Capabilities + +The host must not expose capabilities for: + +- approval decisions +- budget override +- auth bypass +- issue checkout lock override +- direct DB access +- direct filesystem access outside approved workspace services + +## 14.3 Upgrade Rules + +If a plugin upgrade adds capabilities: + +1. the host must mark the plugin `upgrade_pending` +2. the operator must explicitly approve the new capability set +3. the new version does not become `ready` until approval completes + +## 15. Event System + +The host must emit typed domain events that plugins may subscribe to. + +Minimum event set: + +- `company.created` +- `company.updated` +- `project.created` +- `project.updated` +- `project.workspace_created` +- `project.workspace_updated` +- `project.workspace_deleted` +- `issue.created` +- `issue.updated` +- `issue.comment.created` +- `agent.created` +- `agent.updated` +- `agent.status_changed` +- `agent.run.started` +- `agent.run.finished` +- `agent.run.failed` +- `agent.run.cancelled` +- `approval.created` +- `approval.decided` +- `cost_event.created` +- `activity.logged` + +Each event must include: + +- event id +- event type +- occurred at +- actor metadata when applicable +- primary entity metadata +- typed payload + +## 16. Scheduled Jobs + +Plugins may declare scheduled jobs in their manifest. + +Job rules: + +1. Each job has a stable `job_key`. +2. The host is the scheduler of record. +3. The host prevents overlapping execution of the same plugin/job combination unless explicitly allowed later. +4. Every job run is recorded in Postgres. +5. Failed jobs are retryable. + +## 17. Webhooks + +Plugins may declare webhook endpoints in their manifest. + +Webhook route shape: + +- `POST /api/plugins/:pluginId/webhooks/:endpointKey` + +Rules: + +1. The host owns the public route. +2. The worker receives the request body through `handleWebhook`. +3. Signature verification happens in plugin code using secret refs resolved by the host. +4. Every delivery is recorded. +5. Webhook handling must be idempotent. + +## 18. UI Extension Model + +The first plugin UI system is schema-driven. + +The host renders plugin data using built-in UI components. +Plugins return view models, not arbitrary React bundles. + +## 18.1 Global Operator Routes + +- `/settings/plugins` +- `/settings/plugins/:pluginId` + +These routes are instance-level. + +## 18.2 Company-Context Routes + +- `/:companyPrefix/plugins/:pluginId` + +These routes exist because the board UI is organized around companies even though plugin installation is global. + +## 18.3 Detail Tabs + +Plugins may add tabs to: + +- project detail +- issue detail +- agent detail +- goal detail +- run detail + +Recommended route pattern: + +- `/:companyPrefix//:id?tab=` + +## 18.4 Dashboard Widgets + +Plugins may add cards or sections to the dashboard. + +## 18.5 Sidebar Entries + +Plugins may add sidebar links to: + +- global plugin settings +- company-context plugin pages + +## 18.6 Allowed View Model Types + +The host should support a limited set of schema-rendered components: + +- metric cards +- status lists +- tables +- timeseries charts +- markdown text +- key/value blocks +- action bars +- log views +- JSON/debug views + +Arbitrary frontend bundle injection is explicitly out of scope for the first plugin system. + +## 19. Workspace Service APIs + +Workspace service APIs are the foundation for local tooling plugins. + +All workspace APIs must route through the host and validate against known project workspace roots. + +## 19.1 Project Workspace APIs + +Required host APIs: + +- list project workspaces +- get project primary workspace +- resolve project workspace from issue +- resolve current workspace from agent/run when available + +## 19.2 File APIs + +- read file +- write file +- stat path +- search path or filename +- list directory + +All file APIs take a resolved workspace anchor plus a relative path. + +## 19.3 PTY APIs + +- open terminal session +- send input +- resize +- terminate +- subscribe to output + +PTY sessions should default to the selected project workspace when one exists. + +## 19.4 Git APIs + +- status +- diff +- log +- branch create +- worktree create +- commit +- push + +Git APIs require a local `cwd`. +If the workspace is repo-only, the host must reject local git operations until a local checkout exists. + +## 19.5 Process APIs + +- register process +- list processes +- read process metadata +- terminate +- restart +- read logs +- probe health endpoint + +Process tracking should attach to `project_workspace` when possible. + +## 20. Persistence And Postgres + +## 20.1 Database Principles + +1. Core Paperclip data stays in first-party tables. +2. Most plugin-owned data starts in generic extension tables. +3. Plugin data should scope to existing Paperclip objects before new tables are introduced. +4. Arbitrary third-party schema migrations are out of scope for the first plugin system. + +## 20.2 Core Table Reuse + +If data becomes part of the actual Paperclip product model, it should become a first-party table. + +Examples: + +- `project_workspaces` is already first-party +- if Paperclip later decides git state is core product data, it should become a first-party table too + +## 20.3 Required Tables + +### `plugins` + +- `id` uuid pk +- `plugin_key` text unique not null +- `package_name` text not null +- `version` text not null +- `api_version` int not null +- `categories` text[] not null +- `manifest_json` jsonb not null +- `status` enum: `installed | ready | error | upgrade_pending` +- `install_order` int null +- `installed_at` timestamptz not null +- `updated_at` timestamptz not null +- `last_error` text null + +Indexes: + +- unique `plugin_key` +- `status` + +### `plugin_config` + +- `id` uuid pk +- `plugin_id` uuid fk `plugins.id` unique not null +- `config_json` jsonb not null +- `installed_at` timestamptz not null +- `updated_at` timestamptz not null +- `last_error` text null + +### `plugin_state` + +- `id` uuid pk +- `plugin_id` uuid fk `plugins.id` not null +- `scope_kind` enum: `instance | company | project | project_workspace | agent | issue | goal | run` +- `scope_id` uuid/text null +- `namespace` text not null +- `state_key` text not null +- `value_json` jsonb not null +- `updated_at` timestamptz not null + +Constraints: + +- unique `(plugin_id, scope_kind, scope_id, namespace, state_key)` + +Examples: + +- Linear external IDs keyed by `issue` +- GitHub sync cursors keyed by `project` +- file browser preferences keyed by `project_workspace` +- git branch metadata keyed by `project_workspace` +- process metadata keyed by `project_workspace` or `run` + +### `plugin_jobs` + +- `id` uuid pk +- `plugin_id` uuid fk `plugins.id` not null +- `scope_kind` enum nullable +- `scope_id` uuid/text null +- `job_key` text not null +- `schedule` text null +- `status` enum: `idle | queued | running | error` +- `next_run_at` timestamptz null +- `last_started_at` timestamptz null +- `last_finished_at` timestamptz null +- `last_succeeded_at` timestamptz null +- `last_error` text null + +Constraints: + +- unique `(plugin_id, scope_kind, scope_id, job_key)` + +### `plugin_job_runs` + +- `id` uuid pk +- `plugin_job_id` uuid fk `plugin_jobs.id` not null +- `plugin_id` uuid fk `plugins.id` not null +- `status` enum: `queued | running | succeeded | failed | cancelled` +- `trigger` enum: `schedule | manual | retry` +- `started_at` timestamptz null +- `finished_at` timestamptz null +- `error` text null +- `details_json` jsonb null + +Indexes: + +- `(plugin_id, started_at desc)` +- `(plugin_job_id, started_at desc)` + +### `plugin_webhook_deliveries` + +- `id` uuid pk +- `plugin_id` uuid fk `plugins.id` not null +- `scope_kind` enum nullable +- `scope_id` uuid/text null +- `endpoint_key` text not null +- `status` enum: `received | processed | failed | ignored` +- `request_id` text null +- `headers_json` jsonb null +- `body_json` jsonb null +- `received_at` timestamptz not null +- `handled_at` timestamptz null +- `response_code` int null +- `error` text null + +Indexes: + +- `(plugin_id, received_at desc)` +- `(plugin_id, endpoint_key, received_at desc)` + +### `plugin_entities` (optional but recommended) + +- `id` uuid pk +- `plugin_id` uuid fk `plugins.id` not null +- `entity_type` text not null +- `scope_kind` enum not null +- `scope_id` uuid/text null +- `external_id` text null +- `title` text null +- `status` text null +- `data_json` jsonb not null +- `created_at` timestamptz not null +- `updated_at` timestamptz not null + +Indexes: + +- `(plugin_id, entity_type, external_id)` unique when `external_id` is not null +- `(plugin_id, scope_kind, scope_id, entity_type)` + +Use cases: + +- imported Linear issues +- imported GitHub issues +- plugin-owned process records +- plugin-owned external metric bindings + +## 20.4 Activity Log Changes + +The activity log should extend `actor_type` to include `plugin`. + +New actor enum: + +- `agent` +- `user` +- `system` +- `plugin` + +Plugin-originated mutations should write: + +- `actor_type = plugin` +- `actor_id = ` + +## 20.5 Plugin Migrations + +The first plugin system does not allow arbitrary third-party migrations. + +Later, if custom tables become necessary, the system may add a trusted-module-only migration path. + +## 21. Secrets + +Plugin config must never persist raw secret values. + +Rules: + +1. Plugin config stores secret refs only. +2. Secret refs resolve through the existing Paperclip secret provider system. +3. Plugin workers receive resolved secrets only at execution time. +4. Secret values must never be written to: + - plugin config JSON + - activity logs + - webhook delivery rows + - error messages + +## 22. Auditing + +All plugin-originated mutating actions must be auditable. + +Minimum requirements: + +- activity log entry for every mutation +- job run history +- webhook delivery history +- plugin health page +- install/upgrade history in `plugins` + +## 23. Operator UX + +## 23.1 Global Settings + +Global plugin settings page must show: + +- installed plugins +- versions +- status +- requested capabilities +- current errors +- install/upgrade/remove actions + +## 23.2 Plugin Settings Page + +Each plugin may expose: + +- config form derived from `instanceConfigSchema` +- health details +- recent job history +- recent webhook history +- capability list + +Route: + +- `/settings/plugins/:pluginId` + +## 23.3 Company-Context Plugin Page + +Each plugin may expose a company-context main page: + +- `/:companyPrefix/plugins/:pluginId` + +This page is where board users do most day-to-day work. + +## 24. Example Mappings + +This spec directly supports the following plugin types: + +- `@paperclip/plugin-workspace-files` +- `@paperclip/plugin-terminal` +- `@paperclip/plugin-git` +- `@paperclip/plugin-linear` +- `@paperclip/plugin-github-issues` +- `@paperclip/plugin-grafana` +- `@paperclip/plugin-runtime-processes` +- `@paperclip/plugin-stripe` + +## 25. Compatibility And Versioning + +Rules: + +1. Host supports one or more explicit plugin API versions. +2. Plugin manifest declares exactly one `apiVersion`. +3. Host rejects unsupported versions at install time. +4. SDK packages are versioned with the host protocol. +5. Plugin upgrades are explicit operator actions. +6. Capability expansion requires explicit operator approval. + +## 26. Recommended Delivery Order + +## Phase 1 + +- plugin manifest +- install/list/remove/upgrade CLI +- global settings UI +- plugin process manager +- capability enforcement +- `plugins`, `plugin_config`, `plugin_state`, `plugin_jobs`, `plugin_job_runs`, `plugin_webhook_deliveries` +- event bus +- jobs +- webhooks +- settings page +- dashboard widget/page/tab schema rendering + +This phase is enough for: + +- Linear +- GitHub Issues +- Grafana +- Stripe + +## Phase 2 + +- project workspace service built on `project_workspaces` +- file APIs +- PTY APIs +- git APIs +- process APIs +- project-context tabs for plugin pages + +This phase is enough for: + +- file browser +- terminal +- git workflow +- process/server tracking + +## Phase 3 + +- optional `plugin_entities` +- richer action systems +- trusted-module migration path if truly needed +- optional richer frontend extension model +- plugin ecosystem/distribution work + +## 27. Final Design Decision + +Paperclip should not implement a generic in-process hook bag modeled directly after local coding tools. + +Paperclip should implement: + +- trusted platform modules for low-level host integration +- globally installed out-of-process plugins for additive instance-wide capabilities +- schema-driven UI contributions +- project workspace-based local tooling +- generic extension tables for most plugin state +- strict preservation of core governance and audit rules + +That is the complete target design for the Paperclip plugin system. diff --git a/doc/plugins/ideas-from-opencode.md b/doc/plugins/ideas-from-opencode.md new file mode 100644 index 00000000..ac4077f7 --- /dev/null +++ b/doc/plugins/ideas-from-opencode.md @@ -0,0 +1,1637 @@ +# Plugin Ideas From OpenCode + +Status: design report, not a V1 commitment + +Paperclip V1 explicitly excludes a plugin framework in [doc/SPEC-implementation.md](../SPEC-implementation.md), but the long-horizon spec says the architecture should leave room for extensions. This report studies the `opencode` plugin system and translates the useful patterns into a Paperclip-shaped design. + +Assumption for this document: Paperclip is a single-tenant operator-controlled instance. Plugin installation should therefore be global across the instance. "Companies" are still first-class Paperclip objects, but they are organizational records, not tenant-isolation boundaries for plugin trust or installation. + +## Executive Summary + +`opencode` has a real plugin system already. It is intentionally low-friction: + +- plugins are plain JS/TS modules +- they load from local directories and npm packages +- they can hook many runtime events +- they can add custom tools +- they can extend provider auth flows +- they run in-process and can mutate runtime behavior directly + +That model works well for a local coding tool. It should not be copied literally into Paperclip. + +The main conclusion is: + +- Paperclip should copy `opencode`'s typed SDK, deterministic loading, low authoring friction, and clear extension surfaces. +- Paperclip should not copy `opencode`'s trust model, project-local plugin loading, "override by name collision" behavior, or arbitrary in-process mutation hooks for core business logic. +- Paperclip should use multiple extension classes instead of one generic plugin bag: + - trusted in-process modules for low-level platform concerns like agent adapters, storage providers, secret providers, and possibly run-log backends + - out-of-process plugins for most third-party integrations like Linear, GitHub Issues, Grafana, Stripe, and schedulers + - schema-driven UI contributions for dashboard widgets, settings panels, and company pages + - a typed event bus plus scheduled jobs for automation + +If Paperclip does this well, the examples you listed become straightforward: + +- file browser / terminal / git workflow / child process tracking become workspace-runtime plugins built on first-party primitives +- Linear / GitHub / Grafana / Stripe become connector plugins +- future knowledge base and accounting features can also fit the same model + +## Sources Examined + +I cloned `anomalyco/opencode` and reviewed commit: + +- `a965a062595403a8e0083e85770315d5dc9628ab` + +Primary files reviewed: + +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/plugin/src/index.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/plugin/src/tool.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/opencode/src/plugin/index.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/opencode/src/config/config.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/opencode/src/tool/registry.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/opencode/src/provider/auth.ts` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/web/src/content/docs/plugins.mdx` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/web/src/content/docs/custom-tools.mdx` +- `https://github.com/anomalyco/opencode/blob/a965a062595403a8e0083e85770315d5dc9628ab/packages/web/src/content/docs/ecosystem.mdx` + +Relevant Paperclip files reviewed for current extension seams: + +- [server/src/adapters/registry.ts](../../server/src/adapters/registry.ts) +- [ui/src/adapters/registry.ts](../../ui/src/adapters/registry.ts) +- [server/src/storage/provider-registry.ts](../../server/src/storage/provider-registry.ts) +- [server/src/secrets/provider-registry.ts](../../server/src/secrets/provider-registry.ts) +- [server/src/services/run-log-store.ts](../../server/src/services/run-log-store.ts) +- [server/src/services/activity-log.ts](../../server/src/services/activity-log.ts) +- [doc/SPEC.md](../SPEC.md) +- [doc/SPEC-implementation.md](../SPEC-implementation.md) + +## What OpenCode Actually Implements + +## 1. Plugin authoring API + +`opencode` exposes a small package, `@opencode-ai/plugin`, with a typed `Plugin` function and a typed `tool()` helper. + +Core shape: + +- a plugin is an async function that receives a context object +- the plugin returns a `Hooks` object +- hooks are optional +- plugins can also contribute tools and auth providers + +The plugin init context includes: + +- an SDK client +- current project info +- current directory +- current git worktree +- server URL +- Bun shell access + +That is important: `opencode` gives plugins rich runtime power immediately, not a narrow capability API. + +## 2. Hook model + +The hook set is broad. It includes: + +- event subscription +- config-time hook +- message hooks +- model parameter/header hooks +- permission decision hooks +- shell env injection +- tool execution before/after hooks +- tool definition mutation +- compaction prompt customization +- text completion transforms + +The implementation pattern is very simple: + +- core code constructs an `output` object +- each matching plugin hook runs sequentially +- hooks mutate the `output` +- final mutated output is used by core + +This is elegant and easy to extend. + +It is also extremely powerful. A plugin can change auth headers, model params, permission answers, tool inputs, tool descriptions, and shell environment. + +## 3. Plugin discovery and load order + +`opencode` supports two plugin sources: + +- local files +- npm packages + +Local directories: + +- `~/.config/opencode/plugins/` +- `.opencode/plugins/` + +Npm plugins: + +- listed in config under `plugin: []` + +Load order is deterministic and documented: + +1. global config +2. project config +3. global plugin directory +4. project plugin directory + +Important details: + +- config arrays are concatenated rather than replaced +- duplicate plugin names are deduplicated with higher-precedence entries winning +- internal first-party plugins and default plugins are also loaded through the plugin pipeline + +This gives `opencode` a real precedence model rather than "whatever loaded last by accident." + +## 4. Dependency handling + +For local config/plugin directories, `opencode` will: + +- ensure a `package.json` exists +- inject `@opencode-ai/plugin` +- run `bun install` + +That lets local plugins and local custom tools import dependencies. + +This is excellent for local developer ergonomics. + +It is not a safe default for an operator-controlled control plane server. + +## 5. Error handling + +Plugin load failures do not hard-crash the runtime by default. + +Instead, `opencode`: + +- logs the error +- publishes a session error event +- continues loading other plugins + +That is a good operational pattern. One bad plugin should not brick the entire product unless the operator has explicitly configured it as required. + +## 6. Tools are a first-class extension point + +`opencode` has two ways to add tools: + +- export tools directly from a plugin via `hook.tool` +- define local files in `.opencode/tools/` or global tools directories + +The tool API is strong: + +- tools have descriptions +- tools have Zod schemas +- tool execution gets context like session ID, message ID, directory, and worktree +- tools are merged into the same registry as built-in tools +- tool definitions themselves can be mutated by a `tool.definition` hook + +The most aggressive part of the design: + +- custom tools can override built-in tools by name + +That is very powerful for a local coding assistant. +It is too dangerous for Paperclip core actions. + +## 7. Auth is also a plugin surface + +`opencode` allows plugins to register auth methods for providers. + +A plugin can contribute: + +- auth method metadata +- prompt flows +- OAuth flows +- API key flows +- request loaders that adapt provider behavior after auth succeeds + +This is a strong pattern worth copying. Integrations often need custom auth UX and token handling. + +## 8. Ecosystem evidence + +The ecosystem page is the best proof that the model is working in practice. +Community plugins already cover: + +- sandbox/workspace systems +- auth providers +- session headers / telemetry +- memory/context features +- scheduling +- notifications +- worktree helpers +- background agents +- monitoring + +That validates the main thesis: a simple typed plugin API can create real ecosystem velocity. + +## What OpenCode Gets Right + +## 1. Separate plugin SDK from host runtime + +This is one of the best parts of the design. + +- plugin authors code against a clean public package +- host internals can evolve behind the loader +- runtime code and plugin code have a clean contract boundary + +Paperclip should absolutely do this. + +## 2. Deterministic loading and precedence + +`opencode` is explicit about: + +- where plugins come from +- how config merges +- what order wins + +Paperclip should copy this discipline. + +## 3. Low-ceremony authoring + +A plugin author does not have to learn a giant framework. + +- export async function +- return hooks +- optionally export tools + +That simplicity matters. + +## 4. Typed tool definitions + +The `tool()` helper is excellent: + +- typed +- schema-based +- easy to document +- easy for runtime validation + +Paperclip should adopt this style for plugin actions, automations, and UI schemas. + +## 5. Built-in features and plugins use similar shapes + +`opencode` uses the same hook system for internal and external plugin-style behavior in several places. +That reduces special cases. + +Paperclip can benefit from that with adapters, secret backends, storage providers, and connector modules. + +## 6. Incremental extension, not giant abstraction upfront + +`opencode` did not design a giant marketplace platform first. +It added concrete extension points that real features needed. + +That is the correct mindset for Paperclip too. + +## What Paperclip Should Not Copy Directly + +## 1. In-process arbitrary plugin code as the default + +`opencode` is basically a local agent runtime, so unsandboxed plugin execution is acceptable for its audience. + +Paperclip is a control plane for an operator-managed instance with company objects. +The risk profile is different: + +- secrets matter +- approval gates matter +- budgets matter +- mutating actions require auditability + +Default third-party plugins should not run with unrestricted in-process access to server memory, DB handles, and secrets. + +## 2. Project-local plugin loading + +`opencode` has project-local plugin folders because the tool is centered around a codebase. + +Paperclip is not project-scoped. It is instance-scoped. +The comparable unit is: + +- instance-installed plugin package + +Paperclip should not auto-load arbitrary code from a workspace repo like `.paperclip/plugins` or project directories. + +## 3. Arbitrary mutation hooks on core business decisions + +Hooks like: + +- `permission.ask` +- `tool.execute.before` +- `chat.headers` +- `shell.env` + +make sense in `opencode`. + +For Paperclip, equivalent hooks into: + +- approval decisions +- issue checkout semantics +- activity log behavior +- budget enforcement + +would be a mistake. + +Core invariants should stay in core code, not become hook-rewritable. + +## 4. Override-by-name collision + +Allowing a plugin to replace a built-in tool by name is useful in a local agent product. + +Paperclip should not allow plugins to silently replace: + +- core routes +- core mutating actions +- auth behaviors +- permission evaluators +- budget logic +- audit logic + +Extension should be additive or explicitly delegated, never accidental shadowing. + +## 5. Auto-install and execute from user config + +`opencode`'s "install dependencies at startup" flow is ergonomic. +For Paperclip it would be risky because it combines: + +- package installation +- code loading +- execution + +inside the control-plane server startup path. + +Paperclip should require an explicit operator install step. + +## Why Paperclip Needs A Different Shape + +The products are solving different problems. + +| Topic | OpenCode | Paperclip | +|---|---|---| +| Primary unit | local project/worktree | single-tenant operator instance with company objects | +| Trust assumption | local power user on own machine | operator managing one trusted Paperclip instance | +| Failure blast radius | local session/runtime | entire company control plane | +| Extension style | mutate runtime behavior freely | preserve governance and auditability | +| UI model | local app can load local behavior | board UI must stay coherent and safe | +| Security model | host-trusted local plugins | needs capability boundaries and auditability | + +That means Paperclip should borrow the good ideas from `opencode` but use a stricter architecture. + +## Paperclip Already Has Useful Pre-Plugin Seams + +Paperclip has several extension-like seams already: + +- server adapter registry: [server/src/adapters/registry.ts](../../server/src/adapters/registry.ts) +- UI adapter registry: [ui/src/adapters/registry.ts](../../ui/src/adapters/registry.ts) +- storage provider registry: [server/src/storage/provider-registry.ts](../../server/src/storage/provider-registry.ts) +- secret provider registry: [server/src/secrets/provider-registry.ts](../../server/src/secrets/provider-registry.ts) +- pluggable run-log store seam: [server/src/services/run-log-store.ts](../../server/src/services/run-log-store.ts) +- activity log and live event emission: [server/src/services/activity-log.ts](../../server/src/services/activity-log.ts) + +This is good news. +Paperclip does not need to invent extensibility from scratch. +It needs to unify and harden existing seams. + +## Recommended Paperclip Plugin Model + +## 1. Use multiple extension classes + +Do not create one giant `hooks` object for everything. + +Use distinct plugin classes with different trust models. + +| Extension class | Examples | Runtime model | Trust level | Why | +|---|---|---|---|---| +| Platform module | agent adapters, storage providers, secret providers, run-log backends | in-process | highly trusted | tight integration, performance, low-level APIs | +| Connector plugin | Linear, GitHub Issues, Grafana, Stripe | out-of-process worker or sidecar | medium | external sync, safer isolation, clearer failure boundary | +| Workspace plugin | file browser, terminal, git workflow, child process/server tracking | mixed: core workspace services plus plugin descriptors | high | needs local OS/workspace primitives but should reuse core services | +| UI contribution | dashboard widgets, settings forms, company panels | schema-driven first, remote React later if needed | medium | safer than arbitrary frontend code | +| Automation plugin | alerts, schedulers, sync jobs, webhook processors | out-of-process | medium | event-driven automation is a natural plugin fit | + +This split is the most important design recommendation in this report. + +## 2. Keep low-level modules separate from third-party plugins + +Paperclip already has this pattern implicitly: + +- adapters are one thing +- storage providers are another +- secret providers are another + +Keep that separation. + +I would formalize it like this: + +- `module` means trusted code loaded by the host for low-level runtime services +- `plugin` means integration code that talks to Paperclip through a typed plugin protocol and capability model + +This avoids trying to force Stripe, a PTY terminal, and a new agent adapter into the same abstraction. + +## 3. Prefer event-driven extensions over core-logic mutation + +For third-party plugins, the primary API should be: + +- subscribe to typed domain events +- read instance state, including company-bound business records when relevant +- register webhooks +- run scheduled jobs +- write plugin-owned state +- add additive UI surfaces +- invoke explicit Paperclip actions through the API + +Do not make third-party plugins responsible for: + +- deciding whether an approval passes +- intercepting issue checkout semantics +- rewriting activity log behavior +- overriding budget hard-stops + +Those are core invariants. + +## 4. Start with schema-driven UI contributions + +Arbitrary third-party React bundles inside the board UI are possible later, but they should not be the first version. + +First version should let plugins contribute: + +- settings sections defined by JSON schema +- dashboard widgets with server-provided data +- sidebar entries with fixed shell rendering +- detail-page tabs that render plugin data through core UI components + +Why: + +- simpler to secure +- easier to keep visually coherent +- easier to preserve context and auditability +- easier to test + +Later, if needed, Paperclip can support richer frontend extensions through versioned remote modules. + +## 5. Make installation global and keep mappings/config separate + +`opencode` is mostly user-level local config. +Paperclip should treat plugin installation as a global instance-level action. + +Examples: + +- install `@paperclip/plugin-linear` once +- make it available everywhere immediately +- optionally store mappings over Paperclip objects if one company maps to a different Linear team than another + +## 6. Use project workspaces as the primary anchor for local tooling + +Paperclip already has a concrete workspace model for projects: + +- projects expose `workspaces` and `primaryWorkspace` +- the database already has `project_workspaces` +- project routes already support creating, updating, and deleting workspaces +- heartbeat resolution already prefers project workspaces before falling back to task-session or agent-home workspaces + +That means local/runtime plugins should generally anchor themselves to projects first, not invent a parallel workspace model. + +Practical guidance: + +- file browser should browse project workspaces first +- terminal sessions should be launchable from a project workspace +- git should treat the project workspace as the repo root anchor +- dev server and child-process tracking should attach to project workspaces +- issue and agent views can still deep-link into the relevant project workspace context + +In other words: + +- `project` is the business object +- `project_workspace` is the local runtime anchor +- plugins should build on that instead of creating an unrelated workspace model first + +## A Concrete SDK Shape For Paperclip + +An intentionally narrow first pass could look like this: + +```ts +import { definePlugin, z } from "@paperclipai/plugin-sdk"; + +export default definePlugin({ + id: "@paperclip/plugin-linear", + version: "0.1.0", + kind: ["connector", "ui"], + capabilities: [ + "events.subscribe", + "jobs.schedule", + "http.outbound", + "instance.settings", + "dashboard.widget", + "secrets.read-ref", + ], + instanceConfigSchema: z.object({ + linearBaseUrl: z.string().url().optional(), + companyMappings: z.array( + z.object({ + companyId: z.string(), + teamId: z.string(), + apiTokenSecretRef: z.string(), + }), + ).default([]), + }), + async register(ctx) { + ctx.jobs.register("linear-pull", { cron: "*/5 * * * *" }, async (job) => { + // sync Linear issues into plugin-owned state or explicit Paperclip entities + }); + + ctx.events.on("issue.created", async (event) => { + // optional outbound sync + }); + + ctx.ui.registerDashboardWidget({ + id: "linear-health", + title: "Linear", + loader: async ({ companyId }) => ({ status: "ok" }), + }); + }, +}); +``` + +The important point is not the exact syntax. +The important point is the contract shape: + +- typed manifest +- explicit capabilities +- explicit global config with optional company mappings +- event subscriptions +- jobs +- additive UI contributions + +## Recommended Core Extension Surfaces + +## 1. Platform module surfaces + +These should stay close to the current registry style. + +Candidates: + +- `registerAgentAdapter()` +- `registerStorageProvider()` +- `registerSecretProvider()` +- `registerRunLogStore()` +- maybe `registerWorkspaceRuntime()` later + +These are trusted platform modules, not casual plugins. + +## 2. Connector plugin surfaces + +These are the best near-term plugin candidates. + +Capabilities: + +- subscribe to domain events +- define scheduled sync jobs +- expose plugin-specific API routes under `/api/plugins/:pluginId/...` +- use company secret refs +- write plugin state +- publish dashboard data +- log activity through core APIs + +Examples: + +- Linear issue sync +- GitHub issue sync +- Grafana dashboard cards +- Stripe MRR / subscription rollups + +## 3. Workspace-runtime surfaces + +Your local-ops examples need first-party primitives plus plugin contributions. + +Examples: + +- file browser +- terminal +- git workflow +- child process tracking +- local dev server tracking + +These should not be arbitrary third-party code directly poking the host filesystem and PTY layer through ad-hoc hooks. +Instead, Paperclip should add first-party services such as: + +- project workspace service built on `project_workspaces` +- PTY session service +- process registry +- git service +- dev-server registry + +Then plugins can add: + +- UI panels on top of those services +- automations +- annotations +- external sync logic + +This keeps sensitive local-machine behavior centralized and auditable. + +## 4. UI contribution surfaces + +Recommended first version: + +- dashboard widgets +- settings panels +- detail-page tabs +- sidebar sections +- action buttons that invoke plugin routes + +Recommended later version: + +- richer remote UI modules + +Recommended never or only with extreme caution: + +- arbitrary override of core pages +- arbitrary replacement of routing/auth/layout logic + +## Governance And Safety Requirements + +Any Paperclip plugin system has to preserve core control-plane invariants from the repo docs. + +That means: + +- plugin install is global to the instance +- "companies" remain business objects in the API and data model, not tenant boundaries +- approval gates remain core-owned +- budget hard-stops remain core-owned +- mutating actions are activity-logged +- secrets remain ref-based and redacted in logs + +I would require the following for every plugin: + +## 1. Capability declaration + +Every plugin declares a static capability set such as: + +- `companies.read` +- `issues.read` +- `issues.write` +- `events.subscribe` +- `jobs.schedule` +- `http.outbound` +- `webhooks.receive` +- `assets.read` +- `assets.write` +- `workspace.pty` +- `workspace.fs.read` +- `workspace.fs.write` +- `secrets.read-ref` + +The board/operator sees this before installation. + +## 2. Global installation + +A plugin is installed once and becomes available across the instance. +If it needs mappings over specific Paperclip objects, those are plugin data, not enable/disable boundaries. + +## 3. Activity logging + +Plugin-originated mutations should flow through the same activity log mechanism, with actor identity like: + +- `actor_type = system` +- `actor_id = plugin:@paperclip/plugin-linear` + +or a dedicated `plugin` actor type if you want stronger semantics later. + +## 4. Health and failure reporting + +Each plugin should expose: + +- enabled/disabled state +- last successful run +- last error +- recent webhook/job history + +One broken plugin must not break the rest of the company. + +## 5. Secret handling + +Plugins should receive secret refs, not raw secret values in config persistence. +Resolution should go through the existing secret provider abstraction. + +## 6. Resource limits + +Plugins should have: + +- timeout limits +- concurrency limits +- retry policies +- optional per-plugin budgets + +This matters especially for sync connectors and workspace plugins. + +## Data Model Additions To Consider + +I would avoid "arbitrary third-party plugin-defined SQL migrations" in the first version. +That is too much power too early. + +The right mental model is: + +- reuse core tables when the data is clearly part of Paperclip itself +- use generic extension tables for most plugin-owned state +- only allow plugin-specific tables later, and only for trusted platform modules or a tightly controlled migration workflow + +## Recommended Postgres Strategy For Extensions + +### 1. Core tables stay core + +If a concept is becoming part of Paperclip's actual product model, it should get a normal first-party table. + +Examples: + +- `project_workspaces` is already a core table because project workspaces are now part of Paperclip itself +- if a future "project git state" becomes a core feature rather than plugin-owned metadata, that should also be a first-party table + +### 2. Most plugins should start in generic extension tables + +For most plugins, the host should provide a few generic persistence tables and the plugin stores namespaced records there. + +This keeps the system manageable: + +- simpler migrations +- simpler backup/restore +- simpler portability story +- easier operator review +- fewer chances for plugin schema drift to break the instance + +### 3. Scope plugin data to Paperclip objects before adding custom schemas + +A lot of plugin data naturally hangs off existing Paperclip objects: + +- project workspace plugin state should often scope to `project` or `project_workspace` +- issue sync state should scope to `issue` +- metrics widgets may scope to `company`, `project`, or `goal` +- process tracking may scope to `project_workspace`, `agent`, or `run` + +That gives a good default keying model before introducing custom tables. + +### 4. Add trusted module migrations later, not arbitrary plugin migrations now + +If Paperclip eventually needs extension-owned tables, I would only allow that for: + +- trusted first-party packages +- trusted platform modules +- maybe explicitly installed admin-reviewed plugins with pinned versions + +I would not let random third-party plugins run free-form schema migrations on startup. + +Instead, add a controlled mechanism later if it becomes necessary. + +## Suggested baseline extension tables + +## 1. `plugins` + +Instance-level installation record. + +Suggested fields: + +- `id` +- `package_name` +- `version` +- `kind` +- `manifest_json` +- `installed_at` +- `status` + +## 2. `plugin_config` + +Instance-level plugin config. + +Suggested fields: + +- `id` +- `plugin_id` +- `config_json` +- `installed_at` +- `updated_at` +- `last_error` + +## 3. `plugin_state` + +Generic key/value state for plugins. + +Suggested fields: + +- `id` +- `plugin_id` +- `scope_kind` (`instance | company | project | project_workspace | agent | issue | goal | run`) +- `scope_id` nullable +- `namespace` +- `state_key` +- `value_json` +- `updated_at` + +This is enough for many connectors before allowing custom tables. + +Examples: + +- Linear external IDs keyed by `issue` +- GitHub sync cursors keyed by `project` +- file browser preferences keyed by `project_workspace` +- git branch metadata keyed by `project_workspace` +- process metadata keyed by `project_workspace` or `run` + +## 4. `plugin_jobs` + +Scheduled job and run tracking. + +Suggested fields: + +- `id` +- `plugin_id` +- `scope_kind` nullable +- `scope_id` nullable +- `job_key` +- `status` +- `last_started_at` +- `last_finished_at` +- `last_error` + +## 5. `plugin_webhook_deliveries` + +If plugins expose webhooks, delivery history is worth storing. + +Suggested fields: + +- `id` +- `plugin_id` +- `scope_kind` nullable +- `scope_id` nullable +- `endpoint_key` +- `status` +- `received_at` +- `response_code` +- `error` + +## 6. Maybe later: `plugin_entities` + +If generic plugin state becomes too limiting, add a structured, queryable entity table for connector records before allowing arbitrary plugin migrations. + +Suggested fields: + +- `id` +- `plugin_id` +- `entity_type` +- `scope_kind` +- `scope_id` +- `external_id` +- `title` +- `status` +- `data_json` +- `updated_at` + +This is a useful middle ground: + +- much more queryable than opaque key/value state +- still avoids letting every plugin create its own relational schema immediately + +## How The Requested Examples Map To This Model + +| Use case | Best fit | Core primitives needed | Notes | +|---|---|---|---| +| File browser | workspace plugin + schema UI | project workspaces, file API, audit rules | best anchored on project detail pages | +| Terminal | workspace plugin + PTY service | project workspaces, PTY/session service, process limits, audit events | should launch against a project workspace by default | +| Git workflow | workspace plugin | project workspaces, git service, repo/worktree model | project workspace is the natural repo anchor | +| Linear issue tracking | connector plugin | jobs, webhooks, secret refs, issue sync API | very strong plugin candidate | +| GitHub issue tracking | connector plugin | jobs, webhooks, secret refs | very strong plugin candidate | +| Grafana metrics | connector plugin + dashboard widget | outbound HTTP, widget API | probably read-only first | +| Child process/server tracking | workspace plugin | project workspaces, process registry, server heartbeat model | should attach processes to project workspaces when possible | +| Stripe revenue tracking | connector plugin | secret refs, scheduled sync, company metrics API | strong plugin candidate and aligns with future spec direction | + +# Plugin Examples + +## Workspace File Browser + +Package idea: `@paperclip/plugin-workspace-files` + +This plugin lets the board inspect project workspaces, agent workspaces, generated artifacts, and issue-related files without dropping to the shell. It is useful for: + +- browsing files inside project workspaces +- debugging what an agent changed +- reviewing generated outputs before approval +- attaching files from a workspace to issues +- understanding repo layout for a company +- inspecting agent home workspaces in local-trusted mode + +### UX + +- Settings page: `/settings/plugins/workspace-files` +- Main page: `/:companyPrefix/plugins/workspace-files` +- Project tab: `/:companyPrefix/projects/:projectId?tab=files` +- Optional issue tab: `/:companyPrefix/issues/:issueId?tab=files` +- Optional agent tab: `/:companyPrefix/agents/:agentId?tab=workspace` + +Main screens and interactions: + +- Plugin settings: + - choose whether the plugin defaults to `project.primaryWorkspace` + - choose which project workspaces are visible + - choose whether file writes are allowed or read-only + - choose whether hidden files are visible +- Main explorer page: + - project picker at the top + - workspace picker scoped to the selected project's `workspaces` + - tree view on the left + - file preview pane on the right + - search box for filename/path search + - actions: copy path, download file, attach to issue, open diff +- Project tab: + - opens directly into the project's primary workspace + - lets the board switch among all project workspaces + - shows workspace metadata like `cwd`, `repoUrl`, and `repoRef` +- Issue tab: + - resolves the issue's project and opens that project's workspace context + - shows files linked to the issue + - lets the board pull files from the project workspace into issue attachments + - shows the path and last modified info for each linked file +- Agent tab: + - shows the agent's current resolved workspace + - if the run is attached to a project, links back to the project workspace view + - lets the board inspect files the agent is currently touching + +Core workflows: + +- Board opens a project and browses its primary workspace files. +- Board switches from one project workspace to another when a project has multiple checkouts or repo references. +- Board opens an issue, attaches a generated artifact from the file browser, and leaves a review comment. +- Board opens an agent detail page to inspect the exact files behind a failing run. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.detailTab.register` for `project`, `issue`, and `agent` +- `projects.read` +- `project.workspaces.read` +- `workspace.fs.read` +- optional `workspace.fs.write` +- `workspace.fs.stat` +- `workspace.fs.search` +- optional `assets.write` +- `activity.log.write` + +Optional event subscriptions: + +- `events.subscribe(agent.run.started)` +- `events.subscribe(agent.run.finished)` +- `events.subscribe(issue.attachment.created)` + +Important constraint: + +- the plugin should never read arbitrary host paths directly +- it should treat project workspaces as the canonical local file roots when a project is present +- it should only use first-party workspace/file APIs that enforce approved workspace roots + +## Workspace Terminal + +Package idea: `@paperclip/plugin-terminal` + +This plugin gives the board a controlled terminal UI for project workspaces and agent workspaces. It is useful for: + +- debugging stuck runs +- verifying environment state +- running targeted manual commands +- watching long-running commands +- pairing a human operator with an agent workflow + +### UX + +- Settings page: `/settings/plugins/terminal` +- Main page: `/:companyPrefix/plugins/terminal` +- Project tab: `/:companyPrefix/projects/:projectId?tab=terminal` +- Optional agent tab: `/:companyPrefix/agents/:agentId?tab=terminal` +- Optional run tab: `/:companyPrefix/agents/:agentId/runs/:runId?tab=terminal` + +Main screens and interactions: + +- Plugin settings: + - allowed shells and shell policy + - whether commands are read-only, free-form, or allow-listed + - whether terminals require an explicit operator confirmation before launch + - whether new terminal sessions default to the project's primary workspace +- Terminal home page: + - list of active terminal sessions + - button to open a new session + - project picker, then workspace picker from that project's workspaces + - optional agent association + - terminal panel with input, resize, and reconnect support + - controls: interrupt, kill, clear, save transcript +- Project terminal tab: + - opens a session already scoped to the project's primary workspace + - lets the board switch among the project's configured workspaces + - shows recent commands and related process/server state for that project +- Agent terminal tab: + - opens a session already scoped to the agent's workspace + - shows recent related runs and commands +- Run terminal tab: + - lets the board inspect the environment around a specific failed run + +Core workflows: + +- Board opens a terminal against an agent workspace to reproduce a failing command. +- Board opens a project page and launches a terminal directly in that project's primary workspace. +- Board watches a long-running dev server or test command from the terminal page. +- Board kills or interrupts a runaway process from the same UI. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.detailTab.register` for `project`, `agent`, and `run` +- `projects.read` +- `project.workspaces.read` +- `workspace.pty.open` +- `workspace.pty.input` +- `workspace.pty.resize` +- `workspace.pty.terminate` +- `workspace.pty.subscribe` +- `workspace.process.read` +- `activity.log.write` + +Optional event subscriptions: + +- `events.subscribe(agent.run.started)` +- `events.subscribe(agent.run.failed)` +- `events.subscribe(agent.run.cancelled)` + +Important constraint: + +- shell spawning must stay in a first-party PTY service +- the plugin should orchestrate and render sessions, not spawn raw host processes by itself + +## Git Workflow + +Package idea: `@paperclip/plugin-git` + +This plugin adds repo-aware workflow tooling around issues and workspaces. It is useful for: + +- branch creation tied to issues +- quick diff review +- commit and worktree visibility +- PR preparation +- treating the project's primary workspace as the canonical repo anchor +- seeing whether an agent's workspace is clean or dirty + +### UX + +- Settings page: `/settings/plugins/git` +- Main page: `/:companyPrefix/plugins/git` +- Project tab: `/:companyPrefix/projects/:projectId?tab=git` +- Optional issue tab: `/:companyPrefix/issues/:issueId?tab=git` +- Optional agent tab: `/:companyPrefix/agents/:agentId?tab=git` + +Main screens and interactions: + +- Plugin settings: + - branch naming template + - optional remote provider token secret ref + - whether write actions are enabled or read-only + - whether the plugin always uses `project.primaryWorkspace` unless a different project workspace is chosen +- Git overview page: + - project picker and workspace picker + - current branch + - ahead/behind status + - dirty files summary + - recent commits + - active worktrees + - actions: refresh, create branch, create worktree, stage all, commit, open diff +- Project tab: + - opens in the project's primary workspace + - shows workspace metadata and repo binding (`cwd`, `repoUrl`, `repoRef`) + - shows branch, diff, and commit history for that project workspace +- Issue tab: + - resolves the issue's project and uses that project's workspace context + - "create branch from issue" action + - diff view scoped to the project's selected workspace + - link branch/worktree metadata to the issue +- Agent tab: + - shows the agent's branch, worktree, and dirty state + - shows recent commits produced by that agent + - if the agent is working inside a project workspace, links back to the project git tab + +Core workflows: + +- Board creates a branch from an issue and ties it to the project's primary workspace. +- Board opens a project page and reviews the diff for that project's workspace without leaving Paperclip. +- Board reviews the diff after a run without leaving Paperclip. +- Board opens a worktree list to understand parallel branches across agents. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.detailTab.register` for `project`, `issue`, and `agent` +- `ui.action.register` +- `projects.read` +- `project.workspaces.read` +- `workspace.git.status` +- `workspace.git.diff` +- `workspace.git.log` +- `workspace.git.branch.create` +- optional `workspace.git.commit` +- optional `workspace.git.worktree.create` +- optional `workspace.git.push` +- `activity.log.write` + +Optional event subscriptions: + +- `events.subscribe(issue.created)` +- `events.subscribe(issue.updated)` +- `events.subscribe(agent.run.finished)` + +Important constraint: + +- GitHub/GitLab PR creation should likely live in a separate connector plugin rather than overloading the local git plugin + +## Linear Issue Tracking + +Package idea: `@paperclip/plugin-linear` + +This plugin syncs Paperclip work with Linear. It is useful for: + +- importing backlog from Linear +- linking Paperclip issues to Linear issues +- syncing status, comments, and assignees +- mapping company goals/projects to external product planning +- giving board operators a single place to see sync health + +### UX + +- Settings page: `/settings/plugins/linear` +- Main page: `/:companyPrefix/plugins/linear` +- Dashboard widget: `/:companyPrefix/dashboard` +- Optional issue tab: `/:companyPrefix/issues/:issueId?tab=linear` +- Optional project tab: `/:companyPrefix/projects/:projectId?tab=linear` + +Main screens and interactions: + +- Plugin settings: + - Linear API token secret ref + - workspace/team/project mappings + - status mapping between Paperclip and Linear + - sync direction: import only, export only, bidirectional + - comment sync toggle +- Linear overview page: + - sync health card + - recent sync jobs + - mapped projects and teams + - unresolved conflicts queue + - import actions for teams, projects, and issues +- Issue tab: + - linked Linear issue key and URL + - sync status and last synced time + - actions: link existing, create in Linear, resync now, unlink + - timeline of synced comments/status changes +- Dashboard widget: + - open sync errors + - imported vs linked issues count + - recent webhook/job failures + +Core workflows: + +- Board enables the plugin, maps a Linear team, and imports a backlog into Paperclip. +- Paperclip issue status changes push to Linear and Linear comments arrive back through webhooks. +- Board resolves mapping conflicts from the plugin page instead of silently drifting state. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.dashboardWidget.register` +- `ui.detailTab.register` for `issue` and `project` +- `events.subscribe(issue.created)` +- `events.subscribe(issue.updated)` +- `events.subscribe(issue.comment.created)` +- `events.subscribe(project.updated)` +- `jobs.schedule` +- `webhooks.receive` +- `http.outbound` +- `secrets.read-ref` +- `plugin.state.read` +- `plugin.state.write` +- optional `issues.create` +- optional `issues.update` +- optional `issue.comments.create` +- `activity.log.write` + +Important constraint: + +- webhook processing should be idempotent and conflict-aware +- external IDs and sync cursors belong in plugin-owned state, not inline on core issue rows in the first version + +## GitHub Issue Tracking + +Package idea: `@paperclip/plugin-github-issues` + +This plugin syncs Paperclip issues with GitHub Issues and optionally links PRs. It is useful for: + +- importing repo backlogs +- mirroring issue status and comments +- linking PRs to Paperclip issues +- tracking cross-repo work from inside one company view +- bridging engineering workflow with Paperclip task governance + +### UX + +- Settings page: `/settings/plugins/github-issues` +- Main page: `/:companyPrefix/plugins/github-issues` +- Dashboard widget: `/:companyPrefix/dashboard` +- Optional issue tab: `/:companyPrefix/issues/:issueId?tab=github` +- Optional project tab: `/:companyPrefix/projects/:projectId?tab=github` + +Main screens and interactions: + +- Plugin settings: + - GitHub App or PAT secret ref + - org/repo mappings + - label/status mapping + - whether PR linking is enabled + - whether new Paperclip issues should create GitHub issues automatically +- GitHub overview page: + - repo mapping list + - sync health and recent webhook events + - import backlog action + - queue of unlinked GitHub issues +- Issue tab: + - linked GitHub issue and optional linked PRs + - actions: create GitHub issue, link existing issue, unlink, resync + - comment/status sync timeline +- Dashboard widget: + - open PRs linked to active Paperclip issues + - webhook failures + - sync lag metrics + +Core workflows: + +- Board imports GitHub Issues for a repo into Paperclip. +- GitHub webhooks update status/comment state in Paperclip. +- A PR is linked back to the Paperclip issue so the board can follow delivery status. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.dashboardWidget.register` +- `ui.detailTab.register` for `issue` and `project` +- `events.subscribe(issue.created)` +- `events.subscribe(issue.updated)` +- `events.subscribe(issue.comment.created)` +- `jobs.schedule` +- `webhooks.receive` +- `http.outbound` +- `secrets.read-ref` +- `plugin.state.read` +- `plugin.state.write` +- optional `issues.create` +- optional `issues.update` +- optional `issue.comments.create` +- `activity.log.write` + +Important constraint: + +- keep "local git state" and "remote GitHub issue state" in separate plugins even if they work together + +## Grafana Metrics + +Package idea: `@paperclip/plugin-grafana` + +This plugin surfaces external metrics and dashboards inside Paperclip. It is useful for: + +- company KPI visibility +- infrastructure/incident monitoring +- showing deploy, traffic, latency, or revenue charts next to work +- creating Paperclip issues from anomalous metrics + +### UX + +- Settings page: `/settings/plugins/grafana` +- Main page: `/:companyPrefix/plugins/grafana` +- Dashboard widgets: `/:companyPrefix/dashboard` +- Optional goal tab: `/:companyPrefix/goals/:goalId?tab=metrics` + +Main screens and interactions: + +- Plugin settings: + - Grafana base URL + - service account token secret ref + - dashboard and panel mappings + - refresh interval + - optional alert threshold rules +- Dashboard widgets: + - one or more metric cards on the main dashboard + - quick trend view and last refresh time + - link out to Grafana and link in to the full Paperclip plugin page +- Full metrics page: + - selected dashboard panels embedded or proxied + - metric selector + - time range selector + - "create issue from anomaly" action +- Goal tab: + - metric cards relevant to a specific goal or project + +Core workflows: + +- Board sees service degradation or business KPI movement directly on the Paperclip dashboard. +- Board clicks into the full metrics page to inspect the relevant Grafana panels. +- Board creates a Paperclip issue from a threshold breach with a metric snapshot attached. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.dashboardWidget.register` +- `ui.page.register` +- `ui.detailTab.register` for `goal` or `project` +- `jobs.schedule` +- `http.outbound` +- `secrets.read-ref` +- `plugin.state.read` +- `plugin.state.write` +- optional `issues.create` +- optional `assets.write` +- `activity.log.write` + +Optional event subscriptions: + +- `events.subscribe(goal.created)` +- `events.subscribe(project.updated)` + +Important constraint: + +- start read-only first +- do not make Grafana alerting logic part of Paperclip core; keep it as additive signal and issue creation + +## Child Process / Server Tracking + +Package idea: `@paperclip/plugin-runtime-processes` + +This plugin tracks long-lived local processes and dev servers started in project workspaces. It is useful for: + +- seeing which agent started which local service +- tracking ports, health, and uptime +- restarting failed dev servers +- exposing process state alongside issue and run state +- making local development workflows visible to the board + +### UX + +- Settings page: `/settings/plugins/runtime-processes` +- Main page: `/:companyPrefix/plugins/runtime-processes` +- Dashboard widget: `/:companyPrefix/dashboard` +- Process detail page: `/:companyPrefix/plugins/runtime-processes/:processId` +- Project tab: `/:companyPrefix/projects/:projectId?tab=processes` +- Optional agent tab: `/:companyPrefix/agents/:agentId?tab=processes` + +Main screens and interactions: + +- Plugin settings: + - whether manual process registration is allowed + - health check behavior + - whether operators can stop/restart processes + - log retention preferences +- Process list page: + - status table with name, command, cwd, owner agent, port, uptime, and health + - filters for running/exited/crashed processes + - actions: inspect, stop, restart, tail logs +- Project tab: + - filters the process list to the project's workspaces + - shows which workspace each process belongs to + - groups processes by project workspace +- Process detail page: + - process metadata + - live log tail + - health check history + - links to associated issue or run +- Agent tab: + - shows processes started by or assigned to that agent + +Core workflows: + +- An agent starts a dev server; the first-party process service registers it and the plugin renders it. +- Board opens a project and immediately sees the processes attached to that project's workspace. +- Board sees a crashed process on the dashboard and restarts it from the plugin page. +- Board attaches process logs to an issue when debugging a failure. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.sidebar.register` +- `ui.page.register` +- `ui.dashboardWidget.register` +- `ui.detailTab.register` for `project` and `agent` +- `projects.read` +- `project.workspaces.read` +- `workspace.process.register` +- `workspace.process.list` +- `workspace.process.read` +- optional `workspace.process.terminate` +- optional `workspace.process.restart` +- `workspace.process.logs.read` +- optional `workspace.http.probe` +- `plugin.state.read` +- `plugin.state.write` +- `activity.log.write` + +Optional event subscriptions: + +- `events.subscribe(agent.run.started)` +- `events.subscribe(agent.run.finished)` +- `events.subscribe(process.started)` +- `events.subscribe(process.exited)` + +Important constraint: + +- this should be built on a first-party process registry +- the plugin should not own raw child-process spawning on its own + +## Stripe Revenue Tracking + +Package idea: `@paperclip/plugin-stripe` + +This plugin pulls Stripe revenue and subscription data into Paperclip. It is useful for: + +- showing MRR and churn next to company goals +- tracking trials, conversions, and failed payments +- letting the board connect revenue movement to ongoing work +- enabling future financial dashboards beyond token costs + +### UX + +- Settings page: `/settings/plugins/stripe` +- Main page: `/:companyPrefix/plugins/stripe` +- Dashboard widgets: `/:companyPrefix/dashboard` +- Optional company/goal metric tabs if those surfaces exist later + +Main screens and interactions: + +- Plugin settings: + - Stripe secret key secret ref + - account selection if needed + - metric definitions such as MRR treatment and trial handling + - sync interval + - webhook signing secret ref +- Dashboard widgets: + - MRR card + - active subscriptions + - trial-to-paid conversion + - failed payment alerts +- Stripe overview page: + - time series charts + - recent customer/subscription events + - webhook health + - sync history + - action: create issue from billing anomaly + +Core workflows: + +- Board enables the plugin and connects a Stripe account. +- Webhooks and scheduled reconciliation keep plugin state current. +- Revenue widgets appear on the main dashboard and can be linked to company goals. +- Failed payment spikes or churn events can generate Paperclip issues for follow-up. + +### Hooks needed + +Recommended capabilities and extension points: + +- `instance.settings.register` +- `ui.dashboardWidget.register` +- `ui.page.register` +- `jobs.schedule` +- `webhooks.receive` +- `http.outbound` +- `secrets.read-ref` +- `plugin.state.read` +- `plugin.state.write` +- `company.metrics.write` +- optional `issues.create` +- `activity.log.write` + +Important constraint: + +- Stripe data should stay additive to Paperclip core +- it should not leak into core budgeting logic, which is specifically about model/token spend in V1 + +## Specific Patterns From OpenCode Worth Adopting + +## Adopt + +- separate SDK package from runtime loader +- deterministic load order and precedence +- very small authoring API +- typed schemas for plugin inputs/config/tools +- internal extensions using the same registration shapes as external ones when reasonable +- plugin load errors isolated from host startup when possible +- explicit community-facing plugin docs and example templates + +## Adapt, not copy + +- local path loading +- dependency auto-install +- hook mutation model +- built-in override behavior +- broad runtime context objects + +## Avoid + +- project-local arbitrary code loading +- implicit trust of npm packages at startup +- plugins overriding core invariants +- unsandboxed in-process execution as the default extension model + +## Suggested Rollout Plan + +## Phase 0: Harden the seams that already exist + +- formalize adapter/storage/secret/run-log registries as "platform modules" +- remove ad-hoc fallback behavior where possible +- document stable registration contracts + +## Phase 1: Add connector plugins first + +This is the highest-value, lowest-risk plugin category. + +Build: + +- plugin manifest +- global install/update lifecycle +- global plugin config and optional company-mapping storage +- secret ref access +- typed domain event subscription +- scheduled jobs +- webhook endpoints +- activity logging helpers +- dashboard widget and settings-panel contributions + +This phase would immediately cover: + +- Linear +- GitHub +- Grafana +- Stripe + +## Phase 2: Add workspace services and workspace plugins + +Build first-party primitives: + +- project workspace service built on `project_workspaces` +- PTY/session service +- file service +- git service +- process/service tracker + +Then expose additive plugin/UI surfaces on top. + +This phase covers: + +- file browser +- terminal +- git workflow +- child process/server tracking + +## Phase 3: Consider richer UI and plugin packaging + +Only after phases 1 and 2 are stable: + +- richer frontend extension support +- signed/verified plugin packages +- plugin marketplace +- optional custom plugin storage backends or migrations + +## Recommended Architecture Decision + +If I had to collapse this report into one architectural decision, it would be: + +Paperclip should not implement "an OpenCode-style generic in-process hook system." +Paperclip should implement "a plugin platform with multiple trust tiers": + +- trusted platform modules for low-level runtime integration +- typed out-of-process plugins for instance-wide integrations and automation +- schema-driven UI contributions +- core-owned invariants that plugins can observe and act around, but not replace + +That gets the upside of `opencode`'s extensibility without importing the wrong threat model. + +## Concrete Next Steps I Would Take In Paperclip + +1. Write a short extension architecture RFC that formalizes the distinction between `platform modules` and `plugins`. +2. Introduce a small plugin manifest type in `packages/shared` and a `plugins` install/config section in the instance config. +3. Build a typed domain event bus around existing activity/live-event patterns, but keep core invariants non-hookable. +4. Implement connector-plugin MVP only: global install/config, secret refs, jobs, webhooks, settings panel, dashboard widget. +5. Treat workspace features as a separate track that starts by building core workspace primitives, not raw plugin hooks. diff --git a/packages/adapters/openclaw-gateway/doc/ONBOARDING_AND_TEST_PLAN.md b/packages/adapters/openclaw-gateway/doc/ONBOARDING_AND_TEST_PLAN.md index 6c804d22..042b8656 100644 --- a/packages/adapters/openclaw-gateway/doc/ONBOARDING_AND_TEST_PLAN.md +++ b/packages/adapters/openclaw-gateway/doc/ONBOARDING_AND_TEST_PLAN.md @@ -78,26 +78,27 @@ Implication: ## Deep Code Findings (Gaps) -### 1) Onboarding content is still OpenClaw-HTTP specific -`server/src/routes/access.ts` hardcodes onboarding to: -- `recommendedAdapterType: "openclaw"` -- Required `agentDefaultsPayload.headers.x-openclaw-auth` -- HTTP callback URL guidance and `/v1/responses` examples. +### 1) Onboarding manifest/text gateway path (resolved) +Resolved in `server/src/routes/access.ts`: +- `recommendedAdapterType` now points to `openclaw_gateway`. +- Onboarding examples now require `adapterType: "openclaw_gateway"` + `ws://`/`wss://` URL + gateway token header. +- Added fail-fast guidance for short/placeholder tokens. -There is no adapter-specific onboarding manifest/text for `openclaw_gateway`. +### 2) Company settings snippet gateway path (resolved) +Resolved in `ui/src/pages/CompanySettings.tsx`: +- Snippet now instructs OpenClaw Gateway onboarding. +- Snippet explicitly says not to use `/v1/responses` or `/hooks/*` for this flow. -### 2) Company settings snippet is OpenClaw HTTP-first -`ui/src/pages/CompanySettings.tsx` generates one snippet that: -- Assumes OpenClaw HTTP callback setup. -- Instructs enabling `gateway.http.endpoints.responses.enabled=true`. -- Does not provide a dedicated gateway onboarding path. - -### 3) Invite landing “agent join” UX is not wired for OpenClaw adapters +### 3) Invite landing “agent join” UX is not wired for OpenClaw adapters (open) `ui/src/pages/InviteLanding.tsx` shows `openclaw` and `openclaw_gateway` as disabled (“Coming soon”) in join UI. -### 4) Join normalization/replay logic only special-cases `adapterType === "openclaw"` -`server/src/routes/access.ts` helper paths (`buildJoinDefaultsPayloadForAccept`, replay, normalization diagnostics) are OpenClaw-HTTP specific. -No equivalent normalization/diagnostics for gateway defaults. +### 4) Join normalization/replay logic parity (partially resolved) +Resolved: +- `buildJoinDefaultsPayloadForAccept` now normalizes wrapped gateway token headers for `openclaw_gateway`. +- `normalizeAgentDefaultsForJoin` now validates `openclaw_gateway` URL/token and rejects short placeholder tokens at invite-accept time. + +Still open: +- Invite replay path is still special-cased to legacy `openclaw` joins. ### 5) Webhook confusion is expected in current setup For `openclaw` + `streamTransport=webhook`: @@ -257,11 +258,23 @@ POST /api/companies/$CLA_COMPANY_ID/invites ``` 3. Approve join request. -4. Claim API key with `claimSecret`. -5. Save claimed token to OpenClaw expected file path (`~/.openclaw/workspace/paperclip-claimed-api-key.json`) and ensure `PAPERCLIP_API_KEY` + `PAPERCLIP_API_URL` are available for OpenClaw skill execution context. +4. **Hard gate before any task run:** fetch created agent config and validate: +- `adapterType == "openclaw_gateway"` +- `adapterConfig.url` uses `ws://` or `wss://` +- `adapterConfig.headers.x-openclaw-token` exists and is not placeholder/too-short (`len >= 16`) +- token hash matches the OpenClaw `gateway.auth.token` used for join +- pairing mode is explicit: + - default path: `adapterConfig.disableDeviceAuth` is false/absent and stable `adapterConfig.devicePrivateKeyPem` is set so approvals persist across runs + - fallback path: `disableDeviceAuth=true` only for environments that cannot support pairing +5. Trigger one connectivity run. If it returns `pairing required`, approve the pending device request in OpenClaw and retry once. + - Local docker automation path: + - `openclaw devices approve --latest --json --url ws://127.0.0.1:18789 --token ` + - After approval, retries should succeed using the persisted `devicePrivateKeyPem`. +6. Claim API key with `claimSecret`. +7. Save claimed token to OpenClaw expected file path (`~/.openclaw/workspace/paperclip-claimed-api-key.json`) and ensure `PAPERCLIP_API_KEY` + `PAPERCLIP_API_URL` are available for OpenClaw skill execution context. - Write compatibility JSON keys (`token` and `apiKey`) to avoid runtime parser mismatch. -6. Ensure Paperclip skill is installed for OpenClaw runtime. -7. Send one bootstrap prompt to OpenClaw containing all setup instructions needed for this run (auth file usage, heartbeat procedure, required tools). If needed, send one follow-up nudge only. +8. Ensure Paperclip skill is installed for OpenClaw runtime. +9. Send one bootstrap prompt to OpenClaw containing all setup instructions needed for this run (auth file usage, heartbeat procedure, required tools). If needed, send one follow-up nudge only. ## 6) E2E Validation Cases @@ -311,6 +324,8 @@ Responsibilities: - CLA company resolution. - Old OpenClaw agent cleanup. - Invite/join/approve/claim orchestration. +- Gateway agent config/token preflight validation before connectivity or case execution. +- Pairing-mode preflight (`disableDeviceAuth=false` + stable `devicePrivateKeyPem` by default). - E2E case execution + assertions. - Final summary with run IDs, issue IDs, agent ID. @@ -347,5 +362,6 @@ Responsibilities: ## Acceptance Criteria - No webhook-mode ambiguity: webhook path does not silently appear as SSE success without explicit compatibility signal. - Gateway onboarding is first-class and copy/pasteable from company settings. +- Gateway join fails fast if token is missing/placeholder, and smoke preflight verifies adapter/token parity before task runs. - Codex can run end-to-end onboarding and validation against CLA with repeatable cleanup. - All three validation cases are documented with pass/fail criteria and reproducible evidence paths. diff --git a/packages/adapters/openclaw-gateway/src/server/execute.ts b/packages/adapters/openclaw-gateway/src/server/execute.ts index 407e455b..ceec0b91 100644 --- a/packages/adapters/openclaw-gateway/src/server/execute.ts +++ b/packages/adapters/openclaw-gateway/src/server/execute.ts @@ -1074,15 +1074,23 @@ export async function execute(ctx: AdapterExecutionContext): Promise&1)"; then + log "pairing approval response: $(printf "%s" "$output" | tr '\n' ' ' | cut -c1-400)" + return 0 + fi + + warn "pairing auto-approve failed: $(printf "%s" "$output" | tr '\n' ' ' | cut -c1-400)" + return 1 +} + trigger_wakeup() { local reason="$1" local issue_id="${2:-}" @@ -764,8 +831,9 @@ run_case_c() { local marker="OPENCLAW_CASE_C_CREATED_$(date +%s)" local ack_marker="OPENCLAW_CASE_C_ACK_$(date +%s)" + local original_issue_reference="the original case issue you are currently reading" local description - description="Case C validation.\n\nTreat this run as a fresh/new session.\nCreate a NEW Paperclip issue in this same company with title exactly:\n${marker}\nUse description: 'created by case C smoke'.\n\nThen post a comment on this issue containing exactly: ${ack_marker}\nThen mark this issue done." + description="Case C validation.\n\nTreat this run as a fresh/new session.\nCreate a NEW Paperclip issue in this same company with title exactly:\n${marker}\nUse description: 'created by case C smoke'.\n\nThen post a comment on ${original_issue_reference} containing exactly: ${ack_marker}\nDo NOT post the ACK comment on the newly created issue.\nThen mark the original case issue done." local created created="$(create_issue_for_case "[OpenClaw Gateway Smoke] Case C" "$description")" @@ -840,14 +908,32 @@ main() { create_and_approve_gateway_join "$gateway_token" log "joined/approved agent ${AGENT_ID} invite=${INVITE_ID} joinRequest=${JOIN_REQUEST_ID}" + validate_joined_gateway_agent "$gateway_token" - trigger_wakeup "openclaw_gateway_smoke_connectivity" - if [[ -n "$RUN_ID" ]]; then - local connect_status + local connect_status="unknown" + local connect_attempt + for connect_attempt in 1 2; do + trigger_wakeup "openclaw_gateway_smoke_connectivity_attempt_${connect_attempt}" + if [[ -z "$RUN_ID" ]]; then + connect_status="unknown" + break + fi connect_status="$(wait_for_run_terminal "$RUN_ID" "$RUN_TIMEOUT_SEC")" - [[ "$connect_status" == "succeeded" ]] || fail "connectivity wake run failed: ${connect_status}" - log "connectivity wake run ${RUN_ID} succeeded" - fi + if [[ "$connect_status" == "succeeded" ]]; then + log "connectivity wake run ${RUN_ID} succeeded (attempt=${connect_attempt})" + break + fi + + if [[ "$PAIRING_AUTO_APPROVE" == "1" && "$connect_attempt" -eq 1 ]] && run_log_contains_pairing_required "$RUN_ID"; then + log "connectivity run hit pairing gate; attempting one-time pairing approval" + approve_latest_pairing_request "$gateway_token" || fail "pairing approval failed after pairing-required run ${RUN_ID}" + sleep 2 + continue + fi + + fail "connectivity wake run failed: ${connect_status} (attempt=${connect_attempt}, runId=${RUN_ID})" + done + [[ "$connect_status" == "succeeded" ]] || fail "connectivity wake run did not succeed after retries" run_case_a run_case_b diff --git a/server/src/__tests__/invite-accept-openclaw-defaults.test.ts b/server/src/__tests__/invite-accept-openclaw-defaults.test.ts index b94dd55d..dc7b58e1 100644 --- a/server/src/__tests__/invite-accept-openclaw-defaults.test.ts +++ b/server/src/__tests__/invite-accept-openclaw-defaults.test.ts @@ -1,5 +1,8 @@ import { describe, expect, it } from "vitest"; -import { buildJoinDefaultsPayloadForAccept } from "../routes/access.js"; +import { + buildJoinDefaultsPayloadForAccept, + normalizeAgentDefaultsForJoin, +} from "../routes/access.js"; describe("buildJoinDefaultsPayloadForAccept", () => { it("maps OpenClaw compatibility fields into agent defaults", () => { @@ -208,4 +211,84 @@ describe("buildJoinDefaultsPayloadForAccept", () => { expect(result).toEqual(defaultsPayload); }); + + it("normalizes wrapped gateway token headers for openclaw_gateway", () => { + const result = buildJoinDefaultsPayloadForAccept({ + adapterType: "openclaw_gateway", + defaultsPayload: { + url: "ws://127.0.0.1:18789", + headers: { + "x-openclaw-token": { + value: "gateway-token-1234567890", + }, + }, + }, + }) as Record; + + expect(result).toMatchObject({ + url: "ws://127.0.0.1:18789", + headers: { + "x-openclaw-token": "gateway-token-1234567890", + }, + }); + }); + + it("accepts inbound x-openclaw-token for openclaw_gateway", () => { + const result = buildJoinDefaultsPayloadForAccept({ + adapterType: "openclaw_gateway", + defaultsPayload: { + url: "ws://127.0.0.1:18789", + }, + inboundOpenClawTokenHeader: "gateway-token-1234567890", + }) as Record; + + expect(result).toMatchObject({ + headers: { + "x-openclaw-token": "gateway-token-1234567890", + }, + }); + }); + + it("generates persistent device key for openclaw_gateway when device auth is enabled", () => { + const normalized = normalizeAgentDefaultsForJoin({ + adapterType: "openclaw_gateway", + defaultsPayload: { + url: "ws://127.0.0.1:18789", + headers: { + "x-openclaw-token": "gateway-token-1234567890", + }, + disableDeviceAuth: false, + }, + deploymentMode: "authenticated", + deploymentExposure: "private", + bindHost: "127.0.0.1", + allowedHostnames: [], + }); + + expect(normalized.fatalErrors).toEqual([]); + expect(normalized.normalized?.disableDeviceAuth).toBe(false); + expect(typeof normalized.normalized?.devicePrivateKeyPem).toBe("string"); + expect((normalized.normalized?.devicePrivateKeyPem as string).length).toBeGreaterThan(64); + }); + + it("does not generate device key when openclaw_gateway has disableDeviceAuth=true", () => { + const normalized = normalizeAgentDefaultsForJoin({ + adapterType: "openclaw_gateway", + defaultsPayload: { + url: "ws://127.0.0.1:18789", + headers: { + "x-openclaw-token": "gateway-token-1234567890", + }, + disableDeviceAuth: true, + }, + deploymentMode: "authenticated", + deploymentExposure: "private", + bindHost: "127.0.0.1", + allowedHostnames: [], + }); + + expect(normalized.fatalErrors).toEqual([]); + expect(normalized.normalized?.disableDeviceAuth).toBe(true); + expect(normalized.normalized?.devicePrivateKeyPem).toBeUndefined(); + }); }); diff --git a/server/src/__tests__/invite-onboarding-text.test.ts b/server/src/__tests__/invite-onboarding-text.test.ts index 6f22c2d5..8ba30115 100644 --- a/server/src/__tests__/invite-onboarding-text.test.ts +++ b/server/src/__tests__/invite-onboarding-text.test.ts @@ -37,21 +37,22 @@ describe("buildInviteOnboardingTextDocument", () => { allowedHostnames: [], }); - expect(text).toContain("Paperclip OpenClaw Onboarding"); + expect(text).toContain("Paperclip OpenClaw Gateway Onboarding"); expect(text).toContain("/api/invites/token-123/accept"); expect(text).toContain("/api/join-requests/{requestId}/claim-api-key"); expect(text).toContain("/api/invites/token-123/onboarding.txt"); - expect(text).toContain("/api/invites/token-123/test-resolution"); expect(text).toContain("Suggested Paperclip base URLs to try"); expect(text).toContain("http://localhost:3100"); expect(text).toContain("host.docker.internal"); expect(text).toContain("paperclipApiUrl"); - expect(text).toContain("You MUST include agentDefaultsPayload.headers.x-openclaw-auth"); - expect(text).toContain("will fail with 401 Unauthorized"); + expect(text).toContain("adapterType \"openclaw_gateway\""); + expect(text).toContain("headers.x-openclaw-token"); + expect(text).toContain("Do NOT use /v1/responses or /hooks/*"); expect(text).toContain("set the first reachable candidate as agentDefaultsPayload.paperclipApiUrl"); expect(text).toContain("~/.openclaw/workspace/paperclip-claimed-api-key.json"); expect(text).toContain("PAPERCLIP_API_KEY"); expect(text).toContain("saved token field"); + expect(text).toContain("Gateway token unexpectedly short"); }); it("includes loopback diagnostics for authenticated/private onboarding", () => { diff --git a/server/src/routes/access.ts b/server/src/routes/access.ts index 186b8515..406e4bd3 100644 --- a/server/src/routes/access.ts +++ b/server/src/routes/access.ts @@ -1,4 +1,9 @@ -import { createHash, randomBytes, timingSafeEqual } from "node:crypto"; +import { + createHash, + generateKeyPairSync, + randomBytes, + timingSafeEqual +} from "node:crypto"; import fs from "node:fs"; import path from "node:path"; import { fileURLToPath } from "node:url"; @@ -311,6 +316,32 @@ function toAuthorizationHeaderValue(rawToken: string): string { return /^bearer\s+/i.test(trimmed) ? trimmed : `Bearer ${trimmed}`; } +function tokenFromAuthorizationHeader(rawHeader: string | null): string | null { + const trimmed = nonEmptyTrimmedString(rawHeader); + if (!trimmed) return null; + const bearerMatch = trimmed.match(/^bearer\s+(.+)$/i); + if (bearerMatch?.[1]) { + return nonEmptyTrimmedString(bearerMatch[1]); + } + return trimmed; +} + +function parseBooleanLike(value: unknown): boolean | null { + if (typeof value === "boolean") return value; + if (typeof value !== "string") return null; + const normalized = value.trim().toLowerCase(); + if (normalized === "true" || normalized === "1") return true; + if (normalized === "false" || normalized === "0") return false; + return null; +} + +function generateEd25519PrivateKeyPem(): string { + const generated = generateKeyPairSync("ed25519"); + return generated.privateKey + .export({ type: "pkcs8", format: "pem" }) + .toString(); +} + export function buildJoinDefaultsPayloadForAccept(input: { adapterType: string | null; defaultsPayload: unknown; @@ -322,6 +353,59 @@ export function buildJoinDefaultsPayloadForAccept(input: { inboundOpenClawAuthHeader?: string | null; inboundOpenClawTokenHeader?: string | null; }): unknown { + if (input.adapterType === "openclaw_gateway") { + const merged = isPlainObject(input.defaultsPayload) + ? { ...(input.defaultsPayload as Record) } + : ({} as Record); + + if (!nonEmptyTrimmedString(merged.paperclipApiUrl)) { + const legacyPaperclipApiUrl = nonEmptyTrimmedString(input.paperclipApiUrl); + if (legacyPaperclipApiUrl) merged.paperclipApiUrl = legacyPaperclipApiUrl; + } + + const mergedHeaders = normalizeHeaderMap(merged.headers) ?? {}; + + const inboundOpenClawAuthHeader = nonEmptyTrimmedString( + input.inboundOpenClawAuthHeader + ); + const inboundOpenClawTokenHeader = nonEmptyTrimmedString( + input.inboundOpenClawTokenHeader + ); + if ( + inboundOpenClawTokenHeader && + !headerMapHasKeyIgnoreCase(mergedHeaders, "x-openclaw-token") + ) { + mergedHeaders["x-openclaw-token"] = inboundOpenClawTokenHeader; + } + if ( + inboundOpenClawAuthHeader && + !headerMapHasKeyIgnoreCase(mergedHeaders, "x-openclaw-auth") + ) { + mergedHeaders["x-openclaw-auth"] = inboundOpenClawAuthHeader; + } + + const discoveredToken = + headerMapGetIgnoreCase(mergedHeaders, "x-openclaw-token") ?? + headerMapGetIgnoreCase(mergedHeaders, "x-openclaw-auth") ?? + tokenFromAuthorizationHeader( + headerMapGetIgnoreCase(mergedHeaders, "authorization") + ); + if ( + discoveredToken && + !headerMapHasKeyIgnoreCase(mergedHeaders, "x-openclaw-token") + ) { + mergedHeaders["x-openclaw-token"] = discoveredToken; + } + + if (Object.keys(mergedHeaders).length > 0) { + merged.headers = mergedHeaders; + } else { + delete merged.headers; + } + + return Object.keys(merged).length > 0 ? merged : null; + } + if (input.adapterType !== "openclaw") { return input.defaultsPayload; } @@ -516,6 +600,43 @@ function summarizeOpenClawDefaultsForLog(defaultsPayload: unknown) { }; } +function summarizeOpenClawGatewayDefaultsForLog(defaultsPayload: unknown) { + const defaults = isPlainObject(defaultsPayload) + ? (defaultsPayload as Record) + : null; + const headers = defaults ? normalizeHeaderMap(defaults.headers) : undefined; + const gatewayTokenValue = headers + ? headerMapGetIgnoreCase(headers, "x-openclaw-token") ?? + headerMapGetIgnoreCase(headers, "x-openclaw-auth") ?? + tokenFromAuthorizationHeader( + headerMapGetIgnoreCase(headers, "authorization") + ) + : null; + return { + present: Boolean(defaults), + keys: defaults ? Object.keys(defaults).sort() : [], + url: defaults ? nonEmptyTrimmedString(defaults.url) : null, + paperclipApiUrl: defaults + ? nonEmptyTrimmedString(defaults.paperclipApiUrl) + : null, + headerKeys: headers ? Object.keys(headers).sort() : [], + sessionKeyStrategy: defaults + ? nonEmptyTrimmedString(defaults.sessionKeyStrategy) + : null, + disableDeviceAuth: defaults + ? parseBooleanLike(defaults.disableDeviceAuth) + : null, + waitTimeoutMs: + defaults && typeof defaults.waitTimeoutMs === "number" + ? defaults.waitTimeoutMs + : null, + devicePrivateKeyPem: defaults + ? summarizeSecretForLog(defaults.devicePrivateKeyPem) + : null, + gatewayToken: summarizeSecretForLog(gatewayTokenValue) + }; +} + function buildJoinConnectivityDiagnostics(input: { deploymentMode: DeploymentMode; deploymentExposure: DeploymentExposure; @@ -589,7 +710,7 @@ function buildJoinConnectivityDiagnostics(input: { return diagnostics; } -function normalizeAgentDefaultsForJoin(input: { +export function normalizeAgentDefaultsForJoin(input: { adapterType: string | null; defaultsPayload: unknown; deploymentMode: DeploymentMode; @@ -597,12 +718,259 @@ function normalizeAgentDefaultsForJoin(input: { bindHost: string; allowedHostnames: string[]; }) { + const fatalErrors: string[] = []; const diagnostics: JoinDiagnostic[] = []; - if (input.adapterType !== "openclaw") { + if ( + input.adapterType !== "openclaw" && + input.adapterType !== "openclaw_gateway" + ) { const normalized = isPlainObject(input.defaultsPayload) ? (input.defaultsPayload as Record) : null; - return { normalized, diagnostics }; + return { normalized, diagnostics, fatalErrors }; + } + + if (input.adapterType === "openclaw_gateway") { + if (!isPlainObject(input.defaultsPayload)) { + diagnostics.push({ + code: "openclaw_gateway_defaults_missing", + level: "warn", + message: + "No OpenClaw gateway config was provided in agentDefaultsPayload.", + hint: + "Include agentDefaultsPayload.url and headers.x-openclaw-token for OpenClaw gateway joins." + }); + fatalErrors.push( + "agentDefaultsPayload is required for adapterType=openclaw_gateway" + ); + return { + normalized: null as Record | null, + diagnostics, + fatalErrors + }; + } + + const defaults = input.defaultsPayload as Record; + const normalized: Record = {}; + + let gatewayUrl: URL | null = null; + const rawGatewayUrl = nonEmptyTrimmedString(defaults.url); + if (!rawGatewayUrl) { + diagnostics.push({ + code: "openclaw_gateway_url_missing", + level: "warn", + message: "OpenClaw gateway URL is missing.", + hint: "Set agentDefaultsPayload.url to ws:// or wss:// gateway URL." + }); + fatalErrors.push("agentDefaultsPayload.url is required"); + } else { + try { + gatewayUrl = new URL(rawGatewayUrl); + if ( + gatewayUrl.protocol !== "ws:" && + gatewayUrl.protocol !== "wss:" + ) { + diagnostics.push({ + code: "openclaw_gateway_url_protocol", + level: "warn", + message: `OpenClaw gateway URL must use ws:// or wss:// (got ${gatewayUrl.protocol}).` + }); + fatalErrors.push( + "agentDefaultsPayload.url must use ws:// or wss:// for openclaw_gateway" + ); + } else { + normalized.url = gatewayUrl.toString(); + diagnostics.push({ + code: "openclaw_gateway_url_configured", + level: "info", + message: `Gateway endpoint set to ${gatewayUrl.toString()}` + }); + } + } catch { + diagnostics.push({ + code: "openclaw_gateway_url_invalid", + level: "warn", + message: `Invalid OpenClaw gateway URL: ${rawGatewayUrl}` + }); + fatalErrors.push("agentDefaultsPayload.url is not a valid URL"); + } + } + + const headers = normalizeHeaderMap(defaults.headers) ?? {}; + const gatewayToken = + headerMapGetIgnoreCase(headers, "x-openclaw-token") ?? + headerMapGetIgnoreCase(headers, "x-openclaw-auth") ?? + tokenFromAuthorizationHeader(headerMapGetIgnoreCase(headers, "authorization")); + if ( + gatewayToken && + !headerMapHasKeyIgnoreCase(headers, "x-openclaw-token") + ) { + headers["x-openclaw-token"] = gatewayToken; + } + if (Object.keys(headers).length > 0) { + normalized.headers = headers; + } + + if (!gatewayToken) { + diagnostics.push({ + code: "openclaw_gateway_auth_header_missing", + level: "warn", + message: "Gateway auth token is missing from agent defaults.", + hint: + "Set agentDefaultsPayload.headers.x-openclaw-token (or legacy x-openclaw-auth)." + }); + fatalErrors.push( + "agentDefaultsPayload.headers.x-openclaw-token (or x-openclaw-auth) is required" + ); + } else if (gatewayToken.trim().length < 16) { + diagnostics.push({ + code: "openclaw_gateway_auth_header_too_short", + level: "warn", + message: `Gateway auth token appears too short (${gatewayToken.trim().length} chars).`, + hint: + "Use the full gateway auth token from ~/.openclaw/openclaw.json (typically long random string)." + }); + fatalErrors.push( + "agentDefaultsPayload.headers.x-openclaw-token is too short; expected a full gateway token" + ); + } else { + diagnostics.push({ + code: "openclaw_gateway_auth_header_configured", + level: "info", + message: "Gateway auth token configured." + }); + } + + if (isPlainObject(defaults.payloadTemplate)) { + normalized.payloadTemplate = defaults.payloadTemplate; + } + + const parsedDisableDeviceAuth = parseBooleanLike(defaults.disableDeviceAuth); + const disableDeviceAuth = parsedDisableDeviceAuth === true; + if (parsedDisableDeviceAuth !== null) { + normalized.disableDeviceAuth = parsedDisableDeviceAuth; + } + + const configuredDevicePrivateKeyPem = nonEmptyTrimmedString( + defaults.devicePrivateKeyPem + ); + if (configuredDevicePrivateKeyPem) { + normalized.devicePrivateKeyPem = configuredDevicePrivateKeyPem; + diagnostics.push({ + code: "openclaw_gateway_device_key_configured", + level: "info", + message: + "Gateway device key configured. Pairing approvals should persist for this agent." + }); + } else if (!disableDeviceAuth) { + try { + normalized.devicePrivateKeyPem = generateEd25519PrivateKeyPem(); + diagnostics.push({ + code: "openclaw_gateway_device_key_generated", + level: "info", + message: + "Generated persistent gateway device key for this join. Pairing approvals should persist for this agent." + }); + } catch (err) { + diagnostics.push({ + code: "openclaw_gateway_device_key_generate_failed", + level: "warn", + message: `Failed to generate gateway device key: ${ + err instanceof Error ? err.message : String(err) + }`, + hint: + "Set agentDefaultsPayload.devicePrivateKeyPem explicitly or set disableDeviceAuth=true." + }); + fatalErrors.push( + "Failed to generate gateway device key. Set devicePrivateKeyPem or disableDeviceAuth=true." + ); + } + } + + const waitTimeoutMs = + typeof defaults.waitTimeoutMs === "number" && + Number.isFinite(defaults.waitTimeoutMs) + ? Math.floor(defaults.waitTimeoutMs) + : typeof defaults.waitTimeoutMs === "string" + ? Number.parseInt(defaults.waitTimeoutMs.trim(), 10) + : NaN; + if (Number.isFinite(waitTimeoutMs) && waitTimeoutMs > 0) { + normalized.waitTimeoutMs = waitTimeoutMs; + } + + const timeoutSec = + typeof defaults.timeoutSec === "number" && Number.isFinite(defaults.timeoutSec) + ? Math.floor(defaults.timeoutSec) + : typeof defaults.timeoutSec === "string" + ? Number.parseInt(defaults.timeoutSec.trim(), 10) + : NaN; + if (Number.isFinite(timeoutSec) && timeoutSec > 0) { + normalized.timeoutSec = timeoutSec; + } + + const sessionKeyStrategy = nonEmptyTrimmedString(defaults.sessionKeyStrategy); + if ( + sessionKeyStrategy === "fixed" || + sessionKeyStrategy === "issue" || + sessionKeyStrategy === "run" + ) { + normalized.sessionKeyStrategy = sessionKeyStrategy; + } + + const sessionKey = nonEmptyTrimmedString(defaults.sessionKey); + if (sessionKey) { + normalized.sessionKey = sessionKey; + } + + const role = nonEmptyTrimmedString(defaults.role); + if (role) { + normalized.role = role; + } + + if (Array.isArray(defaults.scopes)) { + const scopes = defaults.scopes + .filter((entry): entry is string => typeof entry === "string") + .map((entry) => entry.trim()) + .filter(Boolean); + if (scopes.length > 0) { + normalized.scopes = scopes; + } + } + + const rawPaperclipApiUrl = + typeof defaults.paperclipApiUrl === "string" + ? defaults.paperclipApiUrl.trim() + : ""; + if (rawPaperclipApiUrl) { + try { + const parsedPaperclipApiUrl = new URL(rawPaperclipApiUrl); + if ( + parsedPaperclipApiUrl.protocol !== "http:" && + parsedPaperclipApiUrl.protocol !== "https:" + ) { + diagnostics.push({ + code: "openclaw_gateway_paperclip_api_url_protocol", + level: "warn", + message: `paperclipApiUrl must use http:// or https:// (got ${parsedPaperclipApiUrl.protocol}).` + }); + } else { + normalized.paperclipApiUrl = parsedPaperclipApiUrl.toString(); + diagnostics.push({ + code: "openclaw_gateway_paperclip_api_url_configured", + level: "info", + message: `paperclipApiUrl set to ${parsedPaperclipApiUrl.toString()}` + }); + } + } catch { + diagnostics.push({ + code: "openclaw_gateway_paperclip_api_url_invalid", + level: "warn", + message: `Invalid paperclipApiUrl: ${rawPaperclipApiUrl}` + }); + } + } + + return { normalized, diagnostics, fatalErrors }; } if (!isPlainObject(input.defaultsPayload)) { @@ -613,7 +981,11 @@ function normalizeAgentDefaultsForJoin(input: { "No OpenClaw callback config was provided in agentDefaultsPayload.", hint: "Include agentDefaultsPayload.url so Paperclip can invoke the OpenClaw endpoint immediately after approval." }); - return { normalized: null as Record | null, diagnostics }; + return { + normalized: null as Record | null, + diagnostics, + fatalErrors + }; } const defaults = input.defaultsPayload as Record; @@ -790,7 +1162,7 @@ function normalizeAgentDefaultsForJoin(input: { }) ); - return { normalized, diagnostics }; + return { normalized, diagnostics, fatalErrors }; } function toInviteSummaryResponse( @@ -950,10 +1322,6 @@ function buildInviteOnboardingManifest( const onboardingTextUrl = baseUrl ? `${baseUrl}${onboardingTextPath}` : onboardingTextPath; - const testResolutionPath = `/api/invites/${token}/test-resolution`; - const testResolutionUrl = baseUrl - ? `${baseUrl}${testResolutionPath}` - : testResolutionPath; const discoveryDiagnostics = buildOnboardingDiscoveryDiagnostics({ apiBaseUrl: baseUrl, deploymentMode: opts.deploymentMode, @@ -971,16 +1339,16 @@ function buildInviteOnboardingManifest( invite: toInviteSummaryResponse(req, token, invite), onboarding: { instructions: - "Join as an OpenClaw agent, save your one-time claim secret, wait for board approval, then claim your API key. Save the claim response token to ~/.openclaw/workspace/paperclip-claimed-api-key.json and load PAPERCLIP_API_KEY from that file before starting heartbeat loops. You MUST include agentDefaultsPayload.headers.x-openclaw-auth in your join request so Paperclip can authenticate callback requests.", + "Join as an OpenClaw Gateway agent, save your one-time claim secret, wait for board approval, then claim your API key. Save the claim response token to ~/.openclaw/workspace/paperclip-claimed-api-key.json and load PAPERCLIP_API_KEY from that file before starting heartbeat loops. You MUST submit adapterType='openclaw_gateway', set agentDefaultsPayload.url to your ws:// or wss:// OpenClaw gateway endpoint, and include agentDefaultsPayload.headers.x-openclaw-token (or legacy x-openclaw-auth).", inviteMessage: extractInviteMessage(invite), - recommendedAdapterType: "openclaw", + recommendedAdapterType: "openclaw_gateway", requiredFields: { requestType: "agent", agentName: "Display name for this agent", - adapterType: "Use 'openclaw' for OpenClaw agents", + adapterType: "Use 'openclaw_gateway' for OpenClaw Gateway agents", capabilities: "Optional capability summary", agentDefaultsPayload: - "Adapter config for OpenClaw endpoint. MUST include headers.x-openclaw-auth; include streamTransport ('sse' or 'webhook') plus url/method/paperclipApiUrl (and optional webhookAuthHeader/timeoutSec/payloadTemplate)." + "Adapter config for OpenClaw gateway. MUST include url (ws:// or wss://) and headers.x-openclaw-token (or legacy x-openclaw-auth). Optional fields: paperclipApiUrl, waitTimeoutMs, sessionKeyStrategy, sessionKey, role, scopes, disableDeviceAuth, devicePrivateKeyPem." }, registrationEndpoint: { method: "POST", @@ -1001,21 +1369,12 @@ function buildInviteOnboardingManifest( bindHost: opts.bindHost, allowedHostnames: opts.allowedHostnames, connectionCandidates, - testResolutionEndpoint: { - method: "GET", - path: testResolutionPath, - url: testResolutionUrl, - query: { - url: "https://your-openclaw-agent.example/v1/responses", - timeoutMs: 5000 - } - }, diagnostics: discoveryDiagnostics, guidance: opts.deploymentMode === "authenticated" && opts.deploymentExposure === "private" ? "If OpenClaw runs on another machine, ensure the Paperclip hostname is reachable and allowed via `pnpm paperclipai allowed-hostname `." - : "Ensure OpenClaw can reach this Paperclip API base URL for callbacks and claims." + : "Ensure OpenClaw can reach this Paperclip API base URL for invite, claim, and skill bootstrap calls." }, textInstructions: { path: onboardingTextPath, @@ -1076,7 +1435,7 @@ export function buildInviteOnboardingTextDocument( }; appendBlock(` - # Paperclip OpenClaw Onboarding + # Paperclip OpenClaw Gateway Onboarding This document is meant to be readable by both humans and agents. @@ -1104,6 +1463,7 @@ export function buildInviteOnboardingTextDocument( TOKEN="$(node -p 'require(process.env.HOME+\"/.openclaw/openclaw.json\").gateway.auth.token')" test -n "$TOKEN" || (echo "Missing TOKEN" && exit 1) + test "\${#TOKEN}" -ge 16 || (echo "Gateway token unexpectedly short (\${#TOKEN})" && exit 1) 3) IMPORTANT: Don't accidentally drop the token when generating JSON If you build JSON with Node, pass the token explicitly (argv), don't rely on an un-exported env var. @@ -1116,15 +1476,17 @@ export function buildInviteOnboardingTextDocument( const body = { requestType: "agent", agentName: "OpenClaw", - adapterType: "openclaw", + adapterType: "openclaw_gateway", capabilities: "OpenClaw agent adapter", agentDefaultsPayload: { - url: "http://127.0.0.1:18789/v1/responses", + url: "ws://127.0.0.1:18789", paperclipApiUrl: "http://host.docker.internal:3100", - streamTransport: "sse", - method: "POST", - headers: { "x-openclaw-auth": token }, - timeoutSec: 0 + headers: { "x-openclaw-token": token }, + waitTimeoutMs: 120000, + sessionKeyStrategy: "fixed", + sessionKey: "paperclip", + role: "operator", + scopes: ["operator.admin"] } }; process.stdout.write(JSON.stringify(body)); @@ -1135,23 +1497,30 @@ export function buildInviteOnboardingTextDocument( onboarding.registrationEndpoint.url } - IMPORTANT: You MUST include agentDefaultsPayload.headers.x-openclaw-auth with your gateway token. - Without this token, Paperclip callback requests to your OpenClaw endpoint will fail with 401 Unauthorized. - Set "streamTransport" to "sse" for streaming /v1/responses endpoints, or "webhook" for wake-style callbacks. + IMPORTANT: You MUST include agentDefaultsPayload.headers.x-openclaw-token with your gateway token. + Legacy x-openclaw-auth is also accepted, but x-openclaw-token is preferred. + Use adapterType "openclaw_gateway" and a ws:// or wss:// gateway URL. + Pairing mode requirement: + - Keep device auth enabled (recommended). If devicePrivateKeyPem is omitted, Paperclip generates and persists one during join so pairing approvals are stable. + - You may set disableDeviceAuth=true only for special environments that cannot support pairing. + - First run may return "pairing required" once; approve the pending pairing request in OpenClaw, then retry. + Do NOT use /v1/responses or /hooks/* in this gateway join flow. Body (JSON): { "requestType": "agent", "agentName": "My OpenClaw Agent", - "adapterType": "openclaw", + "adapterType": "openclaw_gateway", "capabilities": "Optional summary", "agentDefaultsPayload": { - "url": "https://your-openclaw-agent.example/v1/responses", + "url": "wss://your-openclaw-gateway.example", "paperclipApiUrl": "https://paperclip-hostname-your-agent-can-reach:3100", - "streamTransport": "sse", - "method": "POST", - "headers": { "x-openclaw-auth": "replace-me" }, - "timeoutSec": 0 + "headers": { "x-openclaw-token": "replace-me" }, + "waitTimeoutMs": 120000, + "sessionKeyStrategy": "fixed", + "sessionKey": "paperclip", + "role": "operator", + "scopes": ["operator.admin"] } } @@ -1160,11 +1529,6 @@ export function buildInviteOnboardingTextDocument( - one-time claimSecret - claimApiKeyPath - Verify the response diagnostics include: - 'openclaw_auth_header_configured' - and do not include: - 'openclaw_auth_header_missing' - ## Step 2: Wait for board approval The board approves the join request in Paperclip before key claim is allowed. @@ -1218,17 +1582,6 @@ export function buildInviteOnboardingTextDocument( } `); - if (onboarding.connectivity?.testResolutionEndpoint?.url) { - appendBlock(` - ## Optional: test callback resolution from Paperclip - ${onboarding.connectivity.testResolutionEndpoint.method ?? "GET"} ${ - onboarding.connectivity.testResolutionEndpoint.url - }?url=https%3A%2F%2Fyour-openclaw-agent.example%2Fv1%2Fresponses - - This endpoint checks whether Paperclip can reach your OpenClaw endpoint and reports reachable, timeout, or unreachable. - `); - } - const connectionCandidates = Array.isArray( onboarding.connectivity?.connectionCandidates ) @@ -1271,9 +1624,6 @@ export function buildInviteOnboardingTextDocument( ${onboarding.skill.path} ${manifest.invite.onboardingPath} `); - if (onboarding.connectivity?.testResolutionEndpoint?.path) { - lines.push(`${onboarding.connectivity.testResolutionEndpoint.path}`); - } return `${lines.join("\n")}\n`; } @@ -1958,9 +2308,14 @@ export function accessRoutes( }) : { normalized: null as Record | null, - diagnostics: [] as JoinDiagnostic[] + diagnostics: [] as JoinDiagnostic[], + fatalErrors: [] as string[] }; + if (requestType === "agent" && joinDefaults.fatalErrors.length > 0) { + throw badRequest(joinDefaults.fatalErrors.join("; ")); + } + if (requestType === "agent" && adapterType === "openclaw") { logger.info( { @@ -1977,6 +2332,22 @@ export function accessRoutes( ); } + if (requestType === "agent" && adapterType === "openclaw_gateway") { + logger.info( + { + inviteId: invite.id, + joinRequestDiagnostics: joinDefaults.diagnostics.map((diag) => ({ + code: diag.code, + level: diag.level + })), + normalizedAgentDefaults: summarizeOpenClawGatewayDefaultsForLog( + joinDefaults.normalized + ) + }, + "invite accept normalized OpenClaw gateway defaults" + ); + } + const claimSecret = requestType === "agent" && !inviteAlreadyAccepted ? createClaimSecret() diff --git a/ui/src/pages/CompanySettings.tsx b/ui/src/pages/CompanySettings.tsx index 0b9cd255..c11bd8b9 100644 --- a/ui/src/pages/CompanySettings.tsx +++ b/ui/src/pages/CompanySettings.tsx @@ -461,7 +461,7 @@ Suggested command: Then verify with: curl -fsS /api/health`; const resolutionLine = resolutionTestUrl - ? `\nYou MUST test callback reachability, call: ${resolutionTestUrl}?url= (well using the hostname that worked above). Do not assume your 172.x is necessarily reachable from Paperclip. Test it. ` + ? `\nYou MUST test Paperclip-to-gateway reachability, call: ${resolutionTestUrl}?url= (using the hostname that worked above). Do not assume your 172.x is necessarily reachable from Paperclip. Test it. ` : ""; return `You're invited to join a Paperclip organization. @@ -476,13 +476,20 @@ Verify the hostname works from your runtime with: GET /api/health ${connectivityBlock} For your "agentName", just use your own agent name (OpenClaw or whatever your name is) and send it directly. -You also need to send a webhook URL that Paperclip can access. Check your config and include that URL.${resolutionLine} +You also need to send a gateway URL that Paperclip can access. Check your config and include that URL.${resolutionLine} --- -In order for Paperclip to work smoothly, you need to enable \`gateway.http.endpoints.responses.enabled=true\` in your OpenClaw config. +This onboarding flow is for OpenClaw Gateway. +When you submit the join request, use: +- \`adapterType: "openclaw_gateway"\` +- \`agentDefaultsPayload.url\` as your \`ws://\` or \`wss://\` gateway URL +- \`agentDefaultsPayload.headers["x-openclaw-token"]\` with your gateway token +- (legacy accepted) \`agentDefaultsPayload.headers["x-openclaw-auth"]\` +- Keep device auth enabled (recommended). If \`devicePrivateKeyPem\` is omitted, Paperclip will generate and persist one during join so pairing approvals remain stable. +- Only use \`disableDeviceAuth=true\` for special environments where pairing cannot be completed. -Please enable that setting and notify your user that we did so. +Do NOT use \`/v1/responses\` or \`/hooks/*\` in this join flow. Before you do anything, please respond to your user that you understand the instructions and you're going to work on them. Then do the step above in another session called "paperclip-onboarding" and then tell your user when you're done. Update your user in intermediate steps along the way so they know what's going on.