Merge pull request #790 from paperclipai/paperclip-token-optimization
Optimize heartbeat token usage
This commit is contained in:
@@ -118,10 +118,18 @@ Result:
|
||||
|
||||
Local adapters inject repo skills into runtime skill directories.
|
||||
|
||||
Important `codex_local` nuance:
|
||||
|
||||
- Codex does not read skills directly from the active worktree.
|
||||
- Paperclip discovers repo skills from the current checkout, then symlinks them into `$CODEX_HOME/skills` or `~/.codex/skills`.
|
||||
- If an existing Paperclip skill symlink already points at another live checkout, the current implementation skips it instead of repointing it.
|
||||
- This can leave Codex using stale skill content from a different worktree even after Paperclip-side skill changes land.
|
||||
- That is both a correctness risk and a token-analysis risk, because runtime behavior may not reflect the instructions in the checkout being tested.
|
||||
|
||||
Current repo skill sizes:
|
||||
|
||||
- `skills/paperclip/SKILL.md`: 17,441 bytes
|
||||
- `skills/create-agent-adapter/SKILL.md`: 31,832 bytes
|
||||
- `.agents/skills/create-agent-adapter/SKILL.md`: 31,832 bytes
|
||||
- `skills/paperclip-create-agent/SKILL.md`: 4,718 bytes
|
||||
- `skills/para-memory-files/SKILL.md`: 3,978 bytes
|
||||
|
||||
@@ -215,6 +223,8 @@ This is the right version of the discussion’s bootstrap idea.
|
||||
|
||||
Static instructions and dynamic wake context have different cache behavior and should be modeled separately.
|
||||
|
||||
For `codex_local`, this also requires isolating the Codex skill home per worktree or teaching Paperclip to repoint its own skill symlinks when the source checkout changes. Otherwise prompt and skill improvements in the active worktree may not reach the running agent.
|
||||
|
||||
### Success criteria
|
||||
|
||||
- fresh-session prompts can remain richer without inflating every resumed heartbeat
|
||||
@@ -305,6 +315,9 @@ Even when reuse is desirable, some sessions become too expensive to keep alive i
|
||||
- `para-memory-files`
|
||||
- `create-agent-adapter`
|
||||
- Expose active skill set in agent config and run metadata.
|
||||
- For `codex_local`, either:
|
||||
- run with a worktree-specific `CODEX_HOME`, or
|
||||
- treat Paperclip-owned Codex skill symlinks as repairable when they point at a different checkout
|
||||
|
||||
### Why
|
||||
|
||||
@@ -363,6 +376,7 @@ Initial targets:
|
||||
6. Rewrite `skills/paperclip/SKILL.md` around delta-fetch behavior.
|
||||
7. Add session rotation with carry-forward summaries.
|
||||
8. Replace global skill injection with explicit allowlists.
|
||||
9. Fix `codex_local` skill resolution so worktree-local skill changes reliably reach the runtime.
|
||||
|
||||
## Recommendation
|
||||
|
||||
|
||||
186
doc/plans/2026-03-13-paperclip-skill-tightening-plan.md
Normal file
186
doc/plans/2026-03-13-paperclip-skill-tightening-plan.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# Paperclip Skill Tightening Plan
|
||||
|
||||
## Status
|
||||
|
||||
Deferred follow-up. Do not include in the current token-optimization PR beyond documenting the plan.
|
||||
|
||||
## Why This Is Deferred
|
||||
|
||||
The `paperclip` skill is part of the critical control-plane safety surface. Tightening it may reduce fresh-session token use, but it also carries prompt-regression risk. We do not yet have evals that would let us safely prove behavior preservation across assignment handling, checkout rules, comment etiquette, approval workflows, and escalation paths.
|
||||
|
||||
The current PR should ship the lower-risk infrastructure wins first:
|
||||
|
||||
- telemetry normalization
|
||||
- safe session reuse
|
||||
- incremental issue/comment context
|
||||
- bootstrap versus heartbeat prompt separation
|
||||
- Codex worktree isolation
|
||||
|
||||
## Current Problem
|
||||
|
||||
Fresh runs still spend substantial input tokens even after the context-path fixes. The remaining large startup cost appears to come from loading the full `paperclip` skill and related instruction surface into context at run start.
|
||||
|
||||
The skill currently mixes three kinds of content in one file:
|
||||
|
||||
- hot-path heartbeat procedure used on nearly every run
|
||||
- critical policy and safety invariants
|
||||
- rare workflow/reference material that most runs do not need
|
||||
|
||||
That structure is safe but expensive.
|
||||
|
||||
## Goals
|
||||
|
||||
- reduce first-run instruction tokens without weakening agent safety
|
||||
- preserve all current Paperclip control-plane capabilities
|
||||
- keep common heartbeat behavior explicit and easy for agents to follow
|
||||
- move rare workflows and reference material out of the hot path
|
||||
- create a structure that can later be evaluated systematically
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- changing Paperclip API semantics
|
||||
- removing required governance rules
|
||||
- deleting rare workflows
|
||||
- changing agent defaults in the current PR
|
||||
|
||||
## Recommended Direction
|
||||
|
||||
### 1. Split Hot Path From Lookup Material
|
||||
|
||||
Restructure the skill into:
|
||||
|
||||
- an always-loaded core section for the common heartbeat loop
|
||||
- on-demand material for infrequent workflows and deep reference
|
||||
|
||||
The core should cover only what is needed on nearly every wake:
|
||||
|
||||
- auth and required headers
|
||||
- inbox-first assignment retrieval
|
||||
- mandatory checkout behavior
|
||||
- `heartbeat-context` first
|
||||
- incremental comment retrieval rules
|
||||
- mention/self-assign exception
|
||||
- blocked-task dedup
|
||||
- status/comment/release expectations before exit
|
||||
|
||||
### 2. Normalize The Skill Around One Canonical Procedure
|
||||
|
||||
The same rules are currently expressed multiple times across:
|
||||
|
||||
- heartbeat steps
|
||||
- critical rules
|
||||
- endpoint reference
|
||||
- workflow examples
|
||||
|
||||
Refactor so each operational fact has one primary home:
|
||||
|
||||
- procedure
|
||||
- invariant list
|
||||
- appendix/reference
|
||||
|
||||
This reduces prompt weight and lowers the chance of internal instruction drift.
|
||||
|
||||
### 3. Compress Prose Into High-Signal Instruction Forms
|
||||
|
||||
Rewrite the hot path using compact operational forms:
|
||||
|
||||
- short ordered checklist
|
||||
- flat invariant list
|
||||
- minimal examples only where ambiguity would be risky
|
||||
|
||||
Reduce:
|
||||
|
||||
- narrative explanation
|
||||
- repeated warnings already covered elsewhere
|
||||
- large example payloads for common operations
|
||||
- long endpoint matrices in the main body
|
||||
|
||||
### 4. Move Rare Workflows Behind Explicit Triggers
|
||||
|
||||
These workflows should remain available but should not dominate fresh-run context:
|
||||
|
||||
- OpenClaw invite flow
|
||||
- project setup flow
|
||||
- planning `<plan/>` writeback flow
|
||||
- instructions-path update flow
|
||||
- detailed link-formatting examples
|
||||
|
||||
Recommended approach:
|
||||
|
||||
- keep a short pointer in the main skill
|
||||
- move detailed procedures into sibling skills or referenced docs that agents read only when needed
|
||||
|
||||
### 5. Separate Policy From Reference
|
||||
|
||||
The skill should distinguish:
|
||||
|
||||
- mandatory operating rules
|
||||
- endpoint lookup/reference
|
||||
- business-process playbooks
|
||||
|
||||
That separation makes it easier to evaluate prompt changes later and lets adapters or orchestration choose what must always be loaded.
|
||||
|
||||
## Proposed Target Structure
|
||||
|
||||
1. Purpose and authentication
|
||||
2. Compact heartbeat procedure
|
||||
3. Hard invariants
|
||||
4. Required comment/update style
|
||||
5. Triggered workflow index
|
||||
6. Appendix/reference
|
||||
|
||||
## Rollout Plan
|
||||
|
||||
### Phase 1. Inventory And Measure
|
||||
|
||||
- annotate the current skill by section and estimate token weight
|
||||
- identify which sections are truly hot-path versus rare
|
||||
- capture representative runs to compare before/after prompt size and behavior
|
||||
|
||||
### Phase 2. Structural Refactor Without Semantic Changes
|
||||
|
||||
- rewrite the main skill into the target structure
|
||||
- preserve all existing rules and capabilities
|
||||
- move rare workflow details into referenced companion material
|
||||
- keep wording changes conservative
|
||||
|
||||
### Phase 3. Validate Against Real Scenarios
|
||||
|
||||
Run scenario checks for:
|
||||
|
||||
- normal assigned heartbeat
|
||||
- comment-triggered wake
|
||||
- blocked-task dedup behavior
|
||||
- approval-resolution wake
|
||||
- delegation/subtask creation
|
||||
- board handoff back to user
|
||||
- plan-request handling
|
||||
|
||||
### Phase 4. Decide Default Loading Strategy
|
||||
|
||||
After validation, decide whether:
|
||||
|
||||
- the entire main skill still loads by default, or
|
||||
- only the compact core loads by default and rare sections are fetched on demand
|
||||
|
||||
Do not change this loading policy without validation.
|
||||
|
||||
## Risks
|
||||
|
||||
- prompt degradation on control-plane safety rules
|
||||
- agents forgetting rare but important workflows
|
||||
- accidental removal of repeated wording that was carrying useful behavior
|
||||
- introducing ambiguous instruction precedence between the core skill and companion materials
|
||||
|
||||
## Preconditions Before Implementation
|
||||
|
||||
- define acceptance scenarios for control-plane correctness
|
||||
- add at least lightweight eval or scripted scenario coverage for key Paperclip flows
|
||||
- confirm how adapter/bootstrap layering should load skill content versus references
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- materially lower first-run input tokens for Paperclip-coordinated agents
|
||||
- no regression in checkout discipline, issue updates, blocked handling, or delegation
|
||||
- no increase in malformed API usage or ownership mistakes
|
||||
- agents still complete rare workflows correctly when explicitly asked
|
||||
@@ -30,6 +30,8 @@ Codex uses `previous_response_id` for session continuity. The adapter serializes
|
||||
|
||||
The adapter symlinks Paperclip skills into the global Codex skills directory (`~/.codex/skills`). Existing user skills are not overwritten.
|
||||
|
||||
When Paperclip is running inside a managed worktree instance (`PAPERCLIP_IN_WORKTREE=true`), the adapter instead uses a worktree-isolated `CODEX_HOME` under the Paperclip instance so Codex skills, sessions, logs, and other runtime state do not leak across checkouts. It seeds that isolated home from the user's main Codex home for shared auth/config continuity.
|
||||
|
||||
For manual local CLI usage outside heartbeat runs (for example running as `codexcoder` directly), use:
|
||||
|
||||
```sh
|
||||
|
||||
@@ -6,7 +6,7 @@ summary: Guide to building a custom adapter
|
||||
Build a custom adapter to connect Paperclip to any agent runtime.
|
||||
|
||||
<Tip>
|
||||
If you're using Claude Code, the `create-agent-adapter` skill can guide you through the full adapter creation process interactively. Just ask Claude to create a new adapter and it will walk you through each step.
|
||||
If you're using Claude Code, the `.agents/skills/create-agent-adapter` skill can guide you through the full adapter creation process interactively. Just ask Claude to create a new adapter and it will walk you through each step.
|
||||
</Tip>
|
||||
|
||||
## Package Structure
|
||||
|
||||
@@ -112,6 +112,16 @@ export function renderTemplate(template: string, data: Record<string, unknown>)
|
||||
return template.replace(/{{\s*([a-zA-Z0-9_.-]+)\s*}}/g, (_, path) => resolvePathValue(data, path));
|
||||
}
|
||||
|
||||
export function joinPromptSections(
|
||||
sections: Array<string | null | undefined>,
|
||||
separator = "\n\n",
|
||||
) {
|
||||
return sections
|
||||
.map((value) => (typeof value === "string" ? value.trim() : ""))
|
||||
.filter(Boolean)
|
||||
.join(separator);
|
||||
}
|
||||
|
||||
export function redactEnvForLogs(env: Record<string, string>): Record<string, string> {
|
||||
const redacted: Record<string, string> = {};
|
||||
for (const [key, value] of Object.entries(env)) {
|
||||
|
||||
@@ -99,6 +99,7 @@ export interface AdapterInvocationMeta {
|
||||
commandNotes?: string[];
|
||||
env?: Record<string, string>;
|
||||
prompt?: string;
|
||||
promptMetrics?: Record<string, number>;
|
||||
context?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ import {
|
||||
parseObject,
|
||||
parseJson,
|
||||
buildPaperclipEnv,
|
||||
joinPromptSections,
|
||||
redactEnvForLogs,
|
||||
ensureAbsoluteDirectory,
|
||||
ensureCommandResolvable,
|
||||
@@ -363,7 +364,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
`[paperclip] Claude session "${runtimeSessionId}" was saved for cwd "${runtimeSessionCwd}" and will not be resumed in "${cwd}".\n`,
|
||||
);
|
||||
}
|
||||
const prompt = renderTemplate(promptTemplate, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -371,7 +373,24 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
};
|
||||
const renderedPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const prompt = joinPromptSections([
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
renderedPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
promptChars: prompt.length,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
heartbeatPromptChars: renderedPrompt.length,
|
||||
};
|
||||
|
||||
const buildClaudeArgs = (resumeSessionId: string | null) => {
|
||||
const args = ["--print", "-", "--output-format", "stream-json", "--verbose"];
|
||||
@@ -416,6 +435,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
commandNotes,
|
||||
env: redactEnvForLogs(env),
|
||||
prompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -67,6 +67,7 @@ export function buildClaudeLocalConfig(v: CreateConfigValues): Record<string, un
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
if (v.model) ac.model = v.model;
|
||||
if (v.thinkingEffort) ac.effort = v.thinkingEffort;
|
||||
if (v.chrome) ac.chrome = true;
|
||||
|
||||
101
packages/adapters/codex-local/src/server/codex-home.ts
Normal file
101
packages/adapters/codex-local/src/server/codex-home.ts
Normal file
@@ -0,0 +1,101 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { AdapterExecutionContext } from "@paperclipai/adapter-utils";
|
||||
|
||||
const TRUTHY_ENV_RE = /^(1|true|yes|on)$/i;
|
||||
const COPIED_SHARED_FILES = ["config.json", "config.toml", "instructions.md"] as const;
|
||||
const SYMLINKED_SHARED_FILES = ["auth.json"] as const;
|
||||
|
||||
function nonEmpty(value: string | undefined): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
|
||||
}
|
||||
|
||||
export async function pathExists(candidate: string): Promise<boolean> {
|
||||
return fs.access(candidate).then(() => true).catch(() => false);
|
||||
}
|
||||
|
||||
export function resolveCodexHomeDir(env: NodeJS.ProcessEnv = process.env): string {
|
||||
const fromEnv = nonEmpty(env.CODEX_HOME);
|
||||
if (fromEnv) return path.resolve(fromEnv);
|
||||
return path.join(os.homedir(), ".codex");
|
||||
}
|
||||
|
||||
function isWorktreeMode(env: NodeJS.ProcessEnv): boolean {
|
||||
return TRUTHY_ENV_RE.test(env.PAPERCLIP_IN_WORKTREE ?? "");
|
||||
}
|
||||
|
||||
function resolveWorktreeCodexHomeDir(env: NodeJS.ProcessEnv): string | null {
|
||||
if (!isWorktreeMode(env)) return null;
|
||||
const paperclipHome = nonEmpty(env.PAPERCLIP_HOME);
|
||||
if (!paperclipHome) return null;
|
||||
const instanceId = nonEmpty(env.PAPERCLIP_INSTANCE_ID);
|
||||
if (instanceId) {
|
||||
return path.resolve(paperclipHome, "instances", instanceId, "codex-home");
|
||||
}
|
||||
return path.resolve(paperclipHome, "codex-home");
|
||||
}
|
||||
|
||||
async function ensureParentDir(target: string): Promise<void> {
|
||||
await fs.mkdir(path.dirname(target), { recursive: true });
|
||||
}
|
||||
|
||||
async function ensureSymlink(target: string, source: string): Promise<void> {
|
||||
const existing = await fs.lstat(target).catch(() => null);
|
||||
if (!existing) {
|
||||
await ensureParentDir(target);
|
||||
await fs.symlink(source, target);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!existing.isSymbolicLink()) {
|
||||
return;
|
||||
}
|
||||
|
||||
const linkedPath = await fs.readlink(target).catch(() => null);
|
||||
if (!linkedPath) return;
|
||||
|
||||
const resolvedLinkedPath = path.resolve(path.dirname(target), linkedPath);
|
||||
if (resolvedLinkedPath === source) return;
|
||||
|
||||
await fs.unlink(target);
|
||||
await fs.symlink(source, target);
|
||||
}
|
||||
|
||||
async function ensureCopiedFile(target: string, source: string): Promise<void> {
|
||||
const existing = await fs.lstat(target).catch(() => null);
|
||||
if (existing) return;
|
||||
await ensureParentDir(target);
|
||||
await fs.copyFile(source, target);
|
||||
}
|
||||
|
||||
export async function prepareWorktreeCodexHome(
|
||||
env: NodeJS.ProcessEnv,
|
||||
onLog: AdapterExecutionContext["onLog"],
|
||||
): Promise<string | null> {
|
||||
const targetHome = resolveWorktreeCodexHomeDir(env);
|
||||
if (!targetHome) return null;
|
||||
|
||||
const sourceHome = resolveCodexHomeDir(env);
|
||||
if (path.resolve(sourceHome) === path.resolve(targetHome)) return targetHome;
|
||||
|
||||
await fs.mkdir(targetHome, { recursive: true });
|
||||
|
||||
for (const name of SYMLINKED_SHARED_FILES) {
|
||||
const source = path.join(sourceHome, name);
|
||||
if (!(await pathExists(source))) continue;
|
||||
await ensureSymlink(path.join(targetHome, name), source);
|
||||
}
|
||||
|
||||
for (const name of COPIED_SHARED_FILES) {
|
||||
const source = path.join(sourceHome, name);
|
||||
if (!(await pathExists(source))) continue;
|
||||
await ensureCopiedFile(path.join(targetHome, name), source);
|
||||
}
|
||||
|
||||
await onLog(
|
||||
"stderr",
|
||||
`[paperclip] Using worktree-isolated Codex home "${targetHome}" (seeded from "${sourceHome}").\n`,
|
||||
);
|
||||
return targetHome;
|
||||
}
|
||||
@@ -1,5 +1,4 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import type { AdapterExecutionContext, AdapterExecutionResult } from "@paperclipai/adapter-utils";
|
||||
@@ -18,9 +17,11 @@ import {
|
||||
listPaperclipSkillEntries,
|
||||
removeMaintainerOnlySkillSymlinks,
|
||||
renderTemplate,
|
||||
joinPromptSections,
|
||||
runChildProcess,
|
||||
} from "@paperclipai/adapter-utils/server-utils";
|
||||
import { parseCodexJsonl, isCodexUnknownSessionError } from "./parse.js";
|
||||
import { pathExists, prepareWorktreeCodexHome, resolveCodexHomeDir } from "./codex-home.js";
|
||||
|
||||
const __moduleDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const CODEX_ROLLOUT_NOISE_RE =
|
||||
@@ -60,10 +61,32 @@ function resolveCodexBillingType(env: Record<string, string>): "api" | "subscrip
|
||||
return hasNonEmptyEnvValue(env, "OPENAI_API_KEY") ? "api" : "subscription";
|
||||
}
|
||||
|
||||
function codexHomeDir(): string {
|
||||
const fromEnv = process.env.CODEX_HOME;
|
||||
if (typeof fromEnv === "string" && fromEnv.trim().length > 0) return fromEnv.trim();
|
||||
return path.join(os.homedir(), ".codex");
|
||||
async function isLikelyPaperclipRepoRoot(candidate: string): Promise<boolean> {
|
||||
const [hasWorkspace, hasPackageJson, hasServerDir, hasAdapterUtilsDir] = await Promise.all([
|
||||
pathExists(path.join(candidate, "pnpm-workspace.yaml")),
|
||||
pathExists(path.join(candidate, "package.json")),
|
||||
pathExists(path.join(candidate, "server")),
|
||||
pathExists(path.join(candidate, "packages", "adapter-utils")),
|
||||
]);
|
||||
|
||||
return hasWorkspace && hasPackageJson && hasServerDir && hasAdapterUtilsDir;
|
||||
}
|
||||
|
||||
async function isLikelyPaperclipRuntimeSkillSource(candidate: string, skillName: string): Promise<boolean> {
|
||||
if (path.basename(candidate) !== skillName) return false;
|
||||
const skillsRoot = path.dirname(candidate);
|
||||
if (path.basename(skillsRoot) !== "skills") return false;
|
||||
if (!(await pathExists(path.join(candidate, "SKILL.md")))) return false;
|
||||
|
||||
let cursor = path.dirname(skillsRoot);
|
||||
for (let depth = 0; depth < 6; depth += 1) {
|
||||
if (await isLikelyPaperclipRepoRoot(cursor)) return true;
|
||||
const parent = path.dirname(cursor);
|
||||
if (parent === cursor) break;
|
||||
cursor = parent;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
type EnsureCodexSkillsInjectedOptions = {
|
||||
@@ -79,7 +102,7 @@ export async function ensureCodexSkillsInjected(
|
||||
const skillsEntries = options.skillsEntries ?? await listPaperclipSkillEntries(__moduleDir);
|
||||
if (skillsEntries.length === 0) return;
|
||||
|
||||
const skillsHome = options.skillsHome ?? path.join(codexHomeDir(), "skills");
|
||||
const skillsHome = options.skillsHome ?? path.join(resolveCodexHomeDir(process.env), "skills");
|
||||
await fs.mkdir(skillsHome, { recursive: true });
|
||||
const removedSkills = await removeMaintainerOnlySkillSymlinks(
|
||||
skillsHome,
|
||||
@@ -96,6 +119,31 @@ export async function ensureCodexSkillsInjected(
|
||||
const target = path.join(skillsHome, entry.name);
|
||||
|
||||
try {
|
||||
const existing = await fs.lstat(target).catch(() => null);
|
||||
if (existing?.isSymbolicLink()) {
|
||||
const linkedPath = await fs.readlink(target).catch(() => null);
|
||||
const resolvedLinkedPath = linkedPath
|
||||
? path.resolve(path.dirname(target), linkedPath)
|
||||
: null;
|
||||
if (
|
||||
resolvedLinkedPath &&
|
||||
resolvedLinkedPath !== entry.source &&
|
||||
(await isLikelyPaperclipRuntimeSkillSource(resolvedLinkedPath, entry.name))
|
||||
) {
|
||||
await fs.unlink(target);
|
||||
if (linkSkill) {
|
||||
await linkSkill(entry.source, target);
|
||||
} else {
|
||||
await fs.symlink(entry.source, target);
|
||||
}
|
||||
await onLog(
|
||||
"stderr",
|
||||
`[paperclip] Repaired Codex skill "${entry.name}" into ${skillsHome}\n`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
const result = await ensurePaperclipSkillSymlink(entry.source, target, linkSkill);
|
||||
if (result === "skipped") continue;
|
||||
|
||||
@@ -160,12 +208,25 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
const useConfiguredInsteadOfAgentHome = workspaceSource === "agent_home" && configuredCwd.length > 0;
|
||||
const effectiveWorkspaceCwd = useConfiguredInsteadOfAgentHome ? "" : workspaceCwd;
|
||||
const cwd = effectiveWorkspaceCwd || configuredCwd || process.cwd();
|
||||
await ensureAbsoluteDirectory(cwd, { createIfMissing: true });
|
||||
await ensureCodexSkillsInjected(onLog);
|
||||
const envConfig = parseObject(config.env);
|
||||
const configuredCodexHome =
|
||||
typeof envConfig.CODEX_HOME === "string" && envConfig.CODEX_HOME.trim().length > 0
|
||||
? path.resolve(envConfig.CODEX_HOME.trim())
|
||||
: null;
|
||||
await ensureAbsoluteDirectory(cwd, { createIfMissing: true });
|
||||
const preparedWorktreeCodexHome =
|
||||
configuredCodexHome ? null : await prepareWorktreeCodexHome(process.env, onLog);
|
||||
const effectiveCodexHome = configuredCodexHome ?? preparedWorktreeCodexHome;
|
||||
await ensureCodexSkillsInjected(
|
||||
onLog,
|
||||
effectiveCodexHome ? { skillsHome: path.join(effectiveCodexHome, "skills") } : {},
|
||||
);
|
||||
const hasExplicitApiKey =
|
||||
typeof envConfig.PAPERCLIP_API_KEY === "string" && envConfig.PAPERCLIP_API_KEY.trim().length > 0;
|
||||
const env: Record<string, string> = { ...buildPaperclipEnv(agent) };
|
||||
if (effectiveCodexHome) {
|
||||
env.CODEX_HOME = effectiveCodexHome;
|
||||
}
|
||||
env.PAPERCLIP_RUN_ID = runId;
|
||||
const wakeTaskId =
|
||||
(typeof context.taskId === "string" && context.taskId.trim().length > 0 && context.taskId.trim()) ||
|
||||
@@ -278,6 +339,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
const instructionsFilePath = asString(config.instructionsFilePath, "").trim();
|
||||
const instructionsDir = instructionsFilePath ? `${path.dirname(instructionsFilePath)}/` : "";
|
||||
let instructionsPrefix = "";
|
||||
let instructionsChars = 0;
|
||||
if (instructionsFilePath) {
|
||||
try {
|
||||
const instructionsContents = await fs.readFile(instructionsFilePath, "utf8");
|
||||
@@ -285,6 +347,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
`${instructionsContents}\n\n` +
|
||||
`The above agent instructions were loaded from ${instructionsFilePath}. ` +
|
||||
`Resolve any relative file references from ${instructionsDir}.\n\n`;
|
||||
instructionsChars = instructionsPrefix.length;
|
||||
await onLog(
|
||||
"stderr",
|
||||
`[paperclip] Loaded agent instructions file: ${instructionsFilePath}\n`,
|
||||
@@ -309,7 +372,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
`Configured instructionsFilePath ${instructionsFilePath}, but file could not be read; continuing without injected instructions.`,
|
||||
];
|
||||
})();
|
||||
const renderedPrompt = renderTemplate(promptTemplate, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -317,8 +381,26 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
const prompt = `${instructionsPrefix}${renderedPrompt}`;
|
||||
};
|
||||
const renderedPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const prompt = joinPromptSections([
|
||||
instructionsPrefix,
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
renderedPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
promptChars: prompt.length,
|
||||
instructionsChars,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
heartbeatPromptChars: renderedPrompt.length,
|
||||
};
|
||||
|
||||
const buildArgs = (resumeSessionId: string | null) => {
|
||||
const args = ["exec", "--json"];
|
||||
@@ -346,6 +428,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
}),
|
||||
env: redactEnvForLogs(env),
|
||||
prompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -71,6 +71,7 @@ export function buildCodexLocalConfig(v: CreateConfigValues): Record<string, unk
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
ac.model = v.model || DEFAULT_CODEX_LOCAL_MODEL;
|
||||
if (v.thinkingEffort) ac.modelReasoningEffort = v.thinkingEffort;
|
||||
ac.timeoutSec = 0;
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
listPaperclipSkillEntries,
|
||||
removeMaintainerOnlySkillSymlinks,
|
||||
renderTemplate,
|
||||
joinPromptSections,
|
||||
runChildProcess,
|
||||
} from "@paperclipai/adapter-utils/server-utils";
|
||||
import { DEFAULT_CURSOR_LOCAL_MODEL } from "../index.js";
|
||||
@@ -268,6 +269,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
const instructionsFilePath = asString(config.instructionsFilePath, "").trim();
|
||||
const instructionsDir = instructionsFilePath ? `${path.dirname(instructionsFilePath)}/` : "";
|
||||
let instructionsPrefix = "";
|
||||
let instructionsChars = 0;
|
||||
if (instructionsFilePath) {
|
||||
try {
|
||||
const instructionsContents = await fs.readFile(instructionsFilePath, "utf8");
|
||||
@@ -275,6 +277,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
`${instructionsContents}\n\n` +
|
||||
`The above agent instructions were loaded from ${instructionsFilePath}. ` +
|
||||
`Resolve any relative file references from ${instructionsDir}.\n\n`;
|
||||
instructionsChars = instructionsPrefix.length;
|
||||
await onLog(
|
||||
"stderr",
|
||||
`[paperclip] Loaded agent instructions file: ${instructionsFilePath}\n`,
|
||||
@@ -307,7 +310,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
return notes;
|
||||
})();
|
||||
|
||||
const renderedPrompt = renderTemplate(promptTemplate, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -315,9 +319,29 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
};
|
||||
const renderedPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const paperclipEnvNote = renderPaperclipEnvNote(env);
|
||||
const prompt = `${instructionsPrefix}${paperclipEnvNote}${renderedPrompt}`;
|
||||
const prompt = joinPromptSections([
|
||||
instructionsPrefix,
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
paperclipEnvNote,
|
||||
renderedPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
promptChars: prompt.length,
|
||||
instructionsChars,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
runtimeNoteChars: paperclipEnvNote.length,
|
||||
heartbeatPromptChars: renderedPrompt.length,
|
||||
};
|
||||
|
||||
const buildArgs = (resumeSessionId: string | null) => {
|
||||
const args = ["-p", "--output-format", "stream-json", "--workspace", cwd];
|
||||
@@ -340,6 +364,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
commandArgs: args,
|
||||
env: redactEnvForLogs(env),
|
||||
prompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -62,6 +62,7 @@ export function buildCursorLocalConfig(v: CreateConfigValues): Record<string, un
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
ac.model = v.model || DEFAULT_CURSOR_LOCAL_MODEL;
|
||||
const mode = normalizeMode(v.thinkingEffort);
|
||||
if (mode) ac.mode = mode;
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
ensureAbsoluteDirectory,
|
||||
ensureCommandResolvable,
|
||||
ensurePaperclipSkillSymlink,
|
||||
joinPromptSections,
|
||||
ensurePathInEnv,
|
||||
listPaperclipSkillEntries,
|
||||
removeMaintainerOnlySkillSymlinks,
|
||||
@@ -268,7 +269,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
return notes;
|
||||
})();
|
||||
|
||||
const renderedPrompt = renderTemplate(promptTemplate, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -276,10 +278,31 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
};
|
||||
const renderedPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const paperclipEnvNote = renderPaperclipEnvNote(env);
|
||||
const apiAccessNote = renderApiAccessNote(env);
|
||||
const prompt = `${instructionsPrefix}${paperclipEnvNote}${apiAccessNote}${renderedPrompt}`;
|
||||
const prompt = joinPromptSections([
|
||||
instructionsPrefix,
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
paperclipEnvNote,
|
||||
apiAccessNote,
|
||||
renderedPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
promptChars: prompt.length,
|
||||
instructionsChars: instructionsPrefix.length,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
runtimeNoteChars: paperclipEnvNote.length + apiAccessNote.length,
|
||||
heartbeatPromptChars: renderedPrompt.length,
|
||||
};
|
||||
|
||||
const buildArgs = (resumeSessionId: string | null) => {
|
||||
const args = ["--output-format", "stream-json"];
|
||||
@@ -309,6 +332,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
)),
|
||||
env: redactEnvForLogs(env),
|
||||
prompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -56,6 +56,7 @@ export function buildGeminiLocalConfig(v: CreateConfigValues): Record<string, un
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
ac.model = v.model || DEFAULT_GEMINI_LOCAL_MODEL;
|
||||
ac.timeoutSec = 0;
|
||||
ac.graceSec = 15;
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
asStringArray,
|
||||
parseObject,
|
||||
buildPaperclipEnv,
|
||||
joinPromptSections,
|
||||
redactEnvForLogs,
|
||||
ensureAbsoluteDirectory,
|
||||
ensureCommandResolvable,
|
||||
@@ -233,7 +234,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
];
|
||||
})();
|
||||
|
||||
const renderedPrompt = renderTemplate(promptTemplate, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -241,8 +243,26 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
const prompt = `${instructionsPrefix}${renderedPrompt}`;
|
||||
};
|
||||
const renderedPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const prompt = joinPromptSections([
|
||||
instructionsPrefix,
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
renderedPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
promptChars: prompt.length,
|
||||
instructionsChars: instructionsPrefix.length,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
heartbeatPromptChars: renderedPrompt.length,
|
||||
};
|
||||
|
||||
const buildArgs = (resumeSessionId: string | null) => {
|
||||
const args = ["run", "--format", "json"];
|
||||
@@ -264,6 +284,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
commandArgs: [...args, `<stdin prompt ${prompt.length} chars>`],
|
||||
env: redactEnvForLogs(env),
|
||||
prompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -55,6 +55,7 @@ export function buildOpenCodeLocalConfig(v: CreateConfigValues): Record<string,
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
if (v.model) ac.model = v.model;
|
||||
if (v.thinkingEffort) ac.variant = v.thinkingEffort;
|
||||
// OpenCode sessions can run until the CLI exits naturally; keep timeout disabled (0)
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
asStringArray,
|
||||
parseObject,
|
||||
buildPaperclipEnv,
|
||||
joinPromptSections,
|
||||
redactEnvForLogs,
|
||||
ensureAbsoluteDirectory,
|
||||
ensureCommandResolvable,
|
||||
@@ -270,7 +271,8 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
systemPromptExtension = promptTemplate;
|
||||
}
|
||||
|
||||
const renderedSystemPromptExtension = renderTemplate(systemPromptExtension, {
|
||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||
const templateData = {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
@@ -278,18 +280,26 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
|
||||
// User prompt is simple - just the rendered prompt template without instructions
|
||||
const userPrompt = renderTemplate(promptTemplate, {
|
||||
agentId: agent.id,
|
||||
companyId: agent.companyId,
|
||||
runId,
|
||||
company: { id: agent.companyId },
|
||||
agent,
|
||||
run: { id: runId, source: "on_demand" },
|
||||
context,
|
||||
});
|
||||
};
|
||||
const renderedSystemPromptExtension = renderTemplate(systemPromptExtension, templateData);
|
||||
const renderedHeartbeatPrompt = renderTemplate(promptTemplate, templateData);
|
||||
const renderedBootstrapPrompt =
|
||||
!canResumeSession && bootstrapPromptTemplate.trim().length > 0
|
||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||
: "";
|
||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||
const userPrompt = joinPromptSections([
|
||||
renderedBootstrapPrompt,
|
||||
sessionHandoffNote,
|
||||
renderedHeartbeatPrompt,
|
||||
]);
|
||||
const promptMetrics = {
|
||||
systemPromptChars: renderedSystemPromptExtension.length,
|
||||
promptChars: userPrompt.length,
|
||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||
sessionHandoffChars: sessionHandoffNote.length,
|
||||
heartbeatPromptChars: renderedHeartbeatPrompt.length,
|
||||
};
|
||||
|
||||
const commandNotes = (() => {
|
||||
if (!resolvedInstructionsFilePath) return [] as string[];
|
||||
@@ -345,6 +355,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
||||
commandArgs: args,
|
||||
env: redactEnvForLogs(env),
|
||||
prompt: userPrompt,
|
||||
promptMetrics,
|
||||
context,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -48,6 +48,7 @@ export function buildPiLocalConfig(v: CreateConfigValues): Record<string, unknow
|
||||
if (v.cwd) ac.cwd = v.cwd;
|
||||
if (v.instructionsFilePath) ac.instructionsFilePath = v.instructionsFilePath;
|
||||
if (v.promptTemplate) ac.promptTemplate = v.promptTemplate;
|
||||
if (v.bootstrapPrompt) ac.bootstrapPromptTemplate = v.bootstrapPrompt;
|
||||
if (v.model) ac.model = v.model;
|
||||
if (v.thinkingEffort) ac.thinking = v.thinkingEffort;
|
||||
|
||||
|
||||
@@ -78,6 +78,10 @@ export const wakeAgentSchema = z.object({
|
||||
reason: z.string().optional().nullable(),
|
||||
payload: z.record(z.unknown()).optional().nullable(),
|
||||
idempotencyKey: z.string().optional().nullable(),
|
||||
forceFreshSession: z.preprocess(
|
||||
(value) => (value === null ? undefined : value),
|
||||
z.boolean().optional().default(false),
|
||||
),
|
||||
});
|
||||
|
||||
export type WakeAgent = z.infer<typeof wakeAgentSchema>;
|
||||
|
||||
48
report/2026-03-13-08-46-token-optimization-implementation.md
Normal file
48
report/2026-03-13-08-46-token-optimization-implementation.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Token Optimization Implementation Report
|
||||
|
||||
Implemented the token-optimization plan across heartbeat orchestration, issue context APIs, adapter prompt construction, skill exposure, and agent configuration UX.
|
||||
|
||||
The main behavior changes are:
|
||||
|
||||
- Heartbeat telemetry now normalizes sessioned local adapter usage as per-run deltas instead of blindly trusting cumulative session totals.
|
||||
- Timer and manual wakes now preserve task sessions by default; fresh sessions are forced only for explicit `forceFreshSession` wakes or new issue assignment wakes.
|
||||
- Heartbeat session rotation is now policy-driven in the control plane, with a handoff note injected when a session is compacted and restarted.
|
||||
- Paperclip issue context now has incremental APIs: `GET /api/agents/me/inbox-lite`, `GET /api/issues/:id/heartbeat-context`, and comment delta queries via `GET /api/issues/:id/comments?after=...&order=asc`.
|
||||
- The `paperclip` skill now teaches agents to use those compact/incremental APIs first, while keeping full-thread fetches as a cold-start fallback.
|
||||
- All local adapters now separate first-session bootstrap prompts from per-heartbeat prompt templates, and emit prompt size metrics in invocation metadata.
|
||||
- Adapter create flows now persist `bootstrapPromptTemplate` correctly.
|
||||
- The agent config UI now explains the difference between bootstrap prompts and heartbeat prompts and warns about prompt churn.
|
||||
- Runtime skill defaults now include `paperclip`, `para-memory-files`, and `paperclip-create-agent`. `create-agent-adapter` was moved to `.agents/skills/create-agent-adapter`.
|
||||
|
||||
Important follow-up finding from real-run review:
|
||||
|
||||
- `codex_local` currently injects Paperclip skills into the shared Codex skills home (`$CODEX_HOME/skills` or `~/.codex/skills`) rather than mounting a worktree-local skill directory.
|
||||
- If a Paperclip-owned skill symlink already points at another live checkout, the adapter currently skips it instead of repointing it.
|
||||
- In practice, this means a worktree can contain newer `skills/paperclip/SKILL.md` guidance while Codex still follows an older checkout's skill content.
|
||||
- That likely explains why PAP-507 still showed full issue/comment reload behavior even though the incremental context work was already implemented in this branch.
|
||||
- This should be treated as a separate follow-up item for `codex_local` skill isolation or symlink repair.
|
||||
|
||||
Files with the most important implementation work:
|
||||
|
||||
- `server/src/services/heartbeat.ts`
|
||||
- `server/src/services/issues.ts`
|
||||
- `server/src/routes/issues.ts`
|
||||
- `server/src/routes/agents.ts`
|
||||
- `server/src/routes/access.ts`
|
||||
- `skills/paperclip/SKILL.md`
|
||||
- `packages/adapters/*/src/server/execute.ts`
|
||||
- `packages/adapters/*/src/ui/build-config.ts`
|
||||
- `ui/src/components/AgentConfigForm.tsx`
|
||||
|
||||
Verification completed successfully:
|
||||
|
||||
- `pnpm -r typecheck`
|
||||
- `pnpm test:run`
|
||||
- `pnpm build`
|
||||
|
||||
While verifying, I also fixed two existing embedded-postgres typing mismatches so repo-wide `typecheck` and `build` pass again:
|
||||
|
||||
- `packages/db/src/migration-runtime.ts`
|
||||
- `cli/src/commands/worktree.ts`
|
||||
|
||||
Next useful follow-up is measuring the before/after effect in real runs now that telemetry is less misleading and prompt/session reuse behavior is consistent across adapters.
|
||||
208
server/src/__tests__/codex-local-execute.test.ts
Normal file
208
server/src/__tests__/codex-local-execute.test.ts
Normal file
@@ -0,0 +1,208 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { execute } from "@paperclipai/adapter-codex-local/server";
|
||||
|
||||
async function writeFakeCodexCommand(commandPath: string): Promise<void> {
|
||||
const script = `#!/usr/bin/env node
|
||||
const fs = require("node:fs");
|
||||
|
||||
const capturePath = process.env.PAPERCLIP_TEST_CAPTURE_PATH;
|
||||
const payload = {
|
||||
argv: process.argv.slice(2),
|
||||
prompt: fs.readFileSync(0, "utf8"),
|
||||
codexHome: process.env.CODEX_HOME || null,
|
||||
paperclipEnvKeys: Object.keys(process.env)
|
||||
.filter((key) => key.startsWith("PAPERCLIP_"))
|
||||
.sort(),
|
||||
};
|
||||
if (capturePath) {
|
||||
fs.writeFileSync(capturePath, JSON.stringify(payload), "utf8");
|
||||
}
|
||||
console.log(JSON.stringify({ type: "thread.started", thread_id: "codex-session-1" }));
|
||||
console.log(JSON.stringify({ type: "item.completed", item: { type: "agent_message", text: "hello" } }));
|
||||
console.log(JSON.stringify({ type: "turn.completed", usage: { input_tokens: 1, cached_input_tokens: 0, output_tokens: 1 } }));
|
||||
`;
|
||||
await fs.writeFile(commandPath, script, "utf8");
|
||||
await fs.chmod(commandPath, 0o755);
|
||||
}
|
||||
|
||||
type CapturePayload = {
|
||||
argv: string[];
|
||||
prompt: string;
|
||||
codexHome: string | null;
|
||||
paperclipEnvKeys: string[];
|
||||
};
|
||||
|
||||
describe("codex execute", () => {
|
||||
it("uses a worktree-isolated CODEX_HOME while preserving shared auth and config", async () => {
|
||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-execute-"));
|
||||
const workspace = path.join(root, "workspace");
|
||||
const commandPath = path.join(root, "codex");
|
||||
const capturePath = path.join(root, "capture.json");
|
||||
const sharedCodexHome = path.join(root, "shared-codex-home");
|
||||
const paperclipHome = path.join(root, "paperclip-home");
|
||||
const isolatedCodexHome = path.join(paperclipHome, "instances", "worktree-1", "codex-home");
|
||||
await fs.mkdir(workspace, { recursive: true });
|
||||
await fs.mkdir(sharedCodexHome, { recursive: true });
|
||||
await fs.writeFile(path.join(sharedCodexHome, "auth.json"), '{"token":"shared"}\n', "utf8");
|
||||
await fs.writeFile(path.join(sharedCodexHome, "config.toml"), 'model = "codex-mini-latest"\n', "utf8");
|
||||
await writeFakeCodexCommand(commandPath);
|
||||
|
||||
const previousHome = process.env.HOME;
|
||||
const previousPaperclipHome = process.env.PAPERCLIP_HOME;
|
||||
const previousPaperclipInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
|
||||
const previousPaperclipInWorktree = process.env.PAPERCLIP_IN_WORKTREE;
|
||||
const previousCodexHome = process.env.CODEX_HOME;
|
||||
process.env.HOME = root;
|
||||
process.env.PAPERCLIP_HOME = paperclipHome;
|
||||
process.env.PAPERCLIP_INSTANCE_ID = "worktree-1";
|
||||
process.env.PAPERCLIP_IN_WORKTREE = "true";
|
||||
process.env.CODEX_HOME = sharedCodexHome;
|
||||
|
||||
try {
|
||||
const result = await execute({
|
||||
runId: "run-1",
|
||||
agent: {
|
||||
id: "agent-1",
|
||||
companyId: "company-1",
|
||||
name: "Codex Coder",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
},
|
||||
runtime: {
|
||||
sessionId: null,
|
||||
sessionParams: null,
|
||||
sessionDisplayId: null,
|
||||
taskKey: null,
|
||||
},
|
||||
config: {
|
||||
command: commandPath,
|
||||
cwd: workspace,
|
||||
env: {
|
||||
PAPERCLIP_TEST_CAPTURE_PATH: capturePath,
|
||||
},
|
||||
promptTemplate: "Follow the paperclip heartbeat.",
|
||||
},
|
||||
context: {},
|
||||
authToken: "run-jwt-token",
|
||||
onLog: async () => {},
|
||||
});
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.errorMessage).toBeNull();
|
||||
|
||||
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as CapturePayload;
|
||||
expect(capture.codexHome).toBe(isolatedCodexHome);
|
||||
expect(capture.argv).toEqual(expect.arrayContaining(["exec", "--json", "-"]));
|
||||
expect(capture.prompt).toContain("Follow the paperclip heartbeat.");
|
||||
expect(capture.paperclipEnvKeys).toEqual(
|
||||
expect.arrayContaining([
|
||||
"PAPERCLIP_AGENT_ID",
|
||||
"PAPERCLIP_API_KEY",
|
||||
"PAPERCLIP_API_URL",
|
||||
"PAPERCLIP_COMPANY_ID",
|
||||
"PAPERCLIP_RUN_ID",
|
||||
]),
|
||||
);
|
||||
|
||||
const isolatedAuth = path.join(isolatedCodexHome, "auth.json");
|
||||
const isolatedConfig = path.join(isolatedCodexHome, "config.toml");
|
||||
const isolatedSkill = path.join(isolatedCodexHome, "skills", "paperclip");
|
||||
|
||||
expect((await fs.lstat(isolatedAuth)).isSymbolicLink()).toBe(true);
|
||||
expect(await fs.realpath(isolatedAuth)).toBe(await fs.realpath(path.join(sharedCodexHome, "auth.json")));
|
||||
expect((await fs.lstat(isolatedConfig)).isFile()).toBe(true);
|
||||
expect(await fs.readFile(isolatedConfig, "utf8")).toBe('model = "codex-mini-latest"\n');
|
||||
expect((await fs.lstat(isolatedSkill)).isSymbolicLink()).toBe(true);
|
||||
} finally {
|
||||
if (previousHome === undefined) delete process.env.HOME;
|
||||
else process.env.HOME = previousHome;
|
||||
if (previousPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME;
|
||||
else process.env.PAPERCLIP_HOME = previousPaperclipHome;
|
||||
if (previousPaperclipInstanceId === undefined) delete process.env.PAPERCLIP_INSTANCE_ID;
|
||||
else process.env.PAPERCLIP_INSTANCE_ID = previousPaperclipInstanceId;
|
||||
if (previousPaperclipInWorktree === undefined) delete process.env.PAPERCLIP_IN_WORKTREE;
|
||||
else process.env.PAPERCLIP_IN_WORKTREE = previousPaperclipInWorktree;
|
||||
if (previousCodexHome === undefined) delete process.env.CODEX_HOME;
|
||||
else process.env.CODEX_HOME = previousCodexHome;
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it("respects an explicit CODEX_HOME config override even in worktree mode", async () => {
|
||||
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-execute-explicit-"));
|
||||
const workspace = path.join(root, "workspace");
|
||||
const commandPath = path.join(root, "codex");
|
||||
const capturePath = path.join(root, "capture.json");
|
||||
const sharedCodexHome = path.join(root, "shared-codex-home");
|
||||
const explicitCodexHome = path.join(root, "explicit-codex-home");
|
||||
const paperclipHome = path.join(root, "paperclip-home");
|
||||
await fs.mkdir(workspace, { recursive: true });
|
||||
await fs.mkdir(sharedCodexHome, { recursive: true });
|
||||
await fs.writeFile(path.join(sharedCodexHome, "auth.json"), '{"token":"shared"}\n', "utf8");
|
||||
await writeFakeCodexCommand(commandPath);
|
||||
|
||||
const previousHome = process.env.HOME;
|
||||
const previousPaperclipHome = process.env.PAPERCLIP_HOME;
|
||||
const previousPaperclipInstanceId = process.env.PAPERCLIP_INSTANCE_ID;
|
||||
const previousPaperclipInWorktree = process.env.PAPERCLIP_IN_WORKTREE;
|
||||
const previousCodexHome = process.env.CODEX_HOME;
|
||||
process.env.HOME = root;
|
||||
process.env.PAPERCLIP_HOME = paperclipHome;
|
||||
process.env.PAPERCLIP_INSTANCE_ID = "worktree-1";
|
||||
process.env.PAPERCLIP_IN_WORKTREE = "true";
|
||||
process.env.CODEX_HOME = sharedCodexHome;
|
||||
|
||||
try {
|
||||
const result = await execute({
|
||||
runId: "run-2",
|
||||
agent: {
|
||||
id: "agent-1",
|
||||
companyId: "company-1",
|
||||
name: "Codex Coder",
|
||||
adapterType: "codex_local",
|
||||
adapterConfig: {},
|
||||
},
|
||||
runtime: {
|
||||
sessionId: null,
|
||||
sessionParams: null,
|
||||
sessionDisplayId: null,
|
||||
taskKey: null,
|
||||
},
|
||||
config: {
|
||||
command: commandPath,
|
||||
cwd: workspace,
|
||||
env: {
|
||||
PAPERCLIP_TEST_CAPTURE_PATH: capturePath,
|
||||
CODEX_HOME: explicitCodexHome,
|
||||
},
|
||||
promptTemplate: "Follow the paperclip heartbeat.",
|
||||
},
|
||||
context: {},
|
||||
authToken: "run-jwt-token",
|
||||
onLog: async () => {},
|
||||
});
|
||||
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.errorMessage).toBeNull();
|
||||
|
||||
const capture = JSON.parse(await fs.readFile(capturePath, "utf8")) as CapturePayload;
|
||||
expect(capture.codexHome).toBe(explicitCodexHome);
|
||||
await expect(fs.lstat(path.join(paperclipHome, "instances", "worktree-1", "codex-home"))).rejects.toThrow();
|
||||
} finally {
|
||||
if (previousHome === undefined) delete process.env.HOME;
|
||||
else process.env.HOME = previousHome;
|
||||
if (previousPaperclipHome === undefined) delete process.env.PAPERCLIP_HOME;
|
||||
else process.env.PAPERCLIP_HOME = previousPaperclipHome;
|
||||
if (previousPaperclipInstanceId === undefined) delete process.env.PAPERCLIP_INSTANCE_ID;
|
||||
else process.env.PAPERCLIP_INSTANCE_ID = previousPaperclipInstanceId;
|
||||
if (previousPaperclipInWorktree === undefined) delete process.env.PAPERCLIP_IN_WORKTREE;
|
||||
else process.env.PAPERCLIP_IN_WORKTREE = previousPaperclipInWorktree;
|
||||
if (previousCodexHome === undefined) delete process.env.CODEX_HOME;
|
||||
else process.env.CODEX_HOME = previousCodexHome;
|
||||
await fs.rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
91
server/src/__tests__/codex-local-skill-injection.test.ts
Normal file
91
server/src/__tests__/codex-local-skill-injection.test.ts
Normal file
@@ -0,0 +1,91 @@
|
||||
import fs from "node:fs/promises";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { ensureCodexSkillsInjected } from "@paperclipai/adapter-codex-local/server";
|
||||
|
||||
async function makeTempDir(prefix: string): Promise<string> {
|
||||
return fs.mkdtemp(path.join(os.tmpdir(), prefix));
|
||||
}
|
||||
|
||||
async function createPaperclipRepoSkill(root: string, skillName: string) {
|
||||
await fs.mkdir(path.join(root, "server"), { recursive: true });
|
||||
await fs.mkdir(path.join(root, "packages", "adapter-utils"), { recursive: true });
|
||||
await fs.mkdir(path.join(root, "skills", skillName), { recursive: true });
|
||||
await fs.writeFile(path.join(root, "pnpm-workspace.yaml"), "packages:\n - packages/*\n", "utf8");
|
||||
await fs.writeFile(path.join(root, "package.json"), '{"name":"paperclip"}\n', "utf8");
|
||||
await fs.writeFile(
|
||||
path.join(root, "skills", skillName, "SKILL.md"),
|
||||
`---\nname: ${skillName}\n---\n`,
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
async function createCustomSkill(root: string, skillName: string) {
|
||||
await fs.mkdir(path.join(root, "custom", skillName), { recursive: true });
|
||||
await fs.writeFile(
|
||||
path.join(root, "custom", skillName, "SKILL.md"),
|
||||
`---\nname: ${skillName}\n---\n`,
|
||||
"utf8",
|
||||
);
|
||||
}
|
||||
|
||||
describe("codex local adapter skill injection", () => {
|
||||
const cleanupDirs = new Set<string>();
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(Array.from(cleanupDirs).map((dir) => fs.rm(dir, { recursive: true, force: true })));
|
||||
cleanupDirs.clear();
|
||||
});
|
||||
|
||||
it("repairs a Codex Paperclip skill symlink that still points at another live checkout", async () => {
|
||||
const currentRepo = await makeTempDir("paperclip-codex-current-");
|
||||
const oldRepo = await makeTempDir("paperclip-codex-old-");
|
||||
const skillsHome = await makeTempDir("paperclip-codex-home-");
|
||||
cleanupDirs.add(currentRepo);
|
||||
cleanupDirs.add(oldRepo);
|
||||
cleanupDirs.add(skillsHome);
|
||||
|
||||
await createPaperclipRepoSkill(currentRepo, "paperclip");
|
||||
await createPaperclipRepoSkill(oldRepo, "paperclip");
|
||||
await fs.symlink(path.join(oldRepo, "skills", "paperclip"), path.join(skillsHome, "paperclip"));
|
||||
|
||||
const logs: string[] = [];
|
||||
await ensureCodexSkillsInjected(
|
||||
async (_stream, chunk) => {
|
||||
logs.push(chunk);
|
||||
},
|
||||
{
|
||||
skillsHome,
|
||||
skillsEntries: [{ name: "paperclip", source: path.join(currentRepo, "skills", "paperclip") }],
|
||||
},
|
||||
);
|
||||
|
||||
expect(await fs.realpath(path.join(skillsHome, "paperclip"))).toBe(
|
||||
await fs.realpath(path.join(currentRepo, "skills", "paperclip")),
|
||||
);
|
||||
expect(logs.some((line) => line.includes('Repaired Codex skill "paperclip"'))).toBe(true);
|
||||
});
|
||||
|
||||
it("preserves a custom Codex skill symlink outside Paperclip repo checkouts", async () => {
|
||||
const currentRepo = await makeTempDir("paperclip-codex-current-");
|
||||
const customRoot = await makeTempDir("paperclip-codex-custom-");
|
||||
const skillsHome = await makeTempDir("paperclip-codex-home-");
|
||||
cleanupDirs.add(currentRepo);
|
||||
cleanupDirs.add(customRoot);
|
||||
cleanupDirs.add(skillsHome);
|
||||
|
||||
await createPaperclipRepoSkill(currentRepo, "paperclip");
|
||||
await createCustomSkill(customRoot, "paperclip");
|
||||
await fs.symlink(path.join(customRoot, "custom", "paperclip"), path.join(skillsHome, "paperclip"));
|
||||
|
||||
await ensureCodexSkillsInjected(async () => {}, {
|
||||
skillsHome,
|
||||
skillsEntries: [{ name: "paperclip", source: path.join(currentRepo, "skills", "paperclip") }],
|
||||
});
|
||||
|
||||
expect(await fs.realpath(path.join(skillsHome, "paperclip"))).toBe(
|
||||
await fs.realpath(path.join(customRoot, "custom", "paperclip")),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -93,16 +93,26 @@ describe("shouldResetTaskSessionForWake", () => {
|
||||
expect(shouldResetTaskSessionForWake({ wakeReason: "issue_assigned" })).toBe(true);
|
||||
});
|
||||
|
||||
it("resets session context on timer heartbeats", () => {
|
||||
expect(shouldResetTaskSessionForWake({ wakeSource: "timer" })).toBe(true);
|
||||
it("preserves session context on timer heartbeats", () => {
|
||||
expect(shouldResetTaskSessionForWake({ wakeSource: "timer" })).toBe(false);
|
||||
});
|
||||
|
||||
it("resets session context on manual on-demand invokes", () => {
|
||||
it("preserves session context on manual on-demand invokes by default", () => {
|
||||
expect(
|
||||
shouldResetTaskSessionForWake({
|
||||
wakeSource: "on_demand",
|
||||
wakeTriggerDetail: "manual",
|
||||
}),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("resets session context when a fresh session is explicitly requested", () => {
|
||||
expect(
|
||||
shouldResetTaskSessionForWake({
|
||||
wakeSource: "on_demand",
|
||||
wakeTriggerDetail: "manual",
|
||||
forceFreshSession: true,
|
||||
}),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
|
||||
@@ -97,7 +97,11 @@ function requestBaseUrl(req: Request) {
|
||||
|
||||
function readSkillMarkdown(skillName: string): string | null {
|
||||
const normalized = skillName.trim().toLowerCase();
|
||||
if (normalized !== "paperclip" && normalized !== "paperclip-create-agent")
|
||||
if (
|
||||
normalized !== "paperclip" &&
|
||||
normalized !== "paperclip-create-agent" &&
|
||||
normalized !== "para-memory-files"
|
||||
)
|
||||
return null;
|
||||
const moduleDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
const candidates = [
|
||||
@@ -1610,6 +1614,10 @@ export function accessRoutes(
|
||||
res.json({
|
||||
skills: [
|
||||
{ name: "paperclip", path: "/api/skills/paperclip" },
|
||||
{
|
||||
name: "para-memory-files",
|
||||
path: "/api/skills/para-memory-files"
|
||||
},
|
||||
{
|
||||
name: "paperclip-create-agent",
|
||||
path: "/api/skills/paperclip-create-agent"
|
||||
|
||||
@@ -575,6 +575,34 @@ export function agentRoutes(db: Db) {
|
||||
res.json({ ...agent, chainOfCommand });
|
||||
});
|
||||
|
||||
router.get("/agents/me/inbox-lite", async (req, res) => {
|
||||
if (req.actor.type !== "agent" || !req.actor.agentId || !req.actor.companyId) {
|
||||
res.status(401).json({ error: "Agent authentication required" });
|
||||
return;
|
||||
}
|
||||
|
||||
const issuesSvc = issueService(db);
|
||||
const rows = await issuesSvc.list(req.actor.companyId, {
|
||||
assigneeAgentId: req.actor.agentId,
|
||||
status: "todo,in_progress,blocked",
|
||||
});
|
||||
|
||||
res.json(
|
||||
rows.map((issue) => ({
|
||||
id: issue.id,
|
||||
identifier: issue.identifier,
|
||||
title: issue.title,
|
||||
status: issue.status,
|
||||
priority: issue.priority,
|
||||
projectId: issue.projectId,
|
||||
goalId: issue.goalId,
|
||||
parentId: issue.parentId,
|
||||
updatedAt: issue.updatedAt,
|
||||
activeRun: issue.activeRun,
|
||||
})),
|
||||
);
|
||||
});
|
||||
|
||||
router.get("/agents/:id", async (req, res) => {
|
||||
const id = req.params.id as string;
|
||||
const agent = await svc.getById(id);
|
||||
@@ -1275,6 +1303,7 @@ export function agentRoutes(db: Db) {
|
||||
contextSnapshot: {
|
||||
triggeredBy: req.actor.type,
|
||||
actorId: req.actor.type === "agent" ? req.actor.agentId : req.actor.userId,
|
||||
forceFreshSession: req.body.forceFreshSession === true,
|
||||
},
|
||||
});
|
||||
|
||||
|
||||
@@ -28,6 +28,8 @@ import { assertCompanyAccess, getActorInfo } from "./authz.js";
|
||||
import { shouldWakeAssigneeOnCheckout } from "./issues-checkout-wakeup.js";
|
||||
import { isAllowedContentType, MAX_ATTACHMENT_BYTES } from "../attachment-types.js";
|
||||
|
||||
const MAX_ISSUE_COMMENT_LIMIT = 500;
|
||||
|
||||
export function issueRoutes(db: Db, storage: StorageService) {
|
||||
const router = Router();
|
||||
const svc = issueService(db);
|
||||
@@ -314,6 +316,79 @@ export function issueRoutes(db: Db, storage: StorageService) {
|
||||
});
|
||||
});
|
||||
|
||||
router.get("/issues/:id/heartbeat-context", async (req, res) => {
|
||||
const id = req.params.id as string;
|
||||
const issue = await svc.getById(id);
|
||||
if (!issue) {
|
||||
res.status(404).json({ error: "Issue not found" });
|
||||
return;
|
||||
}
|
||||
assertCompanyAccess(req, issue.companyId);
|
||||
|
||||
const wakeCommentId =
|
||||
typeof req.query.wakeCommentId === "string" && req.query.wakeCommentId.trim().length > 0
|
||||
? req.query.wakeCommentId.trim()
|
||||
: null;
|
||||
|
||||
const [ancestors, project, goal, commentCursor, wakeComment] = await Promise.all([
|
||||
svc.getAncestors(issue.id),
|
||||
issue.projectId ? projectsSvc.getById(issue.projectId) : null,
|
||||
issue.goalId
|
||||
? goalsSvc.getById(issue.goalId)
|
||||
: !issue.projectId
|
||||
? goalsSvc.getDefaultCompanyGoal(issue.companyId)
|
||||
: null,
|
||||
svc.getCommentCursor(issue.id),
|
||||
wakeCommentId ? svc.getComment(wakeCommentId) : null,
|
||||
]);
|
||||
|
||||
res.json({
|
||||
issue: {
|
||||
id: issue.id,
|
||||
identifier: issue.identifier,
|
||||
title: issue.title,
|
||||
description: issue.description,
|
||||
status: issue.status,
|
||||
priority: issue.priority,
|
||||
projectId: issue.projectId,
|
||||
goalId: goal?.id ?? issue.goalId,
|
||||
parentId: issue.parentId,
|
||||
assigneeAgentId: issue.assigneeAgentId,
|
||||
assigneeUserId: issue.assigneeUserId,
|
||||
updatedAt: issue.updatedAt,
|
||||
},
|
||||
ancestors: ancestors.map((ancestor) => ({
|
||||
id: ancestor.id,
|
||||
identifier: ancestor.identifier,
|
||||
title: ancestor.title,
|
||||
status: ancestor.status,
|
||||
priority: ancestor.priority,
|
||||
})),
|
||||
project: project
|
||||
? {
|
||||
id: project.id,
|
||||
name: project.name,
|
||||
status: project.status,
|
||||
targetDate: project.targetDate,
|
||||
}
|
||||
: null,
|
||||
goal: goal
|
||||
? {
|
||||
id: goal.id,
|
||||
title: goal.title,
|
||||
status: goal.status,
|
||||
level: goal.level,
|
||||
parentId: goal.parentId,
|
||||
}
|
||||
: null,
|
||||
commentCursor,
|
||||
wakeComment:
|
||||
wakeComment && wakeComment.issueId === issue.id
|
||||
? wakeComment
|
||||
: null,
|
||||
});
|
||||
});
|
||||
|
||||
router.post("/issues/:id/read", async (req, res) => {
|
||||
const id = req.params.id as string;
|
||||
const issue = await svc.getById(id);
|
||||
@@ -791,7 +866,29 @@ export function issueRoutes(db: Db, storage: StorageService) {
|
||||
return;
|
||||
}
|
||||
assertCompanyAccess(req, issue.companyId);
|
||||
const comments = await svc.listComments(id);
|
||||
const afterCommentId =
|
||||
typeof req.query.after === "string" && req.query.after.trim().length > 0
|
||||
? req.query.after.trim()
|
||||
: typeof req.query.afterCommentId === "string" && req.query.afterCommentId.trim().length > 0
|
||||
? req.query.afterCommentId.trim()
|
||||
: null;
|
||||
const order =
|
||||
typeof req.query.order === "string" && req.query.order.trim().toLowerCase() === "asc"
|
||||
? "asc"
|
||||
: "desc";
|
||||
const limitRaw =
|
||||
typeof req.query.limit === "string" && req.query.limit.trim().length > 0
|
||||
? Number(req.query.limit)
|
||||
: null;
|
||||
const limit =
|
||||
limitRaw && Number.isFinite(limitRaw) && limitRaw > 0
|
||||
? Math.min(Math.floor(limitRaw), MAX_ISSUE_COMMENT_LIMIT)
|
||||
: null;
|
||||
const comments = await svc.listComments(id, {
|
||||
afterCommentId,
|
||||
order,
|
||||
limit,
|
||||
});
|
||||
res.json(comments);
|
||||
});
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ import { logger } from "../middleware/logger.js";
|
||||
import { publishLiveEvent } from "./live-events.js";
|
||||
import { getRunLogStore, type RunLogHandle } from "./run-log-store.js";
|
||||
import { getServerAdapter, runningProcesses } from "../adapters/index.js";
|
||||
import type { AdapterExecutionResult, AdapterInvocationMeta, AdapterSessionCodec } from "../adapters/index.js";
|
||||
import type { AdapterExecutionResult, AdapterInvocationMeta, AdapterSessionCodec, UsageSummary } from "../adapters/index.js";
|
||||
import { createLocalAgentJwt } from "../agent-auth-jwt.js";
|
||||
import { parseObject, asBoolean, asNumber, appendWithCap, MAX_EXCERPT_BYTES } from "../adapters/utils.js";
|
||||
import { costService } from "./costs.js";
|
||||
@@ -47,6 +47,14 @@ const HEARTBEAT_MAX_CONCURRENT_RUNS_MAX = 10;
|
||||
const DEFERRED_WAKE_CONTEXT_KEY = "_paperclipWakeContext";
|
||||
const startLocksByAgent = new Map<string, Promise<void>>();
|
||||
const REPO_ONLY_CWD_SENTINEL = "/__paperclip_repo_only__";
|
||||
const SESSIONED_LOCAL_ADAPTERS = new Set([
|
||||
"claude_local",
|
||||
"codex_local",
|
||||
"cursor",
|
||||
"gemini_local",
|
||||
"opencode_local",
|
||||
"pi_local",
|
||||
]);
|
||||
|
||||
const heartbeatRunListColumns = {
|
||||
id: heartbeatRuns.id,
|
||||
@@ -117,6 +125,26 @@ interface WakeupOptions {
|
||||
contextSnapshot?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
type UsageTotals = {
|
||||
inputTokens: number;
|
||||
cachedInputTokens: number;
|
||||
outputTokens: number;
|
||||
};
|
||||
|
||||
type SessionCompactionPolicy = {
|
||||
enabled: boolean;
|
||||
maxSessionRuns: number;
|
||||
maxRawInputTokens: number;
|
||||
maxSessionAgeHours: number;
|
||||
};
|
||||
|
||||
type SessionCompactionDecision = {
|
||||
rotate: boolean;
|
||||
reason: string | null;
|
||||
handoffMarkdown: string | null;
|
||||
previousRunId: string | null;
|
||||
};
|
||||
|
||||
interface ParsedIssueAssigneeAdapterOverrides {
|
||||
adapterConfig: Record<string, unknown> | null;
|
||||
useProjectWorkspace: boolean | null;
|
||||
@@ -142,6 +170,88 @@ function readNonEmptyString(value: unknown): string | null {
|
||||
return typeof value === "string" && value.trim().length > 0 ? value : null;
|
||||
}
|
||||
|
||||
function normalizeUsageTotals(usage: UsageSummary | null | undefined): UsageTotals | null {
|
||||
if (!usage) return null;
|
||||
return {
|
||||
inputTokens: Math.max(0, Math.floor(asNumber(usage.inputTokens, 0))),
|
||||
cachedInputTokens: Math.max(0, Math.floor(asNumber(usage.cachedInputTokens, 0))),
|
||||
outputTokens: Math.max(0, Math.floor(asNumber(usage.outputTokens, 0))),
|
||||
};
|
||||
}
|
||||
|
||||
function readRawUsageTotals(usageJson: unknown): UsageTotals | null {
|
||||
const parsed = parseObject(usageJson);
|
||||
if (Object.keys(parsed).length === 0) return null;
|
||||
|
||||
const inputTokens = Math.max(
|
||||
0,
|
||||
Math.floor(asNumber(parsed.rawInputTokens, asNumber(parsed.inputTokens, 0))),
|
||||
);
|
||||
const cachedInputTokens = Math.max(
|
||||
0,
|
||||
Math.floor(asNumber(parsed.rawCachedInputTokens, asNumber(parsed.cachedInputTokens, 0))),
|
||||
);
|
||||
const outputTokens = Math.max(
|
||||
0,
|
||||
Math.floor(asNumber(parsed.rawOutputTokens, asNumber(parsed.outputTokens, 0))),
|
||||
);
|
||||
|
||||
if (inputTokens <= 0 && cachedInputTokens <= 0 && outputTokens <= 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
inputTokens,
|
||||
cachedInputTokens,
|
||||
outputTokens,
|
||||
};
|
||||
}
|
||||
|
||||
function deriveNormalizedUsageDelta(current: UsageTotals | null, previous: UsageTotals | null): UsageTotals | null {
|
||||
if (!current) return null;
|
||||
if (!previous) return { ...current };
|
||||
|
||||
const inputTokens = current.inputTokens >= previous.inputTokens
|
||||
? current.inputTokens - previous.inputTokens
|
||||
: current.inputTokens;
|
||||
const cachedInputTokens = current.cachedInputTokens >= previous.cachedInputTokens
|
||||
? current.cachedInputTokens - previous.cachedInputTokens
|
||||
: current.cachedInputTokens;
|
||||
const outputTokens = current.outputTokens >= previous.outputTokens
|
||||
? current.outputTokens - previous.outputTokens
|
||||
: current.outputTokens;
|
||||
|
||||
return {
|
||||
inputTokens: Math.max(0, inputTokens),
|
||||
cachedInputTokens: Math.max(0, cachedInputTokens),
|
||||
outputTokens: Math.max(0, outputTokens),
|
||||
};
|
||||
}
|
||||
|
||||
function formatCount(value: number | null | undefined) {
|
||||
if (typeof value !== "number" || !Number.isFinite(value)) return "0";
|
||||
return value.toLocaleString("en-US");
|
||||
}
|
||||
|
||||
function parseSessionCompactionPolicy(agent: typeof agents.$inferSelect): SessionCompactionPolicy {
|
||||
const runtimeConfig = parseObject(agent.runtimeConfig);
|
||||
const heartbeat = parseObject(runtimeConfig.heartbeat);
|
||||
const compaction = parseObject(
|
||||
heartbeat.sessionCompaction ?? heartbeat.sessionRotation ?? runtimeConfig.sessionCompaction,
|
||||
);
|
||||
const supportsSessions = SESSIONED_LOCAL_ADAPTERS.has(agent.adapterType);
|
||||
const enabled = compaction.enabled === undefined
|
||||
? supportsSessions
|
||||
: asBoolean(compaction.enabled, supportsSessions);
|
||||
|
||||
return {
|
||||
enabled,
|
||||
maxSessionRuns: Math.max(0, Math.floor(asNumber(compaction.maxSessionRuns, 200))),
|
||||
maxRawInputTokens: Math.max(0, Math.floor(asNumber(compaction.maxRawInputTokens, 2_000_000))),
|
||||
maxSessionAgeHours: Math.max(0, Math.floor(asNumber(compaction.maxSessionAgeHours, 72))),
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveRuntimeSessionParamsForWorkspace(input: {
|
||||
agentId: string;
|
||||
previousSessionParams: Record<string, unknown> | null;
|
||||
@@ -246,29 +356,20 @@ function deriveTaskKey(
|
||||
export function shouldResetTaskSessionForWake(
|
||||
contextSnapshot: Record<string, unknown> | null | undefined,
|
||||
) {
|
||||
if (contextSnapshot?.forceFreshSession === true) return true;
|
||||
|
||||
const wakeReason = readNonEmptyString(contextSnapshot?.wakeReason);
|
||||
if (wakeReason === "issue_assigned") return true;
|
||||
|
||||
const wakeSource = readNonEmptyString(contextSnapshot?.wakeSource);
|
||||
if (wakeSource === "timer") return true;
|
||||
|
||||
const wakeTriggerDetail = readNonEmptyString(contextSnapshot?.wakeTriggerDetail);
|
||||
return wakeSource === "on_demand" && wakeTriggerDetail === "manual";
|
||||
return false;
|
||||
}
|
||||
|
||||
function describeSessionResetReason(
|
||||
contextSnapshot: Record<string, unknown> | null | undefined,
|
||||
) {
|
||||
if (contextSnapshot?.forceFreshSession === true) return "forceFreshSession was requested";
|
||||
|
||||
const wakeReason = readNonEmptyString(contextSnapshot?.wakeReason);
|
||||
if (wakeReason === "issue_assigned") return "wake reason is issue_assigned";
|
||||
|
||||
const wakeSource = readNonEmptyString(contextSnapshot?.wakeSource);
|
||||
if (wakeSource === "timer") return "wake source is timer";
|
||||
|
||||
const wakeTriggerDetail = readNonEmptyString(contextSnapshot?.wakeTriggerDetail);
|
||||
if (wakeSource === "on_demand" && wakeTriggerDetail === "manual") {
|
||||
return "this is a manual invoke";
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -501,6 +602,176 @@ export function heartbeatService(db: Db) {
|
||||
.then((rows) => rows[0] ?? null);
|
||||
}
|
||||
|
||||
async function getLatestRunForSession(
|
||||
agentId: string,
|
||||
sessionId: string,
|
||||
opts?: { excludeRunId?: string | null },
|
||||
) {
|
||||
const conditions = [
|
||||
eq(heartbeatRuns.agentId, agentId),
|
||||
eq(heartbeatRuns.sessionIdAfter, sessionId),
|
||||
];
|
||||
if (opts?.excludeRunId) {
|
||||
conditions.push(sql`${heartbeatRuns.id} <> ${opts.excludeRunId}`);
|
||||
}
|
||||
return db
|
||||
.select()
|
||||
.from(heartbeatRuns)
|
||||
.where(and(...conditions))
|
||||
.orderBy(desc(heartbeatRuns.createdAt))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
}
|
||||
|
||||
async function getOldestRunForSession(agentId: string, sessionId: string) {
|
||||
return db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
createdAt: heartbeatRuns.createdAt,
|
||||
})
|
||||
.from(heartbeatRuns)
|
||||
.where(and(eq(heartbeatRuns.agentId, agentId), eq(heartbeatRuns.sessionIdAfter, sessionId)))
|
||||
.orderBy(asc(heartbeatRuns.createdAt), asc(heartbeatRuns.id))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null);
|
||||
}
|
||||
|
||||
async function resolveNormalizedUsageForSession(input: {
|
||||
agentId: string;
|
||||
runId: string;
|
||||
sessionId: string | null;
|
||||
rawUsage: UsageTotals | null;
|
||||
}) {
|
||||
const { agentId, runId, sessionId, rawUsage } = input;
|
||||
if (!sessionId || !rawUsage) {
|
||||
return {
|
||||
normalizedUsage: rawUsage,
|
||||
previousRawUsage: null as UsageTotals | null,
|
||||
derivedFromSessionTotals: false,
|
||||
};
|
||||
}
|
||||
|
||||
const previousRun = await getLatestRunForSession(agentId, sessionId, { excludeRunId: runId });
|
||||
const previousRawUsage = readRawUsageTotals(previousRun?.usageJson);
|
||||
return {
|
||||
normalizedUsage: deriveNormalizedUsageDelta(rawUsage, previousRawUsage),
|
||||
previousRawUsage,
|
||||
derivedFromSessionTotals: previousRawUsage !== null,
|
||||
};
|
||||
}
|
||||
|
||||
async function evaluateSessionCompaction(input: {
|
||||
agent: typeof agents.$inferSelect;
|
||||
sessionId: string | null;
|
||||
issueId: string | null;
|
||||
}): Promise<SessionCompactionDecision> {
|
||||
const { agent, sessionId, issueId } = input;
|
||||
if (!sessionId) {
|
||||
return {
|
||||
rotate: false,
|
||||
reason: null,
|
||||
handoffMarkdown: null,
|
||||
previousRunId: null,
|
||||
};
|
||||
}
|
||||
|
||||
const policy = parseSessionCompactionPolicy(agent);
|
||||
if (!policy.enabled) {
|
||||
return {
|
||||
rotate: false,
|
||||
reason: null,
|
||||
handoffMarkdown: null,
|
||||
previousRunId: null,
|
||||
};
|
||||
}
|
||||
|
||||
const fetchLimit = Math.max(policy.maxSessionRuns > 0 ? policy.maxSessionRuns + 1 : 0, 4);
|
||||
const runs = await db
|
||||
.select({
|
||||
id: heartbeatRuns.id,
|
||||
createdAt: heartbeatRuns.createdAt,
|
||||
usageJson: heartbeatRuns.usageJson,
|
||||
resultJson: heartbeatRuns.resultJson,
|
||||
error: heartbeatRuns.error,
|
||||
})
|
||||
.from(heartbeatRuns)
|
||||
.where(and(eq(heartbeatRuns.agentId, agent.id), eq(heartbeatRuns.sessionIdAfter, sessionId)))
|
||||
.orderBy(desc(heartbeatRuns.createdAt))
|
||||
.limit(fetchLimit);
|
||||
|
||||
if (runs.length === 0) {
|
||||
return {
|
||||
rotate: false,
|
||||
reason: null,
|
||||
handoffMarkdown: null,
|
||||
previousRunId: null,
|
||||
};
|
||||
}
|
||||
|
||||
const latestRun = runs[0] ?? null;
|
||||
const oldestRun =
|
||||
policy.maxSessionAgeHours > 0
|
||||
? await getOldestRunForSession(agent.id, sessionId)
|
||||
: runs[runs.length - 1] ?? latestRun;
|
||||
const latestRawUsage = readRawUsageTotals(latestRun?.usageJson);
|
||||
const sessionAgeHours =
|
||||
latestRun && oldestRun
|
||||
? Math.max(
|
||||
0,
|
||||
(new Date(latestRun.createdAt).getTime() - new Date(oldestRun.createdAt).getTime()) / (1000 * 60 * 60),
|
||||
)
|
||||
: 0;
|
||||
|
||||
let reason: string | null = null;
|
||||
if (policy.maxSessionRuns > 0 && runs.length > policy.maxSessionRuns) {
|
||||
reason = `session exceeded ${policy.maxSessionRuns} runs`;
|
||||
} else if (
|
||||
policy.maxRawInputTokens > 0 &&
|
||||
latestRawUsage &&
|
||||
latestRawUsage.inputTokens >= policy.maxRawInputTokens
|
||||
) {
|
||||
reason =
|
||||
`session raw input reached ${formatCount(latestRawUsage.inputTokens)} tokens ` +
|
||||
`(threshold ${formatCount(policy.maxRawInputTokens)})`;
|
||||
} else if (policy.maxSessionAgeHours > 0 && sessionAgeHours >= policy.maxSessionAgeHours) {
|
||||
reason = `session age reached ${Math.floor(sessionAgeHours)} hours`;
|
||||
}
|
||||
|
||||
if (!reason || !latestRun) {
|
||||
return {
|
||||
rotate: false,
|
||||
reason: null,
|
||||
handoffMarkdown: null,
|
||||
previousRunId: latestRun?.id ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
const latestSummary = summarizeHeartbeatRunResultJson(latestRun.resultJson);
|
||||
const latestTextSummary =
|
||||
readNonEmptyString(latestSummary?.summary) ??
|
||||
readNonEmptyString(latestSummary?.result) ??
|
||||
readNonEmptyString(latestSummary?.message) ??
|
||||
readNonEmptyString(latestRun.error);
|
||||
|
||||
const handoffMarkdown = [
|
||||
"Paperclip session handoff:",
|
||||
`- Previous session: ${sessionId}`,
|
||||
issueId ? `- Issue: ${issueId}` : "",
|
||||
`- Rotation reason: ${reason}`,
|
||||
latestTextSummary ? `- Last run summary: ${latestTextSummary}` : "",
|
||||
"Continue from the current task state. Rebuild only the minimum context you need.",
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join("\n");
|
||||
|
||||
return {
|
||||
rotate: true,
|
||||
reason,
|
||||
handoffMarkdown,
|
||||
previousRunId: latestRun.id,
|
||||
};
|
||||
}
|
||||
|
||||
async function resolveSessionBeforeForWakeup(
|
||||
agent: typeof agents.$inferSelect,
|
||||
taskKey: string | null,
|
||||
@@ -1016,9 +1287,10 @@ export function heartbeatService(db: Db) {
|
||||
run: typeof heartbeatRuns.$inferSelect,
|
||||
result: AdapterExecutionResult,
|
||||
session: { legacySessionId: string | null },
|
||||
normalizedUsage?: UsageTotals | null,
|
||||
) {
|
||||
await ensureRuntimeState(agent);
|
||||
const usage = result.usage;
|
||||
const usage = normalizedUsage ?? normalizeUsageTotals(result.usage);
|
||||
const inputTokens = usage?.inputTokens ?? 0;
|
||||
const outputTokens = usage?.outputTokens ?? 0;
|
||||
const cachedInputTokens = usage?.cachedInputTokens ?? 0;
|
||||
@@ -1270,15 +1542,42 @@ export function heartbeatService(db: Db) {
|
||||
context.projectId = executionWorkspace.projectId;
|
||||
}
|
||||
const runtimeSessionFallback = taskKey || resetTaskSession ? null : runtime.sessionId;
|
||||
const previousSessionDisplayId = truncateDisplayId(
|
||||
let previousSessionDisplayId = truncateDisplayId(
|
||||
taskSessionForRun?.sessionDisplayId ??
|
||||
(sessionCodec.getDisplayId ? sessionCodec.getDisplayId(runtimeSessionParams) : null) ??
|
||||
readNonEmptyString(runtimeSessionParams?.sessionId) ??
|
||||
runtimeSessionFallback,
|
||||
);
|
||||
let runtimeSessionIdForAdapter =
|
||||
readNonEmptyString(runtimeSessionParams?.sessionId) ?? runtimeSessionFallback;
|
||||
let runtimeSessionParamsForAdapter = runtimeSessionParams;
|
||||
|
||||
const sessionCompaction = await evaluateSessionCompaction({
|
||||
agent,
|
||||
sessionId: previousSessionDisplayId ?? runtimeSessionIdForAdapter,
|
||||
issueId,
|
||||
});
|
||||
if (sessionCompaction.rotate) {
|
||||
context.paperclipSessionHandoffMarkdown = sessionCompaction.handoffMarkdown;
|
||||
context.paperclipSessionRotationReason = sessionCompaction.reason;
|
||||
context.paperclipPreviousSessionId = previousSessionDisplayId ?? runtimeSessionIdForAdapter;
|
||||
runtimeSessionIdForAdapter = null;
|
||||
runtimeSessionParamsForAdapter = null;
|
||||
previousSessionDisplayId = null;
|
||||
if (sessionCompaction.reason) {
|
||||
runtimeWorkspaceWarnings.push(
|
||||
`Starting a fresh session because ${sessionCompaction.reason}.`,
|
||||
);
|
||||
}
|
||||
} else {
|
||||
delete context.paperclipSessionHandoffMarkdown;
|
||||
delete context.paperclipSessionRotationReason;
|
||||
delete context.paperclipPreviousSessionId;
|
||||
}
|
||||
|
||||
const runtimeForAdapter = {
|
||||
sessionId: readNonEmptyString(runtimeSessionParams?.sessionId) ?? runtimeSessionFallback,
|
||||
sessionParams: runtimeSessionParams,
|
||||
sessionId: runtimeSessionIdForAdapter,
|
||||
sessionParams: runtimeSessionParamsForAdapter,
|
||||
sessionDisplayId: previousSessionDisplayId,
|
||||
taskKey,
|
||||
};
|
||||
@@ -1522,6 +1821,14 @@ export function heartbeatService(db: Db) {
|
||||
previousDisplayId: runtimeForAdapter.sessionDisplayId,
|
||||
previousLegacySessionId: runtimeForAdapter.sessionId,
|
||||
});
|
||||
const rawUsage = normalizeUsageTotals(adapterResult.usage);
|
||||
const sessionUsageResolution = await resolveNormalizedUsageForSession({
|
||||
agentId: agent.id,
|
||||
runId: run.id,
|
||||
sessionId: nextSessionState.displayId ?? nextSessionState.legacySessionId,
|
||||
rawUsage,
|
||||
});
|
||||
const normalizedUsage = sessionUsageResolution.normalizedUsage;
|
||||
|
||||
let outcome: "succeeded" | "failed" | "cancelled" | "timed_out";
|
||||
const latestRun = await getRun(run.id);
|
||||
@@ -1550,9 +1857,23 @@ export function heartbeatService(db: Db) {
|
||||
: "failed";
|
||||
|
||||
const usageJson =
|
||||
adapterResult.usage || adapterResult.costUsd != null
|
||||
normalizedUsage || adapterResult.costUsd != null
|
||||
? ({
|
||||
...(adapterResult.usage ?? {}),
|
||||
...(normalizedUsage ?? {}),
|
||||
...(rawUsage ? {
|
||||
rawInputTokens: rawUsage.inputTokens,
|
||||
rawCachedInputTokens: rawUsage.cachedInputTokens,
|
||||
rawOutputTokens: rawUsage.outputTokens,
|
||||
} : {}),
|
||||
...(sessionUsageResolution.derivedFromSessionTotals ? { usageSource: "session_delta" } : {}),
|
||||
...((nextSessionState.displayId ?? nextSessionState.legacySessionId)
|
||||
? { persistedSessionId: nextSessionState.displayId ?? nextSessionState.legacySessionId }
|
||||
: {}),
|
||||
sessionReused: runtimeForAdapter.sessionId != null || runtimeForAdapter.sessionDisplayId != null,
|
||||
taskSessionReused: taskSessionForRun != null,
|
||||
freshSession: runtimeForAdapter.sessionId == null && runtimeForAdapter.sessionDisplayId == null,
|
||||
sessionRotated: sessionCompaction.rotate,
|
||||
sessionRotationReason: sessionCompaction.reason,
|
||||
...(adapterResult.costUsd != null ? { costUsd: adapterResult.costUsd } : {}),
|
||||
...(adapterResult.billingType ? { billingType: adapterResult.billingType } : {}),
|
||||
} as Record<string, unknown>)
|
||||
@@ -1609,7 +1930,7 @@ export function heartbeatService(db: Db) {
|
||||
if (finalizedRun) {
|
||||
await updateRuntimeState(agent, finalizedRun, adapterResult, {
|
||||
legacySessionId: nextSessionState.legacySessionId,
|
||||
});
|
||||
}, normalizedUsage);
|
||||
if (taskKey) {
|
||||
if (adapterResult.clearSession || (!nextSessionState.params && !nextSessionState.displayId)) {
|
||||
await clearTaskSessions(agent.companyId, agent.id, {
|
||||
|
||||
@@ -27,6 +27,7 @@ import { resolveIssueGoalId, resolveNextIssueGoalId } from "./issue-goal-fallbac
|
||||
import { getDefaultCompanyGoal } from "./goals.js";
|
||||
|
||||
const ALL_ISSUE_STATUSES = ["backlog", "todo", "in_progress", "in_review", "blocked", "done", "cancelled"];
|
||||
const MAX_ISSUE_COMMENT_PAGE_LIMIT = 500;
|
||||
|
||||
function assertTransition(from: string, to: string) {
|
||||
if (from === to) return;
|
||||
@@ -1060,13 +1061,86 @@ export function issueService(db: Db) {
|
||||
.returning()
|
||||
.then((rows) => rows[0] ?? null),
|
||||
|
||||
listComments: (issueId: string) =>
|
||||
db
|
||||
listComments: async (
|
||||
issueId: string,
|
||||
opts?: {
|
||||
afterCommentId?: string | null;
|
||||
order?: "asc" | "desc";
|
||||
limit?: number | null;
|
||||
},
|
||||
) => {
|
||||
const order = opts?.order === "asc" ? "asc" : "desc";
|
||||
const afterCommentId = opts?.afterCommentId?.trim() || null;
|
||||
const limit =
|
||||
opts?.limit && opts.limit > 0
|
||||
? Math.min(Math.floor(opts.limit), MAX_ISSUE_COMMENT_PAGE_LIMIT)
|
||||
: null;
|
||||
|
||||
const conditions = [eq(issueComments.issueId, issueId)];
|
||||
if (afterCommentId) {
|
||||
const anchor = await db
|
||||
.select({
|
||||
id: issueComments.id,
|
||||
createdAt: issueComments.createdAt,
|
||||
})
|
||||
.from(issueComments)
|
||||
.where(and(eq(issueComments.issueId, issueId), eq(issueComments.id, afterCommentId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
|
||||
if (!anchor) return [];
|
||||
conditions.push(
|
||||
order === "asc"
|
||||
? sql<boolean>`(
|
||||
${issueComments.createdAt} > ${anchor.createdAt}
|
||||
OR (${issueComments.createdAt} = ${anchor.createdAt} AND ${issueComments.id} > ${anchor.id})
|
||||
)`
|
||||
: sql<boolean>`(
|
||||
${issueComments.createdAt} < ${anchor.createdAt}
|
||||
OR (${issueComments.createdAt} = ${anchor.createdAt} AND ${issueComments.id} < ${anchor.id})
|
||||
)`,
|
||||
);
|
||||
}
|
||||
|
||||
const query = db
|
||||
.select()
|
||||
.from(issueComments)
|
||||
.where(and(...conditions))
|
||||
.orderBy(
|
||||
order === "asc" ? asc(issueComments.createdAt) : desc(issueComments.createdAt),
|
||||
order === "asc" ? asc(issueComments.id) : desc(issueComments.id),
|
||||
);
|
||||
|
||||
const comments = limit ? await query.limit(limit) : await query;
|
||||
return comments.map(redactIssueComment);
|
||||
},
|
||||
|
||||
getCommentCursor: async (issueId: string) => {
|
||||
const [latest, countRow] = await Promise.all([
|
||||
db
|
||||
.select({
|
||||
latestCommentId: issueComments.id,
|
||||
latestCommentAt: issueComments.createdAt,
|
||||
})
|
||||
.from(issueComments)
|
||||
.where(eq(issueComments.issueId, issueId))
|
||||
.orderBy(desc(issueComments.createdAt))
|
||||
.then((comments) => comments.map(redactIssueComment)),
|
||||
.orderBy(desc(issueComments.createdAt), desc(issueComments.id))
|
||||
.limit(1)
|
||||
.then((rows) => rows[0] ?? null),
|
||||
db
|
||||
.select({
|
||||
totalComments: sql<number>`count(*)::int`,
|
||||
})
|
||||
.from(issueComments)
|
||||
.where(eq(issueComments.issueId, issueId))
|
||||
.then((rows) => rows[0] ?? null),
|
||||
]);
|
||||
|
||||
return {
|
||||
totalComments: Number(countRow?.totalComments ?? 0),
|
||||
latestCommentId: latest?.latestCommentId ?? null,
|
||||
latestCommentAt: latest?.latestCommentAt ?? null,
|
||||
};
|
||||
},
|
||||
|
||||
getComment: (commentId: string) =>
|
||||
db
|
||||
|
||||
@@ -35,7 +35,7 @@ Follow these steps every time you wake up:
|
||||
- add a markdown comment explaining why it remains open and what happens next.
|
||||
Always include links to the approval and issue in that comment.
|
||||
|
||||
**Step 3 — Get assignments.** `GET /api/companies/{companyId}/issues?assigneeAgentId={your-agent-id}&status=todo,in_progress,blocked`. Results sorted by priority. This is your inbox.
|
||||
**Step 3 — Get assignments.** Prefer `GET /api/agents/me/inbox-lite` for the normal heartbeat inbox. It returns the compact assignment list you need for prioritization. Fall back to `GET /api/companies/{companyId}/issues?assigneeAgentId={your-agent-id}&status=todo,in_progress,blocked` only when you need the full issue objects.
|
||||
|
||||
**Step 4 — Pick work (with mention exception).** Work on `in_progress` first, then `todo`. Skip `blocked` unless you can unblock it.
|
||||
**Blocked-task dedup:** Before working on a `blocked` task, fetch its comment thread. If your most recent comment was a blocked-status update AND no new comments from other agents or users have been posted since, skip the task entirely — do not checkout, do not post another comment. Exit the heartbeat (or move to the next task) instead. Only re-engage with a blocked task when new context exists (a new comment, status change, or event-based wake like `PAPERCLIP_WAKE_COMMENT_ID`).
|
||||
@@ -56,8 +56,15 @@ Headers: Authorization: Bearer $PAPERCLIP_API_KEY, X-Paperclip-Run-Id: $PAPERCLI
|
||||
|
||||
If already checked out by you, returns normally. If owned by another agent: `409 Conflict` — stop, pick a different task. **Never retry a 409.**
|
||||
|
||||
**Step 6 — Understand context.** `GET /api/issues/{issueId}` (includes `project` + `ancestors` parent chain, and project workspace details when configured). `GET /api/issues/{issueId}/comments`. Read ancestors to understand _why_ this task exists.
|
||||
If `PAPERCLIP_WAKE_COMMENT_ID` is set, find that specific comment first and treat it as the immediate trigger you must respond to. Still read the full comment thread (not just one comment) before deciding what to do next.
|
||||
**Step 6 — Understand context.** Prefer `GET /api/issues/{issueId}/heartbeat-context` first. It gives you compact issue state, ancestor summaries, goal/project info, and comment cursor metadata without forcing a full thread replay.
|
||||
|
||||
Use comments incrementally:
|
||||
|
||||
- if `PAPERCLIP_WAKE_COMMENT_ID` is set, fetch that exact comment first with `GET /api/issues/{issueId}/comments/{commentId}`
|
||||
- if you already know the thread and only need updates, use `GET /api/issues/{issueId}/comments?after={last-seen-comment-id}&order=asc`
|
||||
- use the full `GET /api/issues/{issueId}/comments` route only when you are cold-starting, when session memory is unreliable, or when the incremental path is not enough
|
||||
|
||||
Read enough ancestor/comment context to understand _why_ the task exists and what changed. Do not reflexively reload the whole thread on every heartbeat.
|
||||
|
||||
**Step 7 — Do the work.** Use your tools and capabilities.
|
||||
|
||||
@@ -226,10 +233,13 @@ PATCH /api/agents/{agentId}/instructions-path
|
||||
| Action | Endpoint |
|
||||
| ------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| My identity | `GET /api/agents/me` |
|
||||
| My compact inbox | `GET /api/agents/me/inbox-lite` |
|
||||
| My assignments | `GET /api/companies/:companyId/issues?assigneeAgentId=:id&status=todo,in_progress,blocked` |
|
||||
| Checkout task | `POST /api/issues/:issueId/checkout` |
|
||||
| Get task + ancestors | `GET /api/issues/:issueId` |
|
||||
| Get compact heartbeat context | `GET /api/issues/:issueId/heartbeat-context` |
|
||||
| Get comments | `GET /api/issues/:issueId/comments` |
|
||||
| Get comment delta | `GET /api/issues/:issueId/comments?after=:commentId&order=asc` |
|
||||
| Get specific comment | `GET /api/issues/:issueId/comments/:commentId` |
|
||||
| Update task | `PATCH /api/issues/:issueId` (optional `comment` field) |
|
||||
| Add comment | `POST /api/issues/:issueId/comments` |
|
||||
|
||||
@@ -444,6 +444,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
|
||||
/>
|
||||
</Field>
|
||||
{isLocal && (
|
||||
<>
|
||||
<Field label="Prompt Template" hint={help.promptTemplate}>
|
||||
<MarkdownEditor
|
||||
value={eff(
|
||||
@@ -461,6 +462,10 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
|
||||
}}
|
||||
/>
|
||||
</Field>
|
||||
<div className="rounded-md border border-amber-500/25 bg-amber-500/10 px-3 py-2 text-xs text-amber-100">
|
||||
Prompt template is replayed on every heartbeat. Keep it compact and dynamic to avoid recurring token cost and cache churn.
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
@@ -576,6 +581,7 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
|
||||
|
||||
{/* Prompt template (create mode only — edit mode shows this in Identity) */}
|
||||
{isLocal && isCreate && (
|
||||
<>
|
||||
<Field label="Prompt Template" hint={help.promptTemplate}>
|
||||
<MarkdownEditor
|
||||
value={val!.promptTemplate}
|
||||
@@ -589,6 +595,10 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
|
||||
}}
|
||||
/>
|
||||
</Field>
|
||||
<div className="rounded-md border border-amber-500/25 bg-amber-500/10 px-3 py-2 text-xs text-amber-100">
|
||||
Prompt template is replayed on every heartbeat. Prefer small task framing and variables like <code>{"{{ context.* }}"}</code> or <code>{"{{ run.* }}"}</code>; avoid repeating stable instructions here.
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Adapter-specific fields */}
|
||||
@@ -704,6 +714,9 @@ export function AgentConfigForm(props: AgentConfigFormProps) {
|
||||
}}
|
||||
/>
|
||||
</Field>
|
||||
<div className="rounded-md border border-sky-500/25 bg-sky-500/10 px-3 py-2 text-xs text-sky-100">
|
||||
Bootstrap prompt is only sent for fresh sessions. Put stable setup, habits, and longer reusable guidance here. Frequent changes reduce the value of session reuse because new sessions must replay it.
|
||||
</div>
|
||||
{adapterType === "claude_local" && (
|
||||
<ClaudeLocalAdvancedFields {...adapterFieldProps} />
|
||||
)}
|
||||
|
||||
@@ -26,7 +26,7 @@ export const help: Record<string, string> = {
|
||||
capabilities: "Describes what this agent can do. Shown in the org chart and used for task routing.",
|
||||
adapterType: "How this agent runs: local CLI (Claude/Codex/OpenCode), OpenClaw Gateway, spawned process, or generic HTTP webhook.",
|
||||
cwd: "Default working directory fallback for local adapters. Use an absolute path on the machine running Paperclip.",
|
||||
promptTemplate: "The prompt sent to the agent on each heartbeat. Supports {{ agent.id }}, {{ agent.name }}, {{ agent.role }} variables.",
|
||||
promptTemplate: "Sent on every heartbeat. Keep this small and dynamic. Use it for current-task framing, not large static instructions. Supports {{ agent.id }}, {{ agent.name }}, {{ agent.role }} and other template variables.",
|
||||
model: "Override the default model used by the adapter.",
|
||||
thinkingEffort: "Control model reasoning depth. Supported values vary by adapter/model.",
|
||||
chrome: "Enable Claude's Chrome integration by passing --chrome.",
|
||||
@@ -44,7 +44,7 @@ export const help: Record<string, string> = {
|
||||
args: "Command-line arguments, comma-separated.",
|
||||
extraArgs: "Extra CLI arguments for local adapters, comma-separated.",
|
||||
envVars: "Environment variables injected into the adapter process. Use plain values or secret references.",
|
||||
bootstrapPrompt: "Optional prompt prepended on the first run to bootstrap the agent's environment or habits.",
|
||||
bootstrapPrompt: "Only sent when Paperclip starts a fresh session. Use this for stable setup guidance that should not be repeated on every heartbeat.",
|
||||
payloadTemplateJson: "Optional JSON merged into remote adapter request payloads before Paperclip adds its standard wake and workspace fields.",
|
||||
webhookUrl: "The URL that receives POST requests when the agent is invoked.",
|
||||
heartbeatInterval: "Run this agent automatically on a timer. Useful for periodic tasks like checking for new work.",
|
||||
|
||||
Reference in New Issue
Block a user