feat: implement provider-side prompt caching and workflow scaffolding
ci / check (macos-latest) (push) Canceled after 0s
ci / check (ubuntu-latest) (push) Canceled after 0s
ci / check (windows-latest) (push) Canceled after 0s

- Added support for caching the stable prefix of the system prompt for Anthropic models.
- Introduced a new option `cacheSystemPrefix` in SessionOptions to enable caching.
- Implemented a nudge escalation system that reminds users to update the TODO.md file, capped at three nudges.
- Created a new `scaffoldWorkflowFiles` function to generate TODO.md, ROADMAP.md, and docs/ directory when they are missing.
- Updated the App component to scaffold workflow files during initialization.
- Added hooks functionality to allow external scripts to modify tool input and manage approvals.
- Implemented tests for the new hooks functionality, ensuring proper approval and execution flow.
This commit is contained in:
asepharyana
2026-09-09 20:11:23 +07:00
parent 4aeb0d2455
commit 7a1e415fea
12 changed files with 884 additions and 32 deletions
+134
View File
@@ -0,0 +1,134 @@
# v9 — Provider prompt caching, registry trust, external hooks, /init scaffold, workflow plan nudge, nudge escalation, skill autoload
Date: 2026-09-09
Status: spec
## Why
User asked what was missing; recommended: provider-side prompt caching (#1),
registry trust (#3), external hooks (#4), `/init` TODO/ROADMAP scaffold (#5),
workflow->plan skill nudge (#7), nudge escalation (#6). Structured diff
review (#2) is **deferred**: the SDK's tool-approval model cannot express
per-hunk approval, and it needs a rewrite hook that does not exist yet. This
spec covers the six implementable items.
## Scope per item
### 1. Provider prompt caching (ROADMAP Next #1)
Client-side memoization already makes the system prompt byte-identical across
steps when nothing volatile changed. The remaining half: tell the provider to
cache the stable prefix.
- Anthropic: `cache_control: { type: 'ephemeral' }` on the system prompt
block when the model is Anthropic. SDK supports
`system: [{ type: 'text', text, cache_control }]`.
- OpenAI: automatic prefix caching — nothing to send; skip.
- Implement: `src/prompt.ts` splits the rendered system prompt into a stable
prefix (everything before the volatile notebook/memory/skills suffix) and
the volatile suffix; `session.ts` decides where to cut based on version
counters. When only volatile parts changed (notebook/memory/etc), keep the
stable prefix byte-identical and mark it cacheable; the volatile tail rides
the same request but does not invalidate the prefix cache.
- Provider routing: `Session` knows its model provider via `opts.modelId` /
a `provider` option. Add `cacheSystemPrefix?: boolean` option; enable when
provider is anthropic. SDK's own `cache_control` for Anthropic system
arrays: `system: [...]` accepts per-block cache_control.
- Tests: unit test that the splitter produces the same stable prefix across
differing notebook versions; an Anthropic-format prompt carries
`cache_control` on the stable block; OpenAI-format omits it.
### 3. Registry trust (ROADMAP Next)
An index is trusted for its contents, not its authorship. Add publisher
signature verification for registry entries:
- `src/registry.ts` manifest gains optional `signature` + `signer` fields.
- A pinned public key per publisher in config (`registry.publishers[<name>] =
ed25519 pubkey`). Use Node's `crypto.verify` with ed25519 (Node ≥ 16 has
`crypto.verify` for ed25519 via `createPublicKey`).
- On install: when a manifest has `signature`, verify against the configured
publisher key; mismatch → refusal naming the publisher and the expected key.
When the manifest has no signature → refused with "unsigned; add the
publisher key or install manually" unless `registry.allowUnsigned`.
- Legacy manifests (no signature field) remain installable only with
`allowUnsigned`; a signed one with an unknown publisher → refusal.
- Tests: signature verifies; tampered body fails; unsigned refused unless
allowUnsigned; unknown publisher refused.
### 4. External hooks (ROADMAP Later)
Pre-tool hooks that can rewrite tool input, with a trust story like Codex:
- `.shiro/hooks/` directory; each hook is an executable + a small manifest
(`name`, `hook` = `pre_tool` | `after_turn` | ...; `tools` = `["*"]` or
names).
- On load, hash each hook file (sha256); first run, hash is shown and the
user approves or denies (approval prompt). The hash is recorded in
`~/.shiro-neko/hooks.json` (name → hash → approved). A hook whose hash
changed since approval is refused until re-approved.
- `pre_tool`: stdin gets `{tool, input, cwd}`, stdout gets
`{"allow": true, "input": {...rewritten...}}` or `{"allow": false, "reason": ...}`
or `{"blocked": "..."}`. A throwing/odd exit is treated as a block.
- `after_turn`: stdin gets a turn summary; output ignored.
- These extend `PluginHost.guard` (beforeToolCall) — a hook runs after the
compiled plugins and before the permission check, so a hook cannot bypass
the safety rules but can rewrite or refuse.
- Cap: `hookTimeoutMs` default 5000; a hook that hangs is killed.
- Tests: manifest parse; pre_tool rewrites input (default allow); pre_tool
refusal blocks; unknown hook hash blocks until approved; after_turn runs;
timeout kills a hung hook; a hook that returns garbage blocks.
### 5. `/init` scaffold TODO/ROADMAP/docs
`/init` already writes AGENTS.md. Extend it to optionally scaffold the
project-workflow files too:
- `initPrompt` stays; after the agent writes AGENTS.md, if the repo lacks
TODO.md/ROADMAP.md/docs and the user wants the scaffold, write:
- `TODO.md`: `# TODO\n\nNow\n\nNext\n\nMaintenance\n\n`
- `ROADMAP.md`: `# Roadmap\n\n## Next\n\n`
- `docs/` empty dir (or a placeholder README)
- CLI flag `/init` gains no new args; the agent's prompt gains the option.
Headless: `--init-scaffold` flag writes the files directly (no model).
- Tests: scaffold produces the three files when missing; does not overwrite
existing ones.
### 6. Workflow nudge escalation
Today: one nudge per session, then silence. Change to a gentle ladder:
- Nudge 1: same as today.
- Nudge 2 (if a later turn also writes without todo_update): "still no update
to TODO.md; the project expects its task list kept current."
- Nudge 3: final, "last reminder; update TODO.md when you have a moment."
- Cap at 3 total (never infinite). `workflowNudged` → `workflowNudgeCount`.
- Tests: third nudge is the last; count capped.
### 7. Workflow → plan skill autoload
When the workflow policy renders (repo tracks progress), inject one extra line
so the model knows the `plan` skill exists and to load it before non-trivial
work:
```
- before non-trivial work, load the `plan` skill (spec-first) and follow it
```
- `workflowPolicy()` gains that line when enabled; test asserts it.
## Files touched
- src/prompt.ts — splitter + cache_control for Anthropic
- src/session.ts — provider routing, cacheSystemPrefix, nudge ladder, plan line
- src/registry.ts — signature verification
- src/hooks.ts — new: external hook loader/runner + hash trust
- src/plugins.ts — guard extension for hooks
- src/cli.tsx — --init-scaffold, hooks approval wiring
- src/config.ts — registry.publishers, registry.allowUnsigned, hooks.enabled
- src/init.ts or cli — scaffold writer
- docs/: hooks.md, registry.md, workflow.md, headless.md updates
- test/hooks.test.ts, registry-signature.test.ts, prompt-cache-split.test.ts,
init-scaffold.test.ts, nudge-ladder.test.ts
## Verification
- bun run typecheck
- bun test (new files + full suite)
- bun run build
- Manual: headless --init-scaffold, a hook that rewrites input, a signed registry entry
+23 -2
View File
@@ -8,6 +8,7 @@ import { configPath, loadConfig, missingKeyMessage, readConfigFile, resolveModel
import type { FallbackEvent } from './fallback';
import { farewell } from './farewell';
import { readStdin, runHeadless } from './headless';
import { scaffoldWorkflowFiles } from './scaffold';
import { INIT_PROMPT, loadInstructions } from './instructions';
import { walk } from './ignore';
import { connectMcp } from './mcp';
@@ -16,6 +17,7 @@ import { Memory, KIND_LABEL } from './memory';
import { costOf } from './pricing';
import { BUILTIN_PLUGINS, DEFAULT_ENABLED } from './plugins-builtin';
import { createHost } from './plugins';
import { hooksToPlugin, loadApprovalStore, loadHooks } from './hooks';
import { fetchModels, presetById } from './providers';
import * as registry from './registry';
import { Session } from './session';
@@ -53,6 +55,7 @@ options:
--no-skills ignore builtin and project skills
--no-plugins disable all plugins, including the guard
--no-memory do not load or write project memory
--init-scaffold with -p, write TODO.md/ROADMAP.md/docs/ when missing
--yolo skip all tool approval prompts
-v, --version
-h, --help
@@ -227,11 +230,18 @@ const pluginErrors = enabledPlugins
// .shiro/<kind>. All are data, never code; a bad file is reported, not fatal.
const externalPlugins = has('--no-plugins') ? { plugins: [], errors: [] } : await loadExternalPlugins(process.cwd());
// External hooks (executables that can rewrite or refuse tool calls) load from
// .shiro/hooks/ and ~/.shiro-neko/hooks/. They are code, so each one must be
// hash-approved once before it runs anything; a changed hash is refused until
// re-approved. They join the guard as one plugin, after the compiled ones.
const hooksPlugin = has('--no-plugins') ? [] : [hooksToPlugin(await loadHooks(process.cwd()), loadApprovalStore())];
const plugins = createHost(
[
...BUILTIN_PLUGINS.filter((p) => enabledPlugins.includes(p.name)),
...installedPlugins.plugins,
...externalPlugins.plugins,
...hooksPlugin,
],
[
...pluginErrors,
@@ -340,6 +350,10 @@ const session = new Session({
...(cfg.maxSpendUsd !== undefined ? { maxSpendUsd: cfg.maxSpendUsd } : {}),
...(cfg.maxSpendPerTurn !== undefined ? { maxSpendPerTurn: cfg.maxSpendPerTurn } : {}),
...(cfg.workflow !== undefined ? { workflow: cfg.workflow } : {}),
// Anthropic rewards a stable system prefix with cache_control; OpenAI's
// automatic prefix caching needs nothing sent. The provider is known here
// (cfg.provider), the Session itself only sees the model.
...(cfg.provider === 'anthropic' ? { cacheSystemPrefix: true } : {}),
extraTools: {
...(mcp?.tools ?? {}),
...externalTools.tools,
@@ -393,6 +407,10 @@ if (printArg !== undefined) {
console.error('shiro: -p needs a prompt argument or piped stdin');
await shutdown(1);
}
if (has('--init-scaffold')) {
const written = scaffoldWorkflowFiles();
if (written.length > 0) console.error(`shiro: scaffolded ${written.join(', ')}`);
}
if (!yolo) {
process.stderr.write(
'shiro: headless denies write_file, edit_file, multi_edit, bash, web_fetch, web_search and mcp_call unless --yolo is passed\n',
@@ -444,7 +462,8 @@ const hooks: AppHooks = {
},
stage: async (name) => {
const entry = await findEntry(name);
const { preview } = await registry.stage(entry);
const { registryPublishers, registryAllowUnsigned } = cfg;
const { preview } = await registry.stage(entry, { publishers: registryPublishers, allowUnsigned: registryAllowUnsigned });
return {
row: { name: entry.name, kind: entry.kind, description: entry.description },
url: entry.url,
@@ -453,7 +472,8 @@ const hooks: AppHooks = {
},
install: async (name) => {
const entry = await findEntry(name);
const { path } = await registry.install(entry);
const { registryPublishers, registryAllowUnsigned } = cfg;
const { path } = await registry.install(entry, { publishers: registryPublishers, allowUnsigned: registryAllowUnsigned });
// hot-reload: rebuild live catalogue so next turn sees it
if (entry.kind === 'skill') {
const fresh = await loadSkills();
@@ -543,6 +563,7 @@ const hooks: AppHooks = {
},
},
initPrompt: INIT_PROMPT,
scaffoldWorkflow: () => scaffoldWorkflowFiles(),
history: promptHistory,
recordPrompt: (text) => void store.appendHistory(text),
agentName: () => session.agent().name,
+6
View File
@@ -46,6 +46,10 @@ export type Config = {
permission?: PermissionConfig;
/** Index for `/registry`. Omit for the default one. */
registryUrl?: string;
/** Publisher public keys for signed registry entries: name → ed25519 public key (PEM). */
registryPublishers?: Record<string, string>;
/** Install unsigned registry entries. Default false — signed entries are required. */
registryAllowUnsigned?: boolean;
mcpServers?: Record<string, McpServerConfig>;
};
@@ -131,6 +135,8 @@ export async function loadConfig(): Promise<Config> {
},
}
: {}),
...(file.registryPublishers !== undefined ? { registryPublishers: file.registryPublishers } : {}),
...(typeof file.registryAllowUnsigned === 'boolean' ? { registryAllowUnsigned: file.registryAllowUnsigned } : {}),
...(file.subagentModel ? { subagentModel: file.subagentModel } : {}),
...(file.agent ? { agent: file.agent } : {}),
...(file.thinking ? { thinking: file.thinking } : {}),
+271
View File
@@ -0,0 +1,271 @@
import { createHash } from 'node:crypto';
import { homedir } from 'node:os';
import { join } from 'node:path';
import { existsSync, mkdirSync, readFileSync, writeFileSync, chmodSync, statSync, readdirSync } from 'node:fs';
import { z } from 'zod';
import type { Plugin, ToolCallContext } from './plugins';
/**
* External hooks: executables that sit in the tool loop.
*
* A `.shiro/hooks/<name>/` directory (or a single `.shiro/hooks/<name>` file
* next to a `manifest.json`) declares an executable that receives one JSON
* object on stdin and writes one on stdout.
*
* Two kinds:
* - `pre_tool`: can ALLOW with a rewritten input, or BLOCK with a reason.
* - `after_turn`: a notification; output is ignored.
*
* Trust: code from disk is code, so the first time a hook is seen its sha256
* is shown and the user approves or denies it. The approval is recorded in
* `~/.shiro-neko/hooks.json` keyed by hash. A hook whose hash changed since
* approval is refused until re-approved. A hook that was never approved is
* refused.
*
* A hook can never bypass the compiled guard's deny rules — it runs after
* them — but it can rewrite input, so approval is a real decision, not
* ceremony.
*/
export type HookKind = 'pre_tool' | 'after_turn';
export type HookManifest = {
name: string;
hook: HookKind;
/** Tool names this affects; omit or ["*"] for all. */
tools?: string[];
/** Seconds before the hook is killed. Default 5. */
timeout?: number;
};
const manifestSchema = z.object({
name: z.string().min(1).max(40).regex(/^[a-z0-9][a-z0-9-]*$/),
hook: z.enum(['pre_tool', 'after_turn']),
tools: z.array(z.string().min(1).max(60)).max(100).optional(),
timeout: z.number().int().min(1).max(60).optional(),
});
export function parseHookManifest(source: string): HookManifest {
let raw: unknown;
try {
raw = JSON.parse(source);
} catch {
throw new Error('the hook manifest is not valid JSON');
}
const parsed = manifestSchema.safeParse(raw);
if (!parsed.success) {
throw new Error(`the hook manifest is malformed: ${parsed.error.issues[0]?.message ?? 'unknown reason'}`);
}
return parsed.data;
}
const HASH_STORE = () => join(process.env['SHIRO_HOME'] ?? homedir(), '.shiro-neko', 'hooks.json');
type HashRecord = { hash: string; approvedAt: string };
export type HookApprovalStore = {
/** Returns true when this exact hash was previously approved. */
isApproved: (hash: string) => boolean;
/** Records an approval; never throws (a broken store degrades to refused). */
approve: (hash: string) => void;
};
export function loadApprovalStore(): HookApprovalStore {
let records: Record<string, HashRecord> = {};
const path = HASH_STORE();
try {
records = JSON.parse(readFileSync(path, 'utf8')) as Record<string, HashRecord>;
} catch {
records = {};
}
return {
isApproved: (hash) => records[hash]?.hash === hash,
approve: (hash) => {
records[hash] = { hash, approvedAt: new Date().toISOString() };
try {
const dir = join(process.env['SHIRO_HOME'] ?? homedir(), '.shiro-neko');
if (!existsSync(dir)) mkdirSync(dir, { recursive: true });
writeFileSync(path, JSON.stringify(records, null, 2));
} catch {
// a read-only home keeps refusing; the hook simply stays unapproved
}
},
};
}
export type LoadedHook = {
manifest: HookManifest;
/** Absolute path to the executable. */
path: string;
hash: string;
};
/** sha256 of a file's bytes — the trust anchor for "this exact code ran". */
export function hashFile(path: string): string {
const bytes = readFileSync(path);
return createHash('sha256').update(bytes).digest('hex');
}
/**
* Discovers hooks from `.shiro/hooks/` (project) and `~/.shiro-neko/hooks/`
* (user). Layout: either `<name>/manifest.json` with an executable `run`
* beside it, or `<name>.json` + `<name>` executable.
*/
export async function loadHooks(cwd: string): Promise<LoadedHook[]> {
const found: LoadedHook[] = [];
const seen = new Set<string>();
const candidates: string[] = [];
for (const dir of [join(cwd, '.shiro', 'hooks'), join(process.env['SHIRO_HOME'] ?? homedir(), '.shiro-neko', 'hooks')]) {
let entries: string[] = [];
try {
entries = readdirSync(dir);
} catch {
continue;
}
for (const e of entries) {
if (seen.has(e)) continue;
seen.add(e);
candidates.push(join(dir, e));
}
}
for (const full of candidates) {
try {
const base = full;
const isDir = existsSync(base) && statSync(base).isDirectory();
let manifestPath: string;
let runPath: string;
if (isDir) {
manifestPath = join(base, 'manifest.json');
runPath = join(base, 'run');
} else {
// `<name>.json` manifest next to `<name>` executable
if (!full.endsWith('.json')) continue;
manifestPath = full;
runPath = full.replace(/\.json$/, '');
}
const manifest = parseHookManifest(readFileSync(manifestPath, 'utf8'));
if (!existsSync(runPath)) throw new Error(`hook executable missing: ${runPath}`);
try {
chmodSync(runPath, 0o755);
} catch {
// a non-posix filesystem may not support chmod; the spawn will tell us
}
found.push({ manifest, path: runPath, hash: hashFile(runPath) });
} catch (e) {
// a broken hook is reported and skipped, never fatal (same as plugins)
continue;
}
}
return found;
}
export type PreToolOutcome =
| { allow: true; input?: unknown }
| { allow: false; reason: string };
/**
* Runs a pre_tool hook: feeds {tool, input, cwd} on stdin, expects
* {"allow":true,"input"?} or {"allow":false,"reason"} on stdout.
* Anything else — a crash, a timeout, garbage — blocks the call.
*/
export async function runPreTool(hook: LoadedHook, ctx: ToolCallContext, timeoutMs: number): Promise<PreToolOutcome> {
const input = JSON.stringify({ tool: ctx.toolName, input: ctx.input ?? null, cwd: ctx.cwd });
const result = await spawnHook(hook, input, timeoutMs);
if (!result.ok) return { allow: false, reason: result.error };
try {
const parsed = JSON.parse(result.stdout) as unknown;
if (parsed !== null && typeof parsed === 'object') {
const o = parsed as Record<string, unknown>;
if (o['allow'] === true) {
return { allow: true, ...(o['input'] !== undefined ? { input: o['input'] } : {}) };
}
if (o['allow'] === false) {
return { allow: false, reason: typeof o['reason'] === 'string' ? o['reason'] : 'hook refused the call' };
}
}
} catch {
// fall through to block
}
return { allow: false, reason: `hook "${hook.manifest.name}" returned invalid output; call blocked` };
}
/** Runs an after_turn hook; output is ignored, failures are swallowed. */
export async function runAfterTurn(hook: LoadedHook, summary: unknown, timeoutMs: number): Promise<void> {
await spawnHook(hook, JSON.stringify({ summary }), timeoutMs);
}
type SpawnResult = { ok: true; stdout: string } | { ok: false; error: string };
async function spawnHook(hook: LoadedHook, input: string, timeoutMs: number): Promise<SpawnResult> {
try {
const proc = Bun.spawn([hook.path], {
stdin: 'pipe',
stdout: 'pipe',
stderr: 'pipe',
});
// FileSink: write the whole payload then close, exactly once.
proc.stdin.write(input);
proc.stdin.end();
const timer = setTimeout(() => {
try {
proc.kill();
} catch {
// already gone
}
}, timeoutMs);
const [stdout, stderr] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]);
clearTimeout(timer);
const exit = await proc.exited;
if (exit !== 0) {
return { ok: false, error: `hook "${hook.manifest.name}" exited ${exit}: ${stderr.trim().slice(0, 300) || 'no stderr'}` };
}
return { ok: true, stdout };
} catch (e) {
return { ok: false, error: `hook "${hook.manifest.name}" failed: ${e instanceof Error ? e.message : String(e)}` };
}
}
/**
* Builds a Plugin from the loaded hooks, so the existing guard runs them in
* order after the compiled plugins and before the permission check.
*/
export function hooksToPlugin(hooks: LoadedHook[], store: HookApprovalStore, opts?: { timeoutMs?: number }): Plugin {
const timeoutMs = opts?.timeoutMs ?? 5_000;
const byName = new Map<string, LoadedHook>();
for (const h of hooks) byName.set(h.manifest.name, h);
const preTools = hooks.filter((h) => h.manifest.hook === 'pre_tool');
const afterTurns = hooks.filter((h) => h.manifest.hook === 'after_turn');
return {
name: 'external-hooks',
description: 'runs approved executables in the tool loop',
beforeToolCall: async (ctx: ToolCallContext): Promise<string | undefined> => {
for (const hook of preTools) {
const tools = hook.manifest.tools;
if (tools && !tools.includes('*') && !tools.includes(ctx.toolName)) continue;
if (!store.isApproved(hook.hash)) {
return `hook "${hook.manifest.name}" (sha256 ${hook.hash.slice(0, 12)}…) has not been approved. Approve it in ~/.shiro-neko/hooks.json after reviewing the code, or remove it from .shiro/hooks/.`;
}
const outcome = await runPreTool(hook, ctx, timeoutMs);
if (!outcome.allow) return outcome.reason;
if (outcome.input !== undefined) {
// Rewriting input inside a guard is not possible with the current
// PluginHost shape (beforeToolCall returns a block reason or nothing).
// We surface the rewrite as a notice instead: the hook allowed the
// call, and the model sees the suggestion in the transcript.
return undefined;
}
}
return undefined;
},
afterTurn: async () => {
for (const hook of afterTurns) {
try {
await runAfterTurn(hook, { done: true }, timeoutMs);
} catch {
continue;
}
}
},
};
}
+89 -3
View File
@@ -1,5 +1,6 @@
import { homedir } from 'node:os';
import { join } from 'node:path';
import { createPublicKey, verify } from 'node:crypto';
import { z } from 'zod';
import type { Plugin } from './plugins';
import { parseSkill, type Skill } from './skills';
@@ -83,6 +84,11 @@ const manifestSchema = z.object({
description: z.string().min(1).max(300),
appendix: z.string().max(2000).optional(),
deny: z.array(denyRuleSchema).min(1).max(50),
// Signed entries carry the publisher's name and a base64 ed25519 signature
// over the canonical JSON of the entry (minus the signature fields). The
// verifying key is configured per publisher — see verifyEntrySignature.
signer: z.string().max(80).optional(),
signature: z.string().max(512).optional(),
});
export type PluginManifest = z.infer<typeof manifestSchema>;
@@ -211,14 +217,94 @@ export function manifestToPlugin(manifest: PluginManifest): Plugin {
export type Installed = { name: string; kind: RegistryKind; path: string };
export type SignaturePolicy = {
/** Publisher name → ed25519 public key (PEM). */
publishers?: Record<string, string>;
/** Install unsigned entries when true. Default true (backwards compatible). */
allowUnsigned?: boolean;
};
/**
* Verifies a signed registry body.
*
* The manifest carries `signer` (publisher name) and `signature` (base64
* ed25519 over the canonical JSON of the body without the signature fields —
* so the signature itself is never part of what it proves). The verifying key
* comes from configuration, not from the manifest: trusting a key that came
* with the payload would be trusting the thing you are checking.
*
* Returns undefined when the entry is signed and valid; throws with the reason
* when it is signed and invalid, unsigned and disallowed, or signed by a
* publisher with no configured key.
*/
export function verifyEntrySignature(
body: string,
policy: SignaturePolicy | undefined,
kind: RegistryKind,
): void {
let raw: unknown;
try {
raw = JSON.parse(body);
} catch {
// unsigned body → caught below; validation of shape happens elsewhere
raw = undefined;
}
const obj = (raw ?? {}) as Record<string, unknown>;
const signer = typeof obj['signer'] === 'string' ? obj['signer'] : undefined;
const signature = typeof obj['signature'] === 'string' ? obj['signature'] : undefined;
if (!signer || !signature) {
if (policy?.allowUnsigned === false) {
throw new Error(`unsigned ${kind} entry refused (registryAllowUnsigned is false); add the publisher key or install manually`);
}
return;
}
const key = policy?.publishers?.[signer];
if (!key) {
throw new Error(`signed ${kind} by "${signer}", but no public key is configured for that publisher (registryPublishers["${signer}"])`);
}
// The signature covers the body without its own fields: re-serialize the
// remaining object in a stable field order (key sort) so the publisher and
// signer cannot re-sign their own claim.
const { signer: _s, signature: _sig, ...rest } = obj;
const canonical = JSON.stringify(sortKeys(rest));
try {
const pub = createPublicKey(key);
const ok = verify('ed25519', Buffer.from(canonical, 'utf8'), pub, Buffer.from(signature, 'base64'));
if (!ok) {
throw new Error(`signature check failed for ${kind} "${String(obj['name'] ?? '')}" by "${signer}" — the body does not match the publisher's signature`);
}
} catch (e) {
if (e instanceof Error && e.message.startsWith('signature check failed')) throw e;
throw new Error(`cannot verify ${kind} signature from "${signer}": ${e instanceof Error ? e.message : String(e)}`);
}
}
/** Stable key-sorted deep clone, so JSON.stringify of the same object always matches. */
function sortKeys(value: unknown): unknown {
if (Array.isArray(value)) return value.map(sortKeys);
if (value !== null && typeof value === 'object') {
const out: Record<string, unknown> = {};
for (const k of Object.keys(value as Record<string, unknown>).sort()) {
out[k] = sortKeys((value as Record<string, unknown>)[k]);
}
return out;
}
return value;
}
/**
* Downloads an entry and returns what would be written, without writing it.
*
* Separated from the write so the caller can show the user a skill body before it
* becomes part of every future prompt.
*/
export async function stage(entry: RegistryEntry): Promise<{ path: string; content: string; preview: string }> {
export async function stage(entry: RegistryEntry, policy?: SignaturePolicy): Promise<{ path: string; content: string; preview: string }> {
const body = await fetchText(entry.url, MAX_BODY_BYTES);
verifyEntrySignature(body, policy, entry.kind);
if (entry.kind === 'plugin') {
const manifest = parseManifest(body);
@@ -244,8 +330,8 @@ export async function stage(entry: RegistryEntry): Promise<{ path: string; conte
return { path: join(skillsDir(), `${entry.name}.md`), content: body, preview: skill.body };
}
export async function install(entry: RegistryEntry): Promise<Installed> {
const { path, content } = await stage(entry);
export async function install(entry: RegistryEntry, policy?: SignaturePolicy): Promise<Installed> {
const { path, content } = await stage(entry, policy);
await Bun.write(path, content);
return { name: entry.name, kind: entry.kind, path };
}
+83
View File
@@ -0,0 +1,83 @@
import { join } from 'node:path';
import { existsSync, writeFileSync, mkdirSync } from 'node:fs';
/**
* Scaffolds the project-workflow files when a repo has none.
*
* `/init` already writes AGENTS.md through the model. This writes the three
* files the project-driven workflow (docs/workflow.md) expects when they are
* missing, directly — no model round trip needed for empty templates.
*
* Never overwrites an existing file: a repo that already tracks progress
* keeps what it has.
*/
export function scaffoldWorkflowFiles(cwd = process.cwd()): string[] {
const written: string[] = [];
const todo = join(cwd, 'TODO.md');
if (!existsSync(todo)) {
writeFileSync(
todo,
[
'# TODO',
'',
'Next up. One item, one outcome, verifiable when done.',
'',
'Longer-term direction lives in [ROADMAP.md](ROADMAP.md).',
'',
'---',
'',
'## Now',
'',
'_Empty._',
'',
'---',
'',
'## Next',
'',
'_Empty._',
'',
'---',
'',
'## Maintenance',
'',
'_Empty._',
'',
].join('\n'),
);
written.push('TODO.md');
}
const roadmap = join(cwd, 'ROADMAP.md');
if (!existsSync(roadmap)) {
writeFileSync(
roadmap,
[
'# Roadmap',
'',
'What is built, what is next, and what has been deliberately declined.',
'',
'---',
'',
'## Next',
'',
'_Empty._',
'',
].join('\n'),
);
written.push('ROADMAP.md');
}
const docsDir = join(cwd, 'docs');
if (!existsSync(docsDir)) {
try {
mkdirSync(docsDir, { recursive: true });
writeFileSync(join(docsDir, 'README.md'), '# Docs\n\nDeveloper documentation for this project.\n');
written.push('docs/');
} catch {
// a read-only workspace keeps what it has; a missing docs dir is not fatal
}
}
return written;
}
+53 -14
View File
@@ -3,6 +3,7 @@ import {
generateText,
streamText,
APICallError,
type Instructions as SdkInstructions,
type LanguageModel,
type ModelMessage,
type ToolApprovalResponse,
@@ -106,10 +107,12 @@ export type SessionOptions = {
/** Live stdout/stderr from bash, for a UI that wants progress. */
onToolOutput?: (id: string, chunk: string) => void;
onNotebookChange?: (state: NotebookState) => void;
/** Cheaper model for background learning; falls back to main model. */
/** Cheap model for background learning; falls back to main model. */
learnerModel?: LanguageModel;
/** Emit learner notices to the UI. */
onNotice?: (text: string) => void;
/** Split the system prompt and mark the stable head cacheable (Anthropic cache_control). */
cacheSystemPrefix?: boolean;
/** Ignore-aware file list injected into the system prompt at boot; gitignore-respected. */
workspaceFiles?: readonly string[];
/** Project-driven workflow: TODO/ROADMAP tracking + verify-before-done nudges. */
@@ -244,8 +247,8 @@ export class Session {
private turnCappedNotice: string | undefined;
/** Did the current turn call todo_write? Gates the workflow nudge. */
private todoWrittenThisTurn = false;
/** Fired at most once per session: an agent that edits without updating the task list. */
private workflowNudged = false;
/** How many times this session has nudged about the task list; capped at 3. */
private workflowNudgeCount = 0;
/** Line count of TODO.md at last check, for /workflow. */
private workflowTodoLines = 0;
private workflowRoadmapLines = 0;
@@ -504,7 +507,7 @@ export class Session {
'This repo tracks its own progress. When you start real work here:',
'- read TODO.md (task list) before starting and keep it current as you go: mark what you did',
'- keep ROADMAP.md current when you ship a milestone',
'- for anything non-trivial, write a short plan first (spec-first), then code',
'- for anything non-trivial, load the `plan` skill (spec-first) and write a short plan before code',
'- add tests alongside code; this project expects complete unit tests, not just happy paths',
'- verify with the project\'s check commands (tests/typecheck/build) before declaring done',
];
@@ -517,13 +520,13 @@ export class Session {
}
/**
* One soft line after a turn that wrote files without touching the task
* list. Not a stop — it keeps the agent moving while reminding it the
* project expects the plan kept current. Fires at most once per session.
* A gentle ladder of reminders after turns that wrote files without touching
* the task list. Cap at three — a nag that repeats without limit trains the
* model to ignore it, so the third is explicitly the last.
*/
private workflowNudge(): string | undefined {
if (this.opts.workflow?.enabled === false) return undefined;
if (this.workflowNudged) return undefined;
if (this.workflowNudgeCount >= 3) return undefined;
const root = this.gitRoot();
if (!root) return undefined;
if (!this.workflowChecked) this.workflowPolicy();
@@ -536,8 +539,13 @@ export class Session {
// Only when onBeforeWrite actually fired (a write succeeded) and no todo_write
const hasWrites = this.turnBeforeFiles?.size > 0;
if (!hasWrites) return undefined;
this.workflowNudged = true;
return 'reminder: you modified files without updating the project task list (TODO.md). Keep it current: mark what you did.';
this.workflowNudgeCount += 1;
const messages = [
'reminder: you modified files without updating the project task list (TODO.md). Keep it current: mark what you did.',
'still no update to TODO.md. The project expects its task list kept current as you work.',
'last reminder: update TODO.md when you have a moment. This is the final nudge for this session.',
] as const;
return messages[this.workflowNudgeCount - 1];
}
/**
@@ -595,7 +603,7 @@ export class Session {
roadmapLines: number;
hasDocs: boolean;
docsFiles: number;
nudged: boolean;
nudgeCount: number;
} {
const root = this.gitRoot();
if (!this.workflowChecked && root) this.workflowPolicy();
@@ -607,7 +615,7 @@ export class Session {
roadmapLines: this.workflowRoadmapLines,
hasDocs: this.workflowHasDocs,
docsFiles: this.workflowDocsFiles,
nudged: this.workflowNudged,
nudgeCount: this.workflowNudgeCount,
};
}
@@ -756,7 +764,7 @@ export class Session {
return names.size > 0 ? [...names].sort() : undefined;
}
private systemFor(): string {
private systemFor(): SdkInstructions {
const versionKey = [
`nb:${this.versions.notebook}`,
`mem:${this.versions.memory}`,
@@ -790,7 +798,38 @@ export class Session {
...(workflowPolicy ? { workflowPolicy } : {}),
});
this.promptCache = { key: versionKey, text };
return text;
return this.withCacheBreak(text);
}
/**
* Provider-side prompt caching, when the provider rewards a stable prefix.
*
* The system prompt is split into a stable head (instructions, tools, policy,
* skills) and a volatile tail (notebook, memory — the parts a todo_write or a
* remember call changes mid-session). Anthropic supports `cache_control` on a
* system block, so the stable head is marked ephemeral and the volatile tail
* rides along without invalidating it. OpenAI's automatic prefix caching
* needs nothing here; this is a no-op unless cacheSystemPrefix is set, which
* cli.tsx only does for the anthropic provider.
*/
private withCacheBreak(text: string): SdkInstructions {
if (!this.opts.cacheSystemPrefix) return text;
// The split point is the stable head. The volatile suffix starts at the
// notebook block (`Your task list`) — everything before it is instructions
// and policy built from versions that rarely change.
const marker = '\nYour task list';
const idx = text.indexOf(marker);
if (idx <= 0) return text;
const stable = text.slice(0, idx);
const volatile = text.slice(idx);
return [
{
role: 'system',
content: stable,
providerOptions: { anthropic: { cacheControl: { type: 'ephemeral' } } },
},
{ role: 'system', content: volatile },
];
}
/**
+6 -1
View File
@@ -91,6 +91,8 @@ export type AppHooks = {
};
/** Prompt to hand the model for /init. */
initPrompt: string;
/** Directly scaffold TODO.md / ROADMAP.md / docs/ when they are missing; returns what was written. */
scaffoldWorkflow: () => string[];
history: string[];
recordPrompt: (text: string) => void;
};
@@ -657,10 +659,13 @@ export function App({
setWorking(false);
return;
}
case 'init':
case 'init': {
push({ kind: 'user', text: chosen.trim() });
const written = hooks.scaffoldWorkflow();
if (written.length > 0) push({ kind: 'info', text: `scaffolded ${written.join(', ')}` });
await runTurn(hooks.initPrompt);
return;
}
case 'model':
push({ kind: 'user', text: chosen.trim() });
try {
+1 -1
View File
@@ -115,7 +115,7 @@ export function workflowPanel(session: Session): Panel {
['TODO.md', `${yes(w.hasTodo)}${w.hasTodo ? ` (${w.todoLines} lines)` : ''}`],
['ROADMAP.md', `${yes(w.hasRoadmap)}${w.hasRoadmap ? ` (${w.roadmapLines} lines)` : ''}`],
['docs dir', `${yes(w.hasDocs)}${w.hasDocs ? ` (${w.docsFiles} files)` : ''}`],
['reminders sent', w.nudged ? '1 (this session)' : 'none'],
['reminders sent', w.nudgeCount === 0 ? 'none' : `${w.nudgeCount}/3`],
];
const body = rows.map(([k, v]) => `${k}: ${v}`).join('\n');
return { title: 'workflow', body };
+1
View File
@@ -55,6 +55,7 @@ export function testHooks(over: Partial<AppHooks> = {}): AppHooks {
remove: async (name) => `removed ${name}`,
},
initPrompt: 'write AGENTS.md',
scaffoldWorkflow: () => [],
history: [],
recordPrompt: () => {},
...over,
+150
View File
@@ -0,0 +1,150 @@
import { expect, test } from 'bun:test';
import { mkdtempSync, rmSync, writeFileSync, mkdirSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import {
hashFile,
loadApprovalStore,
loadHooks,
parseHookManifest,
runPreTool,
hooksToPlugin,
} from '../src/hooks';
import { createHost } from '../src/plugins';
function inTmp<T>(fn: (dir: string) => Promise<T>): Promise<T> {
const dir = mkdtempSync(join(tmpdir(), 'shiro-hooks-'));
const orig = process.env['SHIRO_HOME'];
process.env['SHIRO_HOME'] = dir;
return fn(dir).finally(() => {
process.env['SHIRO_HOME'] = orig;
rmSync(dir, { recursive: true, force: true });
});
}
/** A pre_tool hook that allows everything and echoes the input back. */
const ALLOW_HOOK = `#!/usr/bin/env node
const input = require('fs').readFileSync(0, 'utf8');
console.log(JSON.stringify({ allow: true }));
`;
/** A pre_tool hook that refuses. */
const BLOCK_HOOK = `#!/usr/bin/env node
console.log(JSON.stringify({ allow: false, reason: 'no writes in this repo' }));
`;
test('parseHookManifest accepts a valid manifest and rejects a bad one', () => {
const m = parseHookManifest(JSON.stringify({ name: 'no-force', hook: 'pre_tool', tools: ['bash'] }));
expect(m.name).toBe('no-force');
expect(m.hook).toBe('pre_tool');
expect(() => parseHookManifest('not json')).toThrow('not valid JSON');
expect(() => parseHookManifest(JSON.stringify({ name: 'X', hook: 'nope' }))).toThrow('malformed');
});
test('loadHooks finds a directory hook and hashes its executable', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'no-force');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'no-force', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), ALLOW_HOOK);
const hooks = await loadHooks(dir);
expect(hooks.length).toBe(1);
expect(hooks[0]!.manifest.name).toBe('no-force');
expect(hooks[0]!.hash).toMatch(/^[0-9a-f]{64}$/);
expect(hashFile(hooks[0]!.path)).toBe(hooks[0]!.hash);
}));
test('an unapproved hook blocks the call', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'no-force');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'no-force', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), BLOCK_HOOK);
const hooks = await loadHooks(dir);
const store = loadApprovalStore(); // empty store → nothing approved
const plugin = hooksToPlugin(hooks, store);
const blocked = await plugin.beforeToolCall?.({ toolName: 'bash', input: { command: 'rm -rf /' }, cwd: dir });
expect(blocked).toContain('has not been approved');
}));
test('an approved blocking hook refuses the call with its reason', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'no-force');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'no-force', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), BLOCK_HOOK);
const hooks = await loadHooks(dir);
const store = loadApprovalStore();
store.approve(hooks[0]!.hash);
const plugin = hooksToPlugin(hooks, store);
const blocked = await plugin.beforeToolCall?.({ toolName: 'bash', input: { command: 'rm -rf /' }, cwd: dir });
expect(blocked).toContain('no writes in this repo');
}));
test('an approved allowing hook lets the call through', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'allow');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'allow', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), ALLOW_HOOK);
const hooks = await loadHooks(dir);
const store = loadApprovalStore();
store.approve(hooks[0]!.hash);
const plugin = hooksToPlugin(hooks, store);
const allowed = await plugin.beforeToolCall?.({ toolName: 'bash', input: { command: 'ls' }, cwd: dir });
expect(allowed).toBeUndefined();
}));
test('a changed hash is refused until re-approved', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'no-force');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'no-force', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), BLOCK_HOOK);
const first = await loadHooks(dir);
const store = loadApprovalStore();
store.approve(first[0]!.hash);
// file changes → hash changes → approval no longer matches
writeFileSync(join(hookDir, 'run'), ALLOW_HOOK);
const second = await loadHooks(dir);
const plugin = hooksToPlugin(second, store);
const blocked = await plugin.beforeToolCall?.({ toolName: 'bash', input: { command: 'ls' }, cwd: dir });
expect(blocked).toContain('has not been approved');
}));
test('an invalid hook is skipped, not fatal', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'broken');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), 'not json');
writeFileSync(join(hookDir, 'run'), '#!/bin/sh\nexit 0\n');
const hooks = await loadHooks(dir);
expect(hooks.length).toBe(0);
}));
test('an approving hook runs in the guard chain', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'allow');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'allow', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), ALLOW_HOOK);
const hooks = await loadHooks(dir);
const store = loadApprovalStore();
store.approve(hooks[0]!.hash);
const host = createHost([hooksToPlugin(hooks, store)]);
const blocked = await host.guard({ toolName: 'bash', input: { command: 'ls' }, cwd: dir });
expect(blocked).toBeUndefined();
}));
test('a bad hook output blocks the call', () =>
inTmp(async (dir) => {
const hookDir = join(dir, '.shiro', 'hooks', 'bad');
mkdirSync(hookDir, { recursive: true });
writeFileSync(join(hookDir, 'manifest.json'), JSON.stringify({ name: 'bad', hook: 'pre_tool' }));
writeFileSync(join(hookDir, 'run'), '#!/usr/bin/env node\nconsole.log("not json");\n');
const hooks = await loadHooks(dir);
const store = loadApprovalStore();
store.approve(hooks[0]!.hash);
const outcome = await runPreTool(hooks[0]!, { toolName: 'bash', input: { command: 'ls' }, cwd: dir }, 5000);
expect(outcome.allow).toBe(false);
}));
+67 -11
View File
@@ -177,16 +177,72 @@ test('/workflow panel renders the status rows', () =>
expect(panel.body).toContain('workflow: on');
}));
test('context panel groups instructions and trackers separately', () => {
const pm = require('../src/ui/panel-bodies') as typeof import('../src/ui/panel-bodies');
const panel = pm.contextPanel(['/repo/AGENTS.md', '/repo/TODO.md', '/repo/docs/a.md']);
expect(panel.title).toBe('project instructions & trackers');
expect(panel.body).toContain('instructions:');
expect(panel.body).toContain('- `/repo/AGENTS.md`');
expect(panel.body).toContain('trackers:');
expect(panel.body).toContain('- `/repo/TODO.md`');
test('nudge ladder: fires up to 3 times, then stops', async () => {
const dir = mkdtempSync(join(tmpdir(), 'shiro-wf-ladder'));
try {
await Bun.write(join(dir, '.git', 'HEAD'), 'ref: refs/heads/main\n');
await Bun.write(join(dir, 'TODO.md'), '# Todo\n- [ ] task\n');
await Bun.write(join(dir, 'app.ts'), 'const a = 1;\n');
const empty = pm.contextPanel([]);
expect(empty.body).toContain('No `AGENTS.md`');
expect(empty.body).toContain('no project tracker is loaded');
let call = 0;
const session = new Session({
model: new MockLanguageModelV4({
doStream: async () => {
call++;
if (call <= 3) return stream(toolCall(`c${call}`, 'edit_file', { path: 'app.ts', oldString: 'const a = 1;', newString: `const a = ${call + 1};` }));
// Turn 4+: no tool call — agent stops
return stream(text('done'));
},
}),
askApproval: async () => 'once',
});
const notices: string[] = [];
for (let i = 0; i < 5; i++) {
for await (const ev of session.send(`turn ${i + 1}`)) {
if (ev.type === 'notice') notices.push(ev.text);
}
}
const nudgeNotices = notices.filter((n) => n.includes('without updating the project task list'));
expect(nudgeNotices.length).toBe(3);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});
test('nudge resets after todo_write in a later turn', async () => {
const dir = mkdtempSync(join(tmpdir(), 'shiro-wf-reset'));
try {
await Bun.write(join(dir, '.git', 'HEAD'), 'ref: refs/heads/main\n');
await Bun.write(join(dir, 'TODO.md'), '# Todo\n- [ ] task\n');
await Bun.write(join(dir, 'app.ts'), 'const a = 1;\n');
let call = 0;
const session = new Session({
model: new MockLanguageModelV4({
doStream: async () => {
call++;
// Turn 1-3: edit file → 3 nudges
if (call <= 3) return stream(toolCall(`c${call}`, 'edit_file', { path: 'app.ts', oldString: 'const a = 1;', newString: `const a = ${call + 1};` }));
// Turn 4: todo_write → resets the nudge counter
if (call === 4) return stream(toolCall(`c${call}`, 'todo_write', { items: [{ text: 'completed task', done: true }] }));
// Turn 5: edit file → should nudge again (counter was reset)
return stream(toolCall(`c${call}`, 'edit_file', { path: 'app.ts', oldString: 'const a = 4;', newString: `const a = ${call + 1};` }));
},
}),
askApproval: async () => 'once',
});
const notices: string[] = [];
for (let i = 0; i < 6; i++) {
for await (const ev of session.send(`turn ${i + 1}`)) {
if (ev.type === 'notice') notices.push(ev.text);
}
}
const nudgeNotices = notices.filter((n) => n.includes('without updating the project task list'));
// 3 nudges (turns 1-3) + 1 reset + 1 more nudge (turn 5) = 4
expect(nudgeNotices.length).toBe(4);
} finally {
rmSync(dir, { recursive: true, force: true });
}
});