Fix the loop stalling after compaction, add an external registry

The compaction bug, which is the important one:

beta.2 taught the pruner to drop any assistant part whose reasoning item it had
removed. That was right about the 400 and wrong about everything else. On a
reasoning model every tool call carries a provider itemId, so past the threshold
the model could no longer see what it had already run, and re-ran the same tools
until maxSteps ended the turn. Reproduced at 12 model calls for a job needing 4,
with nothing but the user message reaching the wire.

The dependency is not the part, it is the itemId. A part carrying one is
serialised as `{ type: 'item_reference', id }`, a pointer to an item stored
provider-side that depends on its reasoning item. Without the itemId the same
content goes out inline and carries no dependency at all. Verified against the
provider's own serialiser: `text` with an itemId becomes item_reference, the
identical part without one becomes output_text.

So `dropOrphanedItems` becomes `detachOrphanedItems`: strip the itemId, keep the
content. Compaction may shorten the history; it must not blank it. The new test
asserts behaviour rather than shape — the loop must end because the model chose
to, and every call after the first must still carry the earlier exchange. A shape
assertion passed the whole time the model was losing its memory.

Registry, via `/registry [list|search|add|remove|installed]`:

Skills and plugins are treated differently on purpose. A skill is prompt text, so
installing one puts a stranger's words into the system prompt of every future
session in this project; the install shows the body first and the origin is
recorded, so /skills always says where an instruction came from. A plugin is a
JSON manifest of deny rules, evaluated by compiled code identical for every
install. Loading TypeScript from a URL is declined outright: a plugin that can
block tool calls could otherwise lie about blocking them.

Validated before anything is written: https only (file: and data: rejected), name
matched against ^[a-z0-9][a-z0-9-]*$ so it cannot escape its directory, size
caps on index and body, every regex compiled, pattern length capped since it runs
on every tool call, and the body's own name checked against the index. Installed
skills rank below your own, so an install can never shadow a skill you wrote.

Interface:
- Context is a percentage of the compaction threshold, amber from two thirds and
  red at 90. A turn about to lose history now says so beforehand.
- Aligned command menu and registry tables; /skills and /plugins name origins.

538 tests, up from 488. The registry is tested against a real local HTTP server,
and the guard is proven to refuse a .env write end to end rather than assumed to.
This commit is contained in:
Muhammad Zakir Ramadhan
2026-09-03 03:07:56 +07:00
parent 8b1895de98
commit 84c60f2022
26 changed files with 1929 additions and 149 deletions
+97 -7
View File
@@ -14,6 +14,7 @@ import { costOf } from './pricing';
import { BUILTIN_PLUGINS, DEFAULT_ENABLED } from './plugins-builtin';
import { createHost } from './plugins';
import { fetchModels, presetById } from './providers';
import * as registry from './registry';
import { Session } from './session';
import { loadSkills } from './skills';
import * as store from './store';
@@ -21,6 +22,7 @@ import { createTaskTool } from './subagent';
import { VERSION, versionLine } from './version';
import { createAskBridge } from './ui/Ask';
import { App, createApprovalBridge, createNoticeBus, createSubagentBus, type AppHooks } from './ui/App';
import type { RegistryRow as AppRegistryRow } from './ui/Panels';
// SDK warnings go straight to stderr, which tears up the Ink render.
(globalThis as { AI_SDK_LOG_WARNINGS?: boolean }).AI_SDK_LOG_WARNINGS = false;
@@ -56,13 +58,14 @@ first run: start shiro with no key and it opens provider setup, or use /provider
config: ${configPath()}
{ "provider": "openai", "model": "gpt-5", "apiKey": "...",
"agent": "default", "thinking": "medium", "plugins": ["guard", "time"],
"toolSets": ["edit-plus", "git"],
"toolSets": ["edit-plus", "git"], "registryUrl": "https://...",
"mcpServers": { "fs": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "."] } } }
env: SHIRO_PROVIDER SHIRO_MODEL SHIRO_BASE_URL SHIRO_API_KEY
ANTHROPIC_API_KEY OPENAI_API_KEY
skills: builtin, plus ~/.shiro-neko/skills/*.md and .shiro/skills/*.md
registry: /registry to browse and install external skills and plugins
sessions: ${store.sessionsDir()}
in-session: /help for the command list`;
@@ -150,6 +153,40 @@ const instructions = has('--no-instructions') ? [] : await loadInstructions();
const skills = has('--no-skills') ? [] : await loadSkills();
const promptHistory = await store.loadHistory();
const installedPlugins = has('--no-plugins') ? { plugins: [], errors: [] } : await registry.loadInstalledPlugins();
/** Installed entries, as `kind:name`, so the registry list can mark what is already here. */
async function installedNames(): Promise<Set<string>> {
const names = new Set<string>();
for (const s of skills) if (s.origin === 'registry') names.add(`skill:${s.name}`);
for (const p of installedPlugins.plugins) names.add(`plugin:${p.name}`);
return names;
}
/**
* One entry by name, from the index.
*
* A name alone is ambiguous when a skill and a plugin share it, so `skill:name`
* disambiguates. Asking rather than guessing would be worse here: the two differ in
* what they can do, and picking one silently is the wrong default.
*/
async function findEntry(name: string): Promise<registry.RegistryEntry> {
const parsed = /^(skill|plugin):(.+)$/.exec(name.trim());
const kind = parsed?.[1] as registry.RegistryKind | undefined;
const bare = (parsed?.[2] ?? name).trim().toLowerCase();
const entries = await registry.fetchIndex(cfg.registryUrl);
const hits = entries.filter((e) => e.name === bare && (kind === undefined || e.kind === kind));
if (hits.length === 0) throw new Error(`no registry entry named "${bare}". Try /registry search ${bare}`);
if (hits.length > 1) {
throw new Error(
`"${bare}" is both a skill and a plugin. Use /registry add skill:${bare} or /registry add plugin:${bare}`,
);
}
return hits[0]!;
}
let agentVariant: AgentVariant;
try {
agentVariant = resolveAgent(flag('--agent') || cfg.agent, flag('--think') || cfg.thinking);
@@ -163,8 +200,8 @@ const pluginErrors = enabledPlugins
.filter((name) => !BUILTIN_PLUGINS.some((p) => p.name === name))
.map((name) => ({ plugin: name, message: 'no such plugin' }));
const plugins = createHost(
BUILTIN_PLUGINS.filter((p) => enabledPlugins.includes(p.name)),
pluginErrors,
[...BUILTIN_PLUGINS.filter((p) => enabledPlugins.includes(p.name)), ...installedPlugins.plugins],
[...pluginErrors, ...installedPlugins.errors],
);
const memory = has('--no-memory') ? undefined : new Memory(process.cwd(), languageModel);
@@ -280,6 +317,55 @@ const hooks: AppHooks = {
for await (const rel of walk({ limit: 5000 })) found.push(rel);
return found;
},
registry: {
list: async () => {
const entries = await registry.fetchIndex(cfg.registryUrl);
const installed = await installedNames();
return entries.map((e) => ({
name: e.name,
kind: e.kind,
description: e.description,
...(e.author ? { author: e.author } : {}),
installed: installed.has(`${e.kind}:${e.name}`),
}));
},
installed: async () => {
const rows: AppRegistryRow[] = [];
for (const s of skills) {
if (s.origin === 'registry') rows.push({ name: s.name, kind: 'skill', description: s.description, installed: true });
}
for (const p of installedPlugins.plugins) {
rows.push({ name: p.name, kind: 'plugin', description: p.description, installed: true });
}
return rows.sort((a, b) => a.name.localeCompare(b.name));
},
stage: async (name) => {
const entry = await findEntry(name);
const { preview } = await registry.stage(entry);
return {
row: { name: entry.name, kind: entry.kind, description: entry.description },
url: entry.url,
preview,
};
},
install: async (name) => {
const entry = await findEntry(name);
const { path } = await registry.install(entry);
// Loaded on the next start rather than hot-swapped: a skill joins the system
// prompt and a plugin joins the guard chain, and both are built once at boot.
return `installed ${entry.kind} ${entry.name} to ${path}\nrestart shiro to load it`;
},
remove: async (name) => {
const parsed = /^(skill|plugin):(.+)$/.exec(name);
const kinds: registry.RegistryKind[] = parsed ? [parsed[1] as registry.RegistryKind] : ['skill', 'plugin'];
const bare = parsed ? parsed[2]! : name;
for (const kind of kinds) {
if (await registry.uninstall(kind, bare)) return `removed ${kind} ${bare}\nrestart shiro to unload it`;
}
throw new Error(`nothing installed under the name "${bare}"`);
},
},
initPrompt: INIT_PROMPT,
history: promptHistory,
recordPrompt: (text) => void store.appendHistory(text),
@@ -302,13 +388,17 @@ const hooks: AppHooks = {
},
listSkills: () => {
if (skills.length === 0) return 'no skills loaded';
return skills.map((s) => `${s.name.padEnd(10)} ${s.origin.padEnd(8)} ${s.description}`).join('\n');
return [
...skills.map((s) => `${s.name.padEnd(12)} ${s.origin.padEnd(9)} ${s.description}`),
'',
'`/registry` to browse and install more',
].join('\n');
},
listPlugins: () => {
const active = plugins.plugins.map((p) => `${p.name.padEnd(8)} ${p.description}`);
const failed = plugins.errors.map((e) => `${e.plugin.padEnd(8)} ${e.message}`);
const active = plugins.plugins.map((p) => `${p.name.padEnd(12)} ${p.description}`);
const failed = plugins.errors.map((e) => `${e.plugin.padEnd(12)} ${e.message}`);
if (active.length === 0 && failed.length === 0) return 'no plugins active';
return [...active, ...failed].join('\n');
return [...active, ...failed, '', '`/registry` to browse and install more'].join('\n');
},
listMemory: async () => {
if (!memory) return 'memory is disabled (--no-memory)';
+47 -7
View File
@@ -16,6 +16,7 @@ export type CommandAction =
| { type: 'notes' }
| { type: 'skills' }
| { type: 'plugins' }
| { type: 'registry'; action: 'list' | 'search' | 'add' | 'remove' | 'installed'; arg?: string }
| { type: 'memory' }
| { type: 'agent'; agent?: string }
| { type: 'think'; level?: string }
@@ -42,6 +43,7 @@ export const COMMANDS: CommandSpec[] = [
{ name: 'model', arg: '<id>', summary: 'switch model by name' },
{ name: 'skills', summary: 'list loaded skills' },
{ name: 'plugins', summary: 'list active plugins' },
{ name: 'registry', arg: '[search|add|remove] [name]', summary: 'browse and install external skills and plugins' },
{ name: 'init', summary: 'have the agent write AGENTS.md for this project' },
{ name: 'context', summary: 'show which instruction files are loaded' },
{ name: 'todos', summary: "show the agent's task list" },
@@ -60,14 +62,14 @@ export const COMMANDS: CommandSpec[] = [
const usage = (c: CommandSpec) => `/${c.name}${c.arg ? ` ${c.arg}` : ''}`;
export const HELP = [
...COMMANDS.map((c) => `${usage(c).padEnd(18)} ${c.summary}`),
...COMMANDS.map((c) => `${usage(c).padEnd(22)} ${c.summary}`),
'',
'esc interrupt the running turn and clear the queue',
'ctrl-c kill the running command, keeping the turn',
'ctrl-r expand or collapse the reasoning panel',
'tab complete the highlighted command or file path',
'up / down recall earlier prompts, or move in an open menu',
'@ complete a workspace path',
'esc interrupt the running turn and clear the queue',
'ctrl-c kill the running command, keeping the turn',
'ctrl-r expand or collapse the reasoning panel',
'tab complete the highlighted command or file path',
'up / down recall earlier prompts, or move in an open menu',
'@ complete a workspace path',
'',
'typing during a turn queues the prompt; queued prompts run in order afterwards',
].join('\n');
@@ -89,6 +91,42 @@ export function matchCommands(input: string): CommandSpec[] {
/** True while the input is a bare command name being typed, so the menu should show. */
export const isMenuOpen = (input: string) => input.startsWith('/') && !input.includes(' ');
/**
* `/registry [list|installed|search <q>|add <name>|remove <name>]`.
*
* A bare `/registry` lists everything. `add` and `remove` need a name, and saying
* so beats fetching the whole index to then complain.
*/
function parseRegistry(arg: string): CommandAction {
const [verb = '', ...rest] = arg.split(/\s+/).filter(Boolean);
const name = rest.join(' ').trim();
switch (verb) {
case '':
case 'list':
return { type: 'registry', action: 'list' };
case 'installed':
return { type: 'registry', action: 'installed' };
case 'search':
return name
? { type: 'registry', action: 'search', arg: name }
: { type: 'info', text: 'usage: /registry search <query>' };
case 'add':
case 'install':
return name
? { type: 'registry', action: 'add', arg: name }
: { type: 'info', text: 'usage: /registry add <name>' };
case 'remove':
case 'uninstall':
return name
? { type: 'registry', action: 'remove', arg: name }
: { type: 'info', text: 'usage: /registry remove <name>' };
default:
// A bare word is almost always a search, and guessing beats a usage line.
return { type: 'registry', action: 'search', arg: arg.trim() };
}
}
/** Pure parser: no IO, so the TUI and headless mode share one definition. */
export function parseCommand(raw: string): CommandAction {
const input = raw.trim();
@@ -134,6 +172,8 @@ export function parseCommand(raw: string): CommandAction {
return { type: 'skills' };
case 'plugins':
return { type: 'plugins' };
case 'registry':
return parseRegistry(arg);
case 'memory':
return { type: 'memory' };
case 'agent':
+3
View File
@@ -27,6 +27,8 @@ export type Config = {
plugins?: string[];
/** Optional tool sets to offer beyond `core`; omit for all of them. */
toolSets?: ToolSetName[];
/** Index for `/registry`. Omit for the default one. */
registryUrl?: string;
mcpServers?: Record<string, McpServerConfig>;
};
@@ -89,6 +91,7 @@ export async function loadConfig(): Promise<Config> {
...(file.thinking ? { thinking: file.thinking } : {}),
...(Array.isArray(file.plugins) ? { plugins: file.plugins } : {}),
...(Array.isArray(file.toolSets) ? { toolSets: file.toolSets.filter(isToolSetName) } : {}),
...(typeof file.registryUrl === 'string' ? { registryUrl: file.registryUrl } : {}),
...(file.mcpServers ? { mcpServers: file.mcpServers } : {}),
};
}
+41 -34
View File
@@ -6,9 +6,6 @@ type Part = {
providerOptions?: Record<string, Record<string, unknown>>;
};
/** Parts the OpenAI responses API refuses to accept without their reasoning item. */
const DEPENDENT = new Set(['text', 'tool-call']);
function itemId(part: Part): string | undefined {
for (const options of Object.values(part.providerOptions ?? {})) {
const id = options['itemId'];
@@ -20,23 +17,39 @@ function itemId(part: Part): string | undefined {
const partsOf = (message: ModelMessage): Part[] =>
message.role === 'assistant' && Array.isArray(message.content) ? (message.content as Part[]) : [];
/** The part again, with every provider `itemId` removed. */
function withoutItemId(part: Part): Part {
const providerOptions: Record<string, Record<string, unknown>> = {};
for (const [provider, options] of Object.entries(part.providerOptions ?? {})) {
const { itemId: _dropped, ...rest } = options;
if (Object.keys(rest).length > 0) providerOptions[provider] = rest;
}
const next: Part = { ...part };
if (Object.keys(providerOptions).length > 0) next.providerOptions = providerOptions;
else delete next.providerOptions;
return next;
}
/**
* Drops assistant parts left orphaned by reasoning removal.
* Detaches assistant parts from reasoning items that pruning removed.
*
* The OpenAI responses API treats a `message` item as a dependent of the `reasoning`
* item from the same response: send the message without its reasoning and the request
* is rejected with 400 "was provided without its required 'reasoning' item".
* `pruneMessages({ reasoning: 'all' })` strips the reasoning and keeps the message,
* producing exactly that request.
* A part carrying a provider `itemId` is not sent inline. The OpenAI responses
* provider serialises it as `{ type: 'item_reference', id }`, pointing at an item
* stored on their side, and that stored item depends on the `reasoning` item from
* the same response. Send the reference without the reasoning and the request is
* rejected with 400 "was provided without its required 'reasoning' item".
*
* The two carry different item ids, so they cannot be matched by id. What links them
* is the assistant message they arrived in: one message is one response, and its
* reasoning item covers every other item in it.
* The repair is to drop the `itemId`, not the part. Without it the same content is
* serialised inline — a plain assistant message, a plain `function_call` — which
* carries no dependency on anything stored. Verified against the provider's own
* serialiser: `text` with an itemId goes out as `item_reference`, and the identical
* part without one goes out as `output_text`.
*
* Reasoning only disappears from turns pruning is already discarding, so dropping the
* orphaned text costs nothing pruning was not already spending.
* Dropping the part instead, which is what this used to do, cost the model its
* memory of the turn: after compaction it could no longer see the tool results it
* had just collected, so it called the same tools again until it hit the step limit.
*/
export function dropOrphanedItems(before: ModelMessage[], after: ModelMessage[]): ModelMessage[] {
export function detachOrphanedItems(before: ModelMessage[], after: ModelMessage[]): ModelMessage[] {
const survivingReasoning = new Set<string>();
for (const message of after) {
for (const part of partsOf(message)) {
@@ -54,7 +67,6 @@ export function dropOrphanedItems(before: ModelMessage[], after: ModelMessage[])
if (reasoning.some((id) => id !== undefined && survivingReasoning.has(id))) continue;
for (const part of parts) {
if (!DEPENDENT.has(part.type)) continue;
const id = itemId(part);
if (id) orphaned.add(id);
}
@@ -62,23 +74,20 @@ export function dropOrphanedItems(before: ModelMessage[], after: ModelMessage[])
if (orphaned.size === 0) return after;
const cleaned: ModelMessage[] = [];
for (const message of after) {
return after.map((message) => {
const parts = partsOf(message);
if (parts.length === 0) {
cleaned.push(message);
continue;
}
if (parts.length === 0) return message;
const kept = parts.filter((part) => {
let changed = false;
const next = parts.map((part) => {
const id = itemId(part);
return id === undefined || !orphaned.has(id);
if (id === undefined || !orphaned.has(id)) return part;
changed = true;
return withoutItemId(part);
});
if (kept.length > 0) cleaned.push({ ...message, content: kept } as ModelMessage);
}
return cleaned;
return changed ? ({ ...message, content: next } as ModelMessage) : message;
});
}
export type PruneOptions = Parameters<typeof pruneMessages>[0];
@@ -93,11 +102,9 @@ const anyParts = (message: ModelMessage): Part[] =>
*
* The OpenAI responses API rejects a `function_call_output` with no `function_call`
* carrying the same call id: 400 "No tool call found for function call output with
* call_id ...". Two things strand a result that way, and both happen on a long turn:
* `pruneMessages({ toolCalls: 'before-last-3-messages' })` counts messages, so the
* cut can land between an assistant tool-call and the tool message answering it, and
* `dropOrphanedItems` removes a tool-call whose reasoning item did not survive while
* the result sits in a separate message it never looks at.
* call_id ...". `pruneMessages({ toolCalls: 'before-last-3-messages' })` counts
* messages, so the cut can land between an assistant tool-call and the tool message
* answering it.
*
* The reverse pairing is left alone on purpose: a call still awaiting its result is
* exactly what a suspended approval looks like, and dropping it would break resume.
@@ -132,5 +139,5 @@ export function dropOrphanedResults(messages: ModelMessage[]): ModelMessage[] {
/** pruneMessages, then repair the provider-item dependencies it breaks. */
export function prunePreservingItems(options: PruneOptions): ModelMessage[] {
const pruned = pruneMessages(options);
return dropOrphanedResults(dropOrphanedItems(options.messages, pruned));
return dropOrphanedResults(detachOrphanedItems(options.messages, pruned));
}
+295
View File
@@ -0,0 +1,295 @@
import { homedir } from 'node:os';
import { join } from 'node:path';
import { z } from 'zod';
import type { Plugin } from './plugins';
import { parseSkill, type Skill } from './skills';
/**
* External skills and plugins, fetched from an index over HTTPS.
*
* Two kinds, and they are not equally safe.
*
* A **skill** is prompt text. Installing one puts a stranger's words into the
* system prompt of every future session in this project, which is prompt injection
* by invitation. The command shows the body before writing it and the origin is
* recorded, so `/skills` always says where an instruction came from.
*
* A **plugin** is declarative: a name, a prompt appendix, and deny rules matched
* against tool input. Never code. Loading arbitrary TypeScript from a URL would let
* an entry read every file the agent can read and lie about blocking anything, so
* that is not offered at any price.
*/
const MAX_INDEX_BYTES = 256 * 1024;
const MAX_BODY_BYTES = 64 * 1024;
/** A pattern from the internet runs on every tool call; a huge one is a denial of service. */
const MAX_PATTERN = 200;
/**
* https only, with localhost allowed so the suite can serve a real index.
*
* `file:` would read a local path and `data:` would inline a payload, neither of
* which is what "fetch this from a registry" means to the person typing it.
*/
export function isFetchable(url: string): boolean {
let parsed: URL;
try {
parsed = new URL(url);
} catch {
return false;
}
if (parsed.protocol === 'https:') return true;
return parsed.protocol === 'http:' && (parsed.hostname === 'localhost' || parsed.hostname === '127.0.0.1');
}
const entrySchema = z.object({
name: z
.string()
.min(1)
.max(40)
// The name becomes a filename. Anything else is a path traversal.
.regex(/^[a-z0-9][a-z0-9-]*$/, 'lowercase letters, digits, and dashes only'),
description: z.string().min(1).max(300),
// z.url() accepts file: and data:, which would read a local path or inline a
// payload. Only https reaches the network the way the user expects.
url: z
.string()
.url()
.refine((u) => isFetchable(u), 'must be an https URL'),
author: z.string().max(80).optional(),
});
const indexSchema = z.object({
skills: z.array(entrySchema).max(500).optional(),
plugins: z.array(entrySchema).max(500).optional(),
});
export type RegistryKind = 'skill' | 'plugin';
export type RegistryEntry = z.infer<typeof entrySchema> & { kind: RegistryKind };
const denyRuleSchema = z
.object({
tools: z.array(z.string().max(60)).min(1).max(30),
pathPattern: z.string().max(MAX_PATTERN).optional(),
commandPattern: z.string().max(MAX_PATTERN).optional(),
reason: z.string().min(1).max(300),
})
.refine((r) => r.pathPattern !== undefined || r.commandPattern !== undefined, {
message: 'a deny rule needs pathPattern or commandPattern',
});
const manifestSchema = z.object({
name: z.string().min(1).max(40).regex(/^[a-z0-9][a-z0-9-]*$/),
description: z.string().min(1).max(300),
appendix: z.string().max(2000).optional(),
deny: z.array(denyRuleSchema).min(1).max(50),
});
export type PluginManifest = z.infer<typeof manifestSchema>;
export const DEFAULT_INDEX_URL =
'https://raw.githubusercontent.com/zakirkun/shiro-neko-registry/main/index.json';
const home = () => process.env['SHIRO_HOME'] ?? homedir();
export const skillsDir = () => join(home(), '.shiro-neko', 'registry', 'skills');
export const pluginsDir = () => join(home(), '.shiro-neko', 'registry', 'plugins');
async function fetchText(url: string, limit: number): Promise<string> {
if (!isFetchable(url)) throw new Error(`refusing a non-https URL: ${url}`);
const res = await fetch(url, { redirect: 'follow', signal: AbortSignal.timeout(20_000) });
if (!res.ok) throw new Error(`${url} returned ${res.status} ${res.statusText}`);
const declared = Number(res.headers.get('content-length') ?? 0);
if (declared > limit) throw new Error(`${url} is ${declared} bytes, over the ${limit} byte limit`);
const text = await res.text();
if (text.length > limit) throw new Error(`${url} is over the ${limit} byte limit`);
return text;
}
/** Parses an index document. Exported so the shape can be tested without a network. */
export function parseIndex(source: string): RegistryEntry[] {
let raw: unknown;
try {
raw = JSON.parse(source);
} catch {
throw new Error('the registry index is not valid JSON');
}
const parsed = indexSchema.safeParse(raw);
if (!parsed.success) {
throw new Error(`the registry index is malformed: ${parsed.error.issues[0]?.message ?? 'unknown reason'}`);
}
const seen = new Set<string>();
const entries: RegistryEntry[] = [];
for (const kind of ['skill', 'plugin'] as const) {
for (const entry of parsed.data[`${kind}s`] ?? []) {
const key = `${kind}:${entry.name}`;
if (seen.has(key)) continue;
seen.add(key);
entries.push({ ...entry, kind });
}
}
return entries;
}
export async function fetchIndex(url = DEFAULT_INDEX_URL): Promise<RegistryEntry[]> {
return parseIndex(await fetchText(url, MAX_INDEX_BYTES));
}
export const searchEntries = (entries: RegistryEntry[], query: string): RegistryEntry[] => {
const needle = query.trim().toLowerCase();
if (!needle) return entries;
return entries.filter(
(e) => e.name.includes(needle) || e.description.toLowerCase().includes(needle),
);
};
/** Validates a plugin manifest, including that every pattern is a usable regex. */
export function parseManifest(source: string): PluginManifest {
let raw: unknown;
try {
raw = JSON.parse(source);
} catch {
throw new Error('the plugin manifest is not valid JSON');
}
const parsed = manifestSchema.safeParse(raw);
if (!parsed.success) {
throw new Error(`the plugin manifest is malformed: ${parsed.error.issues[0]?.message ?? 'unknown reason'}`);
}
for (const rule of parsed.data.deny) {
for (const pattern of [rule.pathPattern, rule.commandPattern]) {
if (pattern === undefined) continue;
try {
new RegExp(pattern, 'i');
} catch (e) {
throw new Error(`invalid pattern "${pattern}": ${(e as Error).message}`);
}
}
}
return parsed.data;
}
/**
* A manifest as a Plugin.
*
* Rules are data, so the guard is the same code for every installed plugin: match
* the tool name, then the path or command against a compiled regex. Nothing from
* the manifest is ever evaluated.
*/
export function manifestToPlugin(manifest: PluginManifest): Plugin {
const rules = manifest.deny.map((rule) => ({
tools: new Set(rule.tools),
path: rule.pathPattern ? new RegExp(rule.pathPattern, 'i') : undefined,
command: rule.commandPattern ? new RegExp(rule.commandPattern, 'i') : undefined,
reason: rule.reason,
}));
return {
name: manifest.name,
description: `${manifest.description} (installed)`,
...(manifest.appendix ? { appendix: manifest.appendix } : {}),
beforeToolCall: ({ toolName, input }) => {
const o = (input ?? {}) as Record<string, unknown>;
const path = typeof o['path'] === 'string' ? o['path'] : '';
const command = typeof o['command'] === 'string' ? o['command'] : '';
for (const rule of rules) {
if (!rule.tools.has(toolName)) continue;
if (rule.path && path && rule.path.test(path)) return rule.reason;
if (rule.command && command && rule.command.test(command)) return rule.reason;
}
return undefined;
},
};
}
export type Installed = { name: string; kind: RegistryKind; path: string };
/**
* Downloads an entry and returns what would be written, without writing it.
*
* Separated from the write so the caller can show the user a skill body before it
* becomes part of every future prompt.
*/
export async function stage(entry: RegistryEntry): Promise<{ path: string; content: string; preview: string }> {
const body = await fetchText(entry.url, MAX_BODY_BYTES);
if (entry.kind === 'plugin') {
const manifest = parseManifest(body);
if (manifest.name !== entry.name) {
throw new Error(`the manifest calls itself "${manifest.name}" but the index calls it "${entry.name}"`);
}
const rules = manifest.deny
.map((r) => `- denies ${r.tools.join(', ')}: ${r.pathPattern ?? r.commandPattern}`)
.join('\n');
return {
path: join(pluginsDir(), `${entry.name}.json`),
content: JSON.stringify(manifest, null, 2),
preview: `${manifest.description}\n\n${rules}`,
};
}
const skill = parseSkill(body, 'registry');
if (!skill) throw new Error('that skill has no name/description frontmatter, so it cannot be loaded');
if (skill.name !== entry.name) {
throw new Error(`the skill calls itself "${skill.name}" but the index calls it "${entry.name}"`);
}
return { path: join(skillsDir(), `${entry.name}.md`), content: body, preview: skill.body };
}
export async function install(entry: RegistryEntry): Promise<Installed> {
const { path, content } = await stage(entry);
await Bun.write(path, content);
return { name: entry.name, kind: entry.kind, path };
}
/** Removes an installed entry. Returns false when there was nothing to remove. */
export async function uninstall(kind: RegistryKind, name: string): Promise<boolean> {
if (!/^[a-z0-9][a-z0-9-]*$/.test(name)) throw new Error(`"${name}" is not a valid entry name`);
const path = kind === 'plugin' ? join(pluginsDir(), `${name}.json`) : join(skillsDir(), `${name}.md`);
const file = Bun.file(path);
if (!(await file.exists())) return false;
await file.delete();
return true;
}
/**
* Declarative plugins from disk.
*
* A malformed file is reported and skipped rather than fatal: one bad install
* should not stop the agent from starting.
*/
export async function loadInstalledPlugins(): Promise<{ plugins: Plugin[]; errors: { plugin: string; message: string }[] }> {
const plugins: Plugin[] = [];
const errors: { plugin: string; message: string }[] = [];
let files: string[] = [];
try {
for await (const f of new Bun.Glob('*.json').scan({ cwd: pluginsDir(), onlyFiles: true })) files.push(f);
} catch {
return { plugins, errors };
}
for (const file of files.sort()) {
const name = file.replace(/\.json$/, '');
try {
plugins.push(manifestToPlugin(parseManifest(await Bun.file(join(pluginsDir(), file)).text())));
} catch (e) {
errors.push({ plugin: name, message: e instanceof Error ? e.message : String(e) });
}
}
return { plugins, errors };
}
/** Installed skills, for `/registry list` — the loader already merges them by name. */
export function installedSkillNames(skills: Skill[]): string[] {
return skills.filter((s) => s.origin === 'registry').map((s) => s.name);
}
+9 -1
View File
@@ -83,6 +83,9 @@ export type SessionOptions = {
const estimateTokens = (messages: ModelMessage[]) => Math.round(JSON.stringify(messages).length / 4);
/** Estimated tokens at which the wire history is pruned. */
const DEFAULT_COMPACT_THRESHOLD = 120_000;
export class Session {
readonly messages: ModelMessage[];
readonly tools: ToolSet;
@@ -164,6 +167,11 @@ export class Session {
return estimateTokens(this.messages);
}
/** Where compaction kicks in, so the status bar can show how close it is. */
compactThreshold(): number {
return this.opts.compactThreshold ?? DEFAULT_COMPACT_THRESHOLD;
}
private systemFor(): string {
return systemPrompt({
cwd: this.opts.cwd ?? process.cwd(),
@@ -234,7 +242,7 @@ export class Session {
this.opts.onChange?.(this.messages);
this.controller = new AbortController();
const signal = this.controller.signal;
const threshold = this.opts.compactThreshold ?? 120_000;
const threshold = this.compactThreshold();
const outputs: Extract<AgentEvent, { type: 'tool-output' }>[] = [];
onBashOutput(({ toolCallId, chunk }) => {
+16 -7
View File
@@ -4,7 +4,7 @@ import { join } from 'node:path';
import { z } from 'zod';
import { BUILTIN_SKILLS } from './skills-builtin';
export type SkillOrigin = 'builtin' | 'user' | 'project';
export type SkillOrigin = 'builtin' | 'registry' | 'user' | 'project';
export type Skill = {
name: string;
@@ -37,14 +37,23 @@ export function parseSkill(source: string, origin: SkillOrigin, path?: string):
return { name, description, origin, ...(path ? { path } : {}), body: match[2]!.trim().slice(0, MAX_BODY) };
}
const skillDirs = (cwd: string) => [
{ dir: join(process.env['SHIRO_HOME'] ?? homedir(), '.shiro-neko', 'skills'), origin: 'user' as const },
{ dir: join(cwd, '.shiro', 'skills'), origin: 'project' as const },
];
/**
* Precedence, low to high. Installed skills sit below your own on purpose: a skill
* you wrote must never be shadowed by one fetched from a registry.
*/
const skillDirs = (cwd: string) => {
const home = join(process.env['SHIRO_HOME'] ?? homedir(), '.shiro-neko');
return [
{ dir: join(home, 'registry', 'skills'), origin: 'registry' as const },
{ dir: join(home, 'skills'), origin: 'user' as const },
{ dir: join(cwd, '.shiro', 'skills'), origin: 'project' as const },
];
};
/**
* Builtin, then user, then project. Later wins, so a project can override a
* bundled skill by using the same name.
* Builtin, then installed, then user, then project. Later wins, so a project can
* override a bundled or installed skill by using the same name — and a skill you
* wrote yourself always beats one fetched from a registry.
*/
export async function loadSkills(cwd = process.cwd()): Promise<Skill[]> {
const byName = new Map<string, Skill>();
+116 -7
View File
@@ -15,7 +15,7 @@ import { AskPanel, type AskBridge, type AskPending } from './Ask';
import { Diff } from './Diff';
import { Markdown } from './Markdown';
import { Onboard, type OnboardResult } from './Onboard';
import { InfoPanel, OutputPanel, QueuePanel, StatusBar, SubagentPanel, ThinkingPanel, TodoPanel, ActiveTool, FileMenu, type SubagentView } from './Panels';
import { InfoPanel, OutputPanel, QueuePanel, RegistryPanel, InstallPrompt, StatusBar, SubagentPanel, ThinkingPanel, TodoPanel, ActiveTool, FileMenu, type RegistryRow, type SubagentView } from './Panels';
import { PromptInput } from './PromptInput';
type Line =
@@ -139,6 +139,15 @@ export type AppHooks = {
instructionFiles: () => string[];
/** Ignore-aware workspace paths for `@` completion, loaded on first use. */
listPaths: () => Promise<string[]>;
/** Registry index, installed set, and the install/remove actions. */
registry: {
list: () => Promise<RegistryRow[]>;
installed: () => Promise<RegistryRow[]>;
/** Fetches and validates without writing, so the body can be shown first. */
stage: (name: string) => Promise<{ row: RegistryRow; url: string; preview: string }>;
install: (name: string) => Promise<string>;
remove: (name: string) => Promise<string>;
};
/** Prompt to hand the model for /init. */
initPrompt: string;
history: string[];
@@ -198,19 +207,42 @@ function Approval({ pending }: { pending: Pending }) {
}
function CommandMenu({ matches, index }: { matches: CommandSpec[]; index: number }) {
const width = Math.max(...matches.map((c) => `/${c.name}${c.arg ? ` ${c.arg}` : ''}`.length)) + 1;
return (
<Box flexDirection="column" marginTop={1}>
{matches.map((c, i) => (
<Text key={c.name} color={i === index ? 'cyan' : undefined} dimColor={i !== index}>
{i === index ? '> ' : ' '}
{`/${c.name}${c.arg ? ` ${c.arg}` : ''}`.padEnd(18)} {c.summary}
</Text>
<Box key={c.name}>
<Text color={i === index ? 'cyan' : undefined}>{i === index ? '> ' : ' '}</Text>
<Text color={i === index ? 'cyan' : undefined} bold={i === index}>
{`/${c.name}${c.arg ? ` ${c.arg}` : ''}`.padEnd(width)}
</Text>
<Text dimColor>{c.summary}</Text>
</Box>
))}
<Text dimColor>up/down move | tab complete | enter run | esc dismiss</Text>
</Box>
);
}
/** Keyboard wrapper around InstallPrompt, so the prompt itself stays presentational. */
function InstallConfirm({
staged,
onDone,
}: {
staged: { row: RegistryRow; url: string; preview: string };
onDone: (yes: boolean) => void;
}) {
useInput((input, key) => {
const c = input.toLowerCase();
if (c === 'y' || key.return) onDone(true);
else if (c === 'n' || key.escape) onDone(false);
});
return (
<InstallPrompt name={staged.row.name} kind={staged.row.kind} url={staged.url} preview={staged.preview} />
);
}
export function App({
session,
bridge,
@@ -260,8 +292,12 @@ export function App({
const [notebook, setNotebook] = useState<NotebookState>(session.notebook.state());
const [agents, setAgents] = useState<SubagentView[]>([]);
const [panel, setPanel] = useState<{ title: string; hint?: string; body: string } | undefined>();
const [registry, setRegistry] = useState<{ title: string; hint?: string; rows: RegistryRow[] } | undefined>();
const [installing, setInstalling] = useState<
{ row: RegistryRow; url: string; preview: string } | undefined
>();
const modal = pending !== undefined || asking !== undefined || onboarding;
const modal = pending !== undefined || asking !== undefined || onboarding || installing !== undefined;
const anyPicker = modelPicker !== undefined || agentPicker || thinkPicker;
const matches = matchCommands(draft);
const menuOpen = matches.length > 0 && !menuDismissed && !busy && !modal && !anyPicker && !panel;
@@ -371,6 +407,10 @@ export function App({
setPanel(undefined);
return true;
}
if (key.escape && registry) {
setRegistry(undefined);
return true;
}
// The file picker gets first refusal: while an `@` token is open its keys
// mean navigation, not history recall or command completion.
@@ -425,7 +465,7 @@ export function App({
}
return false;
},
[draft, fileMatches.length, fileOpen, highlighted, highlightedPath, matches.length, menuOpen, panel, token],
[draft, fileMatches.length, fileOpen, highlighted, highlightedPath, matches.length, menuOpen, panel, registry, token],
);
const onDraftChange = useCallback((value: string, at: number) => {
@@ -536,6 +576,7 @@ export function App({
setFileIndex(0);
setFileDismissed(false);
setPanel(undefined);
setRegistry(undefined);
// Enter on an open menu runs the highlighted entry, so `/mo` + enter works.
const chosen = menuOpen && highlighted ? `/${highlighted.name}` : raw;
@@ -676,6 +717,50 @@ export function App({
push({ kind: 'user', text: chosen.trim() });
setPanel({ title: 'plugins', body: hooks.listPlugins() });
return;
case 'registry': {
push({ kind: 'user', text: chosen.trim() });
setWorking(true);
try {
if (action.action === 'add') {
// Staged, not installed: nothing is written until the prompt is answered.
setInstalling(await hooks.registry.stage(action.arg!));
return;
}
if (action.action === 'remove') {
push({ kind: 'info', text: await hooks.registry.remove(action.arg!) });
return;
}
if (action.action === 'installed') {
const rows = await hooks.registry.installed();
setRegistry({
title: 'installed',
hint: rows.length === 0 ? 'nothing installed yet' : `${rows.length} from the registry`,
rows,
});
return;
}
const all = await hooks.registry.list();
const rows =
action.action === 'search' && action.arg
? all.filter(
(r) =>
r.name.includes(action.arg!.toLowerCase()) ||
r.description.toLowerCase().includes(action.arg!.toLowerCase()),
)
: all;
setRegistry({
title: action.arg ? `registry: ${action.arg}` : 'registry',
hint: `${rows.length} of ${all.length} available`,
rows,
});
} catch (e) {
push({ kind: 'error', text: e instanceof Error ? e.message : String(e) });
} finally {
setWorking(false);
}
return;
}
case 'memory': {
push({ kind: 'user', text: chosen.trim() });
setWorking(true);
@@ -799,6 +884,29 @@ export function App({
<InfoPanel title={panel.title} {...(panel.hint ? { hint: panel.hint } : {})} lines={panel.body} />
)}
{registry && (
<RegistryPanel title={registry.title} {...(registry.hint ? { hint: registry.hint } : {})} rows={registry.rows} />
)}
{installing && (
<InstallConfirm
staged={installing}
onDone={async (yes) => {
const staged = installing;
setInstalling(undefined);
if (!yes) {
push({ kind: 'info', text: `install cancelled: ${staged.row.name}` });
return;
}
try {
push({ kind: 'info', text: await hooks.registry.install(staged.row.name) });
} catch (e) {
push({ kind: 'error', text: e instanceof Error ? e.message : String(e) });
}
}}
/>
)}
{asking && <AskPanel pending={asking} />}
{pending && <Approval pending={pending} />}
@@ -937,6 +1045,7 @@ export function App({
agent={hooks.agentName()}
thinking={hooks.thinkingLevel()}
contextTokens={session.estimatedTokens()}
contextLimit={session.compactThreshold()}
cost={(() => {
const spend = costOf(hooks.config().model, session.inputTokens, session.outputTokens);
return spend === undefined ? 'unpriced' : formatUsd(spend);
+115 -1
View File
@@ -216,6 +216,7 @@ export function StatusBar({
agent,
thinking,
contextTokens,
contextLimit,
cost,
toolCount,
}: {
@@ -223,14 +224,25 @@ export function StatusBar({
agent: string;
thinking: string;
contextTokens: number;
/** Threshold compaction fires at, so the bar means something. */
contextLimit?: number;
cost: string;
toolCount: number;
}) {
const pct = contextLimit ? Math.min(100, Math.round((contextTokens / contextLimit) * 100)) : undefined;
// Amber from two thirds, red once compaction is imminent: the point is to warn
// before a turn silently loses its history, not after.
const contextColor = pct === undefined ? undefined : pct >= 90 ? 'red' : pct >= 66 ? 'yellow' : undefined;
return (
<Box>
<Text dimColor>{`${model} `}</Text>
<Text color="cyan">{agent}</Text>
<Text dimColor>{`/${thinking} ${toolCount} tools ~${contextTokens} ctx ${cost}`}</Text>
<Text dimColor>{`/${thinking} ${toolCount} tools `}</Text>
<Text color={contextColor} dimColor={contextColor === undefined}>
{pct === undefined ? `~${contextTokens} ctx` : `${pct}% ctx`}
</Text>
<Text dimColor>{` ${cost}`}</Text>
</Box>
);
}
@@ -258,3 +270,105 @@ export function InfoPanel({ title, hint, lines }: { title: string; hint?: string
</Box>
);
}
export type RegistryRow = {
name: string;
kind: 'skill' | 'plugin';
description: string;
author?: string;
installed?: boolean;
};
const KIND_COLOR: Record<RegistryRow['kind'], string> = { skill: 'green', plugin: 'magenta' };
/**
* The registry index as a table.
*
* `kind` is coloured rather than spelled out on every row: skill and plugin carry
* very different risk, and a reader scanning the list should see that at a glance.
*/
export function RegistryPanel({
rows,
hint,
title = 'registry',
}: {
rows: readonly RegistryRow[];
hint?: string;
title?: string;
}) {
if (rows.length === 0) {
return <InfoPanel title={title} {...(hint ? { hint } : {})} lines="nothing found" />;
}
const width = Math.min(22, Math.max(...rows.map((r) => r.name.length)) + 1);
return (
<Box flexDirection="column" borderStyle="round" borderColor="cyan" paddingX={1} marginBottom={1}>
<Text color="cyan" bold>
{title}
</Text>
{hint && <Text dimColor>{hint}</Text>}
{rows.map((r) => (
<Box key={`${r.kind}:${r.name}`}>
<Text color={KIND_COLOR[r.kind]}>{r.kind === 'skill' ? 'S' : 'P'} </Text>
<Text bold>{r.name.padEnd(width)}</Text>
<Text dimColor>{r.description.length > 58 ? `${r.description.slice(0, 58)}...` : r.description}</Text>
{r.installed && <Text color="green">{' installed'}</Text>}
</Box>
))}
<Text dimColor>{'S skill P plugin | /registry add <name>'}</Text>
</Box>
);
}
/**
* Confirmation before an install writes anything.
*
* A skill body becomes part of the system prompt of every future session in this
* project, so it is shown in full first. The wording says that plainly rather than
* asking a generic "are you sure".
*/
export function InstallPrompt({
name,
kind,
url,
preview,
lines = 14,
}: {
name: string;
kind: 'skill' | 'plugin';
url: string;
preview: string;
lines?: number;
}) {
const body = preview.split('\n');
const shown = body.slice(0, lines);
const hidden = body.length - shown.length;
return (
<Box flexDirection="column" borderStyle="round" borderColor="yellow" paddingX={1}>
<Text color="yellow" bold>
{`install ${kind} "${name}"?`}
</Text>
<Text dimColor>{url}</Text>
<Box marginTop={1} flexDirection="column">
{shown.map((l, i) => (
<Text key={i} dimColor>
{` ${l}`}
</Text>
))}
{hidden > 0 && <Text dimColor>{` ... ${hidden} more lines`}</Text>}
</Box>
<Box marginTop={1} flexDirection="column">
<Text color="yellow">
{kind === 'skill'
? 'A skill is instructions the agent follows. This text joins your system prompt.'
: 'A plugin adds refusal rules. It is data, not code: nothing here is executed.'}
</Text>
<Text>
<Text color="green">y</Text> install | <Text color="red">n</Text> cancel
</Text>
</Box>
</Box>
);
}