Refactor view modules for improved readability and consistency

- Updated markdown rendering logic to use more concise methods for obtaining vector lengths.
- Changed review status display to use the correct flag from settings.
- Cleaned up sidebar rendering code for better formatting and readability.
- Enhanced status bar rendering with improved string formatting and consistent style application.
- Refined workflow panel rendering, ensuring consistent style usage and improved readability.
- Added architecture overview and detailed documentation for backend, data, dependencies, and frontend structures.
This commit is contained in:
asepharyana committed 2026-07-16 07:56:11 +07:00
1 parent 7d99cd6618
commit a00aa9bec8
141 files changed
+3420 -2172

No files matched your search

+65
View File
@@ -0,0 +1,65 @@
# Architecture Overview
## System Layout
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
```
┌─────────────────────────────────────────────────────────────┐
│ Process Mode │
│ Single-Process ─── Daemon (background) ─── Attach (client) │
└──────────────────────────┬──────────────────────────────────┘
│ IPC (Unix domain socket)
▼
┌─────────────────────────────────────────────────────────────┐
│ src/main.rs │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Controller │──▶│ Runtime │──▶│ View │ │
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ Harness │ │
│ │ (tool dispatch)│ │
│ └───────┬────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
│ │ Tools │ │ Subagents │ │ Workflow │ │
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
│ └───────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## Process Modes
| Mode | Description |
|------|-------------|
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
## Data Flow
1. **Input** → `controller/input.rs` handles key events and autocomplete
2. **Dispatch** → `app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
3. **LLM Stream** → `app/runtime/stream/mod.rs` parses SSE chunks into typed events
4. **Tool Execution** → `app/harness.rs` gates and runs tool calls via the `Tool` trait
5. **Rendering** → `view/` modules read `AppStateRest` and render via ratatui
## Key Files
| File | Purpose |
|------|---------|
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
| `src/app/state/rest.rs` | Single source-of-truth state struct |
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
| `src/app/harness.rs` | Tool harness with safety gating |
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
| `src/view/mod.rs` | TUI render pipeline |
+68
View File
@@ -0,0 +1,68 @@
# Backend Architecture
## Provider Layer
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
## IPC (Inter-Process Communication)
The daemon-client protocol in `src/ipc/`:
- **Transport**: Unix domain sockets
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
Flow:
```
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
│
└──StatePayload──▶ Client (render)
```
## Workflow Engine
Located in `src/app/workflow/`:
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
## MCP (Model Context Protocol)
`src/app/mcp/manager.rs` manages MCP client connections:
- Uses the `rmcp` crate for the MCP protocol
- Supports stdio-based transport (child process) and streamable HTTP
- Tool discovery via `list_tools()` and dynamic tool registration
## LSP Integration
`src/app/lsp/` provides Language Server Protocol support:
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
## Background Bash
`src/app/bgbash/` manages long-running shell jobs:
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
## Review System
`src/app/subagent/auto.rs` spawns background reviews:
- Quick review after every edit
- Background test generation
- Architecture review
- Security review
- All retry once on failure, escalate to blocking error if retry also fails
+89
View File
@@ -0,0 +1,89 @@
# Data Architecture
## State Model
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
```
AppStateRest
├── session: SessionRuntime (hive_mind state, convergence flag)
├── runtime: RuntimeState (mode, provider status)
├── chat: ChatState (messages, scroll)
├── input: InputState (text, cursor, autocomplete)
├── settings: Settings (provider, model, temperature, concise_output)
├── config: AppConfig (endpoints, credentials)
├── scroll: ScrollState (per-panel offset)
├── diff: DiffState (edit review)
├── tools: Vec with outputs
├── statusline, sidebar, etc.
└── toasts: pending notifications
```
**Mutation rules** (per CLAUDE.md):
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
- Read-only from every other module
- No generic update function — direct field mutation only
## Persistence
### SQLite Message Log (`src/model/msglog/`)
| File | Purpose |
|------|---------|
| `schema.rs` | Table definitions (messages, sessions) |
| `mod.rs` | CRUD operations |
| `query.rs` | Query helpers (search, filter) |
| `blobs.rs` | Large message blob storage |
| `summary.rs` | Conversation summary cache |
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
### Memory System (`src/model/memory.rs`)
File-based memory stored under `~/.claude/projects/<project>/memory/`:
- Each memory is one markdown file with frontmatter (name, description, type)
- Types: `user`, `feedback`, `project`, `reference`
- Memory index in MEMORY.md
- Export/import for lesson sharing
- PID-file session lock prevents concurrent access
### Settings & Config (`src/model/`)
| File | Purpose |
|------|---------|
| `settings.rs` | Serialized user preferences (provider, model, theme) |
| `app_config.rs` | Provider endpoints, API key resolution from env |
| `session.rs` | Current session metadata |
| `conversation.rs` | In-memory conversation state |
| `editlog.rs` | Append-only JSONL edit audit trail |
### Edit Log
`src/model/editlog.rs` records every file mutation:
```json
{"ts": 123, "tool": "edit", "path": "src/main.rs",
"reason": "fix bug", "content_sha256": "abc123",
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
```
Max 5000 entries held in memory before pruning oldest.
## Context Management (`src/app/runtime/context/`)
| Module | Purpose |
|--------|---------|
| `tokens.rs` | Token counting via `tiktoken-rs` |
| `window.rs` | Token window resolution (fit within model context) |
| `dedup.rs` | Deduplication of repeated tool outputs |
| `squash.rs` | Compression of large JSON tool results |
| `shaping.rs` | Message dropping when context exceeds limits |
## IPC Data Flow
```
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
│
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
```
+99
View File
@@ -0,0 +1,99 @@
# Dependencies
## Rust Crates (30+ direct)
### Core Framework
| Crate | Version | Purpose |
|-------|---------|---------|
| `ratatui` | 0.30.2 | TUI framework |
| `crossterm` | 0.29 | Terminal manipulation |
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
### HTTP & Networking
| Crate | Version | Purpose |
|-------|---------|---------|
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
| `webbrowser` | 1 | Open URLs in browser |
| `url` | 2 | URL parsing |
| `percent-encoding` | 2 | URL encoding |
### HTML/Markdown
| Crate | Version | Purpose |
|-------|---------|---------|
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
| `scraper` | 0.27.0 | HTML parsing/selecting |
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
### Serialization
| Crate | Version | Purpose |
|-------|---------|---------|
| `serde` | 1 | Serialization framework |
| `serde_json` | 1 | JSON serialization |
| `serde_yaml_ng` | 0.10 | YAML serialization |
### Storage & Files
| Crate | Version | Purpose |
|-------|---------|---------|
| `rusqlite` | 0.40 | SQLite (bundled) |
| `ignore` | 0.4 | `.gitignore`-aware file walking |
| `globset` | 0.4 | Glob pattern matching |
| `include_dir` | 0.7 | Embed directory contents in binary |
| `infer` | 0.19 | File type detection |
| `dirs` | 6 | Standard OS directories |
### Text & Search
| Crate | Version | Purpose |
|-------|---------|---------|
| `regex` | 1 | Regular expressions |
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
| `similar` | 3 | Diff computation |
| `syntect` | 5 | Syntax highlighting |
| `tiktoken-rs` | 0.12 | OpenAI token counting |
### Cryptography & Encoding
| Crate | Version | Purpose |
|-------|---------|---------|
| `base64` | 0.22 | Base64 encoding |
| `sha2` | 0.11 | SHA-256 hashing |
| `hex` | 0.4 | Hex encoding |
| `uuid` | 1 | UUID generation (v4, v5) |
| `libc` | 0.2 | Raw C FFI bindings |
### Error Handling & Logging
| Crate | Version | Purpose |
|-------|---------|---------|
| `anyhow` | 1 | Error handling |
| `tracing` | 0.1 | Structured logging |
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
| `chrono` | 0.4 | Date/time with serde |
### Other
| Crate | Version | Purpose |
|-------|---------|---------|
| `lsp-types` | 0.97 | LSP protocol types |
| `futures-util` | 0.3 | Async stream combinators |
## External Services
| Service | Purpose |
|---------|---------|
| **Anthropic API** | Primary LLM provider |
| **OpenAI API** | Alternative LLM provider (including OAuth) |
| **GitHub** | Release artifacts via semantic-release CI |
| **MCP Servers** | External tool servers (stdio or HTTP) |
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
## Build Configuration
### Compiler Lints (`.cargo/config.toml`)
All unused code, dead code, and deprecation warnings promoted to errors:
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
### Release Profile
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
### CI/CD
- **CI**: cargo build + test + clippy on every push
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
+79
View File
@@ -0,0 +1,79 @@
# Frontend (TUI) Architecture
## Render Pipeline
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
```
Timer tick
│
▼
main.rs: fn tui_loop()
│
├── controller/input.rs: handle_key() → action
├── app/runtime/actions/mod.rs: apply_action()
│ │
│ └── state mutates (AppStateRest)
│
└── view/mod.rs: build TUI layout
│
├── view/chat.rs: Chat transcript
├── view/sidebar.rs: Usage dashboard
├── view/status.rs: Status bar
├── view/markdown.rs: Message renderer
├── view/workflow.rs: Hive-mind progress
└── view/theme.rs: Tokyo Night palette
```
## Overlay System
16 overlays managed by `app/mode/`:
| Overlay | File | Purpose |
|---------|------|---------|
| Chat input | `mod.rs` | Main input bar with autocomplete |
| Bash | `bash.rs` | Interactive shell panel |
| Editor | `editor.rs` | Built-in file editor |
| Effort | `effort.rs` | LLM effort selector |
| Help | `help.rs` | Keybindings help |
| Key Input | `key_input.rs` | Custom key binding |
| Learning | `learning.rs` | Lesson viewer |
| Loading | `loading.rs` | Spinner overlay |
| MCP | `mcp.rs` | MCP server management |
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
| Rewind | `rewind.rs` | Message/history rewind |
| Settings | `settings.rs` | Settings panel |
| Todo | `todo.rs` | Task/TODO list |
| Workflow | (via view) | Workflow progress |
## Layout Structure
```
┌─────────────────────────────────────────────┐
│ Status Bar (view/status.rs) │
├──────────────────────┬──────────────────────┤
│ │ │
│ Chat Transcript │ Sidebar │
│ (view/chat.rs) │ (view/sidebar.rs) │
│ scrollable, │ tokens, status, │
│ inline-log style │ agent info │
│ │ │
├──────────────────────┴──────────────────────┤
│ Input Bar + Autocomplete dropdown │
│ (view/mod.rs) │
└─────────────────────────────────────────────┘
```
## Input Handling
`controller/input.rs`:
- Normal mode: keystrokes go to the active overlay
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
- Tab cycles autocomplete candidates
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
- Arrow keys scroll chat, sidebar, and other scrollable panels
## Theme
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
+14 -4
View File
@@ -1,4 +1,9 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)] #![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Global registry of running background bash jobs, and control operations //! Global registry of running background bash jobs, and control operations
//! (output polling, kill) exposed to the rest of the app. //! (output polling, kill) exposed to the rest of the app.
//! //!
@@ -10,7 +15,6 @@
//! Why: a single static map (rather than storing jobs in `AppStateRest`) //! Why: a single static map (rather than storing jobs in `AppStateRest`)
//! lets background jobs outlive the borrow of any particular state mutation //! lets background jobs outlive the borrow of any particular state mutation
//! and be looked up by id from tool calls issued at arbitrary points. //! and be looked up by id from tool calls issued at arbitrary points.
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::Mutex; use std::sync::Mutex;
use std::sync::OnceLock; use std::sync::OnceLock;
@@ -44,7 +48,11 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
while let Some(line) = job.try_read_line() { while let Some(line) = job.try_read_line() {
lines.push(line); lines.push(line);
} }
if lines.is_empty() { None } else { Some(lines) } if lines.is_empty() {
None
} else {
Some(lines)
}
} }
/// Terminate a running background bash job and remove it from the registry. /// Terminate a running background bash job and remove it from the registry.
@@ -58,7 +66,9 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
/// Return: `Ok(())` on success, `Err` if the lock is poisoned or no job /// Return: `Ok(())` on success, `Err` if the lock is poisoned or no job
/// with that id exists. /// with that id exists.
pub fn bash_kill(id: &str) -> anyhow::Result<()> { pub fn bash_kill(id: &str) -> anyhow::Result<()> {
let mut map = bash_jobs_map().lock().map_err(|e| anyhow::anyhow!("lock error: {e}"))?; let mut map = bash_jobs_map()
.lock()
.map_err(|e| anyhow::anyhow!("lock error: {e}"))?;
let job = map.remove(id); let job = map.remove(id);
match job { match job {
Some(job) => { Some(job) => {
+19 -13
View File
@@ -9,11 +9,10 @@
//! Why: running bash commands on a detached thread with a channel (rather //! Why: running bash commands on a detached thread with a channel (rather
//! than synchronously) lets the TUI stay responsive while long-running //! than synchronously) lets the TUI stay responsive while long-running
//! shell commands execute in the background. //! shell commands execute in the background.
use std::io::BufRead;
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use std::sync::mpsc; use std::sync::mpsc;
use std::thread; use std::thread;
use std::io::BufRead;
/// Maximum number of output lines buffered in memory per background job. /// Maximum number of output lines buffered in memory per background job.
/// Beyond this limit, old output is dropped to prevent OOM (CWE-770). /// Beyond this limit, old output is dropped to prevent OOM (CWE-770).
@@ -59,17 +58,23 @@ pub fn spawn_bash_job(command: String) -> BashJob {
// Spawn a named thread for easier debugging. If Builder::spawn fails // Spawn a named thread for easier debugging. If Builder::spawn fails
// (e.g. OS resource limit), fall back to unnameable thread::spawn. // (e.g. OS resource limit), fall back to unnameable thread::spawn.
let thread_name = format!("bgbash-{}", &thread_id[..8.min(thread_id.len())]); let thread_name = format!("bgbash-{}", &thread_id[..8.min(thread_id.len())]);
if thread::Builder::new().name(thread_name).spawn({ if thread::Builder::new()
// Clone everything the closure captures so we can also pass it .name(thread_name)
// to the fallback thread without moving. .spawn({
let cmd = cmd.clone(); // Clone everything the closure captures so we can also pass it
let output_tx = output_tx.clone(); // to the fallback thread without moving.
let pid_tx = pid_tx.clone(); let cmd = cmd.clone();
let id_for_log = id_for_log.clone(); let output_tx = output_tx.clone();
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log) let pid_tx = pid_tx.clone();
}).is_err() let id_for_log = id_for_log.clone();
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log)
})
.is_err()
{ {
tracing::warn!("[bgbash:{}] failed to spawn named thread, using unnamed fallback", id_for_log); tracing::warn!(
"[bgbash:{}] failed to spawn named thread, using unnamed fallback",
id_for_log
);
thread::spawn(move || { thread::spawn(move || {
spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log); spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log);
}); });
@@ -139,7 +144,8 @@ fn spawn_bash_thread_body(
if output_tx.try_send(line).is_err() { if output_tx.try_send(line).is_err() {
tracing::debug!( tracing::debug!(
"[bgbash:{}] output buffer full ({} lines), discarding remaining output", "[bgbash:{}] output buffer full ({} lines), discarding remaining output",
id_for_log, MAX_OUTPUT_LINES, id_for_log,
MAX_OUTPUT_LINES,
); );
break; break;
} }
-1
View File
@@ -1,5 +1,4 @@
//! Background bash: run shell commands off the main thread, poll their //! Background bash: run shell commands off the main thread, poll their
//! output non-blockingly, and terminate them on demand. //! output non-blockingly, and terminate them on demand.
pub mod control; pub mod control;
pub mod job; pub mod job;
+231 -158
View File
@@ -84,10 +84,17 @@ const ASSUMPTION_PATTERNS: &[&str] = &[
/// Network-exfiltration and credential-disclosure patterns for bash. /// Network-exfiltration and credential-disclosure patterns for bash.
const EXFIL_PATTERNS: &[&str] = &[ const EXFIL_PATTERNS: &[&str] = &[
"curl ", "wget ", "nc -e ", "ncat ", "/dev/tcp/", "curl ",
"base64 -d |", "base64 --decode |", "wget ",
"openssl s_client", "ssh -R ", "nc -e ",
"scp /", "rsync /", "ncat ",
"/dev/tcp/",
"base64 -d |",
"base64 --decode |",
"openssl s_client",
"ssh -R ",
"scp /",
"rsync /",
]; ];
/// Substrings of well-known credential / secret files that bash must not read. /// Substrings of well-known credential / secret files that bash must not read.
@@ -115,169 +122,63 @@ impl Harness {
/// Decide whether a tool call is allowed to execute. /// Decide whether a tool call is allowed to execute.
/// ///
/// Flow: ALL tools are gated (not just risky ones), closing the bypass /// Flow: ALL tools are gated (not just risky ones), closing the bypass
/// for MCP tools (which are never in the risky list). Basic path /// for MCP tools (which are never in the risky list). Delegates to
/// traversal and reason validation applies to any tool with a `path` /// smaller helper methods for each concern: path traversal, output
/// argument. Heavy content scanning (stub/denial/assumption/exfiltration) /// path validation, content scanning, bash safety, and reason checks.
/// only applies to risky tools. MCP tools (mcp__ prefix) are treated
/// as risky because their behaviour is unknown.
/// ///
/// Return: `Verdict::Allow` or `Verdict::Block(reason)`. /// Return: `Verdict::Allow` or `Verdict::Block(reason)`.
#[allow(clippy::too_many_lines, clippy::unnecessary_debug_formatting)]
pub fn gate_tool_call( pub fn gate_tool_call(
tool_name: &str, tool_name: &str,
args: &serde_json::Value, args: &serde_json::Value,
workspace_roots: &[&std::path::Path], workspace_roots: &[&std::path::Path],
) -> Verdict { ) -> Verdict {
let is_risky = crate::tool::tool_is_risky(tool_name); let is_risky = crate::tool::tool_is_risky(tool_name);
let is_mcp = tool_name.starts_with("mcp__"); let is_mcp = tool_name.starts_with("mcp__");
// ── Universal checks applied to EVERY tool ── // Universal checks applied to EVERY tool.
if let Some(v) = Self::check_path_traversal(args, workspace_roots) {
// Path traversal: check ANY tool that accepts a path argument, return v;
// not just write/edit/delete, so tools like read, MCP tools, }
// and future tools are also protected. if let Some(v) = Self::check_output_path(tool_name, args, workspace_roots) {
if let Some(path) = args.get("path").and_then(|v| v.as_str()) { return v;
if path.contains("..") {
return Verdict::Block(
"path traversal detected in 'path' argument".to_string(),
);
}
if !workspace_roots.is_empty() {
let abs_check = std::path::PathBuf::from(path);
if abs_check.is_absolute()
&& !workspace_roots.iter().any(|r| abs_check.starts_with(r))
{
return Verdict::Block(format!(
"absolute path '{path}' is outside all workspace roots"
));
}
}
} }
// Workspace-root validation for output path. // Non-risky, non-MCP tools pass after universal checks.
if let Some(out_path) = Self::find_output_path(tool_name, args) {
if !workspace_roots.is_empty()
&& !out_path.starts_with("/tmp")
&& !out_path.is_absolute()
{
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
if !allowed {
return Verdict::Block(format!(
"output path '{out_path:?}' is outside all workspace roots"
));
}
}
}
// ── Risky / MCP tool checks ──
// Non-risky, non-MCP tools (read, grep, glob, recall, etc.) are
// allowed after universal checks above.
if !is_risky && !is_mcp { if !is_risky && !is_mcp {
return Verdict::Allow; return Verdict::Allow;
} }
// File-mutating tools: write / edit / delete // File-mutating tools: require a meaningful reason.
if matches!(tool_name, "write" | "edit" | "delete") { if matches!(tool_name, "write" | "edit" | "delete") {
match Self::validate_reason(tool_name, args) { if let Err(msg) = Self::validate_reason(tool_name, args) {
Ok(()) => {} return Verdict::Block(msg);
Err(msg) => return Verdict::Block(msg),
} }
} }
// write / edit content must not contain stubs, denial language, or // write / edit content scanning for stub/denial/assumption patterns.
// assumption language. if let Some(v) = Self::check_content_safety(tool_name, args) {
if matches!(tool_name, "write" | "edit") { return v;
if let Some(content) = Self::extract_content(tool_name, args) {
if let Some(pat) = Self::first_match(&content, STUB_PATTERNS) {
return Verdict::Block(format!(
"content contains stub/placeholder pattern '{pat}'; \
production code must be fully implemented — \
replace the stub with a real implementation"
));
}
if let Some(pat) = Self::first_match(&content, DENIAL_PATTERNS) {
return Verdict::Block(format!(
"content contains denial/punt pattern '{pat}'; \
implement the change properly instead of skipping"
));
}
if let Some(pat) = Self::first_match(&content, ASSUMPTION_PATTERNS) {
return Verdict::Block(format!(
"content contains assumption pattern '{pat}'; \
verify against data/tests instead of guessing"
));
}
}
} }
// Bash: destructive patterns, exfiltration (ALL commands checked, // Bash-specific destructive / exfiltration checks.
// no safe-command whitelist), sensitive-path reads. if let Some(v) = Self::check_bash_safety(args) {
if tool_name == "bash" { return v;
let cmd = args.get("command").and_then(|v| v.as_str()).unwrap_or(""); }
if cmd.contains("..") {
return Verdict::Block( // git_operator: require a non-trivial reason.
"path traversal detected in bash command".to_string(), if tool_name == "git_operator" && !Self::has_valid_reason(args, MIN_REASON_LEN) {
); if args.get("reason").and_then(|v| v.as_str()).is_some() {
}
// Exfiltration patterns are checked on EVERY bash command,
// regardless of prefix. The safe-command whitelist was removed
// because it could be bypassed with command chaining.
for pat in EXFIL_PATTERNS {
if cmd.contains(pat) {
return Verdict::Block(format!(
"potential data-exfiltration command blocked (matched '{pat}')"
));
}
}
for pat in SENSITIVE_PATH_PATTERNS {
if cmd.contains(pat) {
return Verdict::Block(format!(
"refused to read/write sensitive path '{pat}'"
));
}
}
let dangerous_patterns = [
"rm -rf /", "rm -rf --no-preserve-root",
"rm -rf ~", "rm -fr /", "mkfs.", "dd if=",
":(){", "> /dev/sda", "chmod -R 000 /",
"shutdown ", "poweroff ", "reboot ", "halt ",
];
for pat in &dangerous_patterns {
if cmd.contains(pat) {
return Verdict::Block(format!(
"destructive command pattern blocked: {pat}"
));
}
}
// Also scan heredocs / -c / inline content for stub/denial
// language (e.g. `bash -c 'echo todo!()'`)
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
return Verdict::Block(format!( return Verdict::Block(format!(
"bash command contains stub pattern '{pat}'" "git_operator requires a non-trivial 'reason' \
(>= {MIN_REASON_LEN} chars) explaining the operation"
)); ));
} }
return Verdict::Block(
"git_operator requires a 'reason' argument explaining the operation".to_string(),
);
} }
// git_operator: require a non-trivial reason as well. // MCP tools: require a reason when they take meaningful arguments.
if tool_name == "git_operator" {
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
if reason.trim().len() < MIN_REASON_LEN {
return Verdict::Block(format!(
"git_operator requires a non-trivial 'reason' \
(>= {MIN_REASON_LEN} chars) explaining the operation"
));
}
} else {
return Verdict::Block(
"git_operator requires a 'reason' argument explaining the operation"
.to_string(),
);
}
}
// MCP tools: unknown behaviour — require a reason if they take
// arguments, to discourage lazy invocations.
if is_mcp { if is_mcp {
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) { if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
if reason.trim().len() < MIN_REASON_LEN { if reason.trim().len() < MIN_REASON_LEN {
@@ -287,7 +188,6 @@ impl Harness {
)); ));
} }
} else if args.as_object().is_some_and(|m| !m.is_empty()) { } else if args.as_object().is_some_and(|m| !m.is_empty()) {
// Only require reason when there are meaningful arguments
return Verdict::Block(format!( return Verdict::Block(format!(
"MCP tool '{tool_name}' requires a 'reason' argument \ "MCP tool '{tool_name}' requires a 'reason' argument \
explaining the operation" explaining the operation"
@@ -298,6 +198,163 @@ impl Harness {
Verdict::Allow Verdict::Allow
} }
/// Check for path traversal in the `path` argument and verify it stays
/// within workspace roots.
///
/// Flow: reject any path containing `..` → if workspace roots are set,
/// reject absolute paths outside every root.
///
/// Return: `Some(Verdict::Block)` on violation, `None` if the check
/// passes or the tool has no `path` argument.
fn check_path_traversal(
args: &serde_json::Value,
workspace_roots: &[&std::path::Path],
) -> Option<Verdict> {
let path = args.get("path")?.as_str()?;
if path.contains("..") {
return Some(Verdict::Block(
"path traversal detected in 'path' argument".to_string(),
));
}
if !workspace_roots.is_empty() {
let abs_check = std::path::PathBuf::from(path);
if abs_check.is_absolute() && !workspace_roots.iter().any(|r| abs_check.starts_with(r))
{
return Some(Verdict::Block(format!(
"absolute path '{path}' is outside all workspace roots"
)));
}
}
None
}
/// Verify that a tool's output path (if any) stays within workspace roots.
///
/// Flow: if `find_output_path` yields a path, reject it unless it's
/// under `/tmp`, already absolute, or within a workspace root.
///
/// Return: `Some(Verdict::Block)` on violation, `None` otherwise.
fn check_output_path(
tool_name: &str,
args: &serde_json::Value,
workspace_roots: &[&std::path::Path],
) -> Option<Verdict> {
let out_path = Self::find_output_path(tool_name, args)?;
if !workspace_roots.is_empty() && !out_path.starts_with("/tmp") && !out_path.is_absolute() {
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
if !allowed {
return Some(Verdict::Block(format!(
"output path '{}' is outside all workspace roots",
out_path.display(),
)));
}
}
None
}
/// Check write/edit content for stub, denial, and assumption patterns.
///
/// Return: `Some(Verdict::Block)` with a description of the first
/// matched pattern, `None` if the content is clean or not applicable.
fn check_content_safety(tool_name: &str, args: &serde_json::Value) -> Option<Verdict> {
if !matches!(tool_name, "write" | "edit") {
return None;
}
let content = Self::extract_content(tool_name, args)?;
for (patterns, msg_prefix) in [
(&STUB_PATTERNS, "stub/placeholder"),
(&DENIAL_PATTERNS, "denial/punt"),
(&ASSUMPTION_PATTERNS, "assumption"),
] {
if let Some(pat) = Self::first_match(&content, patterns) {
let msg = match msg_prefix {
"stub/placeholder" => format!(
"content contains stub/placeholder pattern '{pat}'; \
production code must be fully implemented — \
replace the stub with a real implementation"
),
"denial/punt" => format!(
"content contains denial/punt pattern '{pat}'; \
implement the change properly instead of skipping"
),
_ => format!(
"content contains assumption pattern '{pat}'; \
verify against data/tests instead of guessing"
),
};
return Some(Verdict::Block(msg));
}
}
None
}
/// Check bash commands for path traversal, exfiltration, sensitive
/// path reads, destructive patterns, and stub language.
///
/// Flow: extract the `command` argument → check each category in
/// sequence, returning the first violation found.
///
/// Return: `Some(Verdict::Block)` on any violation, `None` if the
/// tool is not bash or the command is safe.
fn check_bash_safety(args: &serde_json::Value) -> Option<Verdict> {
let cmd = args.get("command")?.as_str()?;
if cmd.contains("..") {
return Some(Verdict::Block(
"path traversal detected in bash command".to_string(),
));
}
for pat in EXFIL_PATTERNS {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"potential data-exfiltration command blocked (matched '{pat}')"
)));
}
}
for pat in SENSITIVE_PATH_PATTERNS {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"refused to read/write sensitive path '{pat}'"
)));
}
}
let dangerous_patterns = [
"rm -rf /",
"rm -rf --no-preserve-root",
"rm -rf ~",
"rm -fr /",
"mkfs.",
"dd if=",
":(){",
"> /dev/sda",
"chmod -R 000 /",
"shutdown ",
"poweroff ",
"reboot ",
"halt ",
];
for pat in &dangerous_patterns {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"destructive command pattern blocked: {pat}"
)));
}
}
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
return Some(Verdict::Block(format!(
"bash command contains stub pattern '{pat}'"
)));
}
None
}
/// Check whether the given `args` contain a non-trivial `reason`
/// argument meeting the minimum length requirement.
fn has_valid_reason(args: &serde_json::Value, min_len: usize) -> bool {
args.get("reason")
.and_then(|v| v.as_str())
.is_some_and(|r| r.trim().len() >= min_len)
}
/// Validate the `reason` argument for a mutating tool. /// Validate the `reason` argument for a mutating tool.
/// ///
/// Flow: require the field to exist and be a non-empty string ≥ /// Flow: require the field to exist and be a non-empty string ≥
@@ -317,17 +374,13 @@ impl Harness {
Some(v) => match v.as_str() { Some(v) => match v.as_str() {
Some(s) => s, Some(s) => s,
None => { None => {
return Err(format!( return Err(format!("{tool_name} 'reason' must be a string"));
"{tool_name} 'reason' must be a string"
));
} }
}, },
}; };
let trimmed = reason.trim(); let trimmed = reason.trim();
if trimmed.is_empty() { if trimmed.is_empty() {
return Err(format!( return Err(format!("{tool_name} 'reason' must not be empty"));
"{tool_name} 'reason' must not be empty"
));
} }
if trimmed.len() < MIN_REASON_LEN { if trimmed.len() < MIN_REASON_LEN {
return Err(format!( return Err(format!(
@@ -339,9 +392,20 @@ impl Harness {
// Reject generic non-answers // Reject generic non-answers
let lower = trimmed.to_lowercase(); let lower = trimmed.to_lowercase();
let non_answers = [ let non_answers = [
"fix", "update", "change", "edit", "modify", "fix",
"implement", "add", "remove", "delete", "update",
"make it work", "make work", "test", "wip", "tbd", "change",
"edit",
"modify",
"implement",
"add",
"remove",
"delete",
"make it work",
"make work",
"test",
"wip",
"tbd",
]; ];
if non_answers.iter().any(|n| lower == *n) { if non_answers.iter().any(|n| lower == *n) {
return Err(format!( return Err(format!(
@@ -356,7 +420,10 @@ impl Harness {
/// Extract the textual content of a write/edit call, if any. /// Extract the textual content of a write/edit call, if any.
fn extract_content(tool_name: &str, args: &serde_json::Value) -> Option<String> { fn extract_content(tool_name: &str, args: &serde_json::Value) -> Option<String> {
match tool_name { match tool_name {
"write" => args.get("content").and_then(|v| v.as_str()).map(String::from), "write" => args
.get("content")
.and_then(|v| v.as_str())
.map(String::from),
"edit" => { "edit" => {
let old = args.get("old").and_then(|v| v.as_str()).unwrap_or(""); let old = args.get("old").and_then(|v| v.as_str()).unwrap_or("");
let new = args.get("new").and_then(|v| v.as_str()).unwrap_or(""); let new = args.get("new").and_then(|v| v.as_str()).unwrap_or("");
@@ -377,9 +444,10 @@ impl Harness {
/// Extract a candidate output path from a tool call, if one exists. /// Extract a candidate output path from a tool call, if one exists.
fn find_output_path(tool_name: &str, args: &serde_json::Value) -> Option<std::path::PathBuf> { fn find_output_path(tool_name: &str, args: &serde_json::Value) -> Option<std::path::PathBuf> {
match tool_name { match tool_name {
"write" | "edit" | "delete" | "read" => { "write" | "edit" | "delete" | "read" => args
args.get("path").and_then(|v| v.as_str()).map(std::path::PathBuf::from) .get("path")
} .and_then(|v| v.as_str())
.map(std::path::PathBuf::from),
"bash" => { "bash" => {
let cmd = args.get("command").and_then(|v| v.as_str())?; let cmd = args.get("command").and_then(|v| v.as_str())?;
let lower = cmd.to_lowercase(); let lower = cmd.to_lowercase();
@@ -397,7 +465,6 @@ impl Harness {
_ => None, _ => None,
} }
} }
} }
impl Default for Harness { impl Default for Harness {
@@ -418,7 +485,10 @@ mod tests {
return match verdict.to_lowercase().as_str() { return match verdict.to_lowercase().as_str() {
"allow" => Some(Verdict::Allow), "allow" => Some(Verdict::Allow),
"block" => Some(Verdict::Block( "block" => Some(Verdict::Block(
v.get("reason").and_then(|r| r.as_str()).unwrap_or("blocked").to_string() v.get("reason")
.and_then(|r| r.as_str())
.unwrap_or("blocked")
.to_string(),
)), )),
_ => None, _ => None,
}; };
@@ -430,7 +500,11 @@ mod tests {
return Some(Verdict::Allow); return Some(Verdict::Allow);
} }
if l.starts_with("verdict: block") { if l.starts_with("verdict: block") {
let reason = line.split_once(':').map_or("blocked", |x| x.1).trim().to_string(); let reason = line
.split_once(':')
.map_or("blocked", |x| x.1)
.trim()
.to_string();
return Some(Verdict::Block(reason)); return Some(Verdict::Block(reason));
} }
} }
@@ -450,7 +524,6 @@ mod tests {
assert_eq!(result, Verdict::Allow); assert_eq!(result, Verdict::Allow);
} }
#[test] #[test]
fn test_parse_verdict_json_allow() { fn test_parse_verdict_json_allow() {
let v = parse_verdict(r#"{"verdict": "allow"}"#); let v = parse_verdict(r#"{"verdict": "allow"}"#);
+120 -119
View File
@@ -47,13 +47,20 @@ impl LspClient {
cmd.stdout(Stdio::piped()); cmd.stdout(Stdio::piped());
cmd.stderr(Stdio::piped()); cmd.stderr(Stdio::piped());
let mut child = cmd.spawn() let mut child = cmd
.spawn()
.map_err(|e| anyhow::anyhow!("failed to spawn LSP server '{command}': {e}"))?; .map_err(|e| anyhow::anyhow!("failed to spawn LSP server '{command}': {e}"))?;
let stdin = child.stdin.take() let stdin = child
.stdin
.take()
.ok_or_else(|| anyhow::anyhow!("failed to capture stdin for LSP server"))?; .ok_or_else(|| anyhow::anyhow!("failed to capture stdin for LSP server"))?;
let stdout = BufReader::new(child.stdout.take() let stdout = BufReader::new(
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?); child
.stdout
.take()
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?,
);
let mut client = LspClient { let mut client = LspClient {
stdin, stdin,
@@ -106,7 +113,11 @@ impl LspClient {
} }
}); });
let result = client.call_with_timeout("initialize", &init_params, Duration::from_millis(LSP_INIT_TIMEOUT_MS))?; let result = client.call_with_timeout(
"initialize",
&init_params,
Duration::from_millis(LSP_INIT_TIMEOUT_MS),
)?;
client.server_capabilities = result.get("capabilities").cloned().unwrap_or_default(); client.server_capabilities = result.get("capabilities").cloned().unwrap_or_default();
client.notify("initialized", &json!({}))?; client.notify("initialized", &json!({}))?;
@@ -122,7 +133,12 @@ impl LspClient {
self.call_with_timeout(method, params, Duration::from_millis(LSP_CALL_TIMEOUT_MS)) self.call_with_timeout(method, params, Duration::from_millis(LSP_CALL_TIMEOUT_MS))
} }
fn call_with_timeout(&mut self, method: &str, params: &Value, timeout: Duration) -> anyhow::Result<Value> { fn call_with_timeout(
&mut self,
method: &str,
params: &Value,
timeout: Duration,
) -> anyhow::Result<Value> {
self.next_id += 1; self.next_id += 1;
let id = self.next_id; let id = self.next_id;
let req = json!({ let req = json!({
@@ -148,11 +164,14 @@ impl LspClient {
let body = serde_json::to_string(msg) let body = serde_json::to_string(msg)
.map_err(|e| anyhow::anyhow!("failed to serialize LSP message: {e}"))?; .map_err(|e| anyhow::anyhow!("failed to serialize LSP message: {e}"))?;
let header = format!("Content-Length: {}\r\n\r\n", body.len()); let header = format!("Content-Length: {}\r\n\r\n", body.len());
self.stdin.write_all(header.as_bytes()) self.stdin
.write_all(header.as_bytes())
.map_err(|e| anyhow::anyhow!("failed to write LSP frame header: {e}"))?; .map_err(|e| anyhow::anyhow!("failed to write LSP frame header: {e}"))?;
self.stdin.write_all(body.as_bytes()) self.stdin
.write_all(body.as_bytes())
.map_err(|e| anyhow::anyhow!("failed to write LSP frame body: {e}"))?; .map_err(|e| anyhow::anyhow!("failed to write LSP frame body: {e}"))?;
self.stdin.flush() self.stdin
.flush()
.map_err(|e| anyhow::anyhow!("failed to flush LSP stdin: {e}"))?; .map_err(|e| anyhow::anyhow!("failed to flush LSP stdin: {e}"))?;
Ok(()) Ok(())
} }
@@ -166,8 +185,14 @@ impl LspClient {
let frame = self.read_frame()?; let frame = self.read_frame()?;
if frame.get("id") == Some(&json!(expected_id)) { if frame.get("id") == Some(&json!(expected_id)) {
if let Some(err) = frame.get("error") { if let Some(err) = frame.get("error") {
let code = err.get("code").and_then(serde_json::Value::as_i64).unwrap_or(0); let code = err
let msg = err.get("message").and_then(|m| m.as_str()).unwrap_or("unknown error"); .get("code")
.and_then(serde_json::Value::as_i64)
.unwrap_or(0);
let msg = err
.get("message")
.and_then(|m| m.as_str())
.unwrap_or("unknown error");
anyhow::bail!("LSP error {code}: {msg}"); anyhow::bail!("LSP error {code}: {msg}");
} }
return Ok(frame.get("result").cloned().unwrap_or(Value::Null)); return Ok(frame.get("result").cloned().unwrap_or(Value::Null));
@@ -205,8 +230,9 @@ impl LspClient {
// Cap Content-Length at 64 MiB to prevent OOM from a // Cap Content-Length at 64 MiB to prevent OOM from a
// malicious or misconfigured LSP server (CWE-400). // malicious or misconfigured LSP server (CWE-400).
const MAX_CONTENT_LENGTH: usize = 64 * 1024 * 1024; const MAX_CONTENT_LENGTH: usize = 64 * 1024 * 1024;
let length: usize = len_str.trim().parse::<usize>() let length: usize = len_str.trim().parse::<usize>().map_err(|e| {
.map_err(|e| anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e))?; anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e)
})?;
if length > MAX_CONTENT_LENGTH { if length > MAX_CONTENT_LENGTH {
anyhow::bail!( anyhow::bail!(
"Content-Length {length} exceeds maximum allowed size of {MAX_CONTENT_LENGTH} bytes", "Content-Length {length} exceeds maximum allowed size of {MAX_CONTENT_LENGTH} bytes",
@@ -220,7 +246,8 @@ impl LspClient {
.ok_or_else(|| anyhow::anyhow!("missing Content-Length header in LSP response"))?; .ok_or_else(|| anyhow::anyhow!("missing Content-Length header in LSP response"))?;
let mut body = vec![0u8; length]; let mut body = vec![0u8; length];
self.stdout.read_exact(&mut body) self.stdout
.read_exact(&mut body)
.map_err(|e| anyhow::anyhow!("failed to read LSP body ({length} bytes): {e}"))?; .map_err(|e| anyhow::anyhow!("failed to read LSP body ({length} bytes): {e}"))?;
let json_str = String::from_utf8(body) let json_str = String::from_utf8(body)
@@ -230,74 +257,98 @@ impl LspClient {
.map_err(|e| anyhow::anyhow!("invalid JSON in LSP response: {e}")) .map_err(|e| anyhow::anyhow!("invalid JSON in LSP response: {e}"))
} }
pub fn did_open(&mut self, uri: &str, language_id: &str, version: i32, text: &str) -> anyhow::Result<()> { pub fn did_open(
self.notify("textDocument/didOpen", &json!({ &mut self,
"textDocument": { uri: &str,
"uri": uri, language_id: &str,
"languageId": language_id, version: i32,
"version": version, text: &str,
"text": text ) -> anyhow::Result<()> {
} self.notify(
})) "textDocument/didOpen",
&json!({
"textDocument": {
"uri": uri,
"languageId": language_id,
"version": version,
"text": text
}
}),
)
} }
#[allow(dead_code)]
pub fn did_change(&mut self, uri: &str, version: i32, text: &str) -> anyhow::Result<()> { pub fn did_change(&mut self, uri: &str, version: i32, text: &str) -> anyhow::Result<()> {
self.notify("textDocument/didChange", &json!({ self.notify(
"textDocument": { "textDocument/didChange",
"uri": uri, &json!({
"version": version "textDocument": {
}, "uri": uri,
"contentChanges": [{ "version": version
"text": text },
}] "contentChanges": [{
})) "text": text
}]
}),
)
} }
pub fn did_close(&mut self, uri: &str) -> anyhow::Result<()> { pub fn did_close(&mut self, uri: &str) -> anyhow::Result<()> {
self.notify("textDocument/didClose", &json!({ self.notify(
"textDocument": { "textDocument/didClose",
"uri": uri &json!({
} "textDocument": {
})) "uri": uri
}
}),
)
} }
pub fn hover(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> { pub fn hover(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/hover", &json!({ self.call(
"textDocument": { "uri": uri }, "textDocument/hover",
"position": { "line": line, "character": character } &json!({
})) "textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
} }
pub fn completion(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> { pub fn completion(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/completion", &json!({ self.call(
"textDocument": { "uri": uri }, "textDocument/completion",
"position": { "line": line, "character": character } &json!({
})) "textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
} }
pub fn goto_definition(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> { pub fn goto_definition(
self.call("textDocument/definition", &json!({ &mut self,
"textDocument": { "uri": uri }, uri: &str,
"position": { "line": line, "character": character } line: u32,
})) character: u32,
) -> anyhow::Result<Value> {
self.call(
"textDocument/definition",
&json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
} }
pub fn references(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> { pub fn references(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/references", &json!({ self.call(
"textDocument": { "uri": uri }, "textDocument/references",
"position": { "line": line, "character": character }, &json!({
"context": { "textDocument": { "uri": uri },
"includeDeclaration": true "position": { "line": line, "character": character },
} "context": {
})) "includeDeclaration": true
} }
}),
#[allow(dead_code)] )
pub fn document_symbols(&mut self, uri: &str) -> anyhow::Result<Value> {
self.call("textDocument/documentSymbol", &json!({
"textDocument": { "uri": uri }
}))
} }
pub fn collect_diagnostics( pub fn collect_diagnostics(
@@ -313,64 +364,14 @@ impl LspClient {
); );
self.did_close(uri)?; self.did_close(uri)?;
match result { match result {
Ok(params) => Ok(params.get("diagnostics").cloned().unwrap_or_else(|| json!([]))), Ok(params) => Ok(params
.get("diagnostics")
.cloned()
.unwrap_or_else(|| json!([]))),
Err(e) => Err(e), Err(e) => Err(e),
} }
} }
/// Health-check the LSP server.
///
/// Sends a `textDocument/documentSymbol` request on a dummy URI with a
/// 2-second timeout. Returns `true` if the server responds at all —
/// including with an error response such as "file not found", which
/// still proves the process is up and the JSON-RPC channel is live.
/// Returns `false` on timeout, EOF, or any read/write error.
///
/// Flow: build request → `send_frame` → poll frames until id matches
/// (alive) or deadline/read error fires (dead).
#[allow(dead_code)]
pub fn is_alive(&mut self) -> bool {
self.next_id += 1;
let id = self.next_id;
let req = json!({
"jsonrpc": "2.0",
"id": id,
"method": "textDocument/documentSymbol",
"params": {
"textDocument": { "uri": "file:///__zesdex_lsp_health_check__.txt" }
}
});
if self.send_frame(&req).is_err() {
return false;
}
let timeout = Duration::from_secs(2);
let deadline = Instant::now() + timeout;
loop {
if Instant::now() > deadline {
return false;
}
match self.read_frame() {
Ok(frame) => {
if frame.get("id") == Some(&json!(id)) {
return true;
}
// Skip unrelated notifications/responses on the same channel.
}
Err(_) => return false,
}
}
}
/// Send the LSP `exit` notification to request graceful shutdown.
///
/// Per the LSP spec, `exit` is a notification — the server is expected
/// to terminate after receiving it without sending a response. We do
/// not block on any reply.
#[allow(dead_code)]
pub fn exit(&mut self) -> anyhow::Result<()> {
self.notify("exit", &json!({}))
}
pub fn shutdown(&mut self) { pub fn shutdown(&mut self) {
let _ = self.call_with_timeout("shutdown", &json!({}), Duration::from_secs(5)); let _ = self.call_with_timeout("shutdown", &json!({}), Duration::from_secs(5));
let _ = self.notify("exit", &json!({})); let _ = self.notify("exit", &json!({}));
+54 -113
View File
@@ -13,12 +13,6 @@ pub use client::{path_to_lsp_uri, LspClient};
/// to issue LSP requests from threads or async tasks. /// to issue LSP requests from threads or async tasks.
#[derive(Clone)] #[derive(Clone)]
pub struct LspServer { pub struct LspServer {
#[allow(dead_code)]
pub name: String,
#[allow(dead_code)]
pub command: String,
#[allow(dead_code)]
pub args: Vec<String>,
pub language_id: String, pub language_id: String,
pub client: Arc<Mutex<LspClient>>, pub client: Arc<Mutex<LspClient>>,
} }
@@ -38,12 +32,12 @@ pub struct OpenDoc {
/// ///
/// Flow: caller calls `connect*` -> client spawned -> entry pushed to /// Flow: caller calls `connect*` -> client spawned -> entry pushed to
/// `servers` -> `extension_registry` is populated by `register_extensions`. /// `servers` -> `extension_registry` is populated by `register_extensions`.
/// File edits route through `find_server_for_path` / `find_server_for_extension` /// File edits route through `extension_registry` and are dispatched as
/// and are dispatched as `didOpen` / `didChange` notifications. /// `didOpen` / `didChange` notifications.
#[derive(Clone)] #[derive(Clone)]
pub struct LspManager { pub struct LspManager {
pub servers: Vec<LspServer>, pub servers: Vec<LspServer>,
/// Maps file extension (".rs", ".ts", ...) -> server name. /// Maps file extension (".rs", ".ts", ...) -> language id.
pub extension_registry: HashMap<String, String>, pub extension_registry: HashMap<String, String>,
/// Maps document URI -> tracked open document state. /// Maps document URI -> tracked open document state.
pub open_files: HashMap<String, OpenDoc>, pub open_files: HashMap<String, OpenDoc>,
@@ -59,112 +53,73 @@ impl LspManager {
} }
} }
/// Spawn an LSP server and register it under `name`. /// Spawn an LSP server and register it under `language_id`.
/// ///
/// Fails if a server with the same name is already connected. /// Fails if a server with the same `language_id` is already connected.
pub fn connect( pub fn connect(
&mut self, &mut self,
name: &str,
command: &str, command: &str,
args: &[String], args: &[String],
language_id: &str, language_id: &str,
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
if self.servers.iter().any(|s| s.name == name) { if self.servers.iter().any(|s| s.language_id == language_id) {
anyhow::bail!("LSP server '{name}' is already connected"); anyhow::bail!("LSP server for language '{language_id}' is already connected");
} }
let client = LspClient::spawn(command, args)?; let client = LspClient::spawn(command, args)?;
self.servers.push(LspServer { self.servers.push(LspServer {
name: name.to_string(),
command: command.to_string(),
args: args.to_vec(),
language_id: language_id.to_string(), language_id: language_id.to_string(),
client: Arc::new(Mutex::new(client)), client: Arc::new(Mutex::new(client)),
}); });
Ok(()) Ok(())
} }
/// Look up a connected server by name and return a reference to its entry.
#[allow(dead_code)]
pub fn find_server(&self, name: &str) -> Option<&LspServer> {
self.servers.iter().find(|s| s.name == name)
}
/// Return a clone of the `Arc<Mutex<LspClient>>` for a connected server. /// Return a clone of the `Arc<Mutex<LspClient>>` for a connected server.
/// ///
/// Cloning the `Arc` lets callers issue requests without holding a /// Cloning the `Arc` lets callers issue requests without holding a
/// borrow on the manager. /// borrow on the manager.
pub fn get_client(&self, name: &str) -> Option<Arc<Mutex<LspClient>>> { pub fn get_client(&self, language_id: &str) -> Option<Arc<Mutex<LspClient>>> {
self.servers.iter().find(|s| s.name == name).map(|s| s.client.clone()) self.servers
.iter()
.find(|s| s.language_id == language_id)
.map(|s| s.client.clone())
} }
/// Shut down and remove a server by name. Returns true if it existed. /// Shut down and remove a server by language. Returns true if it existed.
pub fn disconnect(&mut self, name: &str) -> bool { pub fn disconnect(&mut self, language_id: &str) -> bool {
if let Some(server) = self.servers.iter().find(|s| s.name == name) { if let Some(server) = self.servers.iter().find(|s| s.language_id == language_id) {
if let Ok(mut client) = server.client.lock() { if let Ok(mut client) = server.client.lock() {
client.shutdown(); client.shutdown();
} }
} }
let len = self.servers.len(); let len = self.servers.len();
self.servers.retain(|s| s.name != name); self.servers.retain(|s| s.language_id != language_id);
self.servers.len() < len self.servers.len() < len
} }
/// Return the language id (e.g. "rust") registered for `name`. /// Return the language id (e.g. "rust") registered for `language_id`.
pub fn get_language_id(&self, name: &str) -> Option<String> { pub fn get_language_id(&self, language_id: &str) -> Option<String> {
self.servers.iter().find(|s| s.name == name).map(|s| s.language_id.clone()) self.servers
} .iter()
.find(|s| s.language_id == language_id)
/// Resolve an extension (".rs", ".ts", ...) to its server's client. .map(|s| s.language_id.clone())
///
/// Flow: lookup `extension_registry` -> resolve server name -> clone client.
/// Returns `None` if no server has been registered for `ext`.
#[allow(dead_code)]
pub fn find_server_for_extension(&self, ext: &str) -> Option<Arc<Mutex<LspClient>>> {
self.extension_registry
.get(ext)
.and_then(|name| self.get_client(name))
}
/// Resolve a file path to its server's client by extension.
///
/// Flow: extract the extension from `path` -> delegate to
/// `find_server_for_extension`. Files without an extension or with
/// an unmapped extension return `None`.
#[allow(dead_code)]
pub fn find_server_for_path(&self, path: &Path) -> Option<Arc<Mutex<LspClient>>> {
path.extension()
.and_then(|e| e.to_str())
.map(|s| format!(".{s}"))
.and_then(|ext| self.find_server_for_extension(&ext))
} }
/// Register a set of file extensions for an already-connected server. /// Register a set of file extensions for an already-connected server.
/// ///
/// Flow: for each `ext`, write `server_name` into `extension_registry`. /// Flow: for each `ext`, write `language_id` into `extension_registry`.
/// Re-registration overwrites the previous target. Unknown server /// Re-registration overwrites the previous target. Unknown language IDs
/// names are accepted at this layer — caller must ensure `server_name` /// are accepted at this layer — caller must ensure a server for
/// is connected or will be connected later. /// `language_id` is connected or will be connected later.
pub fn register_extensions(&mut self, server_name: &str, extensions: &[&str]) { pub fn register_extensions(&mut self, language_id: &str, extensions: &[&str]) {
for ext in extensions { for ext in extensions {
self.extension_registry.insert(ext.to_string(), server_name.to_string()); self.extension_registry
.insert(ext.to_string(), language_id.to_string());
} }
} }
/// Return the registered server name for a given language id.
///
/// Flow: scan `servers` for the first entry whose `language_id` matches.
/// Used when callers have a language hint rather than a file path.
#[allow(dead_code)]
pub fn get_server_name(&self, language: &str) -> Option<String> {
self.servers
.iter()
.find(|s| s.language_id == language)
.map(|s| s.name.clone())
}
/// Notify the relevant LSP server that a file's contents have changed. /// Notify the relevant LSP server that a file's contents have changed.
/// ///
/// Flow: resolve server by extension -> read file contents -> /// Flow: resolve language by extension from the registry -> read file contents ->
/// either send `didOpen` (first time) or `didChange` (already tracked) /// either send `didOpen` (first time) or `didChange` (already tracked)
/// -> update `open_files` with the new version. /// -> update `open_files` with the new version.
/// ///
@@ -172,13 +127,20 @@ impl LspManager {
/// error) are logged with `tracing::warn!` rather than propagated, /// error) are logged with `tracing::warn!` rather than propagated,
/// so a stale notification cannot abort the calling flow. /// so a stale notification cannot abort the calling flow.
pub fn did_change_file(&mut self, path: &Path) { pub fn did_change_file(&mut self, path: &Path) {
let Some(ext) = path.extension().and_then(|e| e.to_str()).map(|s| format!(".{s}")) else { let Some(ext) = path
.extension()
.and_then(|e| e.to_str())
.map(|s| format!(".{s}"))
else {
tracing::warn!("did_change_file: path has no extension: {:?}", path); tracing::warn!("did_change_file: path has no extension: {:?}", path);
return; return;
}; };
let server_name = if let Some(name) = self.extension_registry.get(&ext) { name.clone() } else { let Some(language_id) = self.extension_registry.get(&ext).cloned() else {
tracing::warn!("did_change_file: no LSP server registered for extension '{}'", ext); tracing::warn!(
"did_change_file: no LSP server registered for extension '{}'",
ext
);
return; return;
}; };
@@ -192,12 +154,8 @@ impl LspManager {
} }
}; };
let language_id = self let Some(client) = self.get_client(&language_id) else {
.get_language_id(&server_name) tracing::warn!("did_change_file: no client for language '{}'", language_id);
.unwrap_or_else(|| "plaintext".to_string());
let Some(client) = self.get_client(&server_name) else {
tracing::warn!("did_change_file: server '{}' has no client", server_name);
return; return;
}; };
@@ -210,7 +168,11 @@ impl LspManager {
let mut client = match client.lock() { let mut client = match client.lock() {
Ok(c) => c, Ok(c) => c,
Err(e) => { Err(e) => {
tracing::warn!("did_change_file: client mutex poisoned for '{}': {}", server_name, e); tracing::warn!(
"did_change_file: client mutex poisoned for '{}': {}",
language_id,
e
);
return; return;
} }
}; };
@@ -224,7 +186,7 @@ impl LspManager {
if let Err(e) = send_result { if let Err(e) = send_result {
tracing::warn!( tracing::warn!(
"did_change_file: failed to notify '{}' for {}: {}", "did_change_file: failed to notify '{}' for {}: {}",
server_name, language_id,
uri, uri,
e e
); );
@@ -238,24 +200,6 @@ impl LspManager {
version: next_version, version: next_version,
}, },
); );
}
/// Record that `server_name` has an open document at `uri`.
///
/// Flow: insert/overwrite the `OpenDoc` entry in `open_files`.
/// Does not contact the LSP server — pure local bookkeeping.
#[allow(dead_code)]
pub fn track_open_doc(&mut self, server_name: &str, uri: &str, language: &str, version: i32) {
// server_name retained for future routing extensions; not stored today.
let _ = server_name;
self.open_files.insert(
uri.to_string(),
OpenDoc {
language: language.to_string(),
version,
},
);
} }
/// Shut down every connected server and clear the server list. /// Shut down every connected server and clear the server list.
@@ -272,21 +216,20 @@ impl LspManager {
self.servers.clear(); self.servers.clear();
} }
/// Snapshot the connected servers as `(name, language_id, has_open_docs)` triples. /// Snapshot the connected servers as `(language_id, has_open_docs)` pairs.
/// ///
/// `has_open_docs` is true if any tracked `OpenDoc` was registered /// `has_open_docs` is true if any tracked `OpenDoc` was registered
/// against this server's clients. Useful for status displays. /// against this server's clients. Useful for status displays.
pub fn list_servers(&self) -> Vec<(String, String, bool)> { pub fn list_servers(&self) -> Vec<(String, bool)> {
self.servers self.servers
.iter() .iter()
.map(|s| { .map(|s| {
let name = s.name.clone();
let lang = s.language_id.clone(); let lang = s.language_id.clone();
let has_open = self let has_open = self
.open_files .open_files
.values() .values()
.any(|d| d.language == s.language_id); .any(|d| d.language == s.language_id);
(name, lang, has_open) (lang, has_open)
}) })
.collect() .collect()
} }
@@ -294,18 +237,17 @@ impl LspManager {
/// Connect an LSP server and register its default extensions in one call. /// Connect an LSP server and register its default extensions in one call.
/// ///
/// Flow: invoke `connect` -> on success, register `extensions` against /// Flow: invoke `connect` -> on success, register `extensions` against
/// `name` in `extension_registry`. If `connect` fails, the registries /// `language_id` in `extension_registry`. If `connect` fails, the registries
/// are left untouched and the error is propagated. /// are left untouched and the error is propagated.
pub fn connect_with_extensions( pub fn connect_with_extensions(
&mut self, &mut self,
name: &str,
command: &str, command: &str,
args: &[String], args: &[String],
language_id: &str, language_id: &str,
extensions: &[&str], extensions: &[&str],
) -> anyhow::Result<()> { ) -> anyhow::Result<()> {
self.connect(name, command, args, language_id)?; self.connect(command, args, language_id)?;
self.register_extensions(name, extensions); self.register_extensions(language_id, extensions);
Ok(()) Ok(())
} }
} }
@@ -315,4 +257,3 @@ impl Default for LspManager {
Self::new() Self::new()
} }
} }
+205 -161
View File
@@ -11,7 +11,6 @@
//! Each tier is a fallback for the previous, so we try the most //! Each tier is a fallback for the previous, so we try the most
//! user-friendly path first (rustup component, npm global, etc.) and //! user-friendly path first (rustup component, npm global, etc.) and
//! only fall back to package managers or manual download if those fail. //! only fall back to package managers or manual download if those fail.
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::process::{Command, Stdio}; use std::process::{Command, Stdio};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
@@ -45,14 +44,11 @@ pub enum ProvisionResult {
language: String, language: String,
binary_path: String, binary_path: String,
}, },
/// Every install tier failed — `manual_instructions` tells the user how /// Every install tier failed. Tells the user how to install by hand.
/// to install by hand.
Failed { Failed {
language: String, language: String,
server_name: String, server_name: String,
reason: String, reason: String,
#[allow(dead_code)]
manual_instructions: String,
}, },
} }
@@ -97,28 +93,55 @@ pub struct InstallTier {
pub args: Vec<String>, pub args: Vec<String>,
} }
/// Rust toolchain availability on the host PATH.
#[derive(Debug, Clone)]
pub struct RustToolchain {
pub has_rustup: bool,
pub has_cargo: bool,
}
/// Web / scripting language toolchain availability.
#[derive(Debug, Clone)]
pub struct WebToolchain {
pub has_npm: bool,
pub has_go: bool,
pub has_java: bool,
}
/// General-purpose platform utilities.
#[derive(Debug, Clone)]
pub struct PlatformUtils {
pub has_curl: bool,
pub has_tar: bool,
}
/// Pacman and Brew package managers (Arch / macOS).
#[derive(Debug, Clone)]
pub struct PacmanBrew {
pub has_pacman: bool,
pub has_brew: bool,
}
/// Apt and DNF package managers (Debian / Fedora).
#[derive(Debug, Clone)]
pub struct AptDnf {
pub has_apt: bool,
pub has_dnf: bool,
}
/// Snapshot of the host environment used to decide which install tiers are viable. /// Snapshot of the host environment used to decide which install tiers are viable.
/// ///
/// Populated by `detect_env()` once per `provision_all()` call so we /// Populated by `detect_env()` once per `provision_all_with_progress()` call so we
/// don't re-shell out for every server. `is_linux` / `is_macos` are /// don't re-shell out for every server. `is_linux` / `is_macos` are
/// computed at startup (compile time would also work, but keeping the /// computed at startup (compile time would also work, but keeping the
/// shape uniform with the rest of the struct makes the call sites tidy). /// shape uniform with the rest of the struct makes the call sites tidy).
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
#[allow(dead_code)]
#[allow(clippy::struct_excessive_bools)]
pub struct EnvInfo { pub struct EnvInfo {
pub has_rustup: bool, pub rust: RustToolchain,
pub has_npm: bool, pub web: WebToolchain,
pub has_go: bool, pub platform: PlatformUtils,
pub has_java: bool, pub pacman_brew: PacmanBrew,
pub has_cargo: bool, pub apt_dnf: AptDnf,
pub has_curl: bool,
pub has_wget: bool,
pub has_tar: bool,
pub has_pacman: bool,
pub has_apt: bool,
pub has_brew: bool,
pub has_dnf: bool,
pub is_linux: bool, pub is_linux: bool,
pub is_macos: bool, pub is_macos: bool,
} }
@@ -159,18 +182,27 @@ pub fn which(binary: &str) -> Option<PathBuf> {
/// this only ever runs on Unix-like targets. /// this only ever runs on Unix-like targets.
pub fn detect_env() -> EnvInfo { pub fn detect_env() -> EnvInfo {
EnvInfo { EnvInfo {
has_rustup: which("rustup").is_some(), rust: RustToolchain {
has_npm: which("npm").is_some(), has_rustup: which("rustup").is_some(),
has_go: which("go").is_some(), has_cargo: which("cargo").is_some(),
has_java: which("java").is_some(), },
has_cargo: which("cargo").is_some(), web: WebToolchain {
has_curl: which("curl").is_some(), has_npm: which("npm").is_some(),
has_wget: which("wget").is_some(), has_go: which("go").is_some(),
has_tar: which("tar").is_some(), has_java: which("java").is_some(),
has_pacman: which("pacman").is_some(), },
has_apt: which("apt").is_some() || which("apt-get").is_some(), platform: PlatformUtils {
has_brew: which("brew").is_some(), has_curl: which("curl").is_some(),
has_dnf: which("dnf").is_some(), has_tar: which("tar").is_some(),
},
pacman_brew: PacmanBrew {
has_pacman: which("pacman").is_some(),
has_brew: which("brew").is_some(),
},
apt_dnf: AptDnf {
has_apt: which("apt").is_some() || which("apt-get").is_some(),
has_dnf: which("dnf").is_some(),
},
is_linux: cfg!(target_os = "linux"), is_linux: cfg!(target_os = "linux"),
is_macos: cfg!(target_os = "macos"), is_macos: cfg!(target_os = "macos"),
} }
@@ -179,14 +211,13 @@ pub fn detect_env() -> EnvInfo {
/// Return the static set of supported language servers. /// Return the static set of supported language servers.
/// ///
/// The order is significant: it determines provisioning order and /// The order is significant: it determines provisioning order and
/// the order results appear in `provision_all()`. Tier 1 paths are /// the order results appear in `provision_all_with_progress()`. Tier 1 paths are
/// the canonical/idiomatic install for each ecosystem; later tiers /// the canonical/idiomatic install for each ecosystem; later tiers
/// are fallbacks for hosts that lack the primary tooling. /// are fallbacks for hosts that lack the primary tooling.
/// ///
/// Why hard-coded rather than loaded from settings: the set is small, /// Why hard-coded rather than loaded from settings: the set is small,
/// changes rarely, and bundling it lets the provisioner run before any /// changes rarely, and bundling it lets the provisioner run before any
/// user config has been read (e.g. on first launch). /// user config has been read (e.g. on first launch).
#[allow(clippy::too_many_lines)]
pub fn supported_servers() -> Vec<LanguageServerDef> { pub fn supported_servers() -> Vec<LanguageServerDef> {
vec![ vec![
LanguageServerDef { LanguageServerDef {
@@ -199,13 +230,22 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
label: "rustup component".to_string(), label: "rustup component".to_string(),
requires: vec!["rustup".to_string()], requires: vec!["rustup".to_string()],
command: "rustup".to_string(), command: "rustup".to_string(),
args: vec!["component".to_string(), "add".to_string(), "rust-analyzer".to_string()], args: vec![
"component".to_string(),
"add".to_string(),
"rust-analyzer".to_string(),
],
}, },
InstallTier { InstallTier {
label: "pacman".to_string(), label: "pacman".to_string(),
requires: vec!["pacman".to_string()], requires: vec!["pacman".to_string()],
command: "pacman".to_string(), command: "pacman".to_string(),
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "rust-analyzer".to_string()], args: vec![
"-S".to_string(),
"--noconfirm".to_string(),
"--needed".to_string(),
"rust-analyzer".to_string(),
],
}, },
InstallTier { InstallTier {
label: "brew".to_string(), label: "brew".to_string(),
@@ -217,7 +257,11 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
label: "cargo install".to_string(), label: "cargo install".to_string(),
requires: vec!["cargo".to_string()], requires: vec!["cargo".to_string()],
command: "cargo".to_string(), command: "cargo".to_string(),
args: vec!["install".to_string(), "--locked".to_string(), "rust-analyzer".to_string()], args: vec![
"install".to_string(),
"--locked".to_string(),
"rust-analyzer".to_string(),
],
}, },
InstallTier { InstallTier {
label: "download prebuilt".to_string(), label: "download prebuilt".to_string(),
@@ -268,19 +312,33 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
name: "jdtls".to_string(), name: "jdtls".to_string(),
language: "java".to_string(), language: "java".to_string(),
extensions: vec![".java".to_string()], extensions: vec![".java".to_string()],
binary_names: vec!["jdtls".to_string(), "eclipse-jdt-ls".to_string(), "jdtls-launcher".to_string()], binary_names: vec![
"jdtls".to_string(),
"eclipse-jdt-ls".to_string(),
"jdtls-launcher".to_string(),
],
install_tiers: vec![ install_tiers: vec![
InstallTier { InstallTier {
label: "pacman".to_string(), label: "pacman".to_string(),
requires: vec!["java".to_string(), "pacman".to_string()], requires: vec!["java".to_string(), "pacman".to_string()],
command: "pacman".to_string(), command: "pacman".to_string(),
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "eclipse-jdt-ls".to_string()], args: vec![
"-S".to_string(),
"--noconfirm".to_string(),
"--needed".to_string(),
"eclipse-jdt-ls".to_string(),
],
}, },
InstallTier { InstallTier {
label: "apt".to_string(), label: "apt".to_string(),
requires: vec!["java".to_string(), "apt".to_string()], requires: vec!["java".to_string(), "apt".to_string()],
command: "sudo".to_string(), command: "sudo".to_string(),
args: vec!["apt".to_string(), "install".to_string(), "-y".to_string(), "eclipse-jdt-ls".to_string()], args: vec![
"apt".to_string(),
"install".to_string(),
"-y".to_string(),
"eclipse-jdt-ls".to_string(),
],
}, },
InstallTier { InstallTier {
label: "brew".to_string(), label: "brew".to_string(),
@@ -339,7 +397,9 @@ pub fn run_command(cmd: &str, args: &[&str]) -> std::io::Result<(bool, String)>
let timeout = Duration::from_mins(3); let timeout = Duration::from_mins(3);
let start = Instant::now(); let start = Instant::now();
let status = loop { let status = loop {
if let Some(status) = child.try_wait()? { break Ok(status) } if let Some(status) = child.try_wait()? {
break Ok(status);
}
if start.elapsed() > timeout { if start.elapsed() > timeout {
let _ = child.kill(); let _ = child.kill();
let _ = child.wait(); let _ = child.wait();
@@ -405,9 +465,12 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
info!(url = url, dest = %path_str, "downloading"); info!(url = url, dest = %path_str, "downloading");
let args = [ let args = [
"-fsSL", "-fsSL",
"--connect-timeout", "15", "--connect-timeout",
"--max-time", &max_secs.to_string(), "15",
"-o", &path_str, "--max-time",
&max_secs.to_string(),
"-o",
&path_str,
url, url,
]; ];
let (ok, out) = run_command("curl", &args).map_err(|e| format!("curl spawn: {e}"))?; let (ok, out) = run_command("curl", &args).map_err(|e| format!("curl spawn: {e}"))?;
@@ -419,7 +482,10 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
/// Download rust-analyzer from GitHub releases and install into /// Download rust-analyzer from GitHub releases and install into
/// `~/.local/share/zesdex/lsp/rust-analyzer/bin/rust-analyzer`. /// `~/.local/share/zesdex/lsp/rust-analyzer/bin/rust-analyzer`.
fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> { fn install_rust_analyzer_binary(
env: &EnvInfo,
progress: ProgressFn<'_>,
) -> Result<PathBuf, String> {
let base = lsp_install_dir("rust-analyzer")?; let base = lsp_install_dir("rust-analyzer")?;
std::fs::create_dir_all(&base).map_err(|e| format!("mkdir: {e}"))?; std::fs::create_dir_all(&base).map_err(|e| format!("mkdir: {e}"))?;
@@ -434,9 +500,13 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
let gz = base.join("rust-analyzer.gz"); let gz = base.join("rust-analyzer.gz");
let target = base.join("rust-analyzer"); let target = base.join("rust-analyzer");
if let Some(cb) = progress { cb("Rust: downloading prebuilt binary..."); } if let Some(cb) = progress {
cb("Rust: downloading prebuilt binary...");
}
download_url(url, &gz, 120)?; download_url(url, &gz, 120)?;
if let Some(cb) = progress { cb("Rust: decompressing..."); } if let Some(cb) = progress {
cb("Rust: decompressing...");
}
let (ok, out) = run_command("gunzip", &["-f", &gz.to_string_lossy()]) let (ok, out) = run_command("gunzip", &["-f", &gz.to_string_lossy()])
.map_err(|e| format!("gunzip spawn: {e}"))?; .map_err(|e| format!("gunzip spawn: {e}"))?;
if !ok { if !ok {
@@ -452,7 +522,9 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755)) std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755))
.map_err(|e| format!("chmod: {e}"))?; .map_err(|e| format!("chmod: {e}"))?;
} }
if let Some(cb) = progress { cb("Rust: installed ✓"); } if let Some(cb) = progress {
cb("Rust: installed ✓");
}
Ok(target) Ok(target)
} }
@@ -464,14 +536,24 @@ fn install_jdtls_from_eclipse(progress: ProgressFn) -> Result<PathBuf, String> {
let url = "https://download.eclipse.org/jdtls/snapshots/jdt-language-server-latest.tar.gz"; let url = "https://download.eclipse.org/jdtls/snapshots/jdt-language-server-latest.tar.gz";
let tarball = base.join("jdtls.tar.gz"); let tarball = base.join("jdtls.tar.gz");
if let Some(cb) = progress { cb("Java: downloading JDT-LS (~150MB)..."); } if let Some(cb) = progress {
cb("Java: downloading JDT-LS (~150MB)...");
}
download_url(url, &tarball, 300)?; download_url(url, &tarball, 300)?;
if let Some(cb) = progress { cb("Java: extracting..."); } if let Some(cb) = progress {
cb("Java: extracting...");
}
let (ok, out) = run_command("tar", &[ let (ok, out) = run_command(
"-xzf", tarball.to_str().unwrap_or(""), "tar",
"-C", base.to_str().unwrap_or("."), &[
]).map_err(|e| format!("tar spawn: {e}"))?; "-xzf",
tarball.to_str().unwrap_or(""),
"-C",
base.to_str().unwrap_or("."),
],
)
.map_err(|e| format!("tar spawn: {e}"))?;
if !ok { if !ok {
return Err(format!("tar: {}", out.trim())); return Err(format!("tar: {}", out.trim()));
} }
@@ -510,12 +592,18 @@ exec java \
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755)) std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755))
.map_err(|e| format!("chmod launcher: {e}"))?; .map_err(|e| format!("chmod launcher: {e}"))?;
} }
if let Some(cb) = progress { cb("Java: JDT-LS installed ✓"); } if let Some(cb) = progress {
cb("Java: JDT-LS installed ✓");
}
Ok(launcher) Ok(launcher)
} }
/// Dispatch a sentinel download tier to the correct helper. /// Dispatch a sentinel download tier to the correct helper.
fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> { fn run_download_tier(
name: &str,
env: &EnvInfo,
progress: ProgressFn<'_>,
) -> Result<PathBuf, String> {
match name { match name {
DOWNLOAD_RUST_BIN => install_rust_analyzer_binary(env, progress), DOWNLOAD_RUST_BIN => install_rust_analyzer_binary(env, progress),
DOWNLOAD_JDTLS => install_jdtls_from_eclipse(progress), DOWNLOAD_JDTLS => install_jdtls_from_eclipse(progress),
@@ -523,59 +611,17 @@ fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Res
} }
} }
/// Render the "install by hand" message shown to the user when every fn provision_single_with_progress(
/// automated tier fails. def: &LanguageServerDef,
fn manual_instructions(def: &LanguageServerDef) -> String { env: &EnvInfo,
match def.language.as_str() { progress: ProgressFn<'_>,
"rust" => "Install rust-analyzer:\n \ ) -> ProvisionResult {
Arch: sudo pacman -S rust-analyzer\n \
macOS: brew install rust-analyzer\n \
Any: cargo install --locked rust-analyzer\n \
Rustup: rustup component add rust-analyzer"
.to_string(),
"typescript" => "Install typescript-language-server:\n \
npm install -g typescript typescript-language-server\n \
Arch: sudo pacman -S typescript-language-server"
.to_string(),
"go" => "Install gopls:\n \
go install golang.org/x/tools/gopls@latest\n \
Arch: sudo pacman -S gopls"
.to_string(),
"java" => "Install Eclipse JDT-LS:\n \
Arch: sudo pacman -S eclipse-jdt-ls\n \
Debian: sudo apt install eclipse-jdt-ls\n \
macOS: brew install jdtls\n \
Other: see https://.eclipse.org/jdtls/#download"
.to_string(),
_ => format!("No automated install available for '{}'.", def.language),
}
}
/// Try to provision a single language server.
///
/// Flow: check whether any `binary_names` candidate is already on PATH
/// → if yes, return `AlreadyAvailable` → otherwise walk
/// `install_tiers` in order, skipping tiers whose `requires`
/// binaries are missing → for each viable tier, run the install
/// command (120s timeout) → if it succeeds AND the binary now
/// appears on PATH (or the tier is jdtls-manual returning a
/// launcher path), return Installed → if every tier fails, return
/// Failed with the last error and manual install instructions.
///
/// Why we re-check `which` after the install: `rustup component add`
/// can exit 0 even if the binary wasn't actually placed on PATH (rare,
/// but happens with broken rustup installs). Re-checking gives us a
/// real signal rather than trusting the exit code alone.
#[allow(dead_code)]
pub fn provision_single(def: &LanguageServerDef, env: &EnvInfo) -> ProvisionResult {
provision_single_with_progress(def, env, None)
}
fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progress: ProgressFn<'_>) -> ProvisionResult {
// 1. Check PATH. // 1. Check PATH.
for bin in &def.binary_names { for bin in &def.binary_names {
if let Some(path) = which(bin) { if let Some(path) = which(bin) {
if let Some(cb) = progress { cb(&format!("{}: already installed (PATH)", def.language)); } if let Some(cb) = progress {
cb(&format!("{}: already installed (PATH)", def.language));
}
return ProvisionResult::AlreadyAvailable { return ProvisionResult::AlreadyAvailable {
server_name: def.name.clone(), server_name: def.name.clone(),
language: def.language.clone(), language: def.language.clone(),
@@ -586,7 +632,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
// 2. Check download-install directory (~/.local/share/zesdex/lsp/<name>/...). // 2. Check download-install directory (~/.local/share/zesdex/lsp/<name>/...).
if let Some(path) = previous_download_install(def) { if let Some(path) = previous_download_install(def) {
if let Some(cb) = progress { cb(&format!("{}: found previous install", def.language)); } if let Some(cb) = progress {
cb(&format!("{}: found previous install", def.language));
}
return ProvisionResult::AlreadyAvailable { return ProvisionResult::AlreadyAvailable {
server_name: def.name.clone(), server_name: def.name.clone(),
language: def.language.clone(), language: def.language.clone(),
@@ -594,30 +642,42 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
}; };
} }
if let Some(cb) = progress { cb(&format!("{}: checking install options...", def.language)); } if let Some(cb) = progress {
cb(&format!("{}: checking install options...", def.language));
}
let mut last_reason = String::from("no install tiers succeeded"); let mut last_reason = String::from("no install tiers succeeded");
for tier in &def.install_tiers { for tier in &def.install_tiers {
// Prerequisite gating // Prerequisite gating
let prereqs_met = tier.requires.iter().all(|req| match req.as_str() { let prereqs_met = tier.requires.iter().all(|req| match req.as_str() {
"rustup" => env.has_rustup, "npm" => env.has_npm, "rustup" => env.rust.has_rustup,
"go" => env.has_go, "java" => env.has_java, "npm" => env.web.has_npm,
"cargo" => env.has_cargo, "curl" => env.has_curl, "go" => env.web.has_go,
"tar" => env.has_tar, "pacman" => env.has_pacman, "java" => env.web.has_java,
"apt" => env.has_apt, "brew" => env.has_brew, "cargo" => env.rust.has_cargo,
"dnf" => env.has_dnf, _ => which(req).is_some(), "curl" => env.platform.has_curl,
"tar" => env.platform.has_tar,
"pacman" => env.pacman_brew.has_pacman,
"apt" => env.apt_dnf.has_apt,
"brew" => env.pacman_brew.has_brew,
"dnf" => env.apt_dnf.has_dnf,
_ => which(req).is_some(),
}); });
if !prereqs_met { if !prereqs_met {
let skip = format!("{}: {} — missing prerequisite", def.language, tier.label); let skip = format!("{}: {} — missing prerequisite", def.language, tier.label);
if let Some(cb) = progress { cb(&skip); } if let Some(cb) = progress {
cb(&skip);
}
last_reason = format!("tier '{}' skipped: missing prerequisite", tier.label); last_reason = format!("tier '{}' skipped: missing prerequisite", tier.label);
warn!(server = %def.name, tier = %tier.label, "skipped — missing prerequisites"); warn!(server = %def.name, tier = %tier.label, "skipped — missing prerequisites");
continue; continue;
} }
let trying = format!("{}: {}...", def.language, tier.label); let trying = format!("{}: {}...", def.language, tier.label);
if let Some(cb) = progress { cb(&trying); } if let Some(cb) = progress {
cb(&trying);
}
// Download sentinel → helper. // Download sentinel → helper.
if tier.command.starts_with("__download_") && tier.command.ends_with("__") { if tier.command.starts_with("__download_") && tier.command.ends_with("__") {
@@ -647,7 +707,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
.iter() .iter()
.find_map(|b| which(b).map(|p| p.to_string_lossy().to_string())); .find_map(|b| which(b).map(|p| p.to_string_lossy().to_string()));
if let Some(path) = located { if let Some(path) = located {
if let Some(cb) = progress { cb(&format!("{}: installed ✓", def.language)); } if let Some(cb) = progress {
cb(&format!("{}: installed ✓", def.language));
}
info!(server = %def.name, tier = %tier.label, binary = %path, "installed"); info!(server = %def.name, tier = %tier.label, binary = %path, "installed");
return ProvisionResult::Installed { return ProvisionResult::Installed {
server_name: def.name.clone(), server_name: def.name.clone(),
@@ -671,58 +733,36 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
} }
} }
let manual = manual_instructions(def);
ProvisionResult::Failed { ProvisionResult::Failed {
language: def.language.clone(), server_name: def.name.clone(), language: def.language.clone(),
reason: last_reason, manual_instructions: manual, server_name: def.name.clone(),
reason: last_reason,
} }
} }
/// Provision every supported server in order, returning one /// Provision every supported server with progress callbacks with a human-readable status
/// `ProvisionResult` per server.
///
/// Flow: `detect_env()` once → for each server in `supported_servers()`
/// call `provision_single()` → collect results. Order matches
/// `supported_servers()` (rust, typescript, go, java).
#[allow(dead_code)]
pub fn provision_all() -> Vec<ProvisionResult> {
let env = detect_env();
info!(
linux = env.is_linux,
macos = env.is_macos,
rustup = env.has_rustup,
cargo = env.has_cargo,
npm = env.has_npm,
go = env.has_go,
java = env.has_java,
curl = env.has_curl,
tar = env.has_tar,
pacman = env.has_pacman,
apt = env.has_apt,
brew = env.has_brew,
dnf = env.has_dnf,
"starting LSP provisioning"
);
supported_servers()
.iter()
.map(|def| provision_single(def, &env))
.collect()
}
/// Like `provision_all` but calls `progress` with a human-readable status
/// string at each stage of each server's install attempt. /// string at each stage of each server's install attempt.
pub fn provision_all_with_progress(progress: ProgressFn) -> Vec<ProvisionResult> { pub fn provision_all_with_progress(progress: ProgressFn) -> Vec<ProvisionResult> {
let env = detect_env(); let env = detect_env();
if let Some(cb) = progress { if let Some(cb) = progress {
let flags = [ let flags = [
("rustup", env.has_rustup), ("cargo", env.has_cargo), ("rustup", env.rust.has_rustup),
("npm", env.has_npm), ("go", env.has_go), ("java", env.has_java), ("cargo", env.rust.has_cargo),
("curl", env.has_curl), ("tar", env.has_tar), ("npm", env.web.has_npm),
("pacman", env.has_pacman), ("apt", env.has_apt), ("brew", env.has_brew), ("go", env.web.has_go),
("java", env.web.has_java),
("curl", env.platform.has_curl),
("tar", env.platform.has_tar),
("pacman", env.pacman_brew.has_pacman),
("apt", env.apt_dnf.has_apt),
("brew", env.pacman_brew.has_brew),
]; ];
let avail: String = flags.iter() let avail: String = flags
.filter(|(_, v)| *v).map(|(k, _)| *k) .iter()
.collect::<Vec<_>>().join(", "); .filter(|(_, v)| *v)
.map(|(k, _)| *k)
.collect::<Vec<_>>()
.join(", ");
cb(&format!("LSP: environment ready — {avail}")); cb(&format!("LSP: environment ready — {avail}"));
} }
supported_servers() supported_servers()
@@ -779,9 +819,13 @@ pub fn auto_connect(manager: &Arc<Mutex<LspManager>>, results: &[ProvisionResult
}; };
// Build extension slice for connect_with_extensions. // Build extension slice for connect_with_extensions.
let ext_refs: Vec<&str> = def.extensions.iter().map(std::string::String::as_str).collect(); let ext_refs: Vec<&str> = def
.extensions
.iter()
.map(std::string::String::as_str)
.collect();
match guard.connect_with_extensions(&name, &binary, &[], &language, &ext_refs) { match guard.connect_with_extensions(&binary, &[], &language, &ext_refs) {
Ok(()) => { Ok(()) => {
info!( info!(
name = %name, name = %name,
+129 -91
View File
@@ -1,13 +1,11 @@
//! MCP server connection management: spawning/talking to stdio child //! MCP server connection management: spawning/talking to stdio child
//! processes and HTTP endpoints, and adapting their advertised tools to //! processes and HTTP endpoints, and adapting their advertised tools to
//! the crate's `Tool` trait. //! the crate's `Tool` trait.
use serde_json::{json, Value};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use std::io::{BufRead, BufReader, Write}; use std::io::{BufRead, BufReader, Write};
use std::sync::{Arc, Mutex, OnceLock}; use std::sync::{Arc, Mutex, OnceLock};
const MCP_CONNECT_TIMEOUT_MS: u64 = 20_000; const MCP_CONNECT_TIMEOUT_MS: u64 = 20_000;
const MCP_CALL_TIMEOUT_MS: u64 = 60_000; const MCP_CALL_TIMEOUT_MS: u64 = 60_000;
@@ -35,13 +33,8 @@ fn mcp_static_str(s: &str) -> &'static str {
/// newline-delimited JSON-RPC over stdio, or a remote HTTP endpoint. /// newline-delimited JSON-RPC over stdio, or a remote HTTP endpoint.
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub enum McpTransport { pub enum McpTransport {
Stdio { Stdio { command: String, args: Vec<String> },
command: String, StreamableHttp { url: String },
args: Vec<String>,
},
StreamableHttp {
url: String,
},
} }
/// A single tool advertised by an MCP server, as returned by `tools/list`. /// A single tool advertised by an MCP server, as returned by `tools/list`.
@@ -104,8 +97,8 @@ impl StdioChild {
self.stdin.flush()?; self.stdin.flush()?;
let mut response_line = String::new(); let mut response_line = String::new();
let deadline = std::time::Instant::now() let deadline =
+ std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS); std::time::Instant::now() + std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS);
loop { loop {
if std::time::Instant::now() > deadline { if std::time::Instant::now() > deadline {
anyhow::bail!("MCP call timed out after {MCP_CALL_TIMEOUT_MS}ms"); anyhow::bail!("MCP call timed out after {MCP_CALL_TIMEOUT_MS}ms");
@@ -136,7 +129,9 @@ impl StdioChild {
line_truncated = true; line_truncated = true;
// Consume rest of line to keep stream in sync // Consume rest of line to keep stream in sync
loop { loop {
let buf = self.stdout.fill_buf() let buf = self
.stdout
.fill_buf()
.map_err(|e| anyhow::anyhow!("MCP stdio read error: {e}"))?; .map_err(|e| anyhow::anyhow!("MCP stdio read error: {e}"))?;
if buf.is_empty() { if buf.is_empty() {
anyhow::bail!("MCP stdio child closed mid-line"); anyhow::bail!("MCP stdio child closed mid-line");
@@ -152,9 +147,7 @@ impl StdioChild {
response_line.push(byte as char); response_line.push(byte as char);
} }
if line_truncated { if line_truncated {
anyhow::bail!( anyhow::bail!("MCP response line exceeded {MAX_LINE_LENGTH} byte limit");
"MCP response line exceeded {MAX_LINE_LENGTH} byte limit",
);
} }
let trimmed = response_line.trim(); let trimmed = response_line.trim();
if trimmed.is_empty() { if trimmed.is_empty() {
@@ -172,12 +165,16 @@ impl StdioChild {
})); }));
} }
} }
} // close fn call } // close fn call
} // close impl StdioChild } // close impl StdioChild
pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow::Result<StdioChild> { pub(crate) fn spawn_stdio_child(
command: &str,
extra_args: &[String],
) -> anyhow::Result<StdioChild> {
let parts: Vec<&str> = command.split_whitespace().collect(); let parts: Vec<&str> = command.split_whitespace().collect();
let (prog, prog_args) = parts.split_first() let (prog, prog_args) = parts
.split_first()
.ok_or_else(|| anyhow::anyhow!("MCP stdio command is empty"))?; .ok_or_else(|| anyhow::anyhow!("MCP stdio command is empty"))?;
let mut cmd = std::process::Command::new(prog); let mut cmd = std::process::Command::new(prog);
@@ -189,12 +186,17 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
// rather than discarded silently, making connectivity issues debugable. // rather than discarded silently, making connectivity issues debugable.
cmd.stderr(std::process::Stdio::piped()); cmd.stderr(std::process::Stdio::piped());
let mut child = cmd.spawn() let mut child = cmd
.spawn()
.map_err(|e| anyhow::anyhow!("failed to spawn MCP stdio server '{command}': {e}"))?; .map_err(|e| anyhow::anyhow!("failed to spawn MCP stdio server '{command}': {e}"))?;
let stdin = child.stdin.take() let stdin = child
.stdin
.take()
.ok_or_else(|| anyhow::anyhow!("failed to get stdin for MCP server"))?; .ok_or_else(|| anyhow::anyhow!("failed to get stdin for MCP server"))?;
let stdout = child.stdout.take() let stdout = child
.stdout
.take()
.ok_or_else(|| anyhow::anyhow!("failed to get stdout for MCP server"))?; .ok_or_else(|| anyhow::anyhow!("failed to get stdout for MCP server"))?;
let mut mcp = StdioChild { let mut mcp = StdioChild {
@@ -203,17 +205,20 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
next_id: 0, next_id: 0,
}; };
let deadline = std::time::Instant::now() let deadline =
+ std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS); std::time::Instant::now() + std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS);
let init_result = mcp.call("initialize", &json!({ let init_result = mcp.call(
"protocolVersion": "2024-11-05", "initialize",
"capabilities": {}, &json!({
"clientInfo": { "protocolVersion": "2024-11-05",
"name": "zesdex", "capabilities": {},
"version": "0.1.0" "clientInfo": {
} "name": "zesdex",
})); "version": "0.1.0"
}
}),
);
if std::time::Instant::now() > deadline { if std::time::Instant::now() > deadline {
anyhow::bail!("MCP initialize timed out"); anyhow::bail!("MCP initialize timed out");
@@ -236,21 +241,29 @@ fn call_via_stdio(
// Reuse the persistent child handle if available; otherwise spawn a new one. // Reuse the persistent child handle if available; otherwise spawn a new one.
let mut guard; let mut guard;
let child: &mut StdioChild = if let Some(mtx) = existing_handle { let child: &mut StdioChild = if let Some(mtx) = existing_handle {
guard = mtx.lock().map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?; guard = mtx
.lock()
.map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?;
&mut guard &mut guard
} else { } else {
let mut fresh = spawn_stdio_child(command, extra_args)?; let mut fresh = spawn_stdio_child(command, extra_args)?;
let result = fresh.call("tools/call", &json!({ let result = fresh.call(
"name": tool_name, "tools/call",
"arguments": tool_args &json!({
}))?; "name": tool_name,
"arguments": tool_args
}),
)?;
return Ok(extract_text_content(&result)); return Ok(extract_text_content(&result));
}; };
let result = child.call("tools/call", &json!({ let result = child.call(
"name": tool_name, "tools/call",
"arguments": tool_args &json!({
}))?; "name": tool_name,
"arguments": tool_args
}),
)?;
Ok(extract_text_content(&result)) Ok(extract_text_content(&result))
} }
@@ -289,7 +302,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
} }
}); });
let resp = client.post(url) let resp = client
.post(url)
.header("Content-Type", "application/json") .header("Content-Type", "application/json")
.json(&body) .json(&body)
.send() .send()
@@ -304,7 +318,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
anyhow::bail!("MCP HTTP server returned {status}: {text}"); anyhow::bail!("MCP HTTP server returned {status}: {text}");
} }
let response: Value = resp.json() let response: Value = resp
.json()
.map_err(|e| anyhow::anyhow!("invalid JSON from MCP HTTP server: {e}"))?; .map_err(|e| anyhow::anyhow!("invalid JSON from MCP HTTP server: {e}"))?;
if let Some(err) = response.get("error") { if let Some(err) = response.get("error") {
@@ -321,13 +336,18 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
fn extract_text_content(result: &Value) -> String { fn extract_text_content(result: &Value) -> String {
if let Some(content) = result.get("content") { if let Some(content) = result.get("content") {
if let Some(arr) = content.as_array() { if let Some(arr) = content.as_array() {
let text: Vec<String> = arr.iter().filter_map(|item| { let text: Vec<String> = arr
if item.get("type").and_then(|t| t.as_str()) == Some("text") { .iter()
item.get("text").and_then(|t| t.as_str()).map(std::string::ToString::to_string) .filter_map(|item| {
} else { if item.get("type").and_then(|t| t.as_str()) == Some("text") {
None item.get("text")
} .and_then(|t| t.as_str())
}).collect(); .map(std::string::ToString::to_string)
} else {
None
}
})
.collect();
if !text.is_empty() { if !text.is_empty() {
return text.join("\n"); return text.join("\n");
} }
@@ -372,12 +392,17 @@ impl crate::tool::Tool for McpToolAdapter {
fn run(&self, _ctx: &crate::tool::ToolCtx, args: &Value) -> anyhow::Result<String> { fn run(&self, _ctx: &crate::tool::ToolCtx, args: &Value) -> anyhow::Result<String> {
match &self.transport { match &self.transport {
McpTransport::Stdio { command, args: extra_args } => { McpTransport::Stdio {
call_via_stdio(self.child_handle.as_ref().map(std::convert::AsRef::as_ref), command, extra_args, &self.tool_name, args) command,
} args: extra_args,
McpTransport::StreamableHttp { url } => { } => call_via_stdio(
call_via_http(url, &self.tool_name, args) self.child_handle.as_ref().map(std::convert::AsRef::as_ref),
} command,
extra_args,
&self.tool_name,
args,
),
McpTransport::StreamableHttp { url } => call_via_http(url, &self.tool_name, args),
} }
} }
} }
@@ -400,27 +425,35 @@ impl McpManager {
/// ///
/// Return: boxed `Tool` trait objects ready to merge into the harness's tool list. /// Return: boxed `Tool` trait objects ready to merge into the harness's tool list.
pub fn as_tools(&self) -> Vec<Box<dyn crate::tool::Tool>> { pub fn as_tools(&self) -> Vec<Box<dyn crate::tool::Tool>> {
self.servers.iter().flat_map(|server| { self.servers
let handle = server.child_handle.clone(); .iter()
server.tools.iter().map(move |info| { .flat_map(|server| {
let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter { let handle = server.child_handle.clone();
tool_name: info.name.clone(), server.tools.iter().map(move |info| {
server_name: server.name.clone(), let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter {
transport: server.transport.clone(), tool_name: info.name.clone(),
description: info.description.clone(), server_name: server.name.clone(),
parameters: info.input_schema.clone(), transport: server.transport.clone(),
child_handle: handle.clone(), description: info.description.clone(),
}); parameters: info.input_schema.clone(),
adapter child_handle: handle.clone(),
});
adapter
})
}) })
}).collect() .collect()
} }
/// Connects to an MCP server via stdio by spawning the child process, running /// Connects to an MCP server via stdio by spawning the child process, running
/// the `initialize` handshake, calling `tools/list`, and registering the server /// the `initialize` handshake, calling `tools/list`, and registering the server
/// with its advertised tools in `self.servers`. The child process stays alive /// with its advertised tools in `self.servers`. The child process stays alive
/// for subsequent `tools/call` invocations via the stored `McpServer.tools`. /// for subsequent `tools/call` invocations via the stored `McpServer.tools`.
pub fn connect_stdio(&mut self, name: &str, command: &str, extra_args: &[String]) -> anyhow::Result<()> { pub fn connect_stdio(
&mut self,
name: &str,
command: &str,
extra_args: &[String],
) -> anyhow::Result<()> {
let transport = McpTransport::Stdio { let transport = McpTransport::Stdio {
command: command.to_string(), command: command.to_string(),
args: extra_args.to_vec(), args: extra_args.to_vec(),
@@ -430,19 +463,32 @@ impl McpManager {
let result = child.call("tools/list", &json!({}))?; let result = child.call("tools/list", &json!({}))?;
let tools = if let Some(tool_list) = result.get("tools").and_then(|v| v.as_array()) { let tools = if let Some(tool_list) = result.get("tools").and_then(|v| v.as_array()) {
tool_list.iter().filter_map(|t| { tool_list
Some(McpToolInfo { .iter()
name: t.get("name")?.as_str()?.to_string(), .filter_map(|t| {
description: t.get("description").and_then(|v| v.as_str()).unwrap_or_else(|| { Some(McpToolInfo {
tracing::warn!("[mcp] tool {} missing description", t.get("name").and_then(|n| n.as_str()).unwrap_or("?")); name: t.get("name")?.as_str()?.to_string(),
"" description: t
}).to_string(), .get("description")
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| { .and_then(|v| v.as_str())
tracing::warn!("[mcp] tool {} missing inputSchema", t.get("name").and_then(|n| n.as_str()).unwrap_or("?")); .unwrap_or_else(|| {
serde_json::Value::Null tracing::warn!(
}), "[mcp] tool {} missing description",
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
);
""
})
.to_string(),
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| {
tracing::warn!(
"[mcp] tool {} missing inputSchema",
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
);
serde_json::Value::Null
}),
})
}) })
}).collect() .collect()
} else { } else {
Vec::new() Vec::new()
}; };
@@ -458,12 +504,4 @@ impl McpManager {
Ok(()) Ok(())
} }
/// Removes a server by name. Returns `true` if a server was found and removed.
#[allow(dead_code)]
pub fn disconnect(&mut self, name: &str) -> bool {
let len = self.servers.len();
self.servers.retain(|s| s.name != name);
self.servers.len() < len
}
} }
-1
View File
@@ -1,4 +1,3 @@
//! Model Context Protocol (MCP) client: connects to external MCP servers //! Model Context Protocol (MCP) client: connects to external MCP servers
//! (stdio or HTTP) and exposes their tools through the crate's `Tool` trait. //! (stdio or HTTP) and exposes their tools through the crate's `Tool` trait.
pub mod manager; pub mod manager;
+5 -5
View File
@@ -1,13 +1,13 @@
//! Top-level application module: harness, modes, runtime loop, state, //! Top-level application module: harness, modes, runtime loop, state,
//! workflows, subagents, review, background bash, MCP integration, and //! workflows, subagents, review, background bash, MCP integration, and
//! native LSP client. //! native LSP client.
pub mod bgbash;
pub mod harness; pub mod harness;
pub mod lsp;
pub mod mcp;
pub mod mode; pub mod mode;
pub mod review;
pub mod runtime; pub mod runtime;
pub mod state; pub mod state;
pub mod workflow;
pub mod subagent; pub mod subagent;
pub mod review; pub mod workflow;
pub mod bgbash;
pub mod mcp;
pub mod lsp;
-1
View File
@@ -1,5 +1,4 @@
//! Bash mode: handles submitting a shell command from the bash input panel. //! Bash mode: handles submitting a shell command from the bash input panel.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
/// Launch a background bash job for the submitted command. /// Launch a background bash job for the submitted command.
+5 -5
View File
@@ -1,6 +1,5 @@
//! Editor mode: a minimal in-TUI line editor for viewing/modifying a file, //! Editor mode: a minimal in-TUI line editor for viewing/modifying a file,
//! with bounded undo history. //! with bounded undo history.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
use crate::app::state::types::Overlay; use crate::app::state::types::Overlay;
@@ -66,7 +65,9 @@ impl EditorState {
self.cursor_line += 1; self.cursor_line += 1;
} }
self.cursor_col = self.cursor_col.min( self.cursor_col = self.cursor_col.min(
self.content.get(self.cursor_line).map_or(0, std::string::String::len), self.content
.get(self.cursor_line)
.map_or(0, std::string::String::len),
); );
} }
@@ -114,10 +115,9 @@ impl EditorState {
/// the char directly → mark state dirty. /// the char directly → mark state dirty.
pub fn handle_editor_input(state: &mut AppStateRest, text: &str) { pub fn handle_editor_input(state: &mut AppStateRest, text: &str) {
let editor = &mut state.misc.editor; let editor = &mut state.misc.editor;
if editor.is_none() { let Some(ed) = editor.as_mut() else {
return; return;
} };
let ed = editor.as_mut().unwrap();
for c in text.chars() { for c in text.chars() {
match c { match c {
'\n' | '\r' => { '\n' | '\r' => {
+6 -2
View File
@@ -1,7 +1,11 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)] #![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Effort mode: cycles the agent's reasoning effort level, which scales the //! Effort mode: cycles the agent's reasoning effort level, which scales the
//! LLM's temperature and `max_tokens` for subsequent turns. //! LLM's temperature and `max_tokens` for subsequent turns.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
pub const EFFORT_LEVELS: &[&str] = &["low", "medium", "high", "xhigh", "max"]; pub const EFFORT_LEVELS: &[&str] = &["low", "medium", "high", "xhigh", "max"];
-1
View File
@@ -1,5 +1,4 @@
//! Help mode: static help text and the action that opens/closes the help overlay. //! Help mode: static help text and the action that opens/closes the help overlay.
use crate::app::runtime::actions::Action; use crate::app::runtime::actions::Action;
use crate::app::state::types::Overlay; use crate::app::state::types::Overlay;
-1
View File
@@ -1,5 +1,4 @@
//! Key input mode: raw text capture overlay used for one-off key/text prompts. //! Key input mode: raw text capture overlay used for one-off key/text prompts.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
/// Replace the input buffer with the given text and mark state dirty. /// Replace the input buffer with the given text and mark state dirty.
+4 -2
View File
@@ -33,14 +33,16 @@ pub fn get_learning_items(state: &AppStateRest) -> Vec<LearningItem> {
let scope_str = match p.lesson.scope { let scope_str = match p.lesson.scope {
crate::app::review::LessonScope::Project => "project", crate::app::review::LessonScope::Project => "project",
crate::app::review::LessonScope::Global => "global", crate::app::review::LessonScope::Global => "global",
}.to_string(); }
.to_string();
let conf_str = match p.lesson.confidence { let conf_str = match p.lesson.confidence {
crate::app::review::Confidence::Human => "human", crate::app::review::Confidence::Human => "human",
crate::app::review::Confidence::Verified => "verified", crate::app::review::Confidence::Verified => "verified",
crate::app::review::Confidence::Unverified => "unverified", crate::app::review::Confidence::Unverified => "unverified",
crate::app::review::Confidence::Auto => "auto", crate::app::review::Confidence::Auto => "auto",
}.to_string(); }
.to_string();
items.push(LearningItem::Pending { items.push(LearningItem::Pending {
name: p.lesson.name, name: p.lesson.name,
-1
View File
@@ -1,5 +1,4 @@
//! Loading mode: transient overlay shown while waiting on an async operation. //! Loading mode: transient overlay shown while waiting on an async operation.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
pub const LOADING_MESSAGES: &[&str] = &[ pub const LOADING_MESSAGES: &[&str] = &[
-1
View File
@@ -1,5 +1,4 @@
//! MCP mode: overlay for connecting to a configured MCP server. //! MCP mode: overlay for connecting to a configured MCP server.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
/// Placeholder entry point for connecting to an MCP server by name. /// Placeholder entry point for connecting to an MCP server by name.
+1 -2
View File
@@ -1,14 +1,13 @@
//! TUI mode definitions and per-mode input/action handlers, one submodule //! TUI mode definitions and per-mode input/action handlers, one submodule
//! per overlay/mode (bash, editor, effort, mcp, quit confirm, rewind, etc.). //! per overlay/mode (bash, editor, effort, mcp, quit confirm, rewind, etc.).
pub mod bash; pub mod bash;
pub mod editor; pub mod editor;
pub mod effort; pub mod effort;
pub mod key_input; pub mod key_input;
pub mod mcp; pub mod mcp;
pub mod learning;
pub mod quit_confirm; pub mod quit_confirm;
pub mod rewind; pub mod rewind;
pub mod settings; pub mod settings;
pub mod todo; pub mod todo;
pub mod learning;
-1
View File
@@ -1,5 +1,4 @@
//! Quit-confirm mode: the "are you sure?" overlay shown before exiting. //! Quit-confirm mode: the "are you sure?" overlay shown before exiting.
use crate::app::runtime::actions::Action; use crate::app::runtime::actions::Action;
/// Translate the user's yes/no answer on the quit-confirm overlay into an action. /// Translate the user's yes/no answer on the quit-confirm overlay into an action.
+18 -7
View File
@@ -1,13 +1,19 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)] #![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Rewind mode: restores a file to a pre-edit snapshot stored in the //! Rewind mode: restores a file to a pre-edit snapshot stored in the
//! session's `SQLite` blob store. //! session's `SQLite` blob store.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
use sha2::Digest; use sha2::Digest;
/// Returns the number of stored pre-edit blobs (snapshots) for this session. /// Returns the number of stored pre-edit blobs (snapshots) for this session.
pub fn rewind_count(state: &AppStateRest) -> usize { pub fn rewind_count(state: &AppStateRest) -> usize {
let Ok(conn) = open_session_db(&state.session_dir) else { return 0 }; let Ok(conn) = open_session_db(&state.session_dir) else {
return 0;
};
crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id) crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id)
.ok() .ok()
.map_or(0, |keys| keys.len()) .map_or(0, |keys| keys.len())
@@ -51,7 +57,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
} }
let blob_key = &keys[index]; let blob_key = &keys[index];
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key) { let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key)
{
Ok(Some(b)) => b, Ok(Some(b)) => b,
Ok(None) => { Ok(None) => {
state.push_toast(crate::app::state::types::Toast::new( state.push_toast(crate::app::state::types::Toast::new(
@@ -74,8 +81,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
// Look up the path from the edit log — the blob key is the tool_call_id. // Look up the path from the edit log — the blob key is the tool_call_id.
// The edit log doesn't store the tool_call_id directly, so fall back to the // The edit log doesn't store the tool_call_id directly, so fall back to the
// path from the most recent write/edit entry. // path from the most recent write/edit entry.
let restore_path = find_edit_path(state, blob_key) let restore_path =
.unwrap_or_else(|| state.session_dir.join("snapshot.dat")); find_edit_path(state, blob_key).unwrap_or_else(|| state.session_dir.join("snapshot.dat"));
match std::fs::write(&restore_path, &bytes) { match std::fs::write(&restore_path, &bytes) {
Ok(()) => { Ok(()) => {
@@ -119,6 +126,10 @@ fn open_session_db(session_dir: &std::path::Path) -> anyhow::Result<rusqlite::Co
fn find_edit_path(state: &AppStateRest, _blob_key: &str) -> Option<std::path::PathBuf> { fn find_edit_path(state: &AppStateRest, _blob_key: &str) -> Option<std::path::PathBuf> {
let el = crate::model::editlog::EditLog::new(&state.session_dir); let el = crate::model::editlog::EditLog::new(&state.session_dir);
let entry = el.entries.iter().rev().find(|e| e.tool == "write" || e.tool == "edit")?; let entry = el
.entries
.iter()
.rev()
.find(|e| e.tool == "write" || e.tool == "edit")?;
Some(std::path::PathBuf::from(&entry.path)) Some(std::path::PathBuf::from(&entry.path))
} }
+1 -2
View File
@@ -3,8 +3,7 @@
//! Flow: exposes small mutation functions (currently just cycling the //! Flow: exposes small mutation functions (currently just cycling the
//! internet access mode) invoked by keybindings while the settings overlay //! internet access mode) invoked by keybindings while the settings overlay
//! is active. //! is active.
use crate::model::settings::{InternetMode, Settings};
use crate::model::settings::{Settings, InternetMode};
/// Advance the internet access mode to the next value in the cycle. /// Advance the internet access mode to the next value in the cycle.
/// ///
-1
View File
@@ -2,7 +2,6 @@
//! //!
//! Flow: exposes the toggle handler invoked by a keybinding to show/hide //! Flow: exposes the toggle handler invoked by a keybinding to show/hide
//! the todo overlay. //! the todo overlay.
use crate::app::state::rest::AppStateRest; use crate::app::state::rest::AppStateRest;
use crate::app::state::types::Overlay; use crate::app::state::types::Overlay;
+1 -1
View File
@@ -76,7 +76,7 @@ pub fn should_trigger_review(state: &AppStateRest, origin: Origin) -> bool {
return false; return false;
} }
let Some(runtime) = &state.session_runtime else { return false }; let Some(runtime) = &state.session_runtime else { return false };
if !state.settings.review_enabled { if !state.settings.flags.review_enabled {
return false; return false;
} }
if runtime.edit_count > 0 && runtime.edit_count % 5 == 0 { if runtime.edit_count > 0 && runtime.edit_count % 5 == 0 {
+68 -133
View File
@@ -545,44 +545,21 @@ pub fn apply_action(state: &mut AppStateRest, action: Action) {
state.push_toast(Toast::new(ToastKind::Warning, "Aborting generation...".to_string())); state.push_toast(Toast::new(ToastKind::Warning, "Aborting generation...".to_string()));
} }
Action::Compact => { Action::Compact => {
let Some(messages) = state.session_runtime.as_ref().map(|rt| rt.messages.clone()) else { let max_wire_tokens = state.app_config.model_roles.values()
return; .find(|role| role.provider == state.settings.provider && role.model == state.settings.model)
}; .and_then(|role| role.context_window)
if messages.is_empty() { .unwrap_or(state.app_config.default_context_window) as usize;
return;
} if let Some(ref mut rt) = state.session_runtime {
let (api_key, model, base_url) = match resolve_llm_client_config(state) { let total_chars: usize = rt.messages.iter()
Ok(v) => v, .filter_map(|m| m.content.as_deref())
Err(msg) => { .map(str::len)
state.push_toast(Toast::new(ToastKind::Error, msg));
return;
}
};
let max_wire_tokens = crate::app::runtime::context::window::resolve(&state.app_config, &state.settings);
let turn_events = state.turn_events.clone();
state.push_toast(Toast::new(ToastKind::Info, "Compacting conversation history...".to_string()));
// Manual /compact previously ran synchronously and always
// passed `client: None` to shape_messages, so it never got
// LLM summarization — only automatic mid-turn compaction did.
// Running this on a background thread (same pattern as
// spawn_turn) fixes that asymmetry: both paths now summarize
// dropped history with the LLM instead of one silently
// falling back to a bare placeholder.
std::thread::spawn(move || {
let client = crate::service::provider::LlmClient::new(api_key, model, base_url);
let (deduped, _) = crate::app::runtime::context::dedup::collapse(&messages);
let token_count: usize = deduped.iter()
.map(crate::app::runtime::context::tokens::count_message_tokens)
.sum(); .sum();
let compacted = crate::app::runtime::context::shaping::shape_messages( let token_estimate = total_chars / 3;
&deduped, token_count, max_wire_tokens, true, Some(&client), rt.messages = crate::app::runtime::shortsend::shape_messages(&rt.messages, token_estimate, max_wire_tokens, true, None);
); state.push_toast(Toast::new(ToastKind::Success, "Conversation history compacted.".to_string()));
if let Ok(mut q) = turn_events.lock() { state.dirty = true;
q.push_back(TurnEvent::Compacted(compacted)); }
}
});
} }
Action::LessonAccept { name } => { Action::LessonAccept { name } => {
if let Some(ref rt) = state.session_runtime { if let Some(ref rt) = state.session_runtime {
@@ -623,45 +600,6 @@ pub fn apply_action(state: &mut AppStateRest, action: Action) {
} }
} }
/// Resolve the API key, model name, and base URL for the currently
/// configured provider.
///
/// Flow: look up the provider's `ProviderConfig` for its `api_base` ->
/// resolve the API key from `Settings.api_keys`, falling back to the
/// provider's `api_key_env` environment variable, then its
/// `default_api_key`, then the crate-wide empty-string default.
///
/// Why: this exact resolution was duplicated between `spawn_turn` and
/// needed again for `Action::Compact`'s background-thread LLM call —
/// factored out so both stay in sync.
///
/// Return: `Ok((api_key, model, base_url))`, or `Err(message)` — a
/// user-facing string — if the configured provider has no entry in
/// `AppConfig.providers` at all.
fn resolve_llm_client_config(state: &AppStateRest) -> Result<(String, String, Option<String>), String> {
let base_url = state.app_config.providers.get(&state.settings.provider).map(|p| p.api_base.clone());
let Some(base_url) = base_url else {
return Err(format!(
"Provider '{}' is not configured — no matching entry found. \
Pick a different provider in Settings, or configure it.",
state.settings.provider
));
};
let mut api_key = state.settings.api_keys.get(&state.settings.provider).cloned().unwrap_or_default();
if api_key.is_empty() {
if let Some(provider_cfg) = state.app_config.providers.get(&state.settings.provider) {
api_key = provider_cfg.api_key_env.as_ref()
.and_then(|env| std::env::var(env).ok())
.or_else(|| provider_cfg.default_api_key.clone())
.unwrap_or_default();
}
}
if api_key.is_empty() {
api_key = crate::service::provider::DEFAULT_API_KEY.to_string();
}
Ok((api_key, state.settings.model.clone(), Some(base_url)))
}
/// Spawn a background thread that runs one full LLM turn. /// Spawn a background thread that runs one full LLM turn.
/// ///
/// Flow: check that no turn is currently in-flight → bail if so → /// Flow: check that no turn is currently in-flight → bail if so →
@@ -692,17 +630,42 @@ fn spawn_turn(state: &AppStateRest) {
if messages.is_empty() { if messages.is_empty() {
return; return;
} }
let (api_key, model, base_url) = match resolve_llm_client_config(state) { let mut api_key = state.settings.api_keys.get(&state.settings.provider).cloned().unwrap_or_default();
Ok(v) => v, let model = state.settings.model.clone();
Err(msg) => { let base_url = state.app_config.providers.get(&state.settings.provider)
if let Ok(mut q) = state.turn_events.lock() { .map(|p| p.api_base.clone());
q.push_back(TurnEvent::Error(msg)); let context_window = state.app_config.model_roles.values()
} .find(|role| role.provider == state.settings.provider && role.model == state.settings.model)
return; .and_then(|role| role.context_window)
.unwrap_or(state.app_config.default_context_window) as usize;
// The selected provider has no entry in app_config at all (e.g. the
// Claude-settings auto-detection that registers "claude" found nothing
// this run). Without this check, LlmClient::new silently falls back to
// the zen default base URL while keeping this provider's model name —
// a mismatched request that reaches a real server and comes back as a
// confusing "Missing API key" 401 from an unrelated provider, instead
// of the actual problem: the configured provider doesn't exist.
if base_url.is_none() {
if let Ok(mut q) = state.turn_events.lock() {
q.push_back(TurnEvent::Error(format!(
"Provider '{}' is not configured — no matching entry found. \
Pick a different provider in Settings, or configure it.",
state.settings.provider
)));
} }
}; return;
let context_window = crate::app::runtime::context::window::resolve(&state.app_config, &state.settings); }
let concise_output = state.settings.concise_output; if api_key.is_empty() {
if let Some(provider_cfg) = state.app_config.providers.get(&state.settings.provider) {
api_key = provider_cfg.api_key_env.as_ref()
.and_then(|env| std::env::var(env).ok())
.or_else(|| provider_cfg.default_api_key.clone())
.unwrap_or_default();
}
}
if api_key.is_empty() {
api_key = crate::service::provider::DEFAULT_API_KEY.to_string();
}
let (temperature, max_tokens) = crate::app::mode::effort::generation_params( let (temperature, max_tokens) = crate::app::mode::effort::generation_params(
state.misc.effort_level, state.misc.effort_level,
state.settings.max_tokens, state.settings.max_tokens,
@@ -747,7 +710,6 @@ fn spawn_turn(state: &AppStateRest) {
max_tokens, max_tokens,
abort_flag, abort_flag,
hive_mind_converged, hive_mind_converged,
concise_output,
}; };
let result = run_agent_turn(&tc, &messages, &events_q); let result = run_agent_turn(&tc, &messages, &events_q);
if let Err(e) = result { if let Err(e) = result {
@@ -780,10 +742,6 @@ struct TurnCtx {
/// of this turn — whether a hive-mind convergence already completed /// of this turn — whether a hive-mind convergence already completed
/// earlier in this session. /// earlier in this session.
hive_mind_converged: bool, hive_mind_converged: bool,
/// Snapshot of `Settings.concise_output` taken at the start of this
/// turn, so the system-prompt assembly above can read it without
/// `TurnCtx` needing a `Settings` reference.
concise_output: bool,
} }
/// Build an ASCII tree of the workspace directory structure for the /// Build an ASCII tree of the workspace directory structure for the
@@ -933,9 +891,8 @@ const HIVE_MIND_KICKOFF_NOTE: &str = "The Hive is stirring — Core Intelligence
/// handle tool calls, and loop until the LLM produces a non-tool response /// handle tool calls, and loop until the LLM produces a non-tool response
/// or runs out of unfinished todo items. /// or runs out of unfinished todo items.
/// ///
/// Flow: build system prompt with workspace tree → deduplicate messages /// Flow: build system prompt with workspace tree → optionally shape
/// via `context::dedup::collapse` → optionally shape (compact) messages via /// (compact) messages via `shortsend` → call `chat_with_tools_streaming`
/// `context::shaping::{should_shape, shape_messages}` → call `chat_with_tools_streaming`
/// with a callback that pushes `StreamStart`, `StreamToken`, `Reasoning`, /// with a callback that pushes `StreamStart`, `StreamToken`, `Reasoning`,
/// and `Usage` events → on streaming success, handle tool calls (gated /// and `Usage` events → on streaming success, handle tool calls (gated
/// through `Harness::gate_tool_call`) or unwrap the final assistant /// through `Harness::gate_tool_call`) or unwrap the final assistant
@@ -970,23 +927,12 @@ fn run_agent_turn(
// workspace tree and reads all memory files each time). // workspace tree and reads all memory files each time).
let tree_info = generate_workspace_tree(&tc.workspace_roots); let tree_info = generate_workspace_tree(&tc.workspace_roots);
let memory_section = build_memory_section(&tc.ctx.memory_dir); let memory_section = build_memory_section(&tc.ctx.memory_dir);
let concise_section = if tc.concise_output {
"\n\nWrite tersely: drop articles (a/an/the), filler words (just/really/basically/\
actually/simply), pleasantries (sure/certainly/of course/happy to), and hedging. \
Fragments are fine. Code, commands, file paths, and error text must stay byte-exact \
— never abbreviate or paraphrase those. Exception: for destructive-operation \
confirmations and security-relevant warnings, always give full detail regardless of \
this instruction — clarity matters more than brevity when something risky is at stake."
} else {
""
};
let system_text = format!( let system_text = format!(
"{}\n\n{}\n\n{}{}{}", "{}\n\n{}\n\n{}{}",
crate::resources::SYSTEM_PROMPT, crate::resources::SYSTEM_PROMPT,
crate::resources::SYSTEM_TOOLS, crate::resources::SYSTEM_TOOLS,
tree_info, tree_info,
memory_section, memory_section,
concise_section,
); );
if !msgs.iter().any(|m| matches!(m.role, crate::dto::chat::message::Role::System)) { if !msgs.iter().any(|m| matches!(m.role, crate::dto::chat::message::Role::System)) {
let sys = ChatMessage::system(system_text); let sys = ChatMessage::system(system_text);
@@ -1198,43 +1144,33 @@ fn run_agent_turn(
let mut todo_retry_count = 0usize; let mut todo_retry_count = 0usize;
loop { loop {
// Dedup runs every iteration, unconditionally — repeated let total_chars: usize = msgs.iter()
// read-only tool calls (same tool + same arguments) are .filter_map(|m| m.content.as_deref())
// collapsed to their latest result before anything else, so .map(str::len)
// context stays minimal from turn 1 instead of only shrinking
// once shaping's budget threshold trips.
let (deduped, dedup_changed) = crate::app::runtime::context::dedup::collapse(&msgs);
let token_count: usize = deduped.iter()
.map(crate::app::runtime::context::tokens::count_message_tokens)
.sum(); .sum();
let token_estimate = total_chars / 4;
let max_wire_tokens = tc.context_window; let max_wire_tokens = tc.context_window;
// Skip shaping if abort was requested — the non-streaming LLM // Skip message compaction if abort was requested — the non-streaming
// call for summarization would block without checking abort_flag. // LLM call for summarization would block without checking abort_flag.
let wire_msgs = if !tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst) let wire_msgs = if !tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
&& crate::app::runtime::context::shaping::should_shape(token_count, max_wire_tokens, prev_shaped) && crate::app::runtime::shortsend::should_shape(token_estimate, max_wire_tokens, prev_shaped)
{ {
prev_shaped = true; prev_shaped = true;
let compacted = crate::app::runtime::context::shaping::shape_messages(&deduped, token_count, max_wire_tokens, false, Some(&tc.client)); let compacted = crate::app::runtime::shortsend::shape_messages(&msgs, token_estimate, max_wire_tokens, false, Some(&tc.client));
// Dispatch to the main thread so the local session history is // Dispatch the compacted messages to the main thread so the local session history
// permanently updated and doesn't re-trigger shaping immediately // is permanently compacted and doesn't trigger shaping again immediately on next turn.
// on the next turn.
if let Ok(mut q) = events_q.lock() { if let Ok(mut q) = events_q.lock() {
q.push_back(TurnEvent::Compacted(compacted.clone())); q.push_back(TurnEvent::Compacted(compacted.clone()));
} }
// Also update our local `msgs` variable so the rest of the loop operates on the compacted version
msgs.clone_from(&compacted); msgs.clone_from(&compacted);
compacted compacted
} else { } else {
prev_shaped = false; prev_shaped = false;
if dedup_changed { msgs.clone()
if let Ok(mut q) = events_q.lock() {
q.push_back(TurnEvent::Compacted(deduped.clone()));
}
msgs.clone_from(&deduped);
}
deduped
}; };
let mut stream_started = false; let mut stream_started = false;
@@ -1481,8 +1417,7 @@ fn run_agent_turn(
} }
} }
let squashed_output = crate::app::runtime::context::squash::apply(&tool_name, &output); let tool_msg = ChatMessage::tool_result(tool_call.id.clone(), output);
let tool_msg = ChatMessage::tool_result(tool_call.id.clone(), squashed_output);
archive_message(tc.db.as_ref(), &tc.session_id, &tool_msg); archive_message(tc.db.as_ref(), &tc.session_id, &tool_msg);
msgs.push(tool_msg); msgs.push(tool_msg);
} }
@@ -1677,7 +1612,7 @@ fn execute_one_tool(
/// `should_trigger_review` on `Tick`), only informs the user that /// `should_trigger_review` on `Tick`), only informs the user that
/// a review has material to examine. /// a review has material to examine.
fn maybe_trigger_review(state: &mut AppStateRest) { fn maybe_trigger_review(state: &mut AppStateRest) {
if !state.settings.review_enabled { if !state.settings.flags.review_enabled {
return; return;
} }
let edit_count = state let edit_count = state
+1 -1
View File
@@ -1,8 +1,8 @@
//! Maps parsed `/` slash commands into one or more `Action` variants //! Maps parsed `/` slash commands into one or more `Action` variants
//! that `apply_action` can process. //! that `apply_action` can process.
use crate::controller::command::Command;
use crate::app::runtime::actions::Action; use crate::app::runtime::actions::Action;
use crate::app::state::types::Overlay; use crate::app::state::types::Overlay;
use crate::controller::command::Command;
/// Convert a parsed `Command` into the corresponding sequence of `Action`s. /// Convert a parsed `Command` into the corresponding sequence of `Action`s.
/// ///
+37 -24
View File
@@ -15,11 +15,10 @@
//! `git_operator`, ...) are never touched, even with identical //! `git_operator`, ...) are never touched, even with identical
//! arguments, because call order and repetition can be semantically //! arguments, because call order and repetition can be semantically
//! meaningful (e.g. retrying a flaky `bash` command until it passes). //! meaningful (e.g. retrying a flaky `bash` command until it passes).
use std::collections::HashMap;
use sha2::Digest;
use crate::app::subagent::division::tool_scope::READ_TOOLS; use crate::app::subagent::division::tool_scope::READ_TOOLS;
use crate::dto::chat::message::{ChatMessage, Role}; use crate::dto::chat::message::{ChatMessage, Role};
use sha2::Digest;
use std::collections::HashMap;
const DUPLICATE_PLACEHOLDER: &str = const DUPLICATE_PLACEHOLDER: &str =
"[duplicate result — superseded by a later identical call, see below]"; "[duplicate result — superseded by a later identical call, see below]";
@@ -50,7 +49,9 @@ pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool) {
continue; continue;
} }
let Some(id) = &m.tool_call_id else { continue }; let Some(id) = &m.tool_call_id else { continue };
let Some((name, args)) = call_info.get(id) else { continue }; let Some((name, args)) = call_info.get(id) else {
continue;
};
if !READ_TOOLS.contains(&name.as_str()) { if !READ_TOOLS.contains(&name.as_str()) {
continue; continue;
} }
@@ -58,22 +59,30 @@ pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool) {
} }
let mut changed = false; let mut changed = false;
let result = messages.iter().enumerate().map(|(idx, m)| { let result = messages
if m.role != Role::Tool { .iter()
return m.clone(); .enumerate()
} .map(|(idx, m)| {
let Some(id) = &m.tool_call_id else { return m.clone() }; if m.role != Role::Tool {
let Some((name, args)) = call_info.get(id) else { return m.clone() }; return m.clone();
if !READ_TOOLS.contains(&name.as_str()) { }
return m.clone(); let Some(id) = &m.tool_call_id else {
} return m.clone();
let key = dedup_key(name, args); };
if last_index_for_key.get(&key) == Some(&idx) { let Some((name, args)) = call_info.get(id) else {
return m.clone(); return m.clone();
} };
changed = true; if !READ_TOOLS.contains(&name.as_str()) {
ChatMessage::tool_result(id.clone(), DUPLICATE_PLACEHOLDER.to_string()) return m.clone();
}).collect(); }
let key = dedup_key(name, args);
if last_index_for_key.get(&key) == Some(&idx) {
return m.clone();
}
changed = true;
ChatMessage::tool_result(id.clone(), DUPLICATE_PLACEHOLDER.to_string())
})
.collect();
(result, changed) (result, changed)
} }
@@ -102,7 +111,10 @@ mod tests {
m.tool_calls = Some(vec![ToolCall { m.tool_calls = Some(vec![ToolCall {
id: id.to_string(), id: id.to_string(),
type_: "function".to_string(), type_: "function".to_string(),
function: ToolFunction { name: name.to_string(), arguments: args }, function: ToolFunction {
name: name.to_string(),
arguments: args,
},
}]); }]);
m m
} }
@@ -172,9 +184,10 @@ mod tests {
#[test] #[test]
fn tool_result_with_no_matching_call_is_left_untouched() { fn tool_result_with_no_matching_call_is_left_untouched() {
let messages = vec![ let messages = vec![ChatMessage::tool_result(
ChatMessage::tool_result("orphan-id".to_string(), "some result".to_string()), "orphan-id".to_string(),
]; "some result".to_string(),
)];
let (result, changed) = collapse(&messages); let (result, changed) = collapse(&messages);
-1
View File
@@ -9,7 +9,6 @@
//! layer would only serve one of the two callers generically — the //! layer would only serve one of the two callers generically — the
//! auto-loop already needs per-stage control to decide when to emit //! auto-loop already needs per-stage control to decide when to emit
//! `TurnEvent::Compacted`. //! `TurnEvent::Compacted`.
pub mod dedup; pub mod dedup;
pub mod shaping; pub mod shaping;
pub mod squash; pub mod squash;
+13 -7
View File
@@ -3,7 +3,6 @@
//! the LLM API. Ported from the former `runtime::shortsend` — behavior //! the LLM API. Ported from the former `runtime::shortsend` — behavior
//! is unchanged, only its token-counting now goes through //! is unchanged, only its token-counting now goes through
//! `context::tokens` instead of an inline heuristic. //! `context::tokens` instead of an inline heuristic.
use super::tokens::count_tokens; use super::tokens::count_tokens;
use crate::dto::chat::message::ChatMessage; use crate::dto::chat::message::ChatMessage;
@@ -101,7 +100,8 @@ pub fn shape_messages(
match llm.chat_with_tools_non_streaming(&req_msgs, None) { match llm.chat_with_tools_non_streaming(&req_msgs, None) {
Ok(resp) => { Ok(resp) => {
if let Some(content) = resp.0.content { if let Some(content) = resp.0.content {
summary_text = format!("[Summary of compacted prior conversation:\n{content}\n]"); summary_text =
format!("[Summary of compacted prior conversation:\n{content}\n]");
} }
} }
Err(e) => { Err(e) => {
@@ -136,7 +136,10 @@ mod tests {
#[test] #[test]
fn should_shape_uses_95_percent_threshold_once_already_shaped() { fn should_shape_uses_95_percent_threshold_once_already_shaped() {
assert!(!should_shape(900, 1000, true), "below 95% and already shaped: no re-trigger yet"); assert!(
!should_shape(900, 1000, true),
"below 95% and already shaped: no re-trigger yet"
);
assert!(should_shape(950, 1000, true)); assert!(should_shape(950, 1000, true));
} }
@@ -186,9 +189,9 @@ mod tests {
messages.push(ChatMessage::user(padded_message(i))); messages.push(ChatMessage::user(padded_message(i)));
} }
let result = shape_messages(&messages, 100_000, 1000, true, None); let result = shape_messages(&messages, 100_000, 1000, true, None);
let has_placeholder = result.iter().any(|m| { let has_placeholder = result
m.content.as_deref() == Some("[prior conversation compacted]") .iter()
}); .any(|m| m.content.as_deref() == Some("[prior conversation compacted]"));
assert!(has_placeholder); assert!(has_placeholder);
} }
@@ -200,6 +203,9 @@ mod tests {
} }
let result = shape_messages(&messages, 100_000, 1000, true, None); let result = shape_messages(&messages, 100_000, 1000, true, None);
let last_content = messages.last().unwrap().content.clone(); let last_content = messages.last().unwrap().content.clone();
assert!(result.iter().any(|m| m.content == last_content), "most recent message must survive shaping"); assert!(
result.iter().any(|m| m.content == last_content),
"most recent message must survive shaping"
);
} }
} }
+46 -14
View File
@@ -10,7 +10,6 @@
//! conversation's token budget even on its first occurrence, long //! conversation's token budget even on its first occurrence, long
//! before `dedup`/`shaping` ever get a chance to act on repeats or //! before `dedup`/`shaping` ever get a chance to act on repeats or
//! overall budget. //! overall budget.
use std::collections::HashSet; use std::collections::HashSet;
use std::fmt::Write; use std::fmt::Write;
@@ -220,12 +219,24 @@ fn squash_log(text: &str) -> String {
level_score + stack_boost level_score + stack_boost
}; };
let mut error_idxs: Vec<usize> = (0..lines.len()).filter(|&i| levels[i] == LogLevel::Error).collect(); let mut error_idxs: Vec<usize> = (0..lines.len())
error_idxs.sort_by(|&a, &b| score(b).partial_cmp(&score(a)).unwrap_or(std::cmp::Ordering::Equal)); .filter(|&i| levels[i] == LogLevel::Error)
.collect();
error_idxs.sort_by(|&a, &b| {
score(b)
.partial_cmp(&score(a))
.unwrap_or(std::cmp::Ordering::Equal)
});
error_idxs.truncate(20); error_idxs.truncate(20);
let mut warn_idxs: Vec<usize> = (0..lines.len()).filter(|&i| levels[i] == LogLevel::Warn).collect(); let mut warn_idxs: Vec<usize> = (0..lines.len())
warn_idxs.sort_by(|&a, &b| score(b).partial_cmp(&score(a)).unwrap_or(std::cmp::Ordering::Equal)); .filter(|&i| levels[i] == LogLevel::Warn)
.collect();
warn_idxs.sort_by(|&a, &b| {
score(b)
.partial_cmp(&score(a))
.unwrap_or(std::cmp::Ordering::Equal)
});
warn_idxs.truncate(10); warn_idxs.truncate(10);
let mut keep: HashSet<usize> = HashSet::new(); let mut keep: HashSet<usize> = HashSet::new();
@@ -257,7 +268,10 @@ fn squash_generic(text: &str, budget: usize) -> String {
let mut keep: HashSet<usize> = (0..head_end).chain(tail_start..lines.len()).collect(); let mut keep: HashSet<usize> = (0..head_end).chain(tail_start..lines.len()).collect();
let mut used: usize = lines[..head_end].iter().map(|l| l.len() + 1).sum::<usize>() let mut used: usize = lines[..head_end].iter().map(|l| l.len() + 1).sum::<usize>()
+ lines[tail_start..].iter().map(|l| l.len() + 1).sum::<usize>(); + lines[tail_start..]
.iter()
.map(|l| l.len() + 1)
.sum::<usize>();
let mut prev = ""; let mut prev = "";
for (i, &line) in lines.iter().enumerate().take(tail_start).skip(head_end) { for (i, &line) in lines.iter().enumerate().take(tail_start).skip(head_end) {
let non_trivial = !line.trim().is_empty() && line != prev; let non_trivial = !line.trim().is_empty() && line != prev;
@@ -324,8 +338,8 @@ mod tests {
assert!(text.len() > SQUASH_FLOOR_BYTES); assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("some_mcp_tool", &text); let result = apply("some_mcp_tool", &text);
let parsed: serde_json::Value = serde_json::from_str(&result) let parsed: serde_json::Value =
.expect("squashed JSON must still be valid JSON"); serde_json::from_str(&result).expect("squashed JSON must still be valid JSON");
assert_eq!(parsed["id"], "abc123", "short values must survive"); assert_eq!(parsed["id"], "abc123", "short values must survive");
assert_eq!(parsed["note"], "hi", "short values must survive"); assert_eq!(parsed["note"], "hi", "short values must survive");
@@ -357,7 +371,11 @@ mod tests {
let items = parsed["items"].as_array().unwrap(); let items = parsed["items"].as_array().unwrap();
assert_eq!(items[0].as_str().unwrap(), identifier, "index 0 is under the array cutoff and identifier-shaped, so it's kept under the normal rule"); assert_eq!(items[0].as_str().unwrap(), identifier, "index 0 is under the array cutoff and identifier-shaped, so it's kept under the normal rule");
assert_eq!(items[2].as_str().unwrap(), identifier, "index 2 is still under the cutoff (past-third means index >= 3)"); assert_eq!(
items[2].as_str().unwrap(),
identifier,
"index 2 is still under the cutoff (past-third means index >= 3)"
);
assert_ne!(items[3].as_str().unwrap(), identifier, "index 3 must be force-elided even though it's identifier-shaped and would survive at any earlier index"); assert_ne!(items[3].as_str().unwrap(), identifier, "index 3 must be force-elided even though it's identifier-shaped and would survive at any earlier index");
} }
@@ -412,20 +430,34 @@ mod tests {
let result = apply("grep", &text); let result = apply("grep", &text);
assert!(result.contains("src/file0.rs:0: error handling for case 0"), "generic keeps head"); assert!(
assert!(result.contains("src/file49.rs:49: error handling for case 49"), "generic keeps tail — squash_log would have dropped this"); result.contains("src/file0.rs:0: error handling for case 0"),
"generic keeps head"
);
assert!(
result.contains("src/file49.rs:49: error handling for case 49"),
"generic keeps tail — squash_log would have dropped this"
);
} }
#[test] #[test]
fn generic_large_text_is_truncated_with_omission_marker() { fn generic_large_text_is_truncated_with_omission_marker() {
let lines: Vec<String> = (0..500).map(|i| format!("line number {i} of plain output")).collect(); let lines: Vec<String> = (0..500)
.map(|i| format!("line number {i} of plain output"))
.collect();
let text = lines.join("\n"); let text = lines.join("\n");
assert!(text.len() > SQUASH_FLOOR_BYTES); assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("bash", &text); let result = apply("bash", &text);
assert!(result.contains("line number 0 of plain output"), "keeps head"); assert!(
assert!(result.contains("line number 499 of plain output"), "keeps tail"); result.contains("line number 0 of plain output"),
"keeps head"
);
assert!(
result.contains("line number 499 of plain output"),
"keeps tail"
);
assert!(result.contains("lines omitted")); assert!(result.contains("lines omitted"));
assert!(result.len() < text.len()); assert!(result.len() < text.len());
} }
+3 -2
View File
@@ -10,7 +10,6 @@
//! `o200k_base` is an approximation for non-OpenAI providers but is far //! `o200k_base` is an approximation for non-OpenAI providers but is far
//! closer than a flat byte-per-token guess; it's only used for the //! closer than a flat byte-per-token guess; it's only used for the
//! 85%/95% budget thresholds, not for billing-accurate counts. //! 85%/95% budget thresholds, not for billing-accurate counts.
use crate::dto::chat::message::ChatMessage; use crate::dto::chat::message::ChatMessage;
/// Count tokens in a single string under `o200k_base`. /// Count tokens in a single string under `o200k_base`.
@@ -21,7 +20,9 @@ use crate::dto::chat::message::ChatMessage;
/// (e.g. literal text `<|endoftext|>` pasted by a user) must be counted /// (e.g. literal text `<|endoftext|>` pasted by a user) must be counted
/// as ordinary text, not interpreted as a control token. /// as ordinary text, not interpreted as a control token.
pub fn count_tokens(text: &str) -> usize { pub fn count_tokens(text: &str) -> usize {
tiktoken_rs::o200k_base_singleton().encode_ordinary(text).len() tiktoken_rs::o200k_base_singleton()
.encode_ordinary(text)
.len()
} }
/// Count tokens in a `ChatMessage`'s text content. /// Count tokens in a `ChatMessage`'s text content.
+31 -18
View File
@@ -5,7 +5,6 @@
//! had their own inline version — the status bar's copy additionally //! had their own inline version — the status bar's copy additionally
//! displayed "?" on no match instead of falling back like the other two, //! displayed "?" on no match instead of falling back like the other two,
//! an inconsistency this unifies away). //! an inconsistency this unifies away).
use crate::model::app_config::AppConfig; use crate::model::app_config::AppConfig;
use crate::model::settings::Settings; use crate::model::settings::Settings;
@@ -18,7 +17,9 @@ use crate::model::settings::Settings;
/// ///
/// Return: always a concrete token count, never "unknown". /// Return: always a concrete token count, never "unknown".
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize { pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize {
app_config.model_roles.values() app_config
.model_roles
.values()
.find(|role| role.provider == settings.provider && role.model == settings.model) .find(|role| role.provider == settings.provider && role.model == settings.model)
.and_then(|role| role.context_window) .and_then(|role| role.context_window)
.unwrap_or(app_config.default_context_window) as usize .unwrap_or(app_config.default_context_window) as usize
@@ -32,13 +33,16 @@ mod tests {
#[test] #[test]
fn resolves_context_window_from_matching_model_role() { fn resolves_context_window_from_matching_model_role() {
let mut app_config = AppConfig::default(); let mut app_config = AppConfig::default();
app_config.model_roles.insert("default".to_string(), ModelRole { app_config.model_roles.insert(
provider: "zen".to_string(), "default".to_string(),
model: "deepseek-v4-flash-free".to_string(), ModelRole {
max_tokens: None, provider: "zen".to_string(),
context_window: Some(128_000), model: "deepseek-v4-flash-free".to_string(),
temperature: None, max_tokens: None,
}); context_window: Some(128_000),
temperature: None,
},
);
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.provider = "zen".to_string(); settings.provider = "zen".to_string();
settings.model = "deepseek-v4-flash-free".to_string(); settings.model = "deepseek-v4-flash-free".to_string();
@@ -53,23 +57,32 @@ mod tests {
settings.provider = "nonexistent".to_string(); settings.provider = "nonexistent".to_string();
settings.model = "nonexistent-model".to_string(); settings.model = "nonexistent-model".to_string();
assert_eq!(resolve(&app_config, &settings), app_config.default_context_window as usize); assert_eq!(
resolve(&app_config, &settings),
app_config.default_context_window as usize
);
} }
#[test] #[test]
fn falls_back_to_default_when_matching_role_has_no_context_window_set() { fn falls_back_to_default_when_matching_role_has_no_context_window_set() {
let mut app_config = AppConfig::default(); let mut app_config = AppConfig::default();
app_config.model_roles.insert("default".to_string(), ModelRole { app_config.model_roles.insert(
provider: "zen".to_string(), "default".to_string(),
model: "deepseek-v4-flash-free".to_string(), ModelRole {
max_tokens: None, provider: "zen".to_string(),
context_window: None, model: "deepseek-v4-flash-free".to_string(),
temperature: None, max_tokens: None,
}); context_window: None,
temperature: None,
},
);
let mut settings = Settings::default(); let mut settings = Settings::default();
settings.provider = "zen".to_string(); settings.provider = "zen".to_string();
settings.model = "deepseek-v4-flash-free".to_string(); settings.model = "deepseek-v4-flash-free".to_string();
assert_eq!(resolve(&app_config, &settings), app_config.default_context_window as usize); assert_eq!(
resolve(&app_config, &settings),
app_config.default_context_window as usize
);
} }
} }
+61 -92
View File
@@ -1,8 +1,6 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
//! SSE stream parser: converts SSE- or JSON-chunked LLM responses into //! SSE stream parser: converts SSE- or JSON-chunked LLM responses into
//! typed `StreamEvent` variants (tokens, reasoning, tool calls, usage, done). //! typed `StreamEvent` variants (tokens, reasoning, tool calls, usage, done).
pub mod turn; pub mod turn;
pub mod tools;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::Value; use serde_json::Value;
@@ -89,7 +87,7 @@ impl SseParser {
/// provider-specific parsing layer. /// provider-specific parsing layer.
/// ///
/// Return: 0, 1, or more `StreamEvent`s from the flushed frame. /// Return: 0, 1, or more `StreamEvent`s from the flushed frame.
#[allow(clippy::too_many_lines)]
fn flush_event(&mut self) -> Vec<StreamEvent> { fn flush_event(&mut self) -> Vec<StreamEvent> {
let data = self.data_lines.join("\n"); let data = self.data_lines.join("\n");
self.data_lines.clear(); self.data_lines.clear();
@@ -112,20 +110,32 @@ impl SseParser {
if let Some(usage) = value.get("usage") { if let Some(usage) = value.get("usage") {
if !usage.is_null() { if !usage.is_null() {
let prompt_tokens = usage.get("prompt_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| { let prompt_tokens = usage
tracing::warn!("[stream] prompt_tokens missing in usage chunk"); .get("prompt_tokens")
0 .and_then(serde_json::Value::as_u64)
}); .unwrap_or_else(|| {
let completion_tokens = usage.get("completion_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| { tracing::warn!("[stream] prompt_tokens missing in usage chunk");
tracing::warn!("[stream] completion_tokens missing in usage chunk"); 0
0 });
}); let completion_tokens = usage
let total_tokens = usage.get("total_tokens").and_then(serde_json::Value::as_u64) .get("completion_tokens")
.and_then(serde_json::Value::as_u64)
.unwrap_or_else(|| {
tracing::warn!("[stream] completion_tokens missing in usage chunk");
0
});
let total_tokens = usage
.get("total_tokens")
.and_then(serde_json::Value::as_u64)
.unwrap_or_else(|| { .unwrap_or_else(|| {
tracing::warn!("[stream] total_tokens missing in usage chunk"); tracing::warn!("[stream] total_tokens missing in usage chunk");
prompt_tokens + completion_tokens prompt_tokens + completion_tokens
}); });
events.push(StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens }); events.push(StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
});
} }
} }
@@ -143,23 +153,32 @@ impl SseParser {
} }
// Reasoning token // Reasoning token
if let Some(reasoning) = d.get("reasoning_content").and_then(|r| r.as_str()) { if let Some(reasoning) =
d.get("reasoning_content").and_then(|r| r.as_str())
{
d_events.push(StreamEvent::Reasoning(reasoning.to_string())); d_events.push(StreamEvent::Reasoning(reasoning.to_string()));
} }
// Tool calls — iterate ALL entries, not just first() // Tool calls — iterate ALL entries, not just first()
if let Some(tool_calls) = d.get("tool_calls").and_then(|tc| tc.as_array()) { if let Some(tool_calls) =
d.get("tool_calls").and_then(|tc| tc.as_array())
{
for tc in tool_calls { for tc in tool_calls {
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| { let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] tool call delta missing index, defaulting to 0"); tracing::warn!("[stream] tool call delta missing index, defaulting to 0");
0 0
}) as usize; }) as usize;
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string); let id = tc
let name = tc.get("function") .get("id")
.and_then(|i| i.as_str())
.map(std::string::ToString::to_string);
let name = tc
.get("function")
.and_then(|f| f.get("name")) .and_then(|f| f.get("name"))
.and_then(|n| n.as_str()) .and_then(|n| n.as_str())
.map(std::string::ToString::to_string); .map(std::string::ToString::to_string);
let args_delta = tc.get("function") let args_delta = tc
.get("function")
.and_then(|f| f.get("arguments")) .and_then(|f| f.get("arguments"))
.and_then(|a| a.as_str()) .and_then(|a| a.as_str())
.unwrap_or("") .unwrap_or("")
@@ -174,7 +193,9 @@ impl SseParser {
} }
// Finish reason // Finish reason
if let Some(reason) = choice.get("finish_reason").and_then(|r| r.as_str()) { if let Some(reason) =
choice.get("finish_reason").and_then(|r| r.as_str())
{
if reason == "stop" || reason == "tool_calls" { if reason == "stop" || reason == "tool_calls" {
d_events.push(StreamEvent::Done); d_events.push(StreamEvent::Done);
} }
@@ -193,72 +214,6 @@ impl SseParser {
events.append(&mut other_events); events.append(&mut other_events);
events events
} }
/// Clears any partially-buffered SSE frame. Reserved for reconnect/retry flows that
/// reuse a parser instance across requests rather than constructing a fresh one.
#[allow(dead_code)]
pub fn reset(&mut self) {
self.buffer.clear();
self.event_type = None;
self.data_lines.clear();
}
}
/// Fallback parser for providers that send bare JSON chunks instead of SSE-framed
/// `data: ...` lines. Not used by the `SseParser` streaming path (which handles
/// standard SSE framing directly), kept for providers/tests that feed raw chunks.
///
/// Flow: parse `data` as JSON → extract first `choices[0].delta` →
/// return a `Token`, `Reasoning`, `Done`, or `ToolCallDelta` event based
/// on the fields present.
///
/// Return: `Some(StreamEvent)` if the chunk contained recognisable
/// content, `None` otherwise.
#[allow(dead_code)]
pub fn parse_stream_chunk(data: &str) -> Option<StreamEvent> {
let value: Value = serde_json::from_str(data).ok()?;
if value == Value::Null {
return None;
}
let choices = value.get("choices")?.as_array()?;
let choice = choices.first()?;
let delta = choice.get("delta")?;
if let Some(content) = delta.get("content").and_then(|c| c.as_str()) {
return Some(StreamEvent::Token(content.to_string()));
}
if let Some(reasoning) = delta.get("reasoning_content").and_then(|r| r.as_str()) {
return Some(StreamEvent::Reasoning(reasoning.to_string()));
}
if let Some(finish) = choice.get("finish_reason").and_then(|r| r.as_str()) {
if finish == "stop" || finish == "tool_calls" {
return Some(StreamEvent::Done);
}
}
if let Some(tool_calls) = delta.get("tool_calls").and_then(|tc| tc.as_array()) {
if let Some(tc) = tool_calls.first() {
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] fallback parser: tool call missing index, defaulting to 0");
0
}) as usize;
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string);
let name = tc.get("function")
.and_then(|f| f.get("name"))
.and_then(|n| n.as_str())
.map(std::string::ToString::to_string);
let args = tc.get("function")
.and_then(|f| f.get("arguments"))
.and_then(|a| a.as_str())
.unwrap_or("")
.to_string();
return Some(StreamEvent::ToolCallDelta {
index,
id,
name,
arguments_delta: args,
});
}
}
None
} }
#[cfg(test)] #[cfg(test)]
@@ -280,7 +235,10 @@ mod tests {
fn feed_handles_chunk_split_mid_line() { fn feed_handles_chunk_split_mid_line() {
let mut p = SseParser::new(); let mut p = SseParser::new();
let e1 = p.feed("data: {\"choices\":[{\"delta\":{\"content\":\"partial"); let e1 = p.feed("data: {\"choices\":[{\"delta\":{\"content\":\"partial");
assert!(e1.is_empty(), "no event until the line and blank separator complete"); assert!(
e1.is_empty(),
"no event until the line and blank separator complete"
);
let e2 = p.feed("\"}}]}\n\n"); let e2 = p.feed("\"}}]}\n\n");
assert_eq!(e2.len(), 1); assert_eq!(e2.len(), 1);
match &e2[0] { match &e2[0] {
@@ -300,9 +258,7 @@ mod tests {
#[test] #[test]
fn feed_emits_done_on_finish_reason_stop() { fn feed_emits_done_on_finish_reason_stop() {
let mut p = SseParser::new(); let mut p = SseParser::new();
let events = p.feed( let events = p.feed("data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n");
"data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
);
assert_eq!(events.len(), 1); assert_eq!(events.len(), 1);
assert!(matches!(events[0], StreamEvent::Done)); assert!(matches!(events[0], StreamEvent::Done));
} }
@@ -315,7 +271,12 @@ mod tests {
); );
assert_eq!(events.len(), 1); assert_eq!(events.len(), 1);
match &events[0] { match &events[0] {
StreamEvent::ToolCallDelta { index, id, name, arguments_delta } => { StreamEvent::ToolCallDelta {
index,
id,
name,
arguments_delta,
} => {
assert_eq!(*index, 0); assert_eq!(*index, 0);
assert_eq!(id.as_deref(), Some("call_1")); assert_eq!(id.as_deref(), Some("call_1"));
assert_eq!(name.as_deref(), Some("bash")); assert_eq!(name.as_deref(), Some("bash"));
@@ -333,7 +294,11 @@ mod tests {
); );
assert_eq!(events.len(), 1); assert_eq!(events.len(), 1);
match &events[0] { match &events[0] {
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens } => { StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
} => {
assert_eq!(*prompt_tokens, 10); assert_eq!(*prompt_tokens, 10);
assert_eq!(*completion_tokens, 5); assert_eq!(*completion_tokens, 5);
assert_eq!(*total_tokens, 15); assert_eq!(*total_tokens, 15);
@@ -351,7 +316,11 @@ mod tests {
assert_eq!(events.len(), 2); assert_eq!(events.len(), 2);
match (&events[0], &events[1]) { match (&events[0], &events[1]) {
( (
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens }, StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
},
StreamEvent::Token(t), StreamEvent::Token(t),
) => { ) => {
assert_eq!(*prompt_tokens, 10); assert_eq!(*prompt_tokens, 10);
-101
View File
@@ -1,101 +0,0 @@
//! Standalone accumulator for streamed tool-call deltas.
//!
//! Flow: `ToolCallAccumulator::add_delta` is fed incremental `(index, id,
//! name, arguments_delta)` chunks as they arrive over SSE → grows its
//! internal `Vec<ParsedToolCall>` as needed → `is_complete` reports once
//! every accumulated call has both a name and arguments.
//!
//! Why: mirrors the accumulation logic built into `StreamedTurn::apply_event`
//! but as an independent, reusable type for callers that want to track
//! tool-call deltas without a full `StreamedTurn` (e.g. a lighter-weight
//! preview). Currently unused (`#[allow(dead_code)]`), kept for that future
//! use case.
use super::turn::ParsedToolCall;
use serde_json::{json, Value};
/// Standalone tool-call delta accumulator, functionally equivalent to the accumulation
/// logic built into `StreamedTurn::apply_event`. Reserved for callers that want to track
/// tool-call deltas independently of a full `StreamedTurn` (e.g. a lighter-weight preview).
#[allow(dead_code)]
pub struct ToolCallAccumulator {
calls: Vec<ParsedToolCall>,
}
#[allow(dead_code)]
impl ToolCallAccumulator {
/// Construct an empty accumulator with no tool calls tracked yet.
///
/// Return: a fresh `ToolCallAccumulator`.
pub fn new() -> Self {
ToolCallAccumulator { calls: Vec::new() }
}
/// Append a delta to the tool call at the given index, growing the
/// calls vector if needed.
pub fn add_delta(
&mut self,
index: usize,
id: Option<&str>,
name: Option<&str>,
arguments_delta: &str,
) {
while self.calls.len() <= index {
self.calls.push(ParsedToolCall {
id: String::new(),
name: String::new(),
arguments: String::new(),
is_complete: false,
});
}
let tc = &mut self.calls[index];
if let Some(new_id) = id {
if !new_id.is_empty() {
tc.id = new_id.to_string();
}
}
if let Some(new_name) = name {
if !new_name.is_empty() {
tc.name = new_name.to_string();
}
}
tc.arguments.push_str(arguments_delta);
}
/// Borrow the accumulated tool calls.
pub fn calls(&self) -> &[ParsedToolCall] {
&self.calls
}
/// Return true once all tool calls have both a name and arguments.
pub fn is_complete(&self) -> bool {
!self.calls.is_empty() && self.calls.iter().all(|tc| !tc.name.is_empty() && !tc.arguments.is_empty())
}
/// Clear all accumulated calls (starting a fresh turn).
pub fn reset(&mut self) {
self.calls.clear();
}
/// Build a JSON-serialisable `Vec<Value>` of pending (non-empty-name)
/// tool calls, suitable for downstream inspection or replay.
pub fn pending_args(&self) -> Vec<Value> {
self.calls
.iter()
.filter(|tc| !tc.name.is_empty())
.map(|tc| {
json!({
"tool_call_id": tc.id,
"name": tc.name,
"arguments": tc.arguments,
})
})
.collect()
}
}
impl Default for ToolCallAccumulator {
fn default() -> Self {
Self::new()
}
}
+31 -35
View File
@@ -101,17 +101,7 @@ pub struct ParsedToolCall {
pub is_complete: bool, pub is_complete: bool,
} }
impl ParsedToolCall { impl ParsedToolCall {}
/// Attempt to parse the accumulated argument string as JSON before
/// the tool call is marked complete — useful for a speculative preview.
///
/// Return: `Some(Value)` if the arguments are parsable JSON, `None`
/// if still partial.
#[allow(dead_code)]
pub fn try_parse(&self) -> Option<Value> {
serde_json::from_str(&self.arguments).ok()
}
}
impl StreamedTurn { impl StreamedTurn {
/// Create an empty turn accumulator. /// Create an empty turn accumulator.
@@ -186,12 +176,12 @@ impl StreamedTurn {
let mut msg = if self.tool_calls.is_empty() { let mut msg = if self.tool_calls.is_empty() {
ChatMessage::assistant(None) ChatMessage::assistant(None)
} else { } else {
let tool_dtos: Vec<ToolCall> = self.tool_calls let tool_dtos: Vec<ToolCall> = self
.tool_calls
.iter() .iter()
.filter(|tc| !tc.name.is_empty()) .filter(|tc| !tc.name.is_empty())
.map(|tc| { .map(|tc| {
let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments) let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments) {
{
Ok(v) => v, Ok(v) => v,
Err(e) => { Err(e) => {
let repaired = repair_incomplete_json(&tc.arguments); let repaired = repair_incomplete_json(&tc.arguments);
@@ -200,7 +190,8 @@ impl StreamedTurn {
tracing::warn!( tracing::warn!(
"[stream] tool call '{}' had truncated JSON \ "[stream] tool call '{}' had truncated JSON \
arguments — repaired successfully: {}", arguments — repaired successfully: {}",
tc.name, e, tc.name,
e,
); );
v v
} }
@@ -209,7 +200,9 @@ impl StreamedTurn {
"[stream] tool call '{}' has invalid JSON \ "[stream] tool call '{}' has invalid JSON \
arguments: {} (after repair: {}) — falling \ arguments: {} (after repair: {}) — falling \
back to raw string", back to raw string",
tc.name, e, e2, tc.name,
e,
e2,
); );
serde_json::Value::String(tc.arguments.clone()) serde_json::Value::String(tc.arguments.clone())
} }
@@ -235,7 +228,10 @@ impl StreamedTurn {
let full_content = if self.accumulated_reasoning.is_empty() { let full_content = if self.accumulated_reasoning.is_empty() {
self.accumulated_content.clone() self.accumulated_content.clone()
} else { } else {
format!("<think>\n{}\n</think>\n\n{}", self.accumulated_reasoning, self.accumulated_content) format!(
"<think>\n{}\n</think>\n\n{}",
self.accumulated_reasoning, self.accumulated_content
)
}; };
let content = if full_content.is_empty() { let content = if full_content.is_empty() {
None None
@@ -259,7 +255,8 @@ impl StreamedTurn {
/// Return: `Some((name, parse_error))` for the first bad tool call, or /// Return: `Some((name, parse_error))` for the first bad tool call, or
/// `None` if every tool call's arguments are complete, parsable JSON. /// `None` if every tool call's arguments are complete, parsable JSON.
pub fn incomplete_tool_call(&self) -> Option<(&str, String)> { pub fn incomplete_tool_call(&self) -> Option<(&str, String)> {
self.tool_calls.iter() self.tool_calls
.iter()
.filter(|tc| !tc.name.is_empty()) .filter(|tc| !tc.name.is_empty())
.find_map(|tc| { .find_map(|tc| {
serde_json::from_str::<Value>(&tc.arguments) serde_json::from_str::<Value>(&tc.arguments)
@@ -267,19 +264,6 @@ impl StreamedTurn {
.map(|e| (tc.name.as_str(), e.to_string())) .map(|e| (tc.name.as_str(), e.to_string()))
}) })
} }
/// Reserved accessor for callers that want to branch mid-stream before the turn
/// completes; the current wiring only inspects the final `build_assistant_message()`.
#[allow(dead_code)]
pub fn has_tool_calls(&self) -> bool {
self.tool_calls.iter().any(|tc| !tc.name.is_empty())
}
/// Reserved accessor mirroring `has_tool_calls` for mid-stream content peeks.
#[allow(dead_code)]
pub fn content(&self) -> &str {
&self.accumulated_content
}
} }
impl Default for StreamedTurn { impl Default for StreamedTurn {
@@ -353,7 +337,10 @@ mod tests {
let tcs = msg.tool_calls.expect("should produce tool calls"); let tcs = msg.tool_calls.expect("should produce tool calls");
assert_eq!(tcs.len(), 1); assert_eq!(tcs.len(), 1);
let args = &tcs[0].function.arguments; let args = &tcs[0].function.arguments;
assert!(args.is_object(), "args should be an object after repair: {args:?}"); assert!(
args.is_object(),
"args should be an object after repair: {args:?}"
);
assert_eq!(args.get("path").and_then(|v| v.as_str()), Some("a.txt")); assert_eq!(args.get("path").and_then(|v| v.as_str()), Some("a.txt"));
assert_eq!(args.get("content").and_then(|v| v.as_str()), Some("short")); assert_eq!(args.get("content").and_then(|v| v.as_str()), Some("short"));
} }
@@ -361,7 +348,10 @@ mod tests {
#[test] #[test]
fn incomplete_tool_call_flags_truncated_json() { fn incomplete_tool_call_flags_truncated_json() {
let mut turn = StreamedTurn::new(); let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm")); turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"unterm",
));
let bad = turn.incomplete_tool_call(); let bad = turn.incomplete_tool_call();
assert_eq!(bad.map(|(name, _)| name), Some("write")); assert_eq!(bad.map(|(name, _)| name), Some("write"));
} }
@@ -369,7 +359,10 @@ mod tests {
#[test] #[test]
fn incomplete_tool_call_accepts_complete_json() { fn incomplete_tool_call_accepts_complete_json() {
let mut turn = StreamedTurn::new(); let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"done\"}")); turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"done\"}",
));
assert!(turn.incomplete_tool_call().is_none()); assert!(turn.incomplete_tool_call().is_none());
} }
@@ -386,7 +379,10 @@ mod tests {
// so it should still flag truncated JSON even though // so it should still flag truncated JSON even though
// `build_assistant_message` will later repair it. // `build_assistant_message` will later repair it.
let mut turn = StreamedTurn::new(); let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm")); turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"unterm",
));
// Even though it's repairable, raw parse should still fail // Even though it's repairable, raw parse should still fail
assert!(serde_json::from_str::<Value>(&turn.tool_calls[0].arguments).is_err()); assert!(serde_json::from_str::<Value>(&turn.tool_calls[0].arguments).is_err());
} }
+4 -4
View File
@@ -225,16 +225,16 @@ impl InputState {
/// if none, close and return → otherwise fuzzy-match `query` against /// if none, close and return → otherwise fuzzy-match `query` against
/// `files` via `nucleo-matcher`, keep the top 10 by score. /// `files` via `nucleo-matcher`, keep the top 10 by score.
pub fn open_mention_autocomplete(&mut self, files: &[String]) { pub fn open_mention_autocomplete(&mut self, files: &[String]) {
use nucleo_matcher::{Config, Matcher};
use nucleo_matcher::pattern::{CaseMatching, Normalization, Pattern};
let Some((start, query)) = self.mention_query_at_cursor() else { let Some((start, query)) = self.mention_query_at_cursor() else {
self.close_autocomplete(); self.close_autocomplete();
return; return;
}; };
use nucleo_matcher::{Config, Matcher};
use nucleo_matcher::pattern::{CaseMatching, Normalization, Pattern};
let mut matcher = Matcher::new(Config::DEFAULT.match_paths()); let mut matcher = Matcher::new(Config::DEFAULT.match_paths());
let pattern = Pattern::parse(&query, CaseMatching::Smart, Normalization::Smart); let pattern = Pattern::parse(&query, CaseMatching::Smart, Normalization::Smart);
let matches = pattern.match_list(files.iter(), &mut matcher); let matched_files = pattern.match_list(files.iter(), &mut matcher);
self.autocomplete_candidates = matches.into_iter().take(10).map(|(f, _)| f.clone()).collect(); self.autocomplete_candidates = matched_files.into_iter().take(10).map(|(f, _)| f.clone()).collect();
self.autocomplete_kind = AutocompleteKind::FileMention; self.autocomplete_kind = AutocompleteKind::FileMention;
self.mention_start = start; self.mention_start = start;
self.autocomplete_idx = 0; self.autocomplete_idx = 0;
+1 -1
View File
@@ -167,7 +167,7 @@ impl AppStateRest {
// async executor entirely. It is deliberately not joined -- startup // async executor entirely. It is deliberately not joined -- startup
// must not block on language server installation, and failures are // must not block on language server installation, and failures are
// logged rather than surfaced, since editing still works without LSP. // logged rather than surfaced, since editing still works without LSP.
if state.settings.lsp_auto_provision { if state.settings.flags.lsp_auto_provision {
let lsp_mgr = state.lsp_manager.clone(); let lsp_mgr = state.lsp_manager.clone();
let msg_queue = state.lsp_provision_msgs.clone(); let msg_queue = state.lsp_provision_msgs.clone();
std::thread::spawn(move || { std::thread::spawn(move || {
+1 -2
View File
@@ -1,9 +1,8 @@
//! Per-session runtime state: message history, pending tool queue, //! Per-session runtime state: message history, pending tool queue,
//! background bash jobs, lesson/review counters, and the `TurnEvent` //! background bash jobs, lesson/review counters, and the `TurnEvent`
//! stream emitted while an agent turn is in flight. //! stream emitted while an agent turn is in flight.
use std::path::PathBuf;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::path::PathBuf;
/// Cumulative token/latency counters for a session, persisted alongside it. /// Cumulative token/latency counters for a session, persisted alongside it.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, Default)] #[derive(Debug, Clone, Copy, Serialize, Deserialize, Default)]
-1
View File
@@ -1,6 +1,5 @@
//! Opaque, serializable snapshot of application state used for //! Opaque, serializable snapshot of application state used for
//! attach/daemon IPC transfer. //! attach/daemon IPC transfer.
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
/// A JSON-boxed snapshot of app state, opaque to the transport layer. /// A JSON-boxed snapshot of app state, opaque to the transport layer.
+6 -3
View File
@@ -1,10 +1,13 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)] #![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Shared small state types: toasts, overlays, the transcript cache, //! Shared small state types: toasts, overlays, the transcript cache,
//! tool execution model, and call origin tags. //! tool execution model, and call origin tags.
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
/// Severity/category of a toast notification, used to pick its color. /// Severity/category of a toast notification, used to pick its color.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum ToastKind { pub enum ToastKind {
+72 -40
View File
@@ -15,27 +15,30 @@
//! wrote this file, let me check if it's correct before continuing"). //! wrote this file, let me check if it's correct before continuing").
//! - Background reviews catch broader concerns (missing tests, architectural //! - Background reviews catch broader concerns (missing tests, architectural
//! drift, security issues) without blocking the main agent's flow. //! drift, security issues) without blocking the main agent's flow.
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicBool, Ordering};
use std::collections::VecDeque;
use crate::app::state::runtime::TurnEvent; use crate::app::state::runtime::TurnEvent;
use crate::app::subagent::context::build_subagent_context; use crate::app::subagent::context::build_subagent_context;
use crate::app::subagent::engine::run_subagent; use crate::app::subagent::engine::run_subagent;
use crate::app::subagent::spawn::AgentDefinition;
use crate::app::subagent::event::SubagentEvent; use crate::app::subagent::event::SubagentEvent;
use crate::app::subagent::spawn::AgentDefinition;
use std::collections::VecDeque;
use std::path::Path;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
/// File extensions that should not trigger auto-review (config, lock, data). /// File extensions that should not trigger auto-review (config, lock, data).
const SKIP_REVIEW_EXTENSIONS: &[&str] = &[ const SKIP_REVIEW_EXTENSIONS: &[&str] = &[
".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml", ".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml", ".svg", ".png", ".jpg", ".ico",
".svg", ".png", ".jpg", ".ico", ".woff", ".woff2", ".woff", ".woff2",
]; ];
/// File names that should not trigger auto-review. /// File names that should not trigger auto-review.
const SKIP_REVIEW_FILES: &[&str] = &[ const SKIP_REVIEW_FILES: &[&str] = &[
"Cargo.lock", "yarn.lock", "package-lock.json", "Cargo.lock",
".gitignore", ".env", ".env.example", "yarn.lock",
"package-lock.json",
".gitignore",
".env",
".env.example",
]; ];
/// Prevents a second background subagent of the same kind from spawning /// Prevents a second background subagent of the same kind from spawning
@@ -137,8 +140,19 @@ fn is_production_code(path: &str) -> bool {
.is_some_and(|ext| { .is_some_and(|ext| {
matches!( matches!(
ext, ext,
"rs" | "ts" | "tsx" | "js" | "jsx" | "go" | "py" | "java" | "kt" | "swift" "rs" | "ts"
| "c" | "cpp" | "h" | "hpp" | "tsx"
| "js"
| "jsx"
| "go"
| "py"
| "java"
| "kt"
| "swift"
| "c"
| "cpp"
| "h"
| "hpp"
) )
}) })
} }
@@ -167,11 +181,8 @@ pub fn spawn_quick_review(
file_path, file_path,
); );
let def = AgentDefinition::new( let def = AgentDefinition::new("quick-reviewer".to_string(), "reviewer".to_string())
"quick-reviewer".to_string(), .with_system_prompt(prompt);
"reviewer".to_string(),
)
.with_system_prompt(prompt);
let mut ctx = build_subagent_context(&def); let mut ctx = build_subagent_context(&def);
ctx.session_dir = session_dir.to_path_buf(); ctx.session_dir = session_dir.to_path_buf();
@@ -187,7 +198,7 @@ pub fn spawn_quick_review(
SubagentEvent::ToolResult { tool, .. } => { SubagentEvent::ToolResult { tool, .. } => {
tracing::debug!("[auto-review] tool result: {}", tool); tracing::debug!("[auto-review] tool result: {}", tool);
} }
SubagentEvent::Completed { .. } => { SubagentEvent::Completed => {
tracing::debug!("[auto-review] completed"); tracing::debug!("[auto-review] completed");
} }
_ => {} _ => {}
@@ -277,7 +288,10 @@ pub fn spawn_background_test_gen(
if file_paths.is_empty() { if file_paths.is_empty() {
return; return;
} }
if TEST_GEN_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() { if TEST_GEN_RUNNING
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!("[bg-test-gen] skipped — a test-gen run is already in flight"); tracing::debug!("[bg-test-gen] skipped — a test-gen run is already in flight");
return; return;
} }
@@ -306,8 +320,7 @@ pub fn spawn_background_test_gen(
"test-generator".to_string(), "test-generator".to_string(),
"coder".to_string(), // needs write access "coder".to_string(), // needs write access
) )
.with_system_prompt(prompt) .with_system_prompt(prompt);
;
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-test-gen", Some(&abort_flag)); let result = run_subagent_with_retry(&def, &sd, &ws, "bg-test-gen", Some(&abort_flag));
let message = match &result { let message = match &result {
@@ -347,7 +360,10 @@ pub fn spawn_background_arch_review(
if file_paths.is_empty() { if file_paths.is_empty() {
return; return;
} }
if ARCH_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() { if ARCH_REVIEW_RUNNING
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!("[bg-arch-review] skipped — an arch-review run is already in flight"); tracing::debug!("[bg-arch-review] skipped — an arch-review run is already in flight");
return; return;
} }
@@ -366,12 +382,8 @@ pub fn spawn_background_arch_review(
file_list, file_list,
); );
let def = AgentDefinition::new( let def = AgentDefinition::new("arch-reviewer".to_string(), "reviewer".to_string())
"arch-reviewer".to_string(), .with_system_prompt(prompt);
"reviewer".to_string(),
)
.with_system_prompt(prompt)
;
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-arch-review", Some(&abort_flag)); let result = run_subagent_with_retry(&def, &sd, &ws, "bg-arch-review", Some(&abort_flag));
let message = match &result { let message = match &result {
@@ -422,8 +434,13 @@ pub fn spawn_background_security_review(
if prod_paths.is_empty() { if prod_paths.is_empty() {
return; return;
} }
if SECURITY_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() { if SECURITY_REVIEW_RUNNING
tracing::debug!("[bg-security-review] skipped — a security-review run is already in flight"); .compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!(
"[bg-security-review] skipped — a security-review run is already in flight"
);
return; return;
} }
@@ -441,14 +458,11 @@ pub fn spawn_background_security_review(
file_list, file_list,
); );
let def = AgentDefinition::new( let def = AgentDefinition::new("security-reviewer".to_string(), "reviewer".to_string())
"security-reviewer".to_string(), .with_system_prompt(prompt);
"reviewer".to_string(),
)
.with_system_prompt(prompt)
;
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag)); let result =
run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag));
let message = match &result { let message = match &result {
Ok(output) => { Ok(output) => {
let first = output.lines().next().unwrap_or(output); let first = output.lines().next().unwrap_or(output);
@@ -493,7 +507,13 @@ pub fn spawn_all_background(
.filter(|p| is_production_code(p)) .filter(|p| is_production_code(p))
.cloned() .cloned()
.collect(); .collect();
spawn_background_test_gen(&source_paths, session_dir, workspaces, turn_events, abort_flag.clone()); spawn_background_test_gen(
&source_paths,
session_dir,
workspaces,
turn_events,
abort_flag.clone(),
);
// Background arch review: for all files that are reviewable // Background arch review: for all files that are reviewable
let reviewable: Vec<String> = file_paths let reviewable: Vec<String> = file_paths
@@ -501,10 +521,22 @@ pub fn spawn_all_background(
.filter(|p| is_reviewable_path(p)) .filter(|p| is_reviewable_path(p))
.cloned() .cloned()
.collect(); .collect();
spawn_background_arch_review(&reviewable, session_dir, workspaces, turn_events, abort_flag.clone()); spawn_background_arch_review(
&reviewable,
session_dir,
workspaces,
turn_events,
abort_flag.clone(),
);
// Background security review: only production source files // Background security review: only production source files
spawn_background_security_review(&source_paths, session_dir, workspaces, turn_events, abort_flag); spawn_background_security_review(
&source_paths,
session_dir,
workspaces,
turn_events,
abort_flag,
);
} }
#[cfg(test)] #[cfg(test)]
+6 -4
View File
@@ -1,9 +1,8 @@
//! Construction of a `SubagentContext` from an `AgentDefinition`, //! Construction of a `SubagentContext` from an `AgentDefinition`,
//! including the default read-only tool set for reviewer agents. //! including the default read-only tool set for reviewer agents.
use std::path::PathBuf;
use std::sync::{Arc, Mutex, atomic::AtomicBool};
use super::spawn::AgentDefinition; use super::spawn::AgentDefinition;
use std::path::PathBuf;
use std::sync::{atomic::AtomicBool, Arc, Mutex};
/// Default read-only tool names granted to `role == "reviewer"` agents. /// Default read-only tool names granted to `role == "reviewer"` agents.
pub const REVIEWER_ALLOWED: &[&str] = &["read", "grep", "glob", "recall", "remember"]; pub const REVIEWER_ALLOWED: &[&str] = &["read", "grep", "glob", "recall", "remember"];
@@ -40,7 +39,10 @@ pub struct SubagentContext {
pub fn build_subagent_context(def: &AgentDefinition) -> SubagentContext { pub fn build_subagent_context(def: &AgentDefinition) -> SubagentContext {
let allowed_tools = def.allowed_tools.clone().unwrap_or_else(|| { let allowed_tools = def.allowed_tools.clone().unwrap_or_else(|| {
if def.role == "reviewer" { if def.role == "reviewer" {
REVIEWER_ALLOWED.iter().map(std::string::ToString::to_string).collect() REVIEWER_ALLOWED
.iter()
.map(std::string::ToString::to_string)
.collect()
} else { } else {
Vec::new() Vec::new()
} }
+60 -14
View File
@@ -22,24 +22,64 @@ pub mod tool_scope {
/// authoritative "safe to deduplicate" classification, so there's a /// authoritative "safe to deduplicate" classification, so there's a
/// single list of read-only tool names in the codebase instead of two. /// single list of read-only tool names in the codebase instead of two.
pub const READ_TOOLS: &[&str] = &[ pub const READ_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall", "read",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition", "grep",
"lsp_references", "read_findings", "glob",
"search",
"seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
]; ];
const WRITE_TOOLS: &[&str] = &[ const WRITE_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall", "read",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition", "grep",
"lsp_references", "read_findings", "glob",
"write", "edit", "bash", "todowrite", "todofinish", "remember", "search",
"seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
"write",
"edit",
"bash",
"todowrite",
"todofinish",
"remember",
]; ];
const FULL_TOOLS: &[&str] = &[ const FULL_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall", "read",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition", "grep",
"lsp_references", "read_findings", "glob",
"write", "edit", "bash", "todowrite", "todofinish", "remember", "search",
"delete", "git_operator", "lsp_completion", "lsp_disconnect", "seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
"write",
"edit",
"bash",
"todowrite",
"todofinish",
"remember",
"delete",
"git_operator",
"lsp_completion",
"lsp_disconnect",
]; ];
/// Resolve a tier name to its concrete tool allowlist. /// Resolve a tier name to its concrete tool allowlist.
@@ -98,7 +138,13 @@ mod tests {
let read: HashSet<_> = tools_for(READ).into_iter().collect(); let read: HashSet<_> = tools_for(READ).into_iter().collect();
let write: HashSet<_> = tools_for(WRITE).into_iter().collect(); let write: HashSet<_> = tools_for(WRITE).into_iter().collect();
let full: HashSet<_> = tools_for(FULL).into_iter().collect(); let full: HashSet<_> = tools_for(FULL).into_iter().collect();
assert!(read.is_subset(&write), "read tier must be a subset of write tier"); assert!(
assert!(write.is_subset(&full), "write tier must be a subset of full tier"); read.is_subset(&write),
"read tier must be a subset of write tier"
);
assert!(
write.is_subset(&full),
"write tier must be a subset of full tier"
);
} }
} }
+1 -1
View File
@@ -632,7 +632,7 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
shared_text.truncate(50_000); shared_text.truncate(50_000);
shared_text.push_str("\n...[truncated]"); shared_text.push_str("\n...[truncated]");
} }
f.push(format!("[Auto-Shared] Sibling drone executed '{}' with args {}:\n{}", tool_name, args_json, shared_text)); f.push(format!("[Auto-Shared] Sibling drone executed '{tool_name}' with args {args_json}:\n{shared_text}"));
} }
} }
} }
+1 -11
View File
@@ -1,6 +1,5 @@
//! Event variants that a running subagent can emit to its parent via the //! Event variants that a running subagent can emit to its parent via the
//! shared mpsc channel. //! shared mpsc channel.
use serde_json::Value; use serde_json::Value;
/// Progress and outcome events emitted by `run_subagent` as it processes /// Progress and outcome events emitted by `run_subagent` as it processes
@@ -8,29 +7,20 @@ use serde_json::Value;
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub enum SubagentEvent { pub enum SubagentEvent {
StepCompleted { StepCompleted {
#[allow(dead_code)]
step: usize,
#[allow(dead_code)]
output: String, output: String,
}, },
StepFailed { StepFailed {
step: usize, step: usize,
error: String, error: String,
}, },
Completed { Completed,
#[allow(dead_code)]
output: String,
},
ToolCall { ToolCall {
tool: String, tool: String,
#[allow(dead_code)]
args: Value, args: Value,
}, },
ToolResult { ToolResult {
tool: String, tool: String,
args: Value, args: Value,
#[allow(dead_code)]
output: String,
}, },
Progress(String), Progress(String),
/// Token usage reported by the LLM after one streaming call inside the /// Token usage reported by the LLM after one streaming call inside the
Loaded 50 of 141 files, more files were not shown because too many files have changed in this diff. Show more