Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
86cc412395 | ||
|
|
1afca97c1b | ||
|
|
a00aa9bec8 | ||
|
|
7d99cd6618 | ||
|
|
e075eb7acc | ||
|
|
d6de9735ab | ||
|
|
85d16ebe97 | ||
|
|
96084f7462 | ||
|
|
82c024a854 | ||
|
|
78e402cf14 | ||
|
|
3a6e32d8f3 | ||
|
|
e080d9fc6b | ||
|
|
ea0b498829 | ||
|
|
682007a250 | ||
|
|
7ffdf44135 | ||
|
|
ceb84790bb | ||
|
|
12a03fd3d1 | ||
|
|
059ca8ea24 | ||
|
|
c219b6ec58 | ||
|
|
8bb697a53f | ||
|
|
75e9cadcd5 | ||
|
|
c4e1d7c2b2 | ||
|
|
da39033e81 | ||
|
|
f62ac3f688 | ||
|
|
aaab953d8a | ||
|
|
e0d0860d7b | ||
|
|
dcfc5b9ec0 | ||
|
|
472162d135 | ||
|
|
71b1613d84 | ||
|
|
d93bdba59b | ||
|
|
3b6abd920e | ||
|
|
7f2509ecd2 | ||
|
|
2eebedfea5 | ||
|
|
930961bd85 | ||
|
|
93be92a5fb | ||
|
|
95cae8fd8a | ||
|
|
5c413cf9a3 | ||
|
|
c386030573 | ||
|
|
199028fa2e | ||
|
|
8d77e4565c | ||
|
|
683715cd7a | ||
|
|
2bb8e6f255 | ||
|
|
c3c0ef632a | ||
|
|
5dd835ff24 | ||
|
|
0a24903eb0 | ||
|
|
de8e3703f2 | ||
|
|
b7fc335bf5 | ||
|
|
821622e80d | ||
|
|
8fbc51534d | ||
|
|
3f284cbb9a | ||
|
|
54484ed137 | ||
|
|
9bf5236680 | ||
|
|
4428e8bc01 | ||
|
|
b2e848d124 |
@@ -1,3 +1,61 @@
|
||||
# [1.14.0](https://github.com/asepharyana/zesdex/compare/v1.13.0...v1.14.0) (2026-07-16)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **context:** batasi squash_log ke tool bash saja ([7d99cd6](https://github.com/asepharyana/zesdex/commit/7d99cd66187b3fafb2ddeb19d8e8aa7db139df64))
|
||||
* **context:** perbaiki fixture test shaping agar men-drop pesan lama ([8bb697a](https://github.com/asepharyana/zesdex/commit/8bb697a53fe89c9d00f709df33ccbd7944482f07))
|
||||
* **plan:** perbaiki bug entropy gate dan fixture test squash.rs ([ceb8479](https://github.com/asepharyana/zesdex/commit/ceb84790bb741352b2b3416cda4bdbc6b42767c9))
|
||||
* **plan:** perbaiki fixture test array-cutoff squash_json ([7ffdf44](https://github.com/asepharyana/zesdex/commit/7ffdf441355cda0972b849bc9b475d3de8f39843))
|
||||
* **plan:** perbaiki fixture test shaping agar benar-benar men-drop pesan ([3a6e32d](https://github.com/asepharyana/zesdex/commit/3a6e32d8f3f0dd47d2c5ff63ab0c184b0ee4a6bc))
|
||||
* **plan:** perkuat fixture test log agar benar-benar uji squash_log ([682007a](https://github.com/asepharyana/zesdex/commit/682007a2507f209c2378796442d4bc879516191a))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **context:** tambah context::dedup untuk hasil tool yang berulang ([12a03fd](https://github.com/asepharyana/zesdex/commit/12a03fd3d1123c7a88289e44064537a71b9f574b))
|
||||
* **context:** tambah context::shaping (port dari shortsend) ([e080d9f](https://github.com/asepharyana/zesdex/commit/e080d9fc6b7daf2ab37fb242d03a3f64e8629acc))
|
||||
* **context:** tambah context::squash untuk kompresi hasil tool ([ea0b498](https://github.com/asepharyana/zesdex/commit/ea0b4988299894e4588d851ebc91704a9e73bc73))
|
||||
* **context:** tambah context::tokens dengan tiktoken-rs ([c219b6e](https://github.com/asepharyana/zesdex/commit/c219b6ec58ddad4770e491a48bbe1b6e3d0c8884))
|
||||
* **context:** tambah context::window::resolve ([059ca8e](https://github.com/asepharyana/zesdex/commit/059ca8ea246768dbec06ae5336433853e4276cec))
|
||||
* **runtime:** kompres hasil tool lewat squash sebelum masuk context ([96084f7](https://github.com/asepharyana/zesdex/commit/96084f74621636a72c209388580d0ebbb335d198))
|
||||
* **settings:** tambah mode ringkas opsional (concise_output) ([d6de973](https://github.com/asepharyana/zesdex/commit/d6de9735aba7745f5f57a8e7a5e625613c1a2569))
|
||||
|
||||
# [1.13.0](https://github.com/asepharyana/zesdex/compare/v1.12.0...v1.13.0) (2026-07-14)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* correct test assertion in dim_false_plain_text_has_no_color ([8d77e45](https://github.com/asepharyana/zesdex/commit/8d77e4565c2e3ca49058022c71e8277bdd790312))
|
||||
* Remove orphaned span_text helper function from markdown test module ([199028f](https://github.com/asepharyana/zesdex/commit/199028fa2eb6056dad2bdf0753554939c153a163))
|
||||
* **state:** cegah panic saat select mention dengan cursor stale ([dcfc5b9](https://github.com/asepharyana/zesdex/commit/dcfc5b9ec0d35f4c66b1648ded00f4f1279e0413))
|
||||
* **state:** jangan bangun mention index di mode attach ([e0d0860](https://github.com/asepharyana/zesdex/commit/e0d0860d7ba0ab6a4c9b9f1562f6a767a6424d99))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* Deteksi trigger [@mention](https://github.com/mention) dan Tab-cycle di input handler ([2eebedf](https://github.com/asepharyana/zesdex/commit/2eebedfea56c7d4827f94a2739dbb5fa006e3069))
|
||||
* **ipc:** dukung Ctrl+Y clipboard copy di mode daemon/attach ([472162d](https://github.com/asepharyana/zesdex/commit/472162d135478666913a888a68867e635e50d14a))
|
||||
* Judul dropdown autocomplete mengikuti jenisnya (Commands vs Files) ([7f2509e](https://github.com/asepharyana/zesdex/commit/7f2509ecd2688495545333debee0c81016b28641))
|
||||
* **state:** Alirkan mention_index lewat ToolCtx dan AppStateRest, bangun index di background thread ([93be92a](https://github.com/asepharyana/zesdex/commit/93be92a5fb8644c9020478c49421585ade2d0fd1))
|
||||
* Tambah Ctrl+Y untuk menyalin pesan assistant terakhir ([d93bdba](https://github.com/asepharyana/zesdex/commit/d93bdba59b7dd14b4a76dc91c6b7f70a3c0f721c))
|
||||
* Tambah field pending_clipboard_copy di MiscState ([3b6abd9](https://github.com/asepharyana/zesdex/commit/3b6abd920ea92329ebcf888c3bb50470675c4152))
|
||||
* Tambah helper truncate_diff untuk membatasi panjang diff ([5dd835f](https://github.com/asepharyana/zesdex/commit/5dd835ff244a138bb5f1a8dbfd7cf252c2671e5e))
|
||||
* Tambah MentionIndex, AutocompleteKind, dan deteksi [@mention](https://github.com/mention) di InputState ([95cae8f](https://github.com/asepharyana/zesdex/commit/95cae8fd8a8ce737ba60c3ac8bea77f4c2e14b1a))
|
||||
* Tambah write_osc52 dan salin ke clipboard di mode single-process ([71b1613](https://github.com/asepharyana/zesdex/commit/71b1613d8467cfcb2383b3fce153a25c7883ac3d))
|
||||
* Tambahkan file baru ke mention_index saat tool write membuatnya ([930961b](https://github.com/asepharyana/zesdex/commit/930961bd85701ab8906b8d81095e46d3635a951d))
|
||||
* Tampilkan unified diff pada hasil tool edit ([c3c0ef6](https://github.com/asepharyana/zesdex/commit/c3c0ef632a605d8bf32712049035e4fc814af9d5))
|
||||
* Tampilkan unified diff saat tool write menimpa file yang sudah ada ([2bb8e6f](https://github.com/asepharyana/zesdex/commit/2bb8e6f2555eac01baf2311d41811afad4aa3041))
|
||||
* **view:** Tambah parameter dim dan pewarnaan baris diff di markdown renderer ([683715c](https://github.com/asepharyana/zesdex/commit/683715cd7af77d87a2ae3834360bb6f7a4388bd0))
|
||||
|
||||
# [1.12.0](https://github.com/asepharyana/zesdex/compare/v1.11.0...v1.12.0) (2026-07-14)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* Add mouse capture functionality to terminal and enhance markdown rendering with table support ([4428e8b](https://github.com/asepharyana/zesdex/commit/4428e8bc01c196415ac408a42f57305227a79760))
|
||||
* Improve markdown rendering with enhanced line wrapping and indentation for code blocks ([b2e848d](https://github.com/asepharyana/zesdex/commit/b2e848d124e726c4d8b644d473e518398fab1dea))
|
||||
|
||||
# [1.11.0](https://github.com/asepharyana/zesdex/compare/v1.10.0...v1.11.0) (2026-07-14)
|
||||
|
||||
|
||||
|
||||
Generated
+51
-2
@@ -196,6 +196,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "5cee35f73844aa3014bb606320a6c1f010249dbdf43342fe54b5a4f6a8ed4b79"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
"regex-automata",
|
||||
"serde_core",
|
||||
]
|
||||
|
||||
@@ -807,6 +808,17 @@ dependencies = [
|
||||
"regex-syntax",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fancy-regex"
|
||||
version = "0.17.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "72cf461f865c862bb7dc573f643dd6a2b6842f7c30b07882b56bd148cc2761b8"
|
||||
dependencies = [
|
||||
"bit-set 0.8.0",
|
||||
"regex-automata",
|
||||
"regex-syntax",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "fast-srgb8"
|
||||
version = "1.0.0"
|
||||
@@ -1915,6 +1927,16 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "nucleo-matcher"
|
||||
version = "0.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "bf33f538733d1a5a3494b836ba913207f14d9d4a1d3cd67030c5061bdd2cac85"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
"unicode-segmentation",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "num-conv"
|
||||
version = "0.2.2"
|
||||
@@ -3149,6 +3171,15 @@ version = "0.1.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
|
||||
|
||||
[[package]]
|
||||
name = "similar"
|
||||
version = "3.1.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e6505efef05804732ed8a3f2d4f279429eb485bd69d5b0cc6b19cc02005cda16"
|
||||
dependencies = [
|
||||
"bstr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "siphasher"
|
||||
version = "1.0.3"
|
||||
@@ -3362,7 +3393,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
|
||||
dependencies = [
|
||||
"fastrand",
|
||||
"getrandom 0.3.4",
|
||||
"getrandom 0.4.3",
|
||||
"once_cell",
|
||||
"rustix",
|
||||
"windows-sys 0.61.2",
|
||||
@@ -3502,6 +3533,21 @@ dependencies = [
|
||||
"cfg-if",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tiktoken-rs"
|
||||
version = "0.12.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "027853bbf8c7763b77c5c595f1c271c7d536ced7d6f83452911b944621e57fc2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"base64",
|
||||
"bstr",
|
||||
"fancy-regex 0.17.0",
|
||||
"lazy_static",
|
||||
"regex",
|
||||
"rustc-hash",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "time"
|
||||
version = "0.3.53"
|
||||
@@ -4436,7 +4482,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "zesdex"
|
||||
version = "1.11.0"
|
||||
version = "1.14.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"base64",
|
||||
@@ -4453,6 +4499,7 @@ dependencies = [
|
||||
"infer",
|
||||
"libc",
|
||||
"lsp-types",
|
||||
"nucleo-matcher",
|
||||
"percent-encoding",
|
||||
"pulldown-cmark",
|
||||
"ratatui",
|
||||
@@ -4465,7 +4512,9 @@ dependencies = [
|
||||
"serde_json",
|
||||
"serde_yaml_ng",
|
||||
"sha2 0.11.0",
|
||||
"similar",
|
||||
"syntect",
|
||||
"tiktoken-rs",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
|
||||
+4
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "zesdex"
|
||||
version = "1.11.0"
|
||||
version = "1.14.0"
|
||||
edition = "2021"
|
||||
authors = ["asepharyana <superaseph@gmail.com>"]
|
||||
|
||||
@@ -40,11 +40,13 @@ uuid = { version = "1", features = ["v4", "v5"] }
|
||||
dirs = "6"
|
||||
futures-util = "0.3"
|
||||
pulldown-cmark = { version = "0.13", default-features = false }
|
||||
similar = "3"
|
||||
syntect = { version = "5", default-features = false, features = ["default-fancy"] }
|
||||
rusqlite = { version = "0.40", features = ["bundled"] }
|
||||
ignore = "0.4"
|
||||
regex = "1"
|
||||
globset = "0.4"
|
||||
nucleo-matcher = "0.3"
|
||||
infer = "0.19"
|
||||
base64 = "0.22"
|
||||
sha2 = "0.11"
|
||||
@@ -56,6 +58,7 @@ chrono = { version = "0.4", features = ["serde"] }
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
webbrowser = "1"
|
||||
lsp-types = "0.97"
|
||||
tiktoken-rs = "0.12"
|
||||
|
||||
[[bin]]
|
||||
name = "zesdex"
|
||||
|
||||
@@ -1,61 +1,65 @@
|
||||
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~750 -->
|
||||
|
||||
# Architecture
|
||||
|
||||
Zesdex is a single-process terminal AI coding agent with optional daemon/client split.
|
||||
# Architecture Overview
|
||||
|
||||
## System Layout
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ main.rs │
|
||||
│ single-process ─┬── daemon ── Unix socket ── client │
|
||||
│ └── attach <id> (TUI-only client) │
|
||||
└──────────────────────┬───────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────▼───────────────────────────────┐
|
||||
│ Event Loop │
|
||||
│ ┌────────┐ ┌───────────┐ ┌──────┐ ┌────────┐ │
|
||||
│ │Input │──▶│ Actions │──▶│State │──▶│ TUI │ │
|
||||
│ │Handler │ │ (dispatch)│ │ │ │ Render │ │
|
||||
│ └────────┘ └─────┬─────┘ └──────┘ └────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────▼──────┐ │
|
||||
│ │ LLM Stream │ │
|
||||
│ │ + Tool Exec │ │
|
||||
│ └──────┬──────┘ │
|
||||
│ ┌────┴────┐ │
|
||||
│ │ │ │
|
||||
│ ┌─────▼──┐ ┌───▼────┐ │
|
||||
│ │ Tools │ │Sub- │ │
|
||||
│ │ (37) │ │agents │ │
|
||||
│ └────────┘ └────────┘ │
|
||||
└───────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Data Flow
|
||||
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
|
||||
|
||||
```
|
||||
User keystroke → Controller (KeyEvent → Action)
|
||||
→ apply_action() mutates AppStateRest
|
||||
→ TUI redraws (ratatui Frame)
|
||||
→ On submit: LLM request → SSE stream → tool calls → tool results → more LLM
|
||||
→ Session persisted to disk (editlog, msglog, memory)
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Process Mode │
|
||||
│ Single-Process ─── Daemon (background) ─── Attach (client) │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│ IPC (Unix domain socket)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ src/main.rs │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
||||
│ │ Controller │──▶│ Runtime │──▶│ View │ │
|
||||
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
|
||||
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────▼────────┐ │
|
||||
│ │ Harness │ │
|
||||
│ │ (tool dispatch)│ │
|
||||
│ └───────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────┼─────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ Tools │ │ Subagents │ │ Workflow │ │
|
||||
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
|
||||
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
|
||||
│ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Process Modes
|
||||
|
||||
| Mode | Impl | Process | IPC |
|
||||
|------|------|---------|-----|
|
||||
| Single | `run_single_process()` | One | No |
|
||||
| Daemon | `run_daemon()` | Server | `ipc/server.rs` |
|
||||
| Attach | `run_attach()` | Client | `ipc/client.rs` |
|
||||
| Mode | Description |
|
||||
|------|-------------|
|
||||
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
|
||||
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
|
||||
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
|
||||
|
||||
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. **Input** → `controller/input.rs` handles key events and autocomplete
|
||||
2. **Dispatch** → `app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
|
||||
3. **LLM Stream** → `app/runtime/stream/mod.rs` parses SSE chunks into typed events
|
||||
4. **Tool Execution** → `app/harness.rs` gates and runs tool calls via the `Tool` trait
|
||||
5. **Rendering** → `view/` modules read `AppStateRest` and render via ratatui
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Lines | Role |
|
||||
|------|-------|------|
|
||||
| `src/main.rs` | 647 | Entry, TUI setup, daemon loop, attach loop |
|
||||
| `src/app/runtime/actions/mod.rs` | 1815 | Action dispatch + LLM stream loop + tool execution |
|
||||
| `src/controller/input.rs` | 365 | Key event → Action mapping |
|
||||
| `src/view/mod.rs` | 975 | TUI rendering (ratatui) |
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
|
||||
| `src/app/state/rest.rs` | Single source-of-truth state struct |
|
||||
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
|
||||
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
|
||||
| `src/app/harness.rs` | Tool harness with safety gating |
|
||||
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
|
||||
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
|
||||
| `src/view/mod.rs` | TUI render pipeline |
|
||||
|
||||
+47
-54
@@ -1,75 +1,68 @@
|
||||
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~850 -->
|
||||
# Backend Architecture
|
||||
|
||||
# Backend / Service Layer
|
||||
## Provider Layer
|
||||
|
||||
## AI Provider
|
||||
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
|
||||
|
||||
`src/service/provider.rs` (310 lines)
|
||||
- `LlmClient::new(api_key, model, base_url)` — constructs blocking reqwest client
|
||||
- `chat_with_tools()` — non-streaming with tool definitions
|
||||
- `chat_stream()` — SSE streaming, returns `SseParser` yielding `StreamEvent`
|
||||
- Retry logic: up to 3 attempts on transient errors, exponential backoff
|
||||
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
|
||||
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
|
||||
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
|
||||
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
|
||||
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
|
||||
|
||||
## OAuth
|
||||
## IPC (Inter-Process Communication)
|
||||
|
||||
`src/service/oauth/manager.rs` (113 lines) + `loopback.rs` + `pkce.rs`
|
||||
- PKCE flow: `CodeVerifier` → challenge → browser auth → loopback server → token exchange
|
||||
- Configurable via `app_config.json` provider definitions (auth URL, token URL, scopes)
|
||||
The daemon-client protocol in `src/ipc/`:
|
||||
|
||||
## IPC / Daemon
|
||||
- **Transport**: Unix domain sockets
|
||||
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
|
||||
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
|
||||
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
|
||||
|
||||
`src/ipc/` (7 files, ~350 lines total)
|
||||
- Unix domain socket, length-prefixed JSON frames
|
||||
- Daemon sends `DaemonFrame` (state payload, stream tokens, system notes)
|
||||
- Clients send `ClientRequest` (key presses, resize, submit, scroll)
|
||||
- State sync uses full-state push from daemon to client after each action
|
||||
Flow:
|
||||
```
|
||||
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
|
||||
│
|
||||
└──StatePayload──▶ Client (render)
|
||||
```
|
||||
|
||||
## Workflow Engine
|
||||
|
||||
`src/app/workflow/engine.rs` (648 lines) + `script.rs`
|
||||
- Inline JS-style DSL executed by a lightweight runtime
|
||||
- `agent()`, `parallel()`, `pipeline()`, `phase()`, `log()` — spawns sub-agents
|
||||
- Max concurrency configurable via `workflow_max_concurrency` setting
|
||||
- Hive-mind orchestrator in `hive_mind.rs`: Core Intelligence compiles a `CognitiveCyclePlan` per task — cycle count and nodes-per-cycle are decided fresh each time based on what the task actually needs
|
||||
Located in `src/app/workflow/`:
|
||||
|
||||
## Sub-Agent System
|
||||
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
|
||||
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
|
||||
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
|
||||
|
||||
`src/app/subagent/` (6 files: `spawn.rs`, `engine.rs`, `context.rs`, `event.rs`, `division.rs`, `auto.rs`, ~450 lines total)
|
||||
- `run_subagent()` — spawns independent agent with its own tool set and context
|
||||
- Communicates via `mpsc<SubagentEvent>` channel (tool calls, results, completion)
|
||||
- Uses `LlmClient` (same as main agent) with tool-use API
|
||||
- Auto-healing: on build/test failure, spawns auto-fix sub-agent
|
||||
- Node access tiers (`division.rs`'s `tool_scope` module): `read`, `write`, `full` — granted per node by the Core Intelligence based on what its directive needs
|
||||
## MCP (Model Context Protocol)
|
||||
|
||||
## MCP Client
|
||||
`src/app/mcp/manager.rs` manages MCP client connections:
|
||||
|
||||
`src/app/mcp/manager.rs` (441+ lines)
|
||||
- Stdio transport: spawns child process, JSON-RPC via stdin/stdout
|
||||
- HTTP transport: streaming HTTP with JSON-RPC
|
||||
- Dynamic tool list refresh and error recovery
|
||||
- Persistent child handle for stdio (reuses connection across calls)
|
||||
- Uses the `rmcp` crate for the MCP protocol
|
||||
- Supports stdio-based transport (child process) and streamable HTTP
|
||||
- Tool discovery via `list_tools()` and dynamic tool registration
|
||||
|
||||
## Self-Review
|
||||
## LSP Integration
|
||||
|
||||
`src/app/review/mod.rs` (495 lines)
|
||||
- Post-tool execution quality check against learned lessons
|
||||
- Invokes `run_subagent()` with reviewer prompt
|
||||
- Staleness detection: skips review after N consecutive empty results
|
||||
- Three review types: code quality, architecture, security
|
||||
`src/app/lsp/` provides Language Server Protocol support:
|
||||
|
||||
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
|
||||
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
|
||||
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
|
||||
|
||||
## Background Bash
|
||||
|
||||
`src/app/bgbash/` (2 files: `job.rs`, `control.rs`)
|
||||
- `spawn_bash_job()` — runs `sh -c` in a thread, collects stdout line-by-line
|
||||
- Channels: output via `mpsc<String>`, PID via `mpsc<u32>`
|
||||
- Killable via PID (SIGTERM)
|
||||
- Output buffering capped at 10,000 lines to prevent memory issues
|
||||
`src/app/bgbash/` manages long-running shell jobs:
|
||||
|
||||
## Gate Guard / Harness
|
||||
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
|
||||
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
|
||||
|
||||
`src/app/harness.rs` (495 lines)
|
||||
- `Harness::gate_tool_call()` — verdict-based tool gating (allow/block)
|
||||
- Path traversal, credential read, and destructive command detection
|
||||
- Pattern detection for stub code, denial language, and assumptions in write/edit content
|
||||
- Reason validation for mutating tools (minimum 8 characters, rejects generic non-answers)
|
||||
- Includes 8 unit tests for verdict parsing formats
|
||||
## Review System
|
||||
|
||||
`src/app/subagent/auto.rs` spawns background reviews:
|
||||
|
||||
- Quick review after every edit
|
||||
- Background test generation
|
||||
- Architecture review
|
||||
- Security review
|
||||
- All retry once on failure, escalate to blocking error if retry also fails
|
||||
|
||||
+80
-42
@@ -1,51 +1,89 @@
|
||||
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~600 -->
|
||||
# Data Architecture
|
||||
|
||||
# Data / Persistence Layer
|
||||
## State Model
|
||||
|
||||
## Storage Overview
|
||||
|
||||
Base directory: `~/.config/zesdex/` (via `dirs::data_dir()`)
|
||||
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
|
||||
|
||||
```
|
||||
~/.config/zesdex/
|
||||
├── settings.json # User preferences (provider, model, tokens)
|
||||
├── app_config.json # Provider definitions (API base, auth, models)
|
||||
├── agents/ # Global agent definitions
|
||||
│ └── *.json
|
||||
├── memory/ # Persistent lesson/reference store
|
||||
│ └── *.md # Markdown with YAML frontmatter
|
||||
├── sessions/ # Per-session data
|
||||
│ └── <session-uuid>/
|
||||
│ ├── edits.jsonl # Edit history (JSONL, append-only)
|
||||
│ ├── msglog.db # SQLite message log
|
||||
│ ├── transcript.json # Chat transcript
|
||||
│ ├── session.json # Session metadata
|
||||
│ ├── agents.json # Session-local agent defs
|
||||
│ └── snapshot.dat # State snapshot (daemon mode)
|
||||
├── run/ # Unix domain sockets
|
||||
│ └── zesdex-*.sock
|
||||
└── store.json # Legacy session index
|
||||
AppStateRest
|
||||
├── session: SessionRuntime (hive_mind state, convergence flag)
|
||||
├── runtime: RuntimeState (mode, provider status)
|
||||
├── chat: ChatState (messages, scroll)
|
||||
├── input: InputState (text, cursor, autocomplete)
|
||||
├── settings: Settings (provider, model, temperature, concise_output)
|
||||
├── config: AppConfig (endpoints, credentials)
|
||||
├── scroll: ScrollState (per-panel offset)
|
||||
├── diff: DiffState (edit review)
|
||||
├── tools: Vec with outputs
|
||||
├── statusline, sidebar, etc.
|
||||
└── toasts: pending notifications
|
||||
```
|
||||
|
||||
## Key Files
|
||||
**Mutation rules** (per CLAUDE.md):
|
||||
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
|
||||
- Read-only from every other module
|
||||
- No generic update function — direct field mutation only
|
||||
|
||||
| File | Lines | Role |
|
||||
|------|-------|------|
|
||||
| `src/model/store.rs` | ~50 | File-system storage (ensure_dirs, base_dir resolution) |
|
||||
| `src/model/settings.rs` | ~60 | `Settings` — load/save JSON, API keys map |
|
||||
| `src/model/app_config.rs` | ~80 | `AppConfig` — provider definitions, model roles, auth |
|
||||
| `src/model/memory.rs` | 440 | Memory CRUD — markdown files with frontmatter |
|
||||
| `src/model/editlog.rs` | 161 | Edit log — append-only JSONL (not JSON array) |
|
||||
| `src/model/msglog/` | 4 files | SQLite-backed message log (schema, query, blobs) |
|
||||
| `src/model/session.rs` | ~60 | Session CRUD, listing, archival |
|
||||
| `src/model/session_lock.rs` | ~50 | flock-based session lock |
|
||||
| `src/model/agent_def/` | 3 files | Agent definitions (builtin, global, session-local) |
|
||||
## Persistence
|
||||
|
||||
## Key Patterns
|
||||
### SQLite Message Log (`src/model/msglog/`)
|
||||
|
||||
- **No ORM** — raw JSON files + SQLite via rusqlite
|
||||
- **settings.json** — loaded at startup, saved on quit / mode switches
|
||||
- **Memory format** — Markdown files with YAML frontmatter (`---\nname: ...\ndescription: ...\n---\ncontent`)
|
||||
- **Edit log** — append-only, stores `(file, old, new, timestamp, tool)`
|
||||
- **Session locking** — flock-based, prevents concurrent access to same session dir
|
||||
- **Message log** — SQLite with attached blobs for tool arguments/outputs
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `schema.rs` | Table definitions (messages, sessions) |
|
||||
| `mod.rs` | CRUD operations |
|
||||
| `query.rs` | Query helpers (search, filter) |
|
||||
| `blobs.rs` | Large message blob storage |
|
||||
| `summary.rs` | Conversation summary cache |
|
||||
|
||||
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
|
||||
|
||||
### Memory System (`src/model/memory.rs`)
|
||||
|
||||
File-based memory stored under `~/.claude/projects/<project>/memory/`:
|
||||
|
||||
- Each memory is one markdown file with frontmatter (name, description, type)
|
||||
- Types: `user`, `feedback`, `project`, `reference`
|
||||
- Memory index in MEMORY.md
|
||||
- Export/import for lesson sharing
|
||||
- PID-file session lock prevents concurrent access
|
||||
|
||||
### Settings & Config (`src/model/`)
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `settings.rs` | Serialized user preferences (provider, model, theme) |
|
||||
| `app_config.rs` | Provider endpoints, API key resolution from env |
|
||||
| `session.rs` | Current session metadata |
|
||||
| `conversation.rs` | In-memory conversation state |
|
||||
| `editlog.rs` | Append-only JSONL edit audit trail |
|
||||
|
||||
### Edit Log
|
||||
|
||||
`src/model/editlog.rs` records every file mutation:
|
||||
|
||||
```json
|
||||
{"ts": 123, "tool": "edit", "path": "src/main.rs",
|
||||
"reason": "fix bug", "content_sha256": "abc123",
|
||||
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
|
||||
```
|
||||
|
||||
Max 5000 entries held in memory before pruning oldest.
|
||||
|
||||
## Context Management (`src/app/runtime/context/`)
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `tokens.rs` | Token counting via `tiktoken-rs` |
|
||||
| `window.rs` | Token window resolution (fit within model context) |
|
||||
| `dedup.rs` | Deduplication of repeated tool outputs |
|
||||
| `squash.rs` | Compression of large JSON tool results |
|
||||
| `shaping.rs` | Message dropping when context exceeds limits |
|
||||
|
||||
## IPC Data Flow
|
||||
|
||||
```
|
||||
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
|
||||
│
|
||||
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
|
||||
```
|
||||
|
||||
@@ -1,41 +1,99 @@
|
||||
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~400 -->
|
||||
|
||||
# Dependencies
|
||||
|
||||
## Rust Crates (Cargo.toml)
|
||||
## Rust Crates (30+ direct)
|
||||
|
||||
### Core Framework
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| ratatui | 0.30 | TUI framework (tui-rs successor) |
|
||||
| crossterm | 0.29 | Terminal manipulation (raw mode, alt screen) |
|
||||
| tokio | 1 | Async runtime (daemon, OAuth loopback) |
|
||||
| reqwest | 0.12 | HTTP client (blocking + streaming, vendored native-tls) |
|
||||
| serde / serde_json | 1 | JSON serialization (state, DTOs, IPC, config) |
|
||||
| serde_yaml_ng | 0.10 | YAML frontmatter parsing (memory files) |
|
||||
| anyhow | 1 | Error handling (no custom error types) |
|
||||
| tracing / tracing-subscriber | 0.1/0.3 | Structured logging → file |
|
||||
| rusqlite | 0.40 | SQLite (bundled, for message log) |
|
||||
| pulldown-cmark | 0.13 | Markdown → HTML (chat rendering) |
|
||||
| syntect | 5 | Syntax highlighting (code blocks in chat) |
|
||||
| sha2 | 0.10 | SHA-256 for PKCE challenge |
|
||||
| base64 | 0.22 | URL-safe base64 for PKCE |
|
||||
| libc | 0.2 | daemon PID file locking |
|
||||
| rmcp | 2.2 | MCP client (stdio + HTTP transports) |
|
||||
| uuid | 1 | Session IDs, job IDs |
|
||||
| chrono | 0.4 | Timestamps (ISO 8601, millis) |
|
||||
| dirs | 6 | Platform data directories |
|
||||
| dom_smoothie | 0.18 | HTML → plain text (web scraping) |
|
||||
| scraper | 0.27 | HTML parsing (web scraping) |
|
||||
| ignore | 0.4 | .gitignore-aware file walking (glob tool) |
|
||||
| regex / globset | 0.4 | Pattern matching (grep/glob tools) |
|
||||
| url / percent-encoding | 2 | URL parsing + encoding (OAuth) |
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation |
|
||||
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
|
||||
### HTTP & Networking
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
|
||||
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
|
||||
| `webbrowser` | 1 | Open URLs in browser |
|
||||
| `url` | 2 | URL parsing |
|
||||
| `percent-encoding` | 2 | URL encoding |
|
||||
|
||||
### HTML/Markdown
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
|
||||
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
|
||||
| `scraper` | 0.27.0 | HTML parsing/selecting |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
|
||||
|
||||
### Serialization
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `serde` | 1 | Serialization framework |
|
||||
| `serde_json` | 1 | JSON serialization |
|
||||
| `serde_yaml_ng` | 0.10 | YAML serialization |
|
||||
|
||||
### Storage & Files
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `rusqlite` | 0.40 | SQLite (bundled) |
|
||||
| `ignore` | 0.4 | `.gitignore`-aware file walking |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed directory contents in binary |
|
||||
| `infer` | 0.19 | File type detection |
|
||||
| `dirs` | 6 | Standard OS directories |
|
||||
|
||||
### Text & Search
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting |
|
||||
| `tiktoken-rs` | 0.12 | OpenAI token counting |
|
||||
|
||||
### Cryptography & Encoding
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `base64` | 0.22 | Base64 encoding |
|
||||
| `sha2` | 0.11 | SHA-256 hashing |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `uuid` | 1 | UUID generation (v4, v5) |
|
||||
| `libc` | 0.2 | Raw C FFI bindings |
|
||||
|
||||
### Error Handling & Logging
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `tracing` | 0.1 | Structured logging |
|
||||
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
|
||||
| `chrono` | 0.4 | Date/time with serde |
|
||||
|
||||
### Other
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `lsp-types` | 0.97 | LSP protocol types |
|
||||
| `futures-util` | 0.3 | Async stream combinators |
|
||||
|
||||
## External Services
|
||||
|
||||
| Service | Integration | Notes |
|
||||
|---------|-------------|-------|
|
||||
| **LLM providers** | HTTP API (OpenAI-compatible) | Configurable via app_config.json |
|
||||
| **MCP servers** | stdio or HTTP | Model Context Protocol |
|
||||
| **git** | CLI (spawns `git`) | Via git_operator/git_worktree/git_cred tools |
|
||||
| **sh** | CLI (spawns `sh`) | Via bash tool |
|
||||
| **webbrowser** | opens URL | OAuth browser flow |
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| **Anthropic API** | Primary LLM provider |
|
||||
| **OpenAI API** | Alternative LLM provider (including OAuth) |
|
||||
| **GitHub** | Release artifacts via semantic-release CI |
|
||||
| **MCP Servers** | External tool servers (stdio or HTTP) |
|
||||
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
|
||||
|
||||
## Build Configuration
|
||||
|
||||
### Compiler Lints (`.cargo/config.toml`)
|
||||
All unused code, dead code, and deprecation warnings promoted to errors:
|
||||
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
|
||||
|
||||
### Release Profile
|
||||
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
|
||||
|
||||
### CI/CD
|
||||
- **CI**: cargo build + test + clippy on every push
|
||||
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
|
||||
|
||||
+64
-49
@@ -1,64 +1,79 @@
|
||||
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~700 -->
|
||||
|
||||
# Frontend / TUI
|
||||
# Frontend (TUI) Architecture
|
||||
|
||||
## Render Pipeline
|
||||
|
||||
```
|
||||
ratatui::Terminal::draw(|frame|)
|
||||
→ view::draw(frame, AppStateRest)
|
||||
→ render_main_panel / render_overlay (based on overlay state)
|
||||
→ render_input_bar
|
||||
→ draw_status_bar
|
||||
→ render_toasts (top-right floating notifications)
|
||||
```
|
||||
|
||||
## Layout
|
||||
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ Chat Panel (main_area: Min 3) │
|
||||
│ ┌────────────────────────────────────────┐ │
|
||||
│ │ User: Hello │ │
|
||||
│ │ Agent: Hi there, how can I help? │ │
|
||||
│ │ │ │
|
||||
│ │ Toast notifications (top-right) │ │
|
||||
│ └────────────────────────────────────────┘ │
|
||||
├──────────────────────────────────────────────┤
|
||||
│ Input Bar (3 lines) │
|
||||
│ > Some text... │
|
||||
├──────────────────────────────────────────────┤
|
||||
│ Status Bar (1 line) │
|
||||
│ ┌ Provider │ Model │ Tokens │ Mode │ Quit ─┤
|
||||
└──────────────────────────────────────────────┘
|
||||
Timer tick
|
||||
│
|
||||
▼
|
||||
main.rs: fn tui_loop()
|
||||
│
|
||||
├── controller/input.rs: handle_key() → action
|
||||
├── app/runtime/actions/mod.rs: apply_action()
|
||||
│ │
|
||||
│ └── state mutates (AppStateRest)
|
||||
│
|
||||
└── view/mod.rs: build TUI layout
|
||||
│
|
||||
├── view/chat.rs: Chat transcript
|
||||
├── view/sidebar.rs: Usage dashboard
|
||||
├── view/status.rs: Status bar
|
||||
├── view/markdown.rs: Message renderer
|
||||
├── view/workflow.rs: Hive-mind progress
|
||||
└── view/theme.rs: Tokyo Night palette
|
||||
```
|
||||
|
||||
## Key Files
|
||||
## Overlay System
|
||||
|
||||
| File | Lines | Purpose |
|
||||
|------|-------|---------|
|
||||
| `src/view/mod.rs` | 623 | Frame draw, overlays (16 types), input bar, toasts |
|
||||
| `src/view/chat.rs` | 155 | Chat transcript rendering with markdown |
|
||||
| `src/view/markdown.rs` | 144 | Markdown → ratatui `Span` rendering (pulldown-cmark + syntect) |
|
||||
| `src/view/status.rs` | ~50 | Status bar with provider/model/tokens |
|
||||
| `src/view/workflow.rs` | 88 | Workflow progress visualization |
|
||||
| `src/view/theme.rs` | 23 | Color palette (23 named colors) |
|
||||
| `src/controller/input.rs` | 281 | Key event → Action mapping |
|
||||
16 overlays managed by `app/mode/`:
|
||||
|
||||
## Overlays (16 types)
|
||||
| Overlay | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| Chat input | `mod.rs` | Main input bar with autocomplete |
|
||||
| Bash | `bash.rs` | Interactive shell panel |
|
||||
| Editor | `editor.rs` | Built-in file editor |
|
||||
| Effort | `effort.rs` | LLM effort selector |
|
||||
| Help | `help.rs` | Keybindings help |
|
||||
| Key Input | `key_input.rs` | Custom key binding |
|
||||
| Learning | `learning.rs` | Lesson viewer |
|
||||
| Loading | `loading.rs` | Spinner overlay |
|
||||
| MCP | `mcp.rs` | MCP server management |
|
||||
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
|
||||
| Rewind | `rewind.rs` | Message/history rewind |
|
||||
| Settings | `settings.rs` | Settings panel |
|
||||
| Todo | `todo.rs` | Task/TODO list |
|
||||
| Workflow | (via view) | Workflow progress |
|
||||
|
||||
`Overlay::Help | Settings | Bash | QuitConfirm | Workflow | KeyInput | Editor | Effort | Mcp | Todo | Rewind | Learning | Usage | Loading | ModelSelector | ClearConfirm`
|
||||
## Layout Structure
|
||||
|
||||
Each overlay renders a centered popup via `render_overlay()`.
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Status Bar (view/status.rs) │
|
||||
├──────────────────────┬──────────────────────┤
|
||||
│ │ │
|
||||
│ Chat Transcript │ Sidebar │
|
||||
│ (view/chat.rs) │ (view/sidebar.rs) │
|
||||
│ scrollable, │ tokens, status, │
|
||||
│ inline-log style │ agent info │
|
||||
│ │ │
|
||||
├──────────────────────┴──────────────────────┤
|
||||
│ Input Bar + Autocomplete dropdown │
|
||||
│ (view/mod.rs) │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## State Mutations
|
||||
## Input Handling
|
||||
|
||||
State is mutated in-place from two locations:
|
||||
- `src/controller/input.rs` — keyboard shortcuts and overlay interactions
|
||||
- `src/app/runtime/actions/mod.rs` — `apply_action()` reducer for all programmatic actions
|
||||
`controller/input.rs`:
|
||||
|
||||
## Toast Notifications
|
||||
- Normal mode: keystrokes go to the active overlay
|
||||
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
|
||||
- Tab cycles autocomplete candidates
|
||||
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
|
||||
- Arrow keys scroll chat, sidebar, and other scrollable panels
|
||||
|
||||
`render_toasts()` — floating stack at top-right, color-coded by severity:
|
||||
- Info: blue, Success: green, Warning: yellow, Error: red, Lesson: cyan
|
||||
- Max 4 visible, auto-expire after 5s lifetime
|
||||
## Theme
|
||||
|
||||
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,114 @@
|
||||
# Clipboard Copy via OSC52 — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/controller/input.rs`, `src/main.rs`,
|
||||
`src/ipc/protocol.rs`
|
||||
|
||||
## Context
|
||||
|
||||
There is no clipboard support anywhere in the TUI today, and mouse capture is enabled
|
||||
(`EnableMouseCapture` in `main.rs`), which in most terminal emulators suppresses native
|
||||
click-drag text selection unless the user holds a modifier — making an in-app copy action
|
||||
more valuable than it would be in a plain scrollback. OSC52 is a terminal escape sequence
|
||||
(`\x1b]52;c;<base64>\x07`) that asks the terminal emulator itself to set the system
|
||||
clipboard; it needs no OS-level clipboard library (no X11/Wayland/win32 dependency) and
|
||||
the `base64` crate is already a dependency (used in `service/oauth/pkce.rs`), so no new
|
||||
crate is needed for this feature.
|
||||
|
||||
Key architectural constraint discovered while designing this: `controller::input::handle_key`
|
||||
runs on the **daemon** process in `--daemon`/`--attach` mode (`main.rs:359`, inside
|
||||
`handle_daemon_client`), not on the process that owns the user's actual terminal. A raw
|
||||
`io::stdout()` write inside `handle_key` would go to the headless daemon's stdout in that
|
||||
mode, not the user's terminal. The copy action therefore can't write the escape sequence
|
||||
directly from `handle_key` — it has to signal intent via state, and the terminal-owning
|
||||
process (single-process `run_loop_inner`, or the attach client's loop) performs the actual
|
||||
write.
|
||||
|
||||
## Goals
|
||||
|
||||
- `Ctrl+Y` copies the most recent `Role::Assistant` message's raw text (not the rendered
|
||||
markdown spans) to the system clipboard via OSC52.
|
||||
- Works identically in single-process mode and in `--daemon`/`--attach` mode.
|
||||
- No new dependency.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No native clipboard fallback (e.g. `arboard`) for terminals that don't honor OSC52 —
|
||||
unsupported terminals silently swallow the escape sequence; no error surfaces to the
|
||||
user beyond the optimistic "Copied to clipboard" toast (there's no ack mechanism in the
|
||||
OSC52 protocol to verify the terminal actually did it).
|
||||
- No copy-last-code-block variant — out of scope for this pass; the whole-message copy
|
||||
covers the common case and is simple to extend later if needed.
|
||||
- No mouse-drag text selection — unrelated, much larger feature; not being built here.
|
||||
|
||||
## State (`misc.rs`)
|
||||
|
||||
- `MiscState` gains `pub pending_clipboard_copy: Option<String>`, initialized to `None` in
|
||||
`MiscState::new()`.
|
||||
|
||||
## `input.rs`
|
||||
|
||||
- New top-level arm alongside the existing `Ctrl+C`/`Ctrl+D` handlers:
|
||||
`KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL)`. It finds the last
|
||||
message in `state.transcript_cache.messages` with `role == Role::Assistant`:
|
||||
- If found: `state.misc.pending_clipboard_copy = Some(msg.content.clone())`.
|
||||
- If not found: push an `Info` toast ("No assistant message to copy yet") and leave
|
||||
`pending_clipboard_copy` as `None`.
|
||||
- Returns `Vec::new()` — this is a direct state mutation inside `handle_key`, matching
|
||||
the existing `Ctrl+S` editor-save precedent (`main.rs`'s editor branch also mutates
|
||||
state/does I/O directly rather than going through an `Action`).
|
||||
|
||||
## OSC52 write helper (`main.rs`)
|
||||
|
||||
```
|
||||
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
|
||||
write!(stdout, "\x1b]52;c;{b64}\x07")?;
|
||||
stdout.flush()
|
||||
}
|
||||
```
|
||||
|
||||
Generic over `impl Write` so both the single-process loop (writing to `io::stdout()`) and
|
||||
tests (writing to a `Vec<u8>` to assert the formatted sequence) can use it without a real
|
||||
terminal.
|
||||
|
||||
## Single-process mode (`run_loop_inner`)
|
||||
|
||||
After the existing `for action in actions { apply_action(state, action); }` block, add:
|
||||
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
state.push_toast(Toast::new(ToastKind::Success, "Copied to clipboard".into()));
|
||||
}
|
||||
```
|
||||
|
||||
## Daemon/attach mode
|
||||
|
||||
- `ipc/protocol.rs`: add `DaemonFrame::ClipboardCopy(String)` (alongside `StateUpdate`,
|
||||
`StreamToken`, `SystemNote`, `Closed` — same `Serialize`/`Deserialize` derive).
|
||||
- `handle_daemon_client` (`main.rs`): after each branch that calls `handle_key`/`apply_action`
|
||||
(`KeyPress` and `Submit`, the only two that can reach the input handler), before the
|
||||
existing `send_daemon_update(&mut conn, state)?;` call, add:
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
conn.send(&DaemonFrame::ClipboardCopy(text))?;
|
||||
}
|
||||
```
|
||||
- Attach-client loop (`main.rs`, the function matching on `DaemonFrame::StateUpdate` /
|
||||
`SystemNote` / `Closed` around line 573): add a `DaemonFrame::ClipboardCopy(text) => {
|
||||
let _ = write_osc52(&mut io::stdout(), &text); client_state.push_toast(...); }` arm,
|
||||
mirroring the existing `SystemNote` handling but performing the actual terminal write
|
||||
since this process — not the daemon — owns the user's terminal.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `input.rs`: `Ctrl+Y` with a transcript containing multiple messages sets
|
||||
`pending_clipboard_copy` to the *last* assistant message's content, ignoring later
|
||||
user/tool messages that might follow it; with no assistant message present, it pushes
|
||||
an info toast and leaves `pending_clipboard_copy` as `None`.
|
||||
- `main.rs`: `write_osc52` writing into a `Vec<u8>` buffer produces the exact expected
|
||||
`\x1b]52;c;<base64>\x07` byte sequence for a known input string.
|
||||
@@ -0,0 +1,115 @@
|
||||
# Diff View for edit/write Tools — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/tool/fs/edit.rs`, `src/tool/fs/write.rs`, `src/view/markdown.rs`, `src/view/chat.rs`
|
||||
|
||||
## Context
|
||||
|
||||
`edit` currently reports only a byte-delta (`"edited {rel} ({N} byte delta)"`), and `write`
|
||||
reports only a byte count. Neither the model nor the user sees what actually changed —
|
||||
just a number. This makes it hard for the model to self-verify an edit landed correctly,
|
||||
and hard for the user to review a change without opening the file. No diff-computing
|
||||
library exists in the dependency tree today.
|
||||
|
||||
## Goals
|
||||
|
||||
- `edit` returns a real unified diff (git-style, 3 lines of context) of the change it just
|
||||
made, in place of the byte-delta note.
|
||||
- `write` returns the same kind of diff when it overwrites a file that already existed
|
||||
with valid UTF-8 content; falls back to the current "wrote N bytes" message for new
|
||||
files or non-UTF-8 (binary) overwrites.
|
||||
- Diffs render in the chat view with real color (green add / red remove / cyan hunk
|
||||
header) instead of being flattened to dim/italic like other tool output.
|
||||
- Large diffs are truncated with a trailing count, matching the existing pattern in
|
||||
`read.rs` (`"... ({N} more lines, total {total})"`).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No diff view for any tool besides `edit`/`write` (e.g. no retroactive diffing of
|
||||
`bash_tools.rs` shell edits).
|
||||
- No side-by-side diff layout — unified format only, matching how every other tool
|
||||
output already renders as a single text stream.
|
||||
- No persistence of diff history; each diff is only the delta of the single tool call
|
||||
that produced it, not a cumulative session diff.
|
||||
- No changes to non-tool (assistant/user/system) message rendering or coloring.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `similar = "3"` (line/word diff crate; permissive MIT/Apache-2.0, no heavy
|
||||
transitive deps). Use `TextDiff::from_lines(old, new).unified_diff().context_radius(3)`,
|
||||
which produces standard `@@ -a,b +c,d @@` hunk headers and `-`/`+`/` `-prefixed lines —
|
||||
no custom diff algorithm needed.
|
||||
|
||||
## Tool changes
|
||||
|
||||
### `edit.rs`
|
||||
|
||||
After computing `new_content` and writing it to disk:
|
||||
|
||||
1. Compute `similar::TextDiff::from_lines(&content, &new_content).unified_diff().context_radius(3).to_string()`.
|
||||
2. Split into lines; if `> MAX_DIFF_LINES` (200), keep the first 200 and append
|
||||
`"... ({N} more lines truncated)"`.
|
||||
3. Wrap the (possibly truncated) diff text in a fenced ` ```diff ` block.
|
||||
4. Replace the byte-delta note in the returned message with this block; keep the
|
||||
existing "Graduated checks matched" / LSP note suffixes in their current position
|
||||
(after the diff block).
|
||||
|
||||
### `write.rs`
|
||||
|
||||
Before overwriting:
|
||||
|
||||
1. If `path.exists()` and `fs::read_to_string(&path)` succeeds (valid UTF-8), capture it
|
||||
as `old_content` and note `is_overwrite = true`.
|
||||
2. If the file doesn't exist, or reading it fails (binary/non-UTF-8), `is_overwrite = false`
|
||||
— no error, just skip the diff path silently.
|
||||
3. After writing, if `is_overwrite`, compute and truncate the diff exactly as in `edit.rs`
|
||||
and append the fenced block to the return message (in addition to the existing
|
||||
"wrote N bytes" line, not instead of it — for `write`, unlike `edit`, the byte count is
|
||||
still useful since it can be a full-file rewrite).
|
||||
4. If not `is_overwrite`, return message is unchanged from today.
|
||||
|
||||
The truncation constant (`MAX_DIFF_LINES = 200`) and truncation message format are
|
||||
shared — factor into a small helper in `tool/fs/helpers.rs` used by both tools.
|
||||
|
||||
## Rendering changes
|
||||
|
||||
### `markdown.rs`
|
||||
|
||||
- `render_markdown` gains a `dim: bool` parameter: `render_markdown(text, width, dim)`.
|
||||
- Capture the fence language from `Tag::CodeBlock(CodeBlockKind::Fenced(lang))` (today
|
||||
matched as `CodeBlock(_)`, discarding the language). Track `in_diff_block: bool` when
|
||||
`lang == "diff"`.
|
||||
- Inside a diff block, process text line-by-line instead of as one blob: a line starting
|
||||
with `+` (not `+++`) is styled green, `-` (not `---`) red, `@@` cyan/muted, everything
|
||||
else (context lines, `+++`/`---` file headers) uses the existing code-block teal.
|
||||
- When `dim` is `true`: every span keeps its assigned color as computed above, but
|
||||
non-diff spans (headings, links, plain text, non-diff code blocks, table cells) fall
|
||||
back to `Theme::TEXT_DIM` + `Modifier::ITALIC` instead of their normal palette color —
|
||||
this replicates today's "tool output is always dim" behavior for everything except
|
||||
diff lines.
|
||||
- When `dim` is `false`: behavior is unchanged from today (full color, used for
|
||||
assistant/user/system messages).
|
||||
|
||||
### `chat.rs`
|
||||
|
||||
- `Role::Tool` branch: replace the two manual span-remapping loops (that force every
|
||||
span to `dim_italic`) with a direct call to `render_markdown(&content, content_width, true)`
|
||||
and use the returned spans as-is.
|
||||
- All other roles: call `render_markdown(&content_str, content_width, false)` — same
|
||||
call as today, just with the new explicit `false` argument.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `edit.rs`: a normal single-replace edit produces a diff block with matching
|
||||
`-`/`+` lines; a `replace_all` across 250+ lines truncates at 200 with the correct
|
||||
trailing count.
|
||||
- `write.rs`: writing a brand-new file keeps the old "wrote N bytes" message with no
|
||||
diff block; overwriting an existing UTF-8 file produces a diff block; overwriting
|
||||
a path that reads as invalid UTF-8 (simulate via non-UTF-8 bytes) falls back to the
|
||||
byte-count message without erroring.
|
||||
- `markdown.rs`: a fenced ` ```diff ` block with `+`/`-`/`@@` lines produces spans with
|
||||
the expected fg colors under `dim=true` (diff lines colored) and confirms non-diff
|
||||
text in the same call falls back to `TEXT_DIM` + italic.
|
||||
@@ -0,0 +1,126 @@
|
||||
# Fuzzy @file-mention Autocomplete — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/app/state/rest.rs`, `src/controller/input.rs`,
|
||||
`src/view/mod.rs`, `src/tool/mod.rs`, `src/tool/fs/write.rs`, `src/main.rs`
|
||||
|
||||
## Context
|
||||
|
||||
The chat input already has a dropdown autocomplete (`InputState` in `misc.rs`), but it
|
||||
only covers slash commands: it requires the whole buffer to start with `/` and filters a
|
||||
fixed `COMMANDS` list by prefix. There's no way to reference a project file from the chat
|
||||
input without typing its exact path from memory. The existing `dir_cache` (used by the
|
||||
`dir_cache_update` tool) looks like it could serve this but doesn't: it's a single,
|
||||
non-recursive directory snapshot, overwritten on each LLM-driven `dir_cache_update` call —
|
||||
not a standing, recursive, whole-workspace file index. `search.rs`'s `Grep`/`Glob` tools
|
||||
already do the recursive, `.gitignore`-respecting walk this feature needs, via
|
||||
`ignore::Walk`.
|
||||
|
||||
Also relevant: there is no persistent async runtime driving the TUI loop. `main.rs`
|
||||
constructs a `tokio::runtime::Runtime` but never `.enter()`s or `block_on`s it in the
|
||||
main loop — `run_loop` is fully synchronous. The one existing async-flavored pattern
|
||||
(`dir_cache_update.rs`) spins up a throwaway one-shot runtime purely to satisfy
|
||||
`tokio::sync::RwLock`'s API, then discards it. This feature does not need that ceremony:
|
||||
a plain `std::sync::RwLock` is enough, since every reader/writer here is synchronous
|
||||
(`handle_key`, `Tool::run`, and the index-build thread all being plain sync code).
|
||||
|
||||
## Goals
|
||||
|
||||
- Typing `@` at a word boundary (start of buffer or after whitespace) in the chat input,
|
||||
followed by non-whitespace characters, opens a dropdown of fuzzy-matched project file
|
||||
paths, live-updating as the query changes.
|
||||
- Selecting a candidate splices `@relative/path ` into the buffer at the mention's
|
||||
position (not a whole-buffer replace) and the user keeps typing.
|
||||
- Candidates come from a background-built, whole-workspace file index — not the
|
||||
LLM-facing `dir_cache`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No auto-reading of the selected file's content into the conversation — the inserted
|
||||
`@path` is plain text; the model reads it via the `read` tool if it wants to, same as
|
||||
any other path reference.
|
||||
- No live re-filter on Backspace/Delete while a mention dropdown is open — mirrors the
|
||||
slash-command dropdown's existing behavior (closes on Backspace/Delete rather than
|
||||
refiltering). Not fixing that for commands here; file mentions just inherit it for
|
||||
consistency.
|
||||
- No periodic re-walk of the index after startup — only single-file incremental updates
|
||||
on file creation (see below). A deleted or renamed file may show a stale entry until
|
||||
restart; acceptable since selecting it just inserts text, it doesn't touch the
|
||||
filesystem.
|
||||
- No fuzzy matching over directories, only files.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `nucleo-matcher = "0.3"` (the fuzzy-matching engine from the Helix editor project;
|
||||
small, actively maintained, no heavy transitive deps).
|
||||
|
||||
## Index storage & construction
|
||||
|
||||
- New type in `misc.rs`: `MentionIndex { entries: Arc<std::sync::RwLock<Vec<String>>> }`, with `MentionIndex::new()`, `set(&self, paths: Vec<String>)`, and `snapshot(&self) -> Vec<String>` (both plain sync `.write()`/`.read()`, no `try_`/async — a std `RwLock` doesn't block indefinitely here since every hold is a quick vec swap or clone).
|
||||
- `AppStateRest` gets a `pub mention_index: MentionIndex` field, initialized in `AppStateRest::new()`, threaded into `ToolCtx`/`ToolCtxBuilder` the same way `dir_cache` is (new `mention_index` field on both, wired through `tool_ctx()`/`tool_ctx_for()`/`build()`).
|
||||
- In `main.rs`, right after `AppStateRest::new(...)` in the single-process TUI path and the daemon path (not the attach-only client path, which has no local `ToolCtx`), spawn `std::thread::spawn` that:
|
||||
1. For each workspace root (index `i`, path `w`): `ignore::Walk::new(w)`, keep only files, strip `w` as prefix, format as `rel` for `i == 0` or `[i]rel` for `i > 0` (matching `resolve_path`'s existing workspace-index convention).
|
||||
2. Stop collecting once the total across all workspaces hits 50,000 entries (repos larger than that are rare here; this is a soft cap to bound memory/scan time, not a hard requirement).
|
||||
3. Call `mention_index.set(all_paths)`.
|
||||
- `write.rs`: after a successful write, if the target path did **not** exist before the write (i.e. this created a new file, not an overwrite), compute its relative/workspace-prefixed form and push it onto `ctx.mention_index`'s vec directly (read-modify-write under the same lock) rather than re-walking.
|
||||
|
||||
## `InputState` changes (`misc.rs`)
|
||||
|
||||
- New `pub enum AutocompleteKind { Command, FileMention }`.
|
||||
- `InputState` gains `pub autocomplete_kind: AutocompleteKind` (default `Command`) and
|
||||
`pub mention_start: usize` (byte offset of the triggering `@`).
|
||||
- New `fn mention_query_at_cursor(&self) -> Option<(usize, String)>`: scans backward from
|
||||
`self.cursor` for an `@`; the scan stops (returns `None`) if it hits whitespace before
|
||||
finding `@`. The `@` only counts as a trigger if it's at buffer start or immediately
|
||||
preceded by whitespace. Returns `(byte offset of '@', query text between '@' and cursor)`.
|
||||
- New `fn open_mention_autocomplete(&mut self, files: &[String])`: calls
|
||||
`mention_query_at_cursor()`; if `None`, calls `close_autocomplete()` and returns. If
|
||||
`Some((start, query))`, fuzzy-matches `query` against `files` via `nucleo-matcher`,
|
||||
keeps the top 10 by score, sets `autocomplete_candidates`, `autocomplete_kind =
|
||||
FileMention`, `mention_start = start`, `autocomplete_visible = !candidates.is_empty()`.
|
||||
- `select_autocomplete()` becomes kind-aware:
|
||||
- `Command` (today's behavior, unchanged): `buffer = candidate.clone()`, `cursor =
|
||||
buffer.len()`.
|
||||
- `FileMention`: `buffer.replace_range(mention_start..cursor, &format!("@{candidate} "))`,
|
||||
`cursor = mention_start + candidate.len() + 2` (the `@` plus the candidate plus the
|
||||
trailing space).
|
||||
- Both paths end with `close_autocomplete()`, same as today.
|
||||
|
||||
## `input.rs` wiring
|
||||
|
||||
- `KeyCode::Char(c)` handler: after `state.input.insert(c)`, keep the existing
|
||||
`if buffer.starts_with('/') { open_autocomplete() }` check, and add an `else if let
|
||||
Some(_) = state.input.mention_query_at_cursor() { state.input.open_mention_autocomplete(&state.mention_index.snapshot()) }` branch. These are mutually exclusive in practice (a
|
||||
buffer starting with `/` is a slash command, not a sentence with an `@mention` in it).
|
||||
- `KeyCode::Backspace` / `KeyCode::Delete`: unchanged — both already just call
|
||||
`close_autocomplete()` when a dropdown is visible, regardless of kind. No new branching
|
||||
needed since `close_autocomplete()` already resets `autocomplete_kind` isn't touched but
|
||||
becomes irrelevant once `autocomplete_visible` is false.
|
||||
- `KeyCode::Tab`: currently gated on `buffer.starts_with('/')`. Extend the condition to
|
||||
also fire when `autocomplete_kind == FileMention && autocomplete_visible` so Tab cycles
|
||||
file-mention candidates too.
|
||||
- `KeyCode::Enter`: unchanged — already calls `select_autocomplete()` whenever
|
||||
`autocomplete_visible`, which is now kind-aware internally.
|
||||
|
||||
## Rendering (`view/mod.rs`)
|
||||
|
||||
- `render_input_bar`'s dropdown block reuses the exact same list-rendering code (already
|
||||
generic over `autocomplete_candidates`/`autocomplete_idx`); only the title changes based
|
||||
on `state.input.autocomplete_kind`: `" ⌘ Commands "` (unchanged) vs `" 📁 Files "`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `misc.rs`: `mention_query_at_cursor` returns the right `(start, query)` for `@` at
|
||||
buffer start, `@` after a space mid-sentence, and correctly returns `None` when the `@`
|
||||
is mid-word (e.g. `foo@bar`) or when whitespace exists between the `@` and the cursor.
|
||||
`select_autocomplete` for `FileMention` splices correctly into a buffer with text before
|
||||
and after the mention span; `Command` selection still replaces the whole buffer as
|
||||
before.
|
||||
- `write.rs`: creating a new file appends its path to the shared `mention_index`;
|
||||
overwriting an existing file does not add a duplicate entry.
|
||||
- Index construction: not unit-tested directly (it's a `std::thread::spawn` walking the
|
||||
real filesystem at startup) — covered implicitly by exercising the app manually per the
|
||||
`verify` skill during implementation.
|
||||
@@ -0,0 +1,281 @@
|
||||
# Context & Compaction Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-16
|
||||
**Scope:** replaces `src/app/runtime/shortsend.rs`; touches `src/app/runtime/actions/mod.rs`,
|
||||
`src/view/status.rs`, `src/model/settings.rs`, `src/app/subagent/division.rs`, `Cargo.toml`
|
||||
|
||||
## Context
|
||||
|
||||
The existing conversation-compaction system (`shortsend.rs`, 129 lines) only acts once the
|
||||
context is already close to the model's window limit, and has accumulated inconsistencies
|
||||
found during a codebase audit:
|
||||
|
||||
1. Three different token-count heuristics for the same job: `/3` inside
|
||||
`shortsend::shape_messages`, `/4` in the auto-compact loop
|
||||
(`actions/mod.rs` ~line 1146), `/4` again in the live status bar (`view/status.rs:68`).
|
||||
2. Manual `/compact` (`Action::Compact`, `actions/mod.rs:547-563`) passes `client: None`
|
||||
because `apply_action` is synchronous, so it never gets LLM summarization — it always
|
||||
falls back to the bare `"[prior conversation compacted]"` placeholder, unlike automatic
|
||||
mid-turn compaction (`Some(&tc.client)`, line 1160). Undocumented asymmetry between the
|
||||
two trigger paths.
|
||||
3. `context_window` resolution (`model_roles.values().find(...).and_then(...).unwrap_or(...)`)
|
||||
duplicated three times (`Action::Compact`, `spawn_turn`, `view/status.rs` twice).
|
||||
4. No repeated-tool-call dedup: reading the same file (or running the same grep) twice in a
|
||||
session keeps both full copies in context forever, until compaction eventually drops the
|
||||
older one wholesale along with everything else from that period.
|
||||
5. No per-result compression: a single large tool output (a big `bash` log, a large `grep`
|
||||
result) is stored verbatim even when most of it is redundant or low-value.
|
||||
6. Zero test coverage on `shortsend.rs`.
|
||||
|
||||
Separately, research into three real, permissively-licensed open-source projects
|
||||
(`rtk-ai/rtk`, Apache-2.0; `headroomlabs-ai/headroom`, Apache-2.0; `JuliusBrussee/caveman`,
|
||||
MIT — verified via `gh api` for authenticity/license, and by cloning and reading source, not
|
||||
taken from marketing blog posts) surfaced techniques worth reimplementing natively:
|
||||
|
||||
- **rtk**: generic line-scan compression (strip comment/blank runs, brace-depth collapse of
|
||||
function bodies, importance-ranked truncation ending in an unambiguous `[N more lines]`
|
||||
marker — their own regression tests show a comment-shaped marker confuses the LLM into
|
||||
retry-looping) plus structured per-toolchain parsing (e.g. `cargo --message-format=json`
|
||||
bucketed into errors/warnings, boilerplate lines dropped).
|
||||
- **headroom**: per-content-type compressors — logs (classify lines by level/stack-trace/
|
||||
summary, score, keep highest-value lines + surrounding context, adaptive cap), grep
|
||||
results (group by file, score matches, cap globally and per-file), JSON (keep all
|
||||
structural tokens — keys, brackets, colons — drop or shrink long low-entropy string
|
||||
values, keep short values and UUID/hash-shaped high-entropy ones).
|
||||
- **caveman**: a pure prompt/persona instruction (no algorithm) that tells the model to
|
||||
write tersely — drop articles/filler/hedging, keep code/commands/errors verbatim — with an
|
||||
explicit carve-out that disables terseness for destructive-op confirmations and security
|
||||
warnings. This compresses *output* tokens, a different axis from everything else in this
|
||||
design, which compresses *input* context.
|
||||
|
||||
This is a from-scratch reimplementation of the underlying ideas, not a port — no code is
|
||||
copied from any of the three projects.
|
||||
|
||||
## Goals
|
||||
|
||||
- One unified, always-on pipeline that keeps context lean from turn 1, not just once near
|
||||
the limit.
|
||||
- Deduplicate repeated tool calls: an older copy of a tool result superseded by an identical
|
||||
later call (same tool name + same arguments) is replaced with a placeholder, for read-only
|
||||
tools only.
|
||||
- Compress large individual tool results (logs, JSON, generic text) at capture time, above a
|
||||
size floor.
|
||||
- Fix the three known inconsistencies (token heuristic, manual/auto asymmetry,
|
||||
`context_window` duplication).
|
||||
- Optional, off-by-default "concise mode" system-prompt toggle for terser model output.
|
||||
- Full inline test coverage per repo convention.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not adding a runtime dependency on `rtk`, `headroom`, or `caveman` themselves (as a binary,
|
||||
proxy, or crate) — everything is implemented natively in Rust inside zesdex.
|
||||
- Not building rtk's per-toolchain structured parsers (`cargo --message-format=json`
|
||||
re-invocation, etc.) — too invasive for a general-purpose `bash` tool that runs arbitrary
|
||||
commands zesdex doesn't control the flags of. Only the generic line-scan/log/JSON layer is
|
||||
built.
|
||||
- Not switching to an exact per-provider tokenizer — `tiktoken-rs` (BPE, cl100k_base/
|
||||
o200k_base) is an approximation good enough for the 85%/95% budget thresholds; it is not
|
||||
used for billing-accurate counts.
|
||||
- `caveman-compress`-style memory-file rewriting (the LLM-round-trip variant of caveman) is
|
||||
out of scope — only the pure-prompt persona mechanism is adopted.
|
||||
|
||||
## Architecture
|
||||
|
||||
Replace `src/app/runtime/shortsend.rs` with `src/app/runtime/context/`:
|
||||
|
||||
```
|
||||
context/
|
||||
mod.rs — module registration only, no facade (see below)
|
||||
tokens.rs — unified token counting (tiktoken-rs)
|
||||
dedup.rs — cross-call tool-result deduplication
|
||||
squash.rs — per-result compression (log/json/generic), applied at
|
||||
tool-result construction time, upstream of prepare()
|
||||
shaping.rs — budget-based drop + LLM summarize (renamed shortsend logic)
|
||||
window.rs — shared context_window resolution
|
||||
```
|
||||
|
||||
### `tokens.rs`
|
||||
|
||||
```rust
|
||||
pub fn count_tokens(text: &str) -> usize
|
||||
pub fn count_message_tokens(msg: &ChatMessage) -> usize
|
||||
```
|
||||
|
||||
Backed by `tiktoken-rs` (new dependency, pure Rust, embedded BPE vocab, no network calls at
|
||||
runtime), using `o200k_base`. Replaces all three existing heuristic call sites: `shortsend`'s
|
||||
internal `/3`, the auto-loop's `/4` (`actions/mod.rs` ~1146), and `status.rs:68`'s `/4`.
|
||||
|
||||
### `dedup.rs`
|
||||
|
||||
```rust
|
||||
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool)
|
||||
```
|
||||
|
||||
The `bool` is `true` iff at least one message was replaced with a placeholder — callers use
|
||||
it to decide whether the result is worth persisting/announcing, without needing `ChatMessage`
|
||||
to implement `PartialEq` (it doesn't today, and adding it purely to diff whole message lists
|
||||
would be needless surface area for what `collapse` already knows precisely mid-walk).
|
||||
|
||||
Flow: walk messages, pair each `Role::Tool` message to its originating `ToolCall` via
|
||||
`tool_call_id`. Key = `(function.name, sha256(canonical_json(function.arguments)))` (`sha2`
|
||||
is already a dependency). Track the last index seen per key. For any earlier occurrence of a
|
||||
key whose tool name is in the read-only set, replace that earlier `Tool` message's `content`
|
||||
with a short placeholder (`"[duplicate result — superseded by a later identical call, see
|
||||
below]"`); the assistant's tool-call entry (name + arguments) is left untouched, so the
|
||||
action/audit trail stays intact. Mutating tools are never touched, even with identical
|
||||
arguments, because call order and repetition can be semantically meaningful (e.g. retrying a
|
||||
flaky `bash` command).
|
||||
|
||||
Read-only classification reuses `subagent::division::tool_scope::READ_TOOLS`
|
||||
(`src/app/subagent/division.rs:21`) rather than a new list — that `const` is made `pub` for
|
||||
this purpose. It already enumerates exactly the read-only tool set (`read`, `grep`, `glob`,
|
||||
`search`, `seqthink`, `recall`, `lsp_*`, `read_findings`).
|
||||
|
||||
Runs every turn, unconditionally, before token counting — not gated on `should_shape`.
|
||||
|
||||
### `squash.rs`
|
||||
|
||||
```rust
|
||||
pub fn apply(tool_name: &str, output: &str) -> String
|
||||
```
|
||||
|
||||
`read` is exempted entirely, always passed through unchanged regardless of size: its output
|
||||
must stay byte-exact because the agent relies on it for exact-match edits afterward, and a
|
||||
squashed view of a JSON config file (or any file whose content happens to parse as JSON)
|
||||
would otherwise be silently altered. Size floor for every other tool: outputs under 1500
|
||||
bytes pass through unchanged (compression only pays off on large output, and touching small
|
||||
results risks losing detail with no token benefit). Above the floor, dispatch by content
|
||||
shape:
|
||||
|
||||
- `squash_json(&str) -> String` — walks a parsed `serde_json::Value` (not a hand-rolled
|
||||
tokenizer — `serde_json` already handles escaping/nesting correctly, reusing it is simpler
|
||||
and more robust); structural tokens (keys, brackets, colons, commas, booleans, null) always
|
||||
kept; string values kept if ≤20 chars or "identifier-shaped" (no internal whitespace *and*
|
||||
Shannon entropy ≥3.0 bits/char — catches UUIDs/hashes/paths), otherwise replaced with `"…"`
|
||||
in place; array elements past the first 3 compressed harder (values elided regardless of
|
||||
length/entropy). Applied when `serde_json::from_str` on the output succeeds. The
|
||||
no-whitespace pre-filter matters: raw per-character entropy alone doesn't separate prose
|
||||
from identifiers — repeated English prose measures ~3.89 bits/char, higher than a UUID's
|
||||
~3.39 — because prose also draws from a wide character set. headroom's own entropy gate is
|
||||
"cheaply pre-filtered by 'no spaces'" before scoring for the same reason; multi-word values
|
||||
never reach the entropy check at all under this rule.
|
||||
- `squash_log(&str) -> String` — line classifier (error/fail/warn/info/debug/trace by
|
||||
keyword + stack-trace-frame detection) → score
|
||||
(`level_score {1.0 error/fail, 0.5 warn, 0.1 info, 0.05 debug/trace} + 0.3 if
|
||||
stack-trace-frame + 0.4 if summary-shaped line`) → keep up to 20 highest-scored error
|
||||
lines, up to 10 highest-scored warning lines, all summary lines, plus a ±2-line context
|
||||
window around each kept line → single `[N lines omitted]` marker for drops (not
|
||||
comment-shaped, per rtk's own finding on LLM confusion). Applied when the output isn't
|
||||
valid JSON, the tool is `bash`, and the output has ≥3 lines matching error/warn/stack-trace
|
||||
patterns. The tool restriction (added after the final whole-branch review) matters: a `grep`
|
||||
result full of matches against error-handling code trips the same ≥3-line keyword threshold
|
||||
as a real build log, but `squash_log`'s hard 20-error/10-warning cap has no byte budget and
|
||||
would silently drop legitimate matches past it — the wrong compressor for search results.
|
||||
Only `bash` (the actual log-producing tool) routes through `squash_log`; every other tool
|
||||
whose output happens to look log-shaped falls through to the gentler, byte-budgeted
|
||||
`squash_generic` instead.
|
||||
- `squash_generic(&str, budget) -> String` — importance-ranked truncation: keeps the first 10
|
||||
and last 10 lines plus any line matching a small "looks important" heuristic (non-blank,
|
||||
not a byte-for-byte repeat of the immediately preceding line), single `[N lines omitted]`
|
||||
marker for the rest, capped to `budget` bytes overall (`budget` = the 1500-byte squash
|
||||
floor doubled, i.e. 3000 bytes, chosen so the fallback path still yields a real reduction
|
||||
on anything that triggered it). Fallback for anything that isn't JSON or log-shaped.
|
||||
|
||||
Called once, at the single tool-result construction site
|
||||
(`actions/mod.rs:1420`, `let tool_msg = ChatMessage::tool_result(tool_call.id.clone(),
|
||||
output);`) — `output` is passed through `squash::apply(&tool_name, &output)` before being
|
||||
wrapped. Runs before the result is ever archived or pushed into `msgs`, so compression is
|
||||
permanent and applies uniformly whether or not compaction ever triggers.
|
||||
|
||||
### `shaping.rs`
|
||||
|
||||
Unchanged behavior from today's `shortsend.rs` (hysteresis `should_shape`, 70%-budget
|
||||
newest-first retention, LLM summarization of dropped messages), moved as-is into this file
|
||||
and updated to source token counts from `tokens.rs` instead of its own heuristic.
|
||||
|
||||
### `window.rs`
|
||||
|
||||
```rust
|
||||
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize
|
||||
```
|
||||
|
||||
Replaces the three duplicated `model_roles.values().find(...).and_then(...).unwrap_or(...)`
|
||||
blocks in `Action::Compact`, `spawn_turn`, and `view/status.rs` (×2).
|
||||
|
||||
### `mod.rs`
|
||||
|
||||
No facade function — just `pub mod dedup; pub mod shaping; pub mod squash; pub mod tokens;
|
||||
pub mod window;`. `dedup`, `shaping`, and `tokens` are called directly from each call site
|
||||
(the auto-loop and `Action::Compact`), matching CLAUDE.md's "No DI — modules call ...
|
||||
directly" convention rather than introducing an orchestration layer that only one of the two
|
||||
callers would use generically (the auto-loop already needs per-stage control today — it
|
||||
inspects `should_shape` itself to decide whether to emit `TurnEvent::Compacted` — and would
|
||||
have to unpack a facade's result anyway).
|
||||
|
||||
## Data flow (per turn)
|
||||
|
||||
1. Tool executes → raw `output: String`.
|
||||
2. `squash::apply(tool_name, &output)` — compress if over the size floor (`read` exempted).
|
||||
3. Wrapped into `ChatMessage::tool_result(...)`, archived, pushed to `msgs`.
|
||||
4. Once per loop iteration: `dedup::collapse(&msgs)` (always) → sum
|
||||
`tokens::count_message_tokens` over the result → `shaping::should_shape` → conditionally
|
||||
`shaping::shape_messages`.
|
||||
5. Result pushed as `TurnEvent::Compacted` if dedup changed anything or shaping triggered,
|
||||
consumed on the main thread to update `SessionRuntime.messages`.
|
||||
|
||||
## Fixing the manual/auto asymmetry
|
||||
|
||||
`Action::Compact` (`actions/mod.rs:547`) currently runs synchronously inside `apply_action`
|
||||
and can't block on an LLM call. Fix: make it spawn a background `std::thread::spawn` — the
|
||||
same pattern `spawn_turn` already uses (`actions/mod.rs:694`) — that runs `dedup::collapse`
|
||||
then unconditionally `shaping::shape_messages(.., force=true, Some(&client))` and reports back
|
||||
via `TurnEvent::Compacted`, identical to the automatic path. The toast sequence becomes
|
||||
"Compacting…" immediately (optimistic, non-blocking) then "History compacted" when the
|
||||
`TurnEvent` arrives. This gives manual `/compact` real LLM summarization instead of always
|
||||
falling back to the placeholder.
|
||||
|
||||
## Concise mode (separate from the `context/` module)
|
||||
|
||||
- `Settings` (`src/model/settings.rs`) gains `pub concise_output: bool`, default `false`,
|
||||
with `#[serde(default)]` for backward-compatible deserialization of existing
|
||||
`settings.json` files (matching the existing `hive_mind_node_timeout_ms` precedent in the
|
||||
same file).
|
||||
- When `true`, `run_agent_turn`'s system-prompt assembly (`actions/mod.rs:930-936`) appends a
|
||||
fourth section to `system_text`: a terse-writing instruction (persona-prompt only, no
|
||||
algorithm — drop articles/filler/hedging/pleasantries, keep code/commands/error text
|
||||
byte-exact) with an explicit carve-out disabling terseness for destructive-operation
|
||||
confirmations and security-relevant warnings, mirroring caveman's own "Auto-Clarity"
|
||||
safety exception.
|
||||
- No UI toggle is in scope for this pass — confirmed no such mechanism exists today for any
|
||||
boolean `Settings` field (`review_enabled`, `session_archive_enabled`,
|
||||
`lsp_auto_provision` are all hand-edited in `settings.json`, same as this one will be).
|
||||
|
||||
## New dependency
|
||||
|
||||
`tiktoken-rs` — pure Rust, embedded BPE vocab (`cl100k_base`/`o200k_base`), no network calls
|
||||
at runtime, MIT/Apache-2.0 dual-licensed. Added to `Cargo.toml`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per repo convention, one per new file:
|
||||
|
||||
- `dedup.rs`: same tool+args → older result replaced; different args → no-op; mutating tool
|
||||
with identical args → both kept in full; unmatched `tool_call_id` (malformed history) →
|
||||
no panic, treated as unpaired.
|
||||
- `squash.rs`: JSON input under/over the size floor; JSON with long low-entropy string values
|
||||
gets them elided while short/UUID-shaped values survive; log input with error/warn lines
|
||||
keeps highest-scored lines and emits exactly one `[N lines omitted]` marker; generic text
|
||||
keeps first/last N lines.
|
||||
- `tokens.rs`: known-string token counts against fixed expected values; empty string → 0.
|
||||
- `shaping.rs`: port the behavioral cases implied by today's hysteresis logic (85% trigger
|
||||
when not previously shaped, 95% once shaped) plus budget-drop ordering.
|
||||
- `window.rs`: role match resolves to the role's `context_window`; no match falls back to
|
||||
`default_context_window`.
|
||||
|
||||
## Migration
|
||||
|
||||
- Delete `src/app/runtime/shortsend.rs`; all three call sites (`actions/mod.rs` auto-loop,
|
||||
`Action::Compact`, and the module path itself) updated to `context::`.
|
||||
- `view/status.rs` switches its live token display to `tokens::count_tokens`, so the status
|
||||
bar finally matches what compaction measures internally.
|
||||
@@ -1,4 +1,9 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
//! Global registry of running background bash jobs, and control operations
|
||||
//! (output polling, kill) exposed to the rest of the app.
|
||||
//!
|
||||
@@ -10,7 +15,6 @@
|
||||
//! Why: a single static map (rather than storing jobs in `AppStateRest`)
|
||||
//! lets background jobs outlive the borrow of any particular state mutation
|
||||
//! and be looked up by id from tool calls issued at arbitrary points.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Mutex;
|
||||
use std::sync::OnceLock;
|
||||
@@ -44,7 +48,11 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
|
||||
while let Some(line) = job.try_read_line() {
|
||||
lines.push(line);
|
||||
}
|
||||
if lines.is_empty() { None } else { Some(lines) }
|
||||
if lines.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(lines)
|
||||
}
|
||||
}
|
||||
|
||||
/// Terminate a running background bash job and remove it from the registry.
|
||||
@@ -58,7 +66,9 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
|
||||
/// Return: `Ok(())` on success, `Err` if the lock is poisoned or no job
|
||||
/// with that id exists.
|
||||
pub fn bash_kill(id: &str) -> anyhow::Result<()> {
|
||||
let mut map = bash_jobs_map().lock().map_err(|e| anyhow::anyhow!("lock error: {e}"))?;
|
||||
let mut map = bash_jobs_map()
|
||||
.lock()
|
||||
.map_err(|e| anyhow::anyhow!("lock error: {e}"))?;
|
||||
let job = map.remove(id);
|
||||
match job {
|
||||
Some(job) => {
|
||||
|
||||
+19
-13
@@ -9,11 +9,10 @@
|
||||
//! Why: running bash commands on a detached thread with a channel (rather
|
||||
//! than synchronously) lets the TUI stay responsive while long-running
|
||||
//! shell commands execute in the background.
|
||||
|
||||
use std::io::BufRead;
|
||||
use std::process::{Command, Stdio};
|
||||
use std::sync::mpsc;
|
||||
use std::thread;
|
||||
use std::io::BufRead;
|
||||
|
||||
/// Maximum number of output lines buffered in memory per background job.
|
||||
/// Beyond this limit, old output is dropped to prevent OOM (CWE-770).
|
||||
@@ -59,17 +58,23 @@ pub fn spawn_bash_job(command: String) -> BashJob {
|
||||
// Spawn a named thread for easier debugging. If Builder::spawn fails
|
||||
// (e.g. OS resource limit), fall back to unnameable thread::spawn.
|
||||
let thread_name = format!("bgbash-{}", &thread_id[..8.min(thread_id.len())]);
|
||||
if thread::Builder::new().name(thread_name).spawn({
|
||||
// Clone everything the closure captures so we can also pass it
|
||||
// to the fallback thread without moving.
|
||||
let cmd = cmd.clone();
|
||||
let output_tx = output_tx.clone();
|
||||
let pid_tx = pid_tx.clone();
|
||||
let id_for_log = id_for_log.clone();
|
||||
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log)
|
||||
}).is_err()
|
||||
if thread::Builder::new()
|
||||
.name(thread_name)
|
||||
.spawn({
|
||||
// Clone everything the closure captures so we can also pass it
|
||||
// to the fallback thread without moving.
|
||||
let cmd = cmd.clone();
|
||||
let output_tx = output_tx.clone();
|
||||
let pid_tx = pid_tx.clone();
|
||||
let id_for_log = id_for_log.clone();
|
||||
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log)
|
||||
})
|
||||
.is_err()
|
||||
{
|
||||
tracing::warn!("[bgbash:{}] failed to spawn named thread, using unnamed fallback", id_for_log);
|
||||
tracing::warn!(
|
||||
"[bgbash:{}] failed to spawn named thread, using unnamed fallback",
|
||||
id_for_log
|
||||
);
|
||||
thread::spawn(move || {
|
||||
spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log);
|
||||
});
|
||||
@@ -139,7 +144,8 @@ fn spawn_bash_thread_body(
|
||||
if output_tx.try_send(line).is_err() {
|
||||
tracing::debug!(
|
||||
"[bgbash:{}] output buffer full ({} lines), discarding remaining output",
|
||||
id_for_log, MAX_OUTPUT_LINES,
|
||||
id_for_log,
|
||||
MAX_OUTPUT_LINES,
|
||||
);
|
||||
break;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Background bash: run shell commands off the main thread, poll their
|
||||
//! output non-blockingly, and terminate them on demand.
|
||||
|
||||
pub mod control;
|
||||
pub mod job;
|
||||
|
||||
+231
-158
@@ -84,10 +84,17 @@ const ASSUMPTION_PATTERNS: &[&str] = &[
|
||||
|
||||
/// Network-exfiltration and credential-disclosure patterns for bash.
|
||||
const EXFIL_PATTERNS: &[&str] = &[
|
||||
"curl ", "wget ", "nc -e ", "ncat ", "/dev/tcp/",
|
||||
"base64 -d |", "base64 --decode |",
|
||||
"openssl s_client", "ssh -R ",
|
||||
"scp /", "rsync /",
|
||||
"curl ",
|
||||
"wget ",
|
||||
"nc -e ",
|
||||
"ncat ",
|
||||
"/dev/tcp/",
|
||||
"base64 -d |",
|
||||
"base64 --decode |",
|
||||
"openssl s_client",
|
||||
"ssh -R ",
|
||||
"scp /",
|
||||
"rsync /",
|
||||
];
|
||||
|
||||
/// Substrings of well-known credential / secret files that bash must not read.
|
||||
@@ -115,169 +122,63 @@ impl Harness {
|
||||
/// Decide whether a tool call is allowed to execute.
|
||||
///
|
||||
/// Flow: ALL tools are gated (not just risky ones), closing the bypass
|
||||
/// for MCP tools (which are never in the risky list). Basic path
|
||||
/// traversal and reason validation applies to any tool with a `path`
|
||||
/// argument. Heavy content scanning (stub/denial/assumption/exfiltration)
|
||||
/// only applies to risky tools. MCP tools (mcp__ prefix) are treated
|
||||
/// as risky because their behaviour is unknown.
|
||||
/// for MCP tools (which are never in the risky list). Delegates to
|
||||
/// smaller helper methods for each concern: path traversal, output
|
||||
/// path validation, content scanning, bash safety, and reason checks.
|
||||
///
|
||||
/// Return: `Verdict::Allow` or `Verdict::Block(reason)`.
|
||||
#[allow(clippy::too_many_lines, clippy::unnecessary_debug_formatting)]
|
||||
pub fn gate_tool_call(
|
||||
tool_name: &str,
|
||||
args: &serde_json::Value,
|
||||
workspace_roots: &[&std::path::Path],
|
||||
) -> Verdict {
|
||||
|
||||
let is_risky = crate::tool::tool_is_risky(tool_name);
|
||||
let is_mcp = tool_name.starts_with("mcp__");
|
||||
|
||||
// ── Universal checks applied to EVERY tool ──
|
||||
|
||||
// Path traversal: check ANY tool that accepts a path argument,
|
||||
// not just write/edit/delete, so tools like read, MCP tools,
|
||||
// and future tools are also protected.
|
||||
if let Some(path) = args.get("path").and_then(|v| v.as_str()) {
|
||||
if path.contains("..") {
|
||||
return Verdict::Block(
|
||||
"path traversal detected in 'path' argument".to_string(),
|
||||
);
|
||||
}
|
||||
if !workspace_roots.is_empty() {
|
||||
let abs_check = std::path::PathBuf::from(path);
|
||||
if abs_check.is_absolute()
|
||||
&& !workspace_roots.iter().any(|r| abs_check.starts_with(r))
|
||||
{
|
||||
return Verdict::Block(format!(
|
||||
"absolute path '{path}' is outside all workspace roots"
|
||||
));
|
||||
}
|
||||
}
|
||||
// Universal checks applied to EVERY tool.
|
||||
if let Some(v) = Self::check_path_traversal(args, workspace_roots) {
|
||||
return v;
|
||||
}
|
||||
if let Some(v) = Self::check_output_path(tool_name, args, workspace_roots) {
|
||||
return v;
|
||||
}
|
||||
|
||||
// Workspace-root validation for output path.
|
||||
if let Some(out_path) = Self::find_output_path(tool_name, args) {
|
||||
if !workspace_roots.is_empty()
|
||||
&& !out_path.starts_with("/tmp")
|
||||
&& !out_path.is_absolute()
|
||||
{
|
||||
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
|
||||
if !allowed {
|
||||
return Verdict::Block(format!(
|
||||
"output path '{out_path:?}' is outside all workspace roots"
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Risky / MCP tool checks ──
|
||||
// Non-risky, non-MCP tools (read, grep, glob, recall, etc.) are
|
||||
// allowed after universal checks above.
|
||||
// Non-risky, non-MCP tools pass after universal checks.
|
||||
if !is_risky && !is_mcp {
|
||||
return Verdict::Allow;
|
||||
}
|
||||
|
||||
// File-mutating tools: write / edit / delete
|
||||
// File-mutating tools: require a meaningful reason.
|
||||
if matches!(tool_name, "write" | "edit" | "delete") {
|
||||
match Self::validate_reason(tool_name, args) {
|
||||
Ok(()) => {}
|
||||
Err(msg) => return Verdict::Block(msg),
|
||||
if let Err(msg) = Self::validate_reason(tool_name, args) {
|
||||
return Verdict::Block(msg);
|
||||
}
|
||||
}
|
||||
|
||||
// write / edit content must not contain stubs, denial language, or
|
||||
// assumption language.
|
||||
if matches!(tool_name, "write" | "edit") {
|
||||
if let Some(content) = Self::extract_content(tool_name, args) {
|
||||
if let Some(pat) = Self::first_match(&content, STUB_PATTERNS) {
|
||||
return Verdict::Block(format!(
|
||||
"content contains stub/placeholder pattern '{pat}'; \
|
||||
production code must be fully implemented — \
|
||||
replace the stub with a real implementation"
|
||||
));
|
||||
}
|
||||
if let Some(pat) = Self::first_match(&content, DENIAL_PATTERNS) {
|
||||
return Verdict::Block(format!(
|
||||
"content contains denial/punt pattern '{pat}'; \
|
||||
implement the change properly instead of skipping"
|
||||
));
|
||||
}
|
||||
if let Some(pat) = Self::first_match(&content, ASSUMPTION_PATTERNS) {
|
||||
return Verdict::Block(format!(
|
||||
"content contains assumption pattern '{pat}'; \
|
||||
verify against data/tests instead of guessing"
|
||||
));
|
||||
}
|
||||
}
|
||||
// write / edit content scanning for stub/denial/assumption patterns.
|
||||
if let Some(v) = Self::check_content_safety(tool_name, args) {
|
||||
return v;
|
||||
}
|
||||
|
||||
// Bash: destructive patterns, exfiltration (ALL commands checked,
|
||||
// no safe-command whitelist), sensitive-path reads.
|
||||
if tool_name == "bash" {
|
||||
let cmd = args.get("command").and_then(|v| v.as_str()).unwrap_or("");
|
||||
if cmd.contains("..") {
|
||||
return Verdict::Block(
|
||||
"path traversal detected in bash command".to_string(),
|
||||
);
|
||||
}
|
||||
// Exfiltration patterns are checked on EVERY bash command,
|
||||
// regardless of prefix. The safe-command whitelist was removed
|
||||
// because it could be bypassed with command chaining.
|
||||
for pat in EXFIL_PATTERNS {
|
||||
if cmd.contains(pat) {
|
||||
return Verdict::Block(format!(
|
||||
"potential data-exfiltration command blocked (matched '{pat}')"
|
||||
));
|
||||
}
|
||||
}
|
||||
for pat in SENSITIVE_PATH_PATTERNS {
|
||||
if cmd.contains(pat) {
|
||||
return Verdict::Block(format!(
|
||||
"refused to read/write sensitive path '{pat}'"
|
||||
));
|
||||
}
|
||||
}
|
||||
let dangerous_patterns = [
|
||||
"rm -rf /", "rm -rf --no-preserve-root",
|
||||
"rm -rf ~", "rm -fr /", "mkfs.", "dd if=",
|
||||
":(){", "> /dev/sda", "chmod -R 000 /",
|
||||
"shutdown ", "poweroff ", "reboot ", "halt ",
|
||||
];
|
||||
for pat in &dangerous_patterns {
|
||||
if cmd.contains(pat) {
|
||||
return Verdict::Block(format!(
|
||||
"destructive command pattern blocked: {pat}"
|
||||
));
|
||||
}
|
||||
}
|
||||
// Also scan heredocs / -c / inline content for stub/denial
|
||||
// language (e.g. `bash -c 'echo todo!()'`)
|
||||
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
|
||||
// Bash-specific destructive / exfiltration checks.
|
||||
if let Some(v) = Self::check_bash_safety(args) {
|
||||
return v;
|
||||
}
|
||||
|
||||
// git_operator: require a non-trivial reason.
|
||||
if tool_name == "git_operator" && !Self::has_valid_reason(args, MIN_REASON_LEN) {
|
||||
if args.get("reason").and_then(|v| v.as_str()).is_some() {
|
||||
return Verdict::Block(format!(
|
||||
"bash command contains stub pattern '{pat}'"
|
||||
"git_operator requires a non-trivial 'reason' \
|
||||
(>= {MIN_REASON_LEN} chars) explaining the operation"
|
||||
));
|
||||
}
|
||||
return Verdict::Block(
|
||||
"git_operator requires a 'reason' argument explaining the operation".to_string(),
|
||||
);
|
||||
}
|
||||
|
||||
// git_operator: require a non-trivial reason as well.
|
||||
if tool_name == "git_operator" {
|
||||
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
|
||||
if reason.trim().len() < MIN_REASON_LEN {
|
||||
return Verdict::Block(format!(
|
||||
"git_operator requires a non-trivial 'reason' \
|
||||
(>= {MIN_REASON_LEN} chars) explaining the operation"
|
||||
));
|
||||
}
|
||||
} else {
|
||||
return Verdict::Block(
|
||||
"git_operator requires a 'reason' argument explaining the operation"
|
||||
.to_string(),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// MCP tools: unknown behaviour — require a reason if they take
|
||||
// arguments, to discourage lazy invocations.
|
||||
// MCP tools: require a reason when they take meaningful arguments.
|
||||
if is_mcp {
|
||||
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
|
||||
if reason.trim().len() < MIN_REASON_LEN {
|
||||
@@ -287,7 +188,6 @@ impl Harness {
|
||||
));
|
||||
}
|
||||
} else if args.as_object().is_some_and(|m| !m.is_empty()) {
|
||||
// Only require reason when there are meaningful arguments
|
||||
return Verdict::Block(format!(
|
||||
"MCP tool '{tool_name}' requires a 'reason' argument \
|
||||
explaining the operation"
|
||||
@@ -298,6 +198,163 @@ impl Harness {
|
||||
Verdict::Allow
|
||||
}
|
||||
|
||||
/// Check for path traversal in the `path` argument and verify it stays
|
||||
/// within workspace roots.
|
||||
///
|
||||
/// Flow: reject any path containing `..` → if workspace roots are set,
|
||||
/// reject absolute paths outside every root.
|
||||
///
|
||||
/// Return: `Some(Verdict::Block)` on violation, `None` if the check
|
||||
/// passes or the tool has no `path` argument.
|
||||
fn check_path_traversal(
|
||||
args: &serde_json::Value,
|
||||
workspace_roots: &[&std::path::Path],
|
||||
) -> Option<Verdict> {
|
||||
let path = args.get("path")?.as_str()?;
|
||||
if path.contains("..") {
|
||||
return Some(Verdict::Block(
|
||||
"path traversal detected in 'path' argument".to_string(),
|
||||
));
|
||||
}
|
||||
if !workspace_roots.is_empty() {
|
||||
let abs_check = std::path::PathBuf::from(path);
|
||||
if abs_check.is_absolute() && !workspace_roots.iter().any(|r| abs_check.starts_with(r))
|
||||
{
|
||||
return Some(Verdict::Block(format!(
|
||||
"absolute path '{path}' is outside all workspace roots"
|
||||
)));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Verify that a tool's output path (if any) stays within workspace roots.
|
||||
///
|
||||
/// Flow: if `find_output_path` yields a path, reject it unless it's
|
||||
/// under `/tmp`, already absolute, or within a workspace root.
|
||||
///
|
||||
/// Return: `Some(Verdict::Block)` on violation, `None` otherwise.
|
||||
fn check_output_path(
|
||||
tool_name: &str,
|
||||
args: &serde_json::Value,
|
||||
workspace_roots: &[&std::path::Path],
|
||||
) -> Option<Verdict> {
|
||||
let out_path = Self::find_output_path(tool_name, args)?;
|
||||
if !workspace_roots.is_empty() && !out_path.starts_with("/tmp") && !out_path.is_absolute() {
|
||||
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
|
||||
if !allowed {
|
||||
return Some(Verdict::Block(format!(
|
||||
"output path '{}' is outside all workspace roots",
|
||||
out_path.display(),
|
||||
)));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Check write/edit content for stub, denial, and assumption patterns.
|
||||
///
|
||||
/// Return: `Some(Verdict::Block)` with a description of the first
|
||||
/// matched pattern, `None` if the content is clean or not applicable.
|
||||
fn check_content_safety(tool_name: &str, args: &serde_json::Value) -> Option<Verdict> {
|
||||
if !matches!(tool_name, "write" | "edit") {
|
||||
return None;
|
||||
}
|
||||
let content = Self::extract_content(tool_name, args)?;
|
||||
for (patterns, msg_prefix) in [
|
||||
(&STUB_PATTERNS, "stub/placeholder"),
|
||||
(&DENIAL_PATTERNS, "denial/punt"),
|
||||
(&ASSUMPTION_PATTERNS, "assumption"),
|
||||
] {
|
||||
if let Some(pat) = Self::first_match(&content, patterns) {
|
||||
let msg = match msg_prefix {
|
||||
"stub/placeholder" => format!(
|
||||
"content contains stub/placeholder pattern '{pat}'; \
|
||||
production code must be fully implemented — \
|
||||
replace the stub with a real implementation"
|
||||
),
|
||||
"denial/punt" => format!(
|
||||
"content contains denial/punt pattern '{pat}'; \
|
||||
implement the change properly instead of skipping"
|
||||
),
|
||||
_ => format!(
|
||||
"content contains assumption pattern '{pat}'; \
|
||||
verify against data/tests instead of guessing"
|
||||
),
|
||||
};
|
||||
return Some(Verdict::Block(msg));
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Check bash commands for path traversal, exfiltration, sensitive
|
||||
/// path reads, destructive patterns, and stub language.
|
||||
///
|
||||
/// Flow: extract the `command` argument → check each category in
|
||||
/// sequence, returning the first violation found.
|
||||
///
|
||||
/// Return: `Some(Verdict::Block)` on any violation, `None` if the
|
||||
/// tool is not bash or the command is safe.
|
||||
fn check_bash_safety(args: &serde_json::Value) -> Option<Verdict> {
|
||||
let cmd = args.get("command")?.as_str()?;
|
||||
if cmd.contains("..") {
|
||||
return Some(Verdict::Block(
|
||||
"path traversal detected in bash command".to_string(),
|
||||
));
|
||||
}
|
||||
for pat in EXFIL_PATTERNS {
|
||||
if cmd.contains(pat) {
|
||||
return Some(Verdict::Block(format!(
|
||||
"potential data-exfiltration command blocked (matched '{pat}')"
|
||||
)));
|
||||
}
|
||||
}
|
||||
for pat in SENSITIVE_PATH_PATTERNS {
|
||||
if cmd.contains(pat) {
|
||||
return Some(Verdict::Block(format!(
|
||||
"refused to read/write sensitive path '{pat}'"
|
||||
)));
|
||||
}
|
||||
}
|
||||
let dangerous_patterns = [
|
||||
"rm -rf /",
|
||||
"rm -rf --no-preserve-root",
|
||||
"rm -rf ~",
|
||||
"rm -fr /",
|
||||
"mkfs.",
|
||||
"dd if=",
|
||||
":(){",
|
||||
"> /dev/sda",
|
||||
"chmod -R 000 /",
|
||||
"shutdown ",
|
||||
"poweroff ",
|
||||
"reboot ",
|
||||
"halt ",
|
||||
];
|
||||
for pat in &dangerous_patterns {
|
||||
if cmd.contains(pat) {
|
||||
return Some(Verdict::Block(format!(
|
||||
"destructive command pattern blocked: {pat}"
|
||||
)));
|
||||
}
|
||||
}
|
||||
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
|
||||
return Some(Verdict::Block(format!(
|
||||
"bash command contains stub pattern '{pat}'"
|
||||
)));
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
/// Check whether the given `args` contain a non-trivial `reason`
|
||||
/// argument meeting the minimum length requirement.
|
||||
fn has_valid_reason(args: &serde_json::Value, min_len: usize) -> bool {
|
||||
args.get("reason")
|
||||
.and_then(|v| v.as_str())
|
||||
.is_some_and(|r| r.trim().len() >= min_len)
|
||||
}
|
||||
|
||||
/// Validate the `reason` argument for a mutating tool.
|
||||
///
|
||||
/// Flow: require the field to exist and be a non-empty string ≥
|
||||
@@ -317,17 +374,13 @@ impl Harness {
|
||||
Some(v) => match v.as_str() {
|
||||
Some(s) => s,
|
||||
None => {
|
||||
return Err(format!(
|
||||
"{tool_name} 'reason' must be a string"
|
||||
));
|
||||
return Err(format!("{tool_name} 'reason' must be a string"));
|
||||
}
|
||||
},
|
||||
};
|
||||
let trimmed = reason.trim();
|
||||
if trimmed.is_empty() {
|
||||
return Err(format!(
|
||||
"{tool_name} 'reason' must not be empty"
|
||||
));
|
||||
return Err(format!("{tool_name} 'reason' must not be empty"));
|
||||
}
|
||||
if trimmed.len() < MIN_REASON_LEN {
|
||||
return Err(format!(
|
||||
@@ -339,9 +392,20 @@ impl Harness {
|
||||
// Reject generic non-answers
|
||||
let lower = trimmed.to_lowercase();
|
||||
let non_answers = [
|
||||
"fix", "update", "change", "edit", "modify",
|
||||
"implement", "add", "remove", "delete",
|
||||
"make it work", "make work", "test", "wip", "tbd",
|
||||
"fix",
|
||||
"update",
|
||||
"change",
|
||||
"edit",
|
||||
"modify",
|
||||
"implement",
|
||||
"add",
|
||||
"remove",
|
||||
"delete",
|
||||
"make it work",
|
||||
"make work",
|
||||
"test",
|
||||
"wip",
|
||||
"tbd",
|
||||
];
|
||||
if non_answers.iter().any(|n| lower == *n) {
|
||||
return Err(format!(
|
||||
@@ -356,7 +420,10 @@ impl Harness {
|
||||
/// Extract the textual content of a write/edit call, if any.
|
||||
fn extract_content(tool_name: &str, args: &serde_json::Value) -> Option<String> {
|
||||
match tool_name {
|
||||
"write" => args.get("content").and_then(|v| v.as_str()).map(String::from),
|
||||
"write" => args
|
||||
.get("content")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(String::from),
|
||||
"edit" => {
|
||||
let old = args.get("old").and_then(|v| v.as_str()).unwrap_or("");
|
||||
let new = args.get("new").and_then(|v| v.as_str()).unwrap_or("");
|
||||
@@ -377,9 +444,10 @@ impl Harness {
|
||||
/// Extract a candidate output path from a tool call, if one exists.
|
||||
fn find_output_path(tool_name: &str, args: &serde_json::Value) -> Option<std::path::PathBuf> {
|
||||
match tool_name {
|
||||
"write" | "edit" | "delete" | "read" => {
|
||||
args.get("path").and_then(|v| v.as_str()).map(std::path::PathBuf::from)
|
||||
}
|
||||
"write" | "edit" | "delete" | "read" => args
|
||||
.get("path")
|
||||
.and_then(|v| v.as_str())
|
||||
.map(std::path::PathBuf::from),
|
||||
"bash" => {
|
||||
let cmd = args.get("command").and_then(|v| v.as_str())?;
|
||||
let lower = cmd.to_lowercase();
|
||||
@@ -397,7 +465,6 @@ impl Harness {
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
impl Default for Harness {
|
||||
@@ -418,7 +485,10 @@ mod tests {
|
||||
return match verdict.to_lowercase().as_str() {
|
||||
"allow" => Some(Verdict::Allow),
|
||||
"block" => Some(Verdict::Block(
|
||||
v.get("reason").and_then(|r| r.as_str()).unwrap_or("blocked").to_string()
|
||||
v.get("reason")
|
||||
.and_then(|r| r.as_str())
|
||||
.unwrap_or("blocked")
|
||||
.to_string(),
|
||||
)),
|
||||
_ => None,
|
||||
};
|
||||
@@ -430,7 +500,11 @@ mod tests {
|
||||
return Some(Verdict::Allow);
|
||||
}
|
||||
if l.starts_with("verdict: block") {
|
||||
let reason = line.split_once(':').map_or("blocked", |x| x.1).trim().to_string();
|
||||
let reason = line
|
||||
.split_once(':')
|
||||
.map_or("blocked", |x| x.1)
|
||||
.trim()
|
||||
.to_string();
|
||||
return Some(Verdict::Block(reason));
|
||||
}
|
||||
}
|
||||
@@ -450,7 +524,6 @@ mod tests {
|
||||
assert_eq!(result, Verdict::Allow);
|
||||
}
|
||||
|
||||
|
||||
#[test]
|
||||
fn test_parse_verdict_json_allow() {
|
||||
let v = parse_verdict(r#"{"verdict": "allow"}"#);
|
||||
|
||||
+120
-119
@@ -47,13 +47,20 @@ impl LspClient {
|
||||
cmd.stdout(Stdio::piped());
|
||||
cmd.stderr(Stdio::piped());
|
||||
|
||||
let mut child = cmd.spawn()
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
.map_err(|e| anyhow::anyhow!("failed to spawn LSP server '{command}': {e}"))?;
|
||||
|
||||
let stdin = child.stdin.take()
|
||||
let stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("failed to capture stdin for LSP server"))?;
|
||||
let stdout = BufReader::new(child.stdout.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?);
|
||||
let stdout = BufReader::new(
|
||||
child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?,
|
||||
);
|
||||
|
||||
let mut client = LspClient {
|
||||
stdin,
|
||||
@@ -106,7 +113,11 @@ impl LspClient {
|
||||
}
|
||||
});
|
||||
|
||||
let result = client.call_with_timeout("initialize", &init_params, Duration::from_millis(LSP_INIT_TIMEOUT_MS))?;
|
||||
let result = client.call_with_timeout(
|
||||
"initialize",
|
||||
&init_params,
|
||||
Duration::from_millis(LSP_INIT_TIMEOUT_MS),
|
||||
)?;
|
||||
client.server_capabilities = result.get("capabilities").cloned().unwrap_or_default();
|
||||
|
||||
client.notify("initialized", &json!({}))?;
|
||||
@@ -122,7 +133,12 @@ impl LspClient {
|
||||
self.call_with_timeout(method, params, Duration::from_millis(LSP_CALL_TIMEOUT_MS))
|
||||
}
|
||||
|
||||
fn call_with_timeout(&mut self, method: &str, params: &Value, timeout: Duration) -> anyhow::Result<Value> {
|
||||
fn call_with_timeout(
|
||||
&mut self,
|
||||
method: &str,
|
||||
params: &Value,
|
||||
timeout: Duration,
|
||||
) -> anyhow::Result<Value> {
|
||||
self.next_id += 1;
|
||||
let id = self.next_id;
|
||||
let req = json!({
|
||||
@@ -148,11 +164,14 @@ impl LspClient {
|
||||
let body = serde_json::to_string(msg)
|
||||
.map_err(|e| anyhow::anyhow!("failed to serialize LSP message: {e}"))?;
|
||||
let header = format!("Content-Length: {}\r\n\r\n", body.len());
|
||||
self.stdin.write_all(header.as_bytes())
|
||||
self.stdin
|
||||
.write_all(header.as_bytes())
|
||||
.map_err(|e| anyhow::anyhow!("failed to write LSP frame header: {e}"))?;
|
||||
self.stdin.write_all(body.as_bytes())
|
||||
self.stdin
|
||||
.write_all(body.as_bytes())
|
||||
.map_err(|e| anyhow::anyhow!("failed to write LSP frame body: {e}"))?;
|
||||
self.stdin.flush()
|
||||
self.stdin
|
||||
.flush()
|
||||
.map_err(|e| anyhow::anyhow!("failed to flush LSP stdin: {e}"))?;
|
||||
Ok(())
|
||||
}
|
||||
@@ -166,8 +185,14 @@ impl LspClient {
|
||||
let frame = self.read_frame()?;
|
||||
if frame.get("id") == Some(&json!(expected_id)) {
|
||||
if let Some(err) = frame.get("error") {
|
||||
let code = err.get("code").and_then(serde_json::Value::as_i64).unwrap_or(0);
|
||||
let msg = err.get("message").and_then(|m| m.as_str()).unwrap_or("unknown error");
|
||||
let code = err
|
||||
.get("code")
|
||||
.and_then(serde_json::Value::as_i64)
|
||||
.unwrap_or(0);
|
||||
let msg = err
|
||||
.get("message")
|
||||
.and_then(|m| m.as_str())
|
||||
.unwrap_or("unknown error");
|
||||
anyhow::bail!("LSP error {code}: {msg}");
|
||||
}
|
||||
return Ok(frame.get("result").cloned().unwrap_or(Value::Null));
|
||||
@@ -205,8 +230,9 @@ impl LspClient {
|
||||
// Cap Content-Length at 64 MiB to prevent OOM from a
|
||||
// malicious or misconfigured LSP server (CWE-400).
|
||||
const MAX_CONTENT_LENGTH: usize = 64 * 1024 * 1024;
|
||||
let length: usize = len_str.trim().parse::<usize>()
|
||||
.map_err(|e| anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e))?;
|
||||
let length: usize = len_str.trim().parse::<usize>().map_err(|e| {
|
||||
anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e)
|
||||
})?;
|
||||
if length > MAX_CONTENT_LENGTH {
|
||||
anyhow::bail!(
|
||||
"Content-Length {length} exceeds maximum allowed size of {MAX_CONTENT_LENGTH} bytes",
|
||||
@@ -220,7 +246,8 @@ impl LspClient {
|
||||
.ok_or_else(|| anyhow::anyhow!("missing Content-Length header in LSP response"))?;
|
||||
|
||||
let mut body = vec![0u8; length];
|
||||
self.stdout.read_exact(&mut body)
|
||||
self.stdout
|
||||
.read_exact(&mut body)
|
||||
.map_err(|e| anyhow::anyhow!("failed to read LSP body ({length} bytes): {e}"))?;
|
||||
|
||||
let json_str = String::from_utf8(body)
|
||||
@@ -230,74 +257,98 @@ impl LspClient {
|
||||
.map_err(|e| anyhow::anyhow!("invalid JSON in LSP response: {e}"))
|
||||
}
|
||||
|
||||
pub fn did_open(&mut self, uri: &str, language_id: &str, version: i32, text: &str) -> anyhow::Result<()> {
|
||||
self.notify("textDocument/didOpen", &json!({
|
||||
"textDocument": {
|
||||
"uri": uri,
|
||||
"languageId": language_id,
|
||||
"version": version,
|
||||
"text": text
|
||||
}
|
||||
}))
|
||||
pub fn did_open(
|
||||
&mut self,
|
||||
uri: &str,
|
||||
language_id: &str,
|
||||
version: i32,
|
||||
text: &str,
|
||||
) -> anyhow::Result<()> {
|
||||
self.notify(
|
||||
"textDocument/didOpen",
|
||||
&json!({
|
||||
"textDocument": {
|
||||
"uri": uri,
|
||||
"languageId": language_id,
|
||||
"version": version,
|
||||
"text": text
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn did_change(&mut self, uri: &str, version: i32, text: &str) -> anyhow::Result<()> {
|
||||
self.notify("textDocument/didChange", &json!({
|
||||
"textDocument": {
|
||||
"uri": uri,
|
||||
"version": version
|
||||
},
|
||||
"contentChanges": [{
|
||||
"text": text
|
||||
}]
|
||||
}))
|
||||
self.notify(
|
||||
"textDocument/didChange",
|
||||
&json!({
|
||||
"textDocument": {
|
||||
"uri": uri,
|
||||
"version": version
|
||||
},
|
||||
"contentChanges": [{
|
||||
"text": text
|
||||
}]
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn did_close(&mut self, uri: &str) -> anyhow::Result<()> {
|
||||
self.notify("textDocument/didClose", &json!({
|
||||
"textDocument": {
|
||||
"uri": uri
|
||||
}
|
||||
}))
|
||||
self.notify(
|
||||
"textDocument/didClose",
|
||||
&json!({
|
||||
"textDocument": {
|
||||
"uri": uri
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn hover(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
|
||||
self.call("textDocument/hover", &json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}))
|
||||
self.call(
|
||||
"textDocument/hover",
|
||||
&json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn completion(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
|
||||
self.call("textDocument/completion", &json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}))
|
||||
self.call(
|
||||
"textDocument/completion",
|
||||
&json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn goto_definition(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
|
||||
self.call("textDocument/definition", &json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}))
|
||||
pub fn goto_definition(
|
||||
&mut self,
|
||||
uri: &str,
|
||||
line: u32,
|
||||
character: u32,
|
||||
) -> anyhow::Result<Value> {
|
||||
self.call(
|
||||
"textDocument/definition",
|
||||
&json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character }
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn references(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
|
||||
self.call("textDocument/references", &json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character },
|
||||
"context": {
|
||||
"includeDeclaration": true
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
pub fn document_symbols(&mut self, uri: &str) -> anyhow::Result<Value> {
|
||||
self.call("textDocument/documentSymbol", &json!({
|
||||
"textDocument": { "uri": uri }
|
||||
}))
|
||||
self.call(
|
||||
"textDocument/references",
|
||||
&json!({
|
||||
"textDocument": { "uri": uri },
|
||||
"position": { "line": line, "character": character },
|
||||
"context": {
|
||||
"includeDeclaration": true
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
pub fn collect_diagnostics(
|
||||
@@ -313,64 +364,14 @@ impl LspClient {
|
||||
);
|
||||
self.did_close(uri)?;
|
||||
match result {
|
||||
Ok(params) => Ok(params.get("diagnostics").cloned().unwrap_or_else(|| json!([]))),
|
||||
Ok(params) => Ok(params
|
||||
.get("diagnostics")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| json!([]))),
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Health-check the LSP server.
|
||||
///
|
||||
/// Sends a `textDocument/documentSymbol` request on a dummy URI with a
|
||||
/// 2-second timeout. Returns `true` if the server responds at all —
|
||||
/// including with an error response such as "file not found", which
|
||||
/// still proves the process is up and the JSON-RPC channel is live.
|
||||
/// Returns `false` on timeout, EOF, or any read/write error.
|
||||
///
|
||||
/// Flow: build request → `send_frame` → poll frames until id matches
|
||||
/// (alive) or deadline/read error fires (dead).
|
||||
#[allow(dead_code)]
|
||||
pub fn is_alive(&mut self) -> bool {
|
||||
self.next_id += 1;
|
||||
let id = self.next_id;
|
||||
let req = json!({
|
||||
"jsonrpc": "2.0",
|
||||
"id": id,
|
||||
"method": "textDocument/documentSymbol",
|
||||
"params": {
|
||||
"textDocument": { "uri": "file:///__zesdex_lsp_health_check__.txt" }
|
||||
}
|
||||
});
|
||||
if self.send_frame(&req).is_err() {
|
||||
return false;
|
||||
}
|
||||
let timeout = Duration::from_secs(2);
|
||||
let deadline = Instant::now() + timeout;
|
||||
loop {
|
||||
if Instant::now() > deadline {
|
||||
return false;
|
||||
}
|
||||
match self.read_frame() {
|
||||
Ok(frame) => {
|
||||
if frame.get("id") == Some(&json!(id)) {
|
||||
return true;
|
||||
}
|
||||
// Skip unrelated notifications/responses on the same channel.
|
||||
}
|
||||
Err(_) => return false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Send the LSP `exit` notification to request graceful shutdown.
|
||||
///
|
||||
/// Per the LSP spec, `exit` is a notification — the server is expected
|
||||
/// to terminate after receiving it without sending a response. We do
|
||||
/// not block on any reply.
|
||||
#[allow(dead_code)]
|
||||
pub fn exit(&mut self) -> anyhow::Result<()> {
|
||||
self.notify("exit", &json!({}))
|
||||
}
|
||||
|
||||
pub fn shutdown(&mut self) {
|
||||
let _ = self.call_with_timeout("shutdown", &json!({}), Duration::from_secs(5));
|
||||
let _ = self.notify("exit", &json!({}));
|
||||
|
||||
+54
-113
@@ -13,12 +13,6 @@ pub use client::{path_to_lsp_uri, LspClient};
|
||||
/// to issue LSP requests from threads or async tasks.
|
||||
#[derive(Clone)]
|
||||
pub struct LspServer {
|
||||
#[allow(dead_code)]
|
||||
pub name: String,
|
||||
#[allow(dead_code)]
|
||||
pub command: String,
|
||||
#[allow(dead_code)]
|
||||
pub args: Vec<String>,
|
||||
pub language_id: String,
|
||||
pub client: Arc<Mutex<LspClient>>,
|
||||
}
|
||||
@@ -38,12 +32,12 @@ pub struct OpenDoc {
|
||||
///
|
||||
/// Flow: caller calls `connect*` -> client spawned -> entry pushed to
|
||||
/// `servers` -> `extension_registry` is populated by `register_extensions`.
|
||||
/// File edits route through `find_server_for_path` / `find_server_for_extension`
|
||||
/// and are dispatched as `didOpen` / `didChange` notifications.
|
||||
/// File edits route through `extension_registry` and are dispatched as
|
||||
/// `didOpen` / `didChange` notifications.
|
||||
#[derive(Clone)]
|
||||
pub struct LspManager {
|
||||
pub servers: Vec<LspServer>,
|
||||
/// Maps file extension (".rs", ".ts", ...) -> server name.
|
||||
/// Maps file extension (".rs", ".ts", ...) -> language id.
|
||||
pub extension_registry: HashMap<String, String>,
|
||||
/// Maps document URI -> tracked open document state.
|
||||
pub open_files: HashMap<String, OpenDoc>,
|
||||
@@ -59,112 +53,73 @@ impl LspManager {
|
||||
}
|
||||
}
|
||||
|
||||
/// Spawn an LSP server and register it under `name`.
|
||||
/// Spawn an LSP server and register it under `language_id`.
|
||||
///
|
||||
/// Fails if a server with the same name is already connected.
|
||||
/// Fails if a server with the same `language_id` is already connected.
|
||||
pub fn connect(
|
||||
&mut self,
|
||||
name: &str,
|
||||
command: &str,
|
||||
args: &[String],
|
||||
language_id: &str,
|
||||
) -> anyhow::Result<()> {
|
||||
if self.servers.iter().any(|s| s.name == name) {
|
||||
anyhow::bail!("LSP server '{name}' is already connected");
|
||||
if self.servers.iter().any(|s| s.language_id == language_id) {
|
||||
anyhow::bail!("LSP server for language '{language_id}' is already connected");
|
||||
}
|
||||
let client = LspClient::spawn(command, args)?;
|
||||
self.servers.push(LspServer {
|
||||
name: name.to_string(),
|
||||
command: command.to_string(),
|
||||
args: args.to_vec(),
|
||||
language_id: language_id.to_string(),
|
||||
client: Arc::new(Mutex::new(client)),
|
||||
});
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Look up a connected server by name and return a reference to its entry.
|
||||
#[allow(dead_code)]
|
||||
pub fn find_server(&self, name: &str) -> Option<&LspServer> {
|
||||
self.servers.iter().find(|s| s.name == name)
|
||||
}
|
||||
|
||||
/// Return a clone of the `Arc<Mutex<LspClient>>` for a connected server.
|
||||
///
|
||||
/// Cloning the `Arc` lets callers issue requests without holding a
|
||||
/// borrow on the manager.
|
||||
pub fn get_client(&self, name: &str) -> Option<Arc<Mutex<LspClient>>> {
|
||||
self.servers.iter().find(|s| s.name == name).map(|s| s.client.clone())
|
||||
pub fn get_client(&self, language_id: &str) -> Option<Arc<Mutex<LspClient>>> {
|
||||
self.servers
|
||||
.iter()
|
||||
.find(|s| s.language_id == language_id)
|
||||
.map(|s| s.client.clone())
|
||||
}
|
||||
|
||||
/// Shut down and remove a server by name. Returns true if it existed.
|
||||
pub fn disconnect(&mut self, name: &str) -> bool {
|
||||
if let Some(server) = self.servers.iter().find(|s| s.name == name) {
|
||||
/// Shut down and remove a server by language. Returns true if it existed.
|
||||
pub fn disconnect(&mut self, language_id: &str) -> bool {
|
||||
if let Some(server) = self.servers.iter().find(|s| s.language_id == language_id) {
|
||||
if let Ok(mut client) = server.client.lock() {
|
||||
client.shutdown();
|
||||
}
|
||||
}
|
||||
let len = self.servers.len();
|
||||
self.servers.retain(|s| s.name != name);
|
||||
self.servers.retain(|s| s.language_id != language_id);
|
||||
self.servers.len() < len
|
||||
}
|
||||
|
||||
/// Return the language id (e.g. "rust") registered for `name`.
|
||||
pub fn get_language_id(&self, name: &str) -> Option<String> {
|
||||
self.servers.iter().find(|s| s.name == name).map(|s| s.language_id.clone())
|
||||
}
|
||||
|
||||
/// Resolve an extension (".rs", ".ts", ...) to its server's client.
|
||||
///
|
||||
/// Flow: lookup `extension_registry` -> resolve server name -> clone client.
|
||||
/// Returns `None` if no server has been registered for `ext`.
|
||||
#[allow(dead_code)]
|
||||
pub fn find_server_for_extension(&self, ext: &str) -> Option<Arc<Mutex<LspClient>>> {
|
||||
self.extension_registry
|
||||
.get(ext)
|
||||
.and_then(|name| self.get_client(name))
|
||||
}
|
||||
|
||||
/// Resolve a file path to its server's client by extension.
|
||||
///
|
||||
/// Flow: extract the extension from `path` -> delegate to
|
||||
/// `find_server_for_extension`. Files without an extension or with
|
||||
/// an unmapped extension return `None`.
|
||||
#[allow(dead_code)]
|
||||
pub fn find_server_for_path(&self, path: &Path) -> Option<Arc<Mutex<LspClient>>> {
|
||||
path.extension()
|
||||
.and_then(|e| e.to_str())
|
||||
.map(|s| format!(".{s}"))
|
||||
.and_then(|ext| self.find_server_for_extension(&ext))
|
||||
/// Return the language id (e.g. "rust") registered for `language_id`.
|
||||
pub fn get_language_id(&self, language_id: &str) -> Option<String> {
|
||||
self.servers
|
||||
.iter()
|
||||
.find(|s| s.language_id == language_id)
|
||||
.map(|s| s.language_id.clone())
|
||||
}
|
||||
|
||||
/// Register a set of file extensions for an already-connected server.
|
||||
///
|
||||
/// Flow: for each `ext`, write `server_name` into `extension_registry`.
|
||||
/// Re-registration overwrites the previous target. Unknown server
|
||||
/// names are accepted at this layer — caller must ensure `server_name`
|
||||
/// is connected or will be connected later.
|
||||
pub fn register_extensions(&mut self, server_name: &str, extensions: &[&str]) {
|
||||
/// Flow: for each `ext`, write `language_id` into `extension_registry`.
|
||||
/// Re-registration overwrites the previous target. Unknown language IDs
|
||||
/// are accepted at this layer — caller must ensure a server for
|
||||
/// `language_id` is connected or will be connected later.
|
||||
pub fn register_extensions(&mut self, language_id: &str, extensions: &[&str]) {
|
||||
for ext in extensions {
|
||||
self.extension_registry.insert(ext.to_string(), server_name.to_string());
|
||||
self.extension_registry
|
||||
.insert(ext.to_string(), language_id.to_string());
|
||||
}
|
||||
}
|
||||
|
||||
/// Return the registered server name for a given language id.
|
||||
///
|
||||
/// Flow: scan `servers` for the first entry whose `language_id` matches.
|
||||
/// Used when callers have a language hint rather than a file path.
|
||||
#[allow(dead_code)]
|
||||
pub fn get_server_name(&self, language: &str) -> Option<String> {
|
||||
self.servers
|
||||
.iter()
|
||||
.find(|s| s.language_id == language)
|
||||
.map(|s| s.name.clone())
|
||||
}
|
||||
|
||||
/// Notify the relevant LSP server that a file's contents have changed.
|
||||
///
|
||||
/// Flow: resolve server by extension -> read file contents ->
|
||||
/// Flow: resolve language by extension from the registry -> read file contents ->
|
||||
/// either send `didOpen` (first time) or `didChange` (already tracked)
|
||||
/// -> update `open_files` with the new version.
|
||||
///
|
||||
@@ -172,13 +127,20 @@ impl LspManager {
|
||||
/// error) are logged with `tracing::warn!` rather than propagated,
|
||||
/// so a stale notification cannot abort the calling flow.
|
||||
pub fn did_change_file(&mut self, path: &Path) {
|
||||
let Some(ext) = path.extension().and_then(|e| e.to_str()).map(|s| format!(".{s}")) else {
|
||||
let Some(ext) = path
|
||||
.extension()
|
||||
.and_then(|e| e.to_str())
|
||||
.map(|s| format!(".{s}"))
|
||||
else {
|
||||
tracing::warn!("did_change_file: path has no extension: {:?}", path);
|
||||
return;
|
||||
};
|
||||
|
||||
let server_name = if let Some(name) = self.extension_registry.get(&ext) { name.clone() } else {
|
||||
tracing::warn!("did_change_file: no LSP server registered for extension '{}'", ext);
|
||||
let Some(language_id) = self.extension_registry.get(&ext).cloned() else {
|
||||
tracing::warn!(
|
||||
"did_change_file: no LSP server registered for extension '{}'",
|
||||
ext
|
||||
);
|
||||
return;
|
||||
};
|
||||
|
||||
@@ -192,12 +154,8 @@ impl LspManager {
|
||||
}
|
||||
};
|
||||
|
||||
let language_id = self
|
||||
.get_language_id(&server_name)
|
||||
.unwrap_or_else(|| "plaintext".to_string());
|
||||
|
||||
let Some(client) = self.get_client(&server_name) else {
|
||||
tracing::warn!("did_change_file: server '{}' has no client", server_name);
|
||||
let Some(client) = self.get_client(&language_id) else {
|
||||
tracing::warn!("did_change_file: no client for language '{}'", language_id);
|
||||
return;
|
||||
};
|
||||
|
||||
@@ -210,7 +168,11 @@ impl LspManager {
|
||||
let mut client = match client.lock() {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
tracing::warn!("did_change_file: client mutex poisoned for '{}': {}", server_name, e);
|
||||
tracing::warn!(
|
||||
"did_change_file: client mutex poisoned for '{}': {}",
|
||||
language_id,
|
||||
e
|
||||
);
|
||||
return;
|
||||
}
|
||||
};
|
||||
@@ -224,7 +186,7 @@ impl LspManager {
|
||||
if let Err(e) = send_result {
|
||||
tracing::warn!(
|
||||
"did_change_file: failed to notify '{}' for {}: {}",
|
||||
server_name,
|
||||
language_id,
|
||||
uri,
|
||||
e
|
||||
);
|
||||
@@ -238,24 +200,6 @@ impl LspManager {
|
||||
version: next_version,
|
||||
},
|
||||
);
|
||||
|
||||
}
|
||||
|
||||
/// Record that `server_name` has an open document at `uri`.
|
||||
///
|
||||
/// Flow: insert/overwrite the `OpenDoc` entry in `open_files`.
|
||||
/// Does not contact the LSP server — pure local bookkeeping.
|
||||
#[allow(dead_code)]
|
||||
pub fn track_open_doc(&mut self, server_name: &str, uri: &str, language: &str, version: i32) {
|
||||
// server_name retained for future routing extensions; not stored today.
|
||||
let _ = server_name;
|
||||
self.open_files.insert(
|
||||
uri.to_string(),
|
||||
OpenDoc {
|
||||
language: language.to_string(),
|
||||
version,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/// Shut down every connected server and clear the server list.
|
||||
@@ -272,21 +216,20 @@ impl LspManager {
|
||||
self.servers.clear();
|
||||
}
|
||||
|
||||
/// Snapshot the connected servers as `(name, language_id, has_open_docs)` triples.
|
||||
/// Snapshot the connected servers as `(language_id, has_open_docs)` pairs.
|
||||
///
|
||||
/// `has_open_docs` is true if any tracked `OpenDoc` was registered
|
||||
/// against this server's clients. Useful for status displays.
|
||||
pub fn list_servers(&self) -> Vec<(String, String, bool)> {
|
||||
pub fn list_servers(&self) -> Vec<(String, bool)> {
|
||||
self.servers
|
||||
.iter()
|
||||
.map(|s| {
|
||||
let name = s.name.clone();
|
||||
let lang = s.language_id.clone();
|
||||
let has_open = self
|
||||
.open_files
|
||||
.values()
|
||||
.any(|d| d.language == s.language_id);
|
||||
(name, lang, has_open)
|
||||
(lang, has_open)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
@@ -294,18 +237,17 @@ impl LspManager {
|
||||
/// Connect an LSP server and register its default extensions in one call.
|
||||
///
|
||||
/// Flow: invoke `connect` -> on success, register `extensions` against
|
||||
/// `name` in `extension_registry`. If `connect` fails, the registries
|
||||
/// `language_id` in `extension_registry`. If `connect` fails, the registries
|
||||
/// are left untouched and the error is propagated.
|
||||
pub fn connect_with_extensions(
|
||||
&mut self,
|
||||
name: &str,
|
||||
command: &str,
|
||||
args: &[String],
|
||||
language_id: &str,
|
||||
extensions: &[&str],
|
||||
) -> anyhow::Result<()> {
|
||||
self.connect(name, command, args, language_id)?;
|
||||
self.register_extensions(name, extensions);
|
||||
self.connect(command, args, language_id)?;
|
||||
self.register_extensions(language_id, extensions);
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
@@ -315,4 +257,3 @@ impl Default for LspManager {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
+205
-161
@@ -11,7 +11,6 @@
|
||||
//! Each tier is a fallback for the previous, so we try the most
|
||||
//! user-friendly path first (rustup component, npm global, etc.) and
|
||||
//! only fall back to package managers or manual download if those fail.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::{Command, Stdio};
|
||||
use std::sync::{Arc, Mutex};
|
||||
@@ -45,14 +44,11 @@ pub enum ProvisionResult {
|
||||
language: String,
|
||||
binary_path: String,
|
||||
},
|
||||
/// Every install tier failed — `manual_instructions` tells the user how
|
||||
/// to install by hand.
|
||||
/// Every install tier failed. Tells the user how to install by hand.
|
||||
Failed {
|
||||
language: String,
|
||||
server_name: String,
|
||||
reason: String,
|
||||
#[allow(dead_code)]
|
||||
manual_instructions: String,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -97,28 +93,55 @@ pub struct InstallTier {
|
||||
pub args: Vec<String>,
|
||||
}
|
||||
|
||||
/// Rust toolchain availability on the host PATH.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RustToolchain {
|
||||
pub has_rustup: bool,
|
||||
pub has_cargo: bool,
|
||||
}
|
||||
|
||||
/// Web / scripting language toolchain availability.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct WebToolchain {
|
||||
pub has_npm: bool,
|
||||
pub has_go: bool,
|
||||
pub has_java: bool,
|
||||
}
|
||||
|
||||
/// General-purpose platform utilities.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PlatformUtils {
|
||||
pub has_curl: bool,
|
||||
pub has_tar: bool,
|
||||
}
|
||||
|
||||
/// Pacman and Brew package managers (Arch / macOS).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PacmanBrew {
|
||||
pub has_pacman: bool,
|
||||
pub has_brew: bool,
|
||||
}
|
||||
|
||||
/// Apt and DNF package managers (Debian / Fedora).
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AptDnf {
|
||||
pub has_apt: bool,
|
||||
pub has_dnf: bool,
|
||||
}
|
||||
|
||||
/// Snapshot of the host environment used to decide which install tiers are viable.
|
||||
///
|
||||
/// Populated by `detect_env()` once per `provision_all()` call so we
|
||||
/// Populated by `detect_env()` once per `provision_all_with_progress()` call so we
|
||||
/// don't re-shell out for every server. `is_linux` / `is_macos` are
|
||||
/// computed at startup (compile time would also work, but keeping the
|
||||
/// shape uniform with the rest of the struct makes the call sites tidy).
|
||||
#[derive(Debug, Clone)]
|
||||
#[allow(dead_code)]
|
||||
#[allow(clippy::struct_excessive_bools)]
|
||||
pub struct EnvInfo {
|
||||
pub has_rustup: bool,
|
||||
pub has_npm: bool,
|
||||
pub has_go: bool,
|
||||
pub has_java: bool,
|
||||
pub has_cargo: bool,
|
||||
pub has_curl: bool,
|
||||
pub has_wget: bool,
|
||||
pub has_tar: bool,
|
||||
pub has_pacman: bool,
|
||||
pub has_apt: bool,
|
||||
pub has_brew: bool,
|
||||
pub has_dnf: bool,
|
||||
pub rust: RustToolchain,
|
||||
pub web: WebToolchain,
|
||||
pub platform: PlatformUtils,
|
||||
pub pacman_brew: PacmanBrew,
|
||||
pub apt_dnf: AptDnf,
|
||||
pub is_linux: bool,
|
||||
pub is_macos: bool,
|
||||
}
|
||||
@@ -159,18 +182,27 @@ pub fn which(binary: &str) -> Option<PathBuf> {
|
||||
/// this only ever runs on Unix-like targets.
|
||||
pub fn detect_env() -> EnvInfo {
|
||||
EnvInfo {
|
||||
has_rustup: which("rustup").is_some(),
|
||||
has_npm: which("npm").is_some(),
|
||||
has_go: which("go").is_some(),
|
||||
has_java: which("java").is_some(),
|
||||
has_cargo: which("cargo").is_some(),
|
||||
has_curl: which("curl").is_some(),
|
||||
has_wget: which("wget").is_some(),
|
||||
has_tar: which("tar").is_some(),
|
||||
has_pacman: which("pacman").is_some(),
|
||||
has_apt: which("apt").is_some() || which("apt-get").is_some(),
|
||||
has_brew: which("brew").is_some(),
|
||||
has_dnf: which("dnf").is_some(),
|
||||
rust: RustToolchain {
|
||||
has_rustup: which("rustup").is_some(),
|
||||
has_cargo: which("cargo").is_some(),
|
||||
},
|
||||
web: WebToolchain {
|
||||
has_npm: which("npm").is_some(),
|
||||
has_go: which("go").is_some(),
|
||||
has_java: which("java").is_some(),
|
||||
},
|
||||
platform: PlatformUtils {
|
||||
has_curl: which("curl").is_some(),
|
||||
has_tar: which("tar").is_some(),
|
||||
},
|
||||
pacman_brew: PacmanBrew {
|
||||
has_pacman: which("pacman").is_some(),
|
||||
has_brew: which("brew").is_some(),
|
||||
},
|
||||
apt_dnf: AptDnf {
|
||||
has_apt: which("apt").is_some() || which("apt-get").is_some(),
|
||||
has_dnf: which("dnf").is_some(),
|
||||
},
|
||||
is_linux: cfg!(target_os = "linux"),
|
||||
is_macos: cfg!(target_os = "macos"),
|
||||
}
|
||||
@@ -179,14 +211,13 @@ pub fn detect_env() -> EnvInfo {
|
||||
/// Return the static set of supported language servers.
|
||||
///
|
||||
/// The order is significant: it determines provisioning order and
|
||||
/// the order results appear in `provision_all()`. Tier 1 paths are
|
||||
/// the order results appear in `provision_all_with_progress()`. Tier 1 paths are
|
||||
/// the canonical/idiomatic install for each ecosystem; later tiers
|
||||
/// are fallbacks for hosts that lack the primary tooling.
|
||||
///
|
||||
/// Why hard-coded rather than loaded from settings: the set is small,
|
||||
/// changes rarely, and bundling it lets the provisioner run before any
|
||||
/// user config has been read (e.g. on first launch).
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub fn supported_servers() -> Vec<LanguageServerDef> {
|
||||
vec![
|
||||
LanguageServerDef {
|
||||
@@ -199,13 +230,22 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
|
||||
label: "rustup component".to_string(),
|
||||
requires: vec!["rustup".to_string()],
|
||||
command: "rustup".to_string(),
|
||||
args: vec!["component".to_string(), "add".to_string(), "rust-analyzer".to_string()],
|
||||
args: vec![
|
||||
"component".to_string(),
|
||||
"add".to_string(),
|
||||
"rust-analyzer".to_string(),
|
||||
],
|
||||
},
|
||||
InstallTier {
|
||||
label: "pacman".to_string(),
|
||||
requires: vec!["pacman".to_string()],
|
||||
command: "pacman".to_string(),
|
||||
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "rust-analyzer".to_string()],
|
||||
args: vec![
|
||||
"-S".to_string(),
|
||||
"--noconfirm".to_string(),
|
||||
"--needed".to_string(),
|
||||
"rust-analyzer".to_string(),
|
||||
],
|
||||
},
|
||||
InstallTier {
|
||||
label: "brew".to_string(),
|
||||
@@ -217,7 +257,11 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
|
||||
label: "cargo install".to_string(),
|
||||
requires: vec!["cargo".to_string()],
|
||||
command: "cargo".to_string(),
|
||||
args: vec!["install".to_string(), "--locked".to_string(), "rust-analyzer".to_string()],
|
||||
args: vec![
|
||||
"install".to_string(),
|
||||
"--locked".to_string(),
|
||||
"rust-analyzer".to_string(),
|
||||
],
|
||||
},
|
||||
InstallTier {
|
||||
label: "download prebuilt".to_string(),
|
||||
@@ -268,19 +312,33 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
|
||||
name: "jdtls".to_string(),
|
||||
language: "java".to_string(),
|
||||
extensions: vec![".java".to_string()],
|
||||
binary_names: vec!["jdtls".to_string(), "eclipse-jdt-ls".to_string(), "jdtls-launcher".to_string()],
|
||||
binary_names: vec![
|
||||
"jdtls".to_string(),
|
||||
"eclipse-jdt-ls".to_string(),
|
||||
"jdtls-launcher".to_string(),
|
||||
],
|
||||
install_tiers: vec![
|
||||
InstallTier {
|
||||
label: "pacman".to_string(),
|
||||
requires: vec!["java".to_string(), "pacman".to_string()],
|
||||
command: "pacman".to_string(),
|
||||
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "eclipse-jdt-ls".to_string()],
|
||||
args: vec![
|
||||
"-S".to_string(),
|
||||
"--noconfirm".to_string(),
|
||||
"--needed".to_string(),
|
||||
"eclipse-jdt-ls".to_string(),
|
||||
],
|
||||
},
|
||||
InstallTier {
|
||||
label: "apt".to_string(),
|
||||
requires: vec!["java".to_string(), "apt".to_string()],
|
||||
command: "sudo".to_string(),
|
||||
args: vec!["apt".to_string(), "install".to_string(), "-y".to_string(), "eclipse-jdt-ls".to_string()],
|
||||
args: vec![
|
||||
"apt".to_string(),
|
||||
"install".to_string(),
|
||||
"-y".to_string(),
|
||||
"eclipse-jdt-ls".to_string(),
|
||||
],
|
||||
},
|
||||
InstallTier {
|
||||
label: "brew".to_string(),
|
||||
@@ -339,7 +397,9 @@ pub fn run_command(cmd: &str, args: &[&str]) -> std::io::Result<(bool, String)>
|
||||
let timeout = Duration::from_mins(3);
|
||||
let start = Instant::now();
|
||||
let status = loop {
|
||||
if let Some(status) = child.try_wait()? { break Ok(status) }
|
||||
if let Some(status) = child.try_wait()? {
|
||||
break Ok(status);
|
||||
}
|
||||
if start.elapsed() > timeout {
|
||||
let _ = child.kill();
|
||||
let _ = child.wait();
|
||||
@@ -405,9 +465,12 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
|
||||
info!(url = url, dest = %path_str, "downloading");
|
||||
let args = [
|
||||
"-fsSL",
|
||||
"--connect-timeout", "15",
|
||||
"--max-time", &max_secs.to_string(),
|
||||
"-o", &path_str,
|
||||
"--connect-timeout",
|
||||
"15",
|
||||
"--max-time",
|
||||
&max_secs.to_string(),
|
||||
"-o",
|
||||
&path_str,
|
||||
url,
|
||||
];
|
||||
let (ok, out) = run_command("curl", &args).map_err(|e| format!("curl spawn: {e}"))?;
|
||||
@@ -419,7 +482,10 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
|
||||
|
||||
/// Download rust-analyzer from GitHub releases and install into
|
||||
/// `~/.local/share/zesdex/lsp/rust-analyzer/bin/rust-analyzer`.
|
||||
fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> {
|
||||
fn install_rust_analyzer_binary(
|
||||
env: &EnvInfo,
|
||||
progress: ProgressFn<'_>,
|
||||
) -> Result<PathBuf, String> {
|
||||
let base = lsp_install_dir("rust-analyzer")?;
|
||||
std::fs::create_dir_all(&base).map_err(|e| format!("mkdir: {e}"))?;
|
||||
|
||||
@@ -434,9 +500,13 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
|
||||
let gz = base.join("rust-analyzer.gz");
|
||||
let target = base.join("rust-analyzer");
|
||||
|
||||
if let Some(cb) = progress { cb("Rust: downloading prebuilt binary..."); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Rust: downloading prebuilt binary...");
|
||||
}
|
||||
download_url(url, &gz, 120)?;
|
||||
if let Some(cb) = progress { cb("Rust: decompressing..."); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Rust: decompressing...");
|
||||
}
|
||||
let (ok, out) = run_command("gunzip", &["-f", &gz.to_string_lossy()])
|
||||
.map_err(|e| format!("gunzip spawn: {e}"))?;
|
||||
if !ok {
|
||||
@@ -452,7 +522,9 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
|
||||
std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755))
|
||||
.map_err(|e| format!("chmod: {e}"))?;
|
||||
}
|
||||
if let Some(cb) = progress { cb("Rust: installed ✓"); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Rust: installed ✓");
|
||||
}
|
||||
Ok(target)
|
||||
}
|
||||
|
||||
@@ -464,14 +536,24 @@ fn install_jdtls_from_eclipse(progress: ProgressFn) -> Result<PathBuf, String> {
|
||||
|
||||
let url = "https://download.eclipse.org/jdtls/snapshots/jdt-language-server-latest.tar.gz";
|
||||
let tarball = base.join("jdtls.tar.gz");
|
||||
if let Some(cb) = progress { cb("Java: downloading JDT-LS (~150MB)..."); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Java: downloading JDT-LS (~150MB)...");
|
||||
}
|
||||
download_url(url, &tarball, 300)?;
|
||||
if let Some(cb) = progress { cb("Java: extracting..."); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Java: extracting...");
|
||||
}
|
||||
|
||||
let (ok, out) = run_command("tar", &[
|
||||
"-xzf", tarball.to_str().unwrap_or(""),
|
||||
"-C", base.to_str().unwrap_or("."),
|
||||
]).map_err(|e| format!("tar spawn: {e}"))?;
|
||||
let (ok, out) = run_command(
|
||||
"tar",
|
||||
&[
|
||||
"-xzf",
|
||||
tarball.to_str().unwrap_or(""),
|
||||
"-C",
|
||||
base.to_str().unwrap_or("."),
|
||||
],
|
||||
)
|
||||
.map_err(|e| format!("tar spawn: {e}"))?;
|
||||
if !ok {
|
||||
return Err(format!("tar: {}", out.trim()));
|
||||
}
|
||||
@@ -510,12 +592,18 @@ exec java \
|
||||
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755))
|
||||
.map_err(|e| format!("chmod launcher: {e}"))?;
|
||||
}
|
||||
if let Some(cb) = progress { cb("Java: JDT-LS installed ✓"); }
|
||||
if let Some(cb) = progress {
|
||||
cb("Java: JDT-LS installed ✓");
|
||||
}
|
||||
Ok(launcher)
|
||||
}
|
||||
|
||||
/// Dispatch a sentinel download tier to the correct helper.
|
||||
fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> {
|
||||
fn run_download_tier(
|
||||
name: &str,
|
||||
env: &EnvInfo,
|
||||
progress: ProgressFn<'_>,
|
||||
) -> Result<PathBuf, String> {
|
||||
match name {
|
||||
DOWNLOAD_RUST_BIN => install_rust_analyzer_binary(env, progress),
|
||||
DOWNLOAD_JDTLS => install_jdtls_from_eclipse(progress),
|
||||
@@ -523,59 +611,17 @@ fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Res
|
||||
}
|
||||
}
|
||||
|
||||
/// Render the "install by hand" message shown to the user when every
|
||||
/// automated tier fails.
|
||||
fn manual_instructions(def: &LanguageServerDef) -> String {
|
||||
match def.language.as_str() {
|
||||
"rust" => "Install rust-analyzer:\n \
|
||||
Arch: sudo pacman -S rust-analyzer\n \
|
||||
macOS: brew install rust-analyzer\n \
|
||||
Any: cargo install --locked rust-analyzer\n \
|
||||
Rustup: rustup component add rust-analyzer"
|
||||
.to_string(),
|
||||
"typescript" => "Install typescript-language-server:\n \
|
||||
npm install -g typescript typescript-language-server\n \
|
||||
Arch: sudo pacman -S typescript-language-server"
|
||||
.to_string(),
|
||||
"go" => "Install gopls:\n \
|
||||
go install golang.org/x/tools/gopls@latest\n \
|
||||
Arch: sudo pacman -S gopls"
|
||||
.to_string(),
|
||||
"java" => "Install Eclipse JDT-LS:\n \
|
||||
Arch: sudo pacman -S eclipse-jdt-ls\n \
|
||||
Debian: sudo apt install eclipse-jdt-ls\n \
|
||||
macOS: brew install jdtls\n \
|
||||
Other: see https://.eclipse.org/jdtls/#download"
|
||||
.to_string(),
|
||||
_ => format!("No automated install available for '{}'.", def.language),
|
||||
}
|
||||
}
|
||||
|
||||
/// Try to provision a single language server.
|
||||
///
|
||||
/// Flow: check whether any `binary_names` candidate is already on PATH
|
||||
/// → if yes, return `AlreadyAvailable` → otherwise walk
|
||||
/// `install_tiers` in order, skipping tiers whose `requires`
|
||||
/// binaries are missing → for each viable tier, run the install
|
||||
/// command (120s timeout) → if it succeeds AND the binary now
|
||||
/// appears on PATH (or the tier is jdtls-manual returning a
|
||||
/// launcher path), return Installed → if every tier fails, return
|
||||
/// Failed with the last error and manual install instructions.
|
||||
///
|
||||
/// Why we re-check `which` after the install: `rustup component add`
|
||||
/// can exit 0 even if the binary wasn't actually placed on PATH (rare,
|
||||
/// but happens with broken rustup installs). Re-checking gives us a
|
||||
/// real signal rather than trusting the exit code alone.
|
||||
#[allow(dead_code)]
|
||||
pub fn provision_single(def: &LanguageServerDef, env: &EnvInfo) -> ProvisionResult {
|
||||
provision_single_with_progress(def, env, None)
|
||||
}
|
||||
|
||||
fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progress: ProgressFn<'_>) -> ProvisionResult {
|
||||
fn provision_single_with_progress(
|
||||
def: &LanguageServerDef,
|
||||
env: &EnvInfo,
|
||||
progress: ProgressFn<'_>,
|
||||
) -> ProvisionResult {
|
||||
// 1. Check PATH.
|
||||
for bin in &def.binary_names {
|
||||
if let Some(path) = which(bin) {
|
||||
if let Some(cb) = progress { cb(&format!("{}: already installed (PATH)", def.language)); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&format!("{}: already installed (PATH)", def.language));
|
||||
}
|
||||
return ProvisionResult::AlreadyAvailable {
|
||||
server_name: def.name.clone(),
|
||||
language: def.language.clone(),
|
||||
@@ -586,7 +632,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
|
||||
|
||||
// 2. Check download-install directory (~/.local/share/zesdex/lsp/<name>/...).
|
||||
if let Some(path) = previous_download_install(def) {
|
||||
if let Some(cb) = progress { cb(&format!("{}: found previous install", def.language)); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&format!("{}: found previous install", def.language));
|
||||
}
|
||||
return ProvisionResult::AlreadyAvailable {
|
||||
server_name: def.name.clone(),
|
||||
language: def.language.clone(),
|
||||
@@ -594,30 +642,42 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
|
||||
};
|
||||
}
|
||||
|
||||
if let Some(cb) = progress { cb(&format!("{}: checking install options...", def.language)); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&format!("{}: checking install options...", def.language));
|
||||
}
|
||||
|
||||
let mut last_reason = String::from("no install tiers succeeded");
|
||||
|
||||
for tier in &def.install_tiers {
|
||||
// Prerequisite gating
|
||||
let prereqs_met = tier.requires.iter().all(|req| match req.as_str() {
|
||||
"rustup" => env.has_rustup, "npm" => env.has_npm,
|
||||
"go" => env.has_go, "java" => env.has_java,
|
||||
"cargo" => env.has_cargo, "curl" => env.has_curl,
|
||||
"tar" => env.has_tar, "pacman" => env.has_pacman,
|
||||
"apt" => env.has_apt, "brew" => env.has_brew,
|
||||
"dnf" => env.has_dnf, _ => which(req).is_some(),
|
||||
"rustup" => env.rust.has_rustup,
|
||||
"npm" => env.web.has_npm,
|
||||
"go" => env.web.has_go,
|
||||
"java" => env.web.has_java,
|
||||
"cargo" => env.rust.has_cargo,
|
||||
"curl" => env.platform.has_curl,
|
||||
"tar" => env.platform.has_tar,
|
||||
"pacman" => env.pacman_brew.has_pacman,
|
||||
"apt" => env.apt_dnf.has_apt,
|
||||
"brew" => env.pacman_brew.has_brew,
|
||||
"dnf" => env.apt_dnf.has_dnf,
|
||||
_ => which(req).is_some(),
|
||||
});
|
||||
if !prereqs_met {
|
||||
let skip = format!("{}: {} — missing prerequisite", def.language, tier.label);
|
||||
if let Some(cb) = progress { cb(&skip); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&skip);
|
||||
}
|
||||
last_reason = format!("tier '{}' skipped: missing prerequisite", tier.label);
|
||||
warn!(server = %def.name, tier = %tier.label, "skipped — missing prerequisites");
|
||||
continue;
|
||||
}
|
||||
|
||||
let trying = format!("{}: {}...", def.language, tier.label);
|
||||
if let Some(cb) = progress { cb(&trying); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&trying);
|
||||
}
|
||||
|
||||
// Download sentinel → helper.
|
||||
if tier.command.starts_with("__download_") && tier.command.ends_with("__") {
|
||||
@@ -647,7 +707,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
|
||||
.iter()
|
||||
.find_map(|b| which(b).map(|p| p.to_string_lossy().to_string()));
|
||||
if let Some(path) = located {
|
||||
if let Some(cb) = progress { cb(&format!("{}: installed ✓", def.language)); }
|
||||
if let Some(cb) = progress {
|
||||
cb(&format!("{}: installed ✓", def.language));
|
||||
}
|
||||
info!(server = %def.name, tier = %tier.label, binary = %path, "installed");
|
||||
return ProvisionResult::Installed {
|
||||
server_name: def.name.clone(),
|
||||
@@ -671,58 +733,36 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
|
||||
}
|
||||
}
|
||||
|
||||
let manual = manual_instructions(def);
|
||||
ProvisionResult::Failed {
|
||||
language: def.language.clone(), server_name: def.name.clone(),
|
||||
reason: last_reason, manual_instructions: manual,
|
||||
language: def.language.clone(),
|
||||
server_name: def.name.clone(),
|
||||
reason: last_reason,
|
||||
}
|
||||
}
|
||||
|
||||
/// Provision every supported server in order, returning one
|
||||
/// `ProvisionResult` per server.
|
||||
///
|
||||
/// Flow: `detect_env()` once → for each server in `supported_servers()`
|
||||
/// call `provision_single()` → collect results. Order matches
|
||||
/// `supported_servers()` (rust, typescript, go, java).
|
||||
#[allow(dead_code)]
|
||||
pub fn provision_all() -> Vec<ProvisionResult> {
|
||||
let env = detect_env();
|
||||
info!(
|
||||
linux = env.is_linux,
|
||||
macos = env.is_macos,
|
||||
rustup = env.has_rustup,
|
||||
cargo = env.has_cargo,
|
||||
npm = env.has_npm,
|
||||
go = env.has_go,
|
||||
java = env.has_java,
|
||||
curl = env.has_curl,
|
||||
tar = env.has_tar,
|
||||
pacman = env.has_pacman,
|
||||
apt = env.has_apt,
|
||||
brew = env.has_brew,
|
||||
dnf = env.has_dnf,
|
||||
"starting LSP provisioning"
|
||||
);
|
||||
supported_servers()
|
||||
.iter()
|
||||
.map(|def| provision_single(def, &env))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Like `provision_all` but calls `progress` with a human-readable status
|
||||
/// Provision every supported server with progress callbacks with a human-readable status
|
||||
/// string at each stage of each server's install attempt.
|
||||
pub fn provision_all_with_progress(progress: ProgressFn) -> Vec<ProvisionResult> {
|
||||
let env = detect_env();
|
||||
if let Some(cb) = progress {
|
||||
let flags = [
|
||||
("rustup", env.has_rustup), ("cargo", env.has_cargo),
|
||||
("npm", env.has_npm), ("go", env.has_go), ("java", env.has_java),
|
||||
("curl", env.has_curl), ("tar", env.has_tar),
|
||||
("pacman", env.has_pacman), ("apt", env.has_apt), ("brew", env.has_brew),
|
||||
("rustup", env.rust.has_rustup),
|
||||
("cargo", env.rust.has_cargo),
|
||||
("npm", env.web.has_npm),
|
||||
("go", env.web.has_go),
|
||||
("java", env.web.has_java),
|
||||
("curl", env.platform.has_curl),
|
||||
("tar", env.platform.has_tar),
|
||||
("pacman", env.pacman_brew.has_pacman),
|
||||
("apt", env.apt_dnf.has_apt),
|
||||
("brew", env.pacman_brew.has_brew),
|
||||
];
|
||||
let avail: String = flags.iter()
|
||||
.filter(|(_, v)| *v).map(|(k, _)| *k)
|
||||
.collect::<Vec<_>>().join(", ");
|
||||
let avail: String = flags
|
||||
.iter()
|
||||
.filter(|(_, v)| *v)
|
||||
.map(|(k, _)| *k)
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
cb(&format!("LSP: environment ready — {avail}"));
|
||||
}
|
||||
supported_servers()
|
||||
@@ -779,9 +819,13 @@ pub fn auto_connect(manager: &Arc<Mutex<LspManager>>, results: &[ProvisionResult
|
||||
};
|
||||
|
||||
// Build extension slice for connect_with_extensions.
|
||||
let ext_refs: Vec<&str> = def.extensions.iter().map(std::string::String::as_str).collect();
|
||||
let ext_refs: Vec<&str> = def
|
||||
.extensions
|
||||
.iter()
|
||||
.map(std::string::String::as_str)
|
||||
.collect();
|
||||
|
||||
match guard.connect_with_extensions(&name, &binary, &[], &language, &ext_refs) {
|
||||
match guard.connect_with_extensions(&binary, &[], &language, &ext_refs) {
|
||||
Ok(()) => {
|
||||
info!(
|
||||
name = %name,
|
||||
|
||||
+129
-91
@@ -1,13 +1,11 @@
|
||||
//! MCP server connection management: spawning/talking to stdio child
|
||||
//! processes and HTTP endpoints, and adapting their advertised tools to
|
||||
//! the crate's `Tool` trait.
|
||||
|
||||
use serde_json::{json, Value};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{json, Value};
|
||||
use std::io::{BufRead, BufReader, Write};
|
||||
use std::sync::{Arc, Mutex, OnceLock};
|
||||
|
||||
|
||||
const MCP_CONNECT_TIMEOUT_MS: u64 = 20_000;
|
||||
const MCP_CALL_TIMEOUT_MS: u64 = 60_000;
|
||||
|
||||
@@ -35,13 +33,8 @@ fn mcp_static_str(s: &str) -> &'static str {
|
||||
/// newline-delimited JSON-RPC over stdio, or a remote HTTP endpoint.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub enum McpTransport {
|
||||
Stdio {
|
||||
command: String,
|
||||
args: Vec<String>,
|
||||
},
|
||||
StreamableHttp {
|
||||
url: String,
|
||||
},
|
||||
Stdio { command: String, args: Vec<String> },
|
||||
StreamableHttp { url: String },
|
||||
}
|
||||
|
||||
/// A single tool advertised by an MCP server, as returned by `tools/list`.
|
||||
@@ -104,8 +97,8 @@ impl StdioChild {
|
||||
self.stdin.flush()?;
|
||||
|
||||
let mut response_line = String::new();
|
||||
let deadline = std::time::Instant::now()
|
||||
+ std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS);
|
||||
let deadline =
|
||||
std::time::Instant::now() + std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS);
|
||||
loop {
|
||||
if std::time::Instant::now() > deadline {
|
||||
anyhow::bail!("MCP call timed out after {MCP_CALL_TIMEOUT_MS}ms");
|
||||
@@ -136,7 +129,9 @@ impl StdioChild {
|
||||
line_truncated = true;
|
||||
// Consume rest of line to keep stream in sync
|
||||
loop {
|
||||
let buf = self.stdout.fill_buf()
|
||||
let buf = self
|
||||
.stdout
|
||||
.fill_buf()
|
||||
.map_err(|e| anyhow::anyhow!("MCP stdio read error: {e}"))?;
|
||||
if buf.is_empty() {
|
||||
anyhow::bail!("MCP stdio child closed mid-line");
|
||||
@@ -152,9 +147,7 @@ impl StdioChild {
|
||||
response_line.push(byte as char);
|
||||
}
|
||||
if line_truncated {
|
||||
anyhow::bail!(
|
||||
"MCP response line exceeded {MAX_LINE_LENGTH} byte limit",
|
||||
);
|
||||
anyhow::bail!("MCP response line exceeded {MAX_LINE_LENGTH} byte limit");
|
||||
}
|
||||
let trimmed = response_line.trim();
|
||||
if trimmed.is_empty() {
|
||||
@@ -172,12 +165,16 @@ impl StdioChild {
|
||||
}));
|
||||
}
|
||||
}
|
||||
} // close fn call
|
||||
} // close impl StdioChild
|
||||
} // close fn call
|
||||
} // close impl StdioChild
|
||||
|
||||
pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow::Result<StdioChild> {
|
||||
pub(crate) fn spawn_stdio_child(
|
||||
command: &str,
|
||||
extra_args: &[String],
|
||||
) -> anyhow::Result<StdioChild> {
|
||||
let parts: Vec<&str> = command.split_whitespace().collect();
|
||||
let (prog, prog_args) = parts.split_first()
|
||||
let (prog, prog_args) = parts
|
||||
.split_first()
|
||||
.ok_or_else(|| anyhow::anyhow!("MCP stdio command is empty"))?;
|
||||
|
||||
let mut cmd = std::process::Command::new(prog);
|
||||
@@ -189,12 +186,17 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
|
||||
// rather than discarded silently, making connectivity issues debugable.
|
||||
cmd.stderr(std::process::Stdio::piped());
|
||||
|
||||
let mut child = cmd.spawn()
|
||||
let mut child = cmd
|
||||
.spawn()
|
||||
.map_err(|e| anyhow::anyhow!("failed to spawn MCP stdio server '{command}': {e}"))?;
|
||||
|
||||
let stdin = child.stdin.take()
|
||||
let stdin = child
|
||||
.stdin
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("failed to get stdin for MCP server"))?;
|
||||
let stdout = child.stdout.take()
|
||||
let stdout = child
|
||||
.stdout
|
||||
.take()
|
||||
.ok_or_else(|| anyhow::anyhow!("failed to get stdout for MCP server"))?;
|
||||
|
||||
let mut mcp = StdioChild {
|
||||
@@ -203,17 +205,20 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
|
||||
next_id: 0,
|
||||
};
|
||||
|
||||
let deadline = std::time::Instant::now()
|
||||
+ std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS);
|
||||
let deadline =
|
||||
std::time::Instant::now() + std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS);
|
||||
|
||||
let init_result = mcp.call("initialize", &json!({
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": {},
|
||||
"clientInfo": {
|
||||
"name": "zesdex",
|
||||
"version": "0.1.0"
|
||||
}
|
||||
}));
|
||||
let init_result = mcp.call(
|
||||
"initialize",
|
||||
&json!({
|
||||
"protocolVersion": "2024-11-05",
|
||||
"capabilities": {},
|
||||
"clientInfo": {
|
||||
"name": "zesdex",
|
||||
"version": "0.1.0"
|
||||
}
|
||||
}),
|
||||
);
|
||||
|
||||
if std::time::Instant::now() > deadline {
|
||||
anyhow::bail!("MCP initialize timed out");
|
||||
@@ -236,21 +241,29 @@ fn call_via_stdio(
|
||||
// Reuse the persistent child handle if available; otherwise spawn a new one.
|
||||
let mut guard;
|
||||
let child: &mut StdioChild = if let Some(mtx) = existing_handle {
|
||||
guard = mtx.lock().map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?;
|
||||
guard = mtx
|
||||
.lock()
|
||||
.map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?;
|
||||
&mut guard
|
||||
} else {
|
||||
let mut fresh = spawn_stdio_child(command, extra_args)?;
|
||||
let result = fresh.call("tools/call", &json!({
|
||||
"name": tool_name,
|
||||
"arguments": tool_args
|
||||
}))?;
|
||||
let result = fresh.call(
|
||||
"tools/call",
|
||||
&json!({
|
||||
"name": tool_name,
|
||||
"arguments": tool_args
|
||||
}),
|
||||
)?;
|
||||
return Ok(extract_text_content(&result));
|
||||
};
|
||||
|
||||
let result = child.call("tools/call", &json!({
|
||||
"name": tool_name,
|
||||
"arguments": tool_args
|
||||
}))?;
|
||||
let result = child.call(
|
||||
"tools/call",
|
||||
&json!({
|
||||
"name": tool_name,
|
||||
"arguments": tool_args
|
||||
}),
|
||||
)?;
|
||||
|
||||
Ok(extract_text_content(&result))
|
||||
}
|
||||
@@ -289,7 +302,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
|
||||
}
|
||||
});
|
||||
|
||||
let resp = client.post(url)
|
||||
let resp = client
|
||||
.post(url)
|
||||
.header("Content-Type", "application/json")
|
||||
.json(&body)
|
||||
.send()
|
||||
@@ -304,7 +318,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
|
||||
anyhow::bail!("MCP HTTP server returned {status}: {text}");
|
||||
}
|
||||
|
||||
let response: Value = resp.json()
|
||||
let response: Value = resp
|
||||
.json()
|
||||
.map_err(|e| anyhow::anyhow!("invalid JSON from MCP HTTP server: {e}"))?;
|
||||
|
||||
if let Some(err) = response.get("error") {
|
||||
@@ -321,13 +336,18 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
|
||||
fn extract_text_content(result: &Value) -> String {
|
||||
if let Some(content) = result.get("content") {
|
||||
if let Some(arr) = content.as_array() {
|
||||
let text: Vec<String> = arr.iter().filter_map(|item| {
|
||||
if item.get("type").and_then(|t| t.as_str()) == Some("text") {
|
||||
item.get("text").and_then(|t| t.as_str()).map(std::string::ToString::to_string)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}).collect();
|
||||
let text: Vec<String> = arr
|
||||
.iter()
|
||||
.filter_map(|item| {
|
||||
if item.get("type").and_then(|t| t.as_str()) == Some("text") {
|
||||
item.get("text")
|
||||
.and_then(|t| t.as_str())
|
||||
.map(std::string::ToString::to_string)
|
||||
} else {
|
||||
None
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
if !text.is_empty() {
|
||||
return text.join("\n");
|
||||
}
|
||||
@@ -372,12 +392,17 @@ impl crate::tool::Tool for McpToolAdapter {
|
||||
|
||||
fn run(&self, _ctx: &crate::tool::ToolCtx, args: &Value) -> anyhow::Result<String> {
|
||||
match &self.transport {
|
||||
McpTransport::Stdio { command, args: extra_args } => {
|
||||
call_via_stdio(self.child_handle.as_ref().map(std::convert::AsRef::as_ref), command, extra_args, &self.tool_name, args)
|
||||
}
|
||||
McpTransport::StreamableHttp { url } => {
|
||||
call_via_http(url, &self.tool_name, args)
|
||||
}
|
||||
McpTransport::Stdio {
|
||||
command,
|
||||
args: extra_args,
|
||||
} => call_via_stdio(
|
||||
self.child_handle.as_ref().map(std::convert::AsRef::as_ref),
|
||||
command,
|
||||
extra_args,
|
||||
&self.tool_name,
|
||||
args,
|
||||
),
|
||||
McpTransport::StreamableHttp { url } => call_via_http(url, &self.tool_name, args),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -400,27 +425,35 @@ impl McpManager {
|
||||
///
|
||||
/// Return: boxed `Tool` trait objects ready to merge into the harness's tool list.
|
||||
pub fn as_tools(&self) -> Vec<Box<dyn crate::tool::Tool>> {
|
||||
self.servers.iter().flat_map(|server| {
|
||||
let handle = server.child_handle.clone();
|
||||
server.tools.iter().map(move |info| {
|
||||
let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter {
|
||||
tool_name: info.name.clone(),
|
||||
server_name: server.name.clone(),
|
||||
transport: server.transport.clone(),
|
||||
description: info.description.clone(),
|
||||
parameters: info.input_schema.clone(),
|
||||
child_handle: handle.clone(),
|
||||
});
|
||||
adapter
|
||||
self.servers
|
||||
.iter()
|
||||
.flat_map(|server| {
|
||||
let handle = server.child_handle.clone();
|
||||
server.tools.iter().map(move |info| {
|
||||
let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter {
|
||||
tool_name: info.name.clone(),
|
||||
server_name: server.name.clone(),
|
||||
transport: server.transport.clone(),
|
||||
description: info.description.clone(),
|
||||
parameters: info.input_schema.clone(),
|
||||
child_handle: handle.clone(),
|
||||
});
|
||||
adapter
|
||||
})
|
||||
})
|
||||
}).collect()
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Connects to an MCP server via stdio by spawning the child process, running
|
||||
/// the `initialize` handshake, calling `tools/list`, and registering the server
|
||||
/// with its advertised tools in `self.servers`. The child process stays alive
|
||||
/// for subsequent `tools/call` invocations via the stored `McpServer.tools`.
|
||||
pub fn connect_stdio(&mut self, name: &str, command: &str, extra_args: &[String]) -> anyhow::Result<()> {
|
||||
pub fn connect_stdio(
|
||||
&mut self,
|
||||
name: &str,
|
||||
command: &str,
|
||||
extra_args: &[String],
|
||||
) -> anyhow::Result<()> {
|
||||
let transport = McpTransport::Stdio {
|
||||
command: command.to_string(),
|
||||
args: extra_args.to_vec(),
|
||||
@@ -430,19 +463,32 @@ impl McpManager {
|
||||
let result = child.call("tools/list", &json!({}))?;
|
||||
|
||||
let tools = if let Some(tool_list) = result.get("tools").and_then(|v| v.as_array()) {
|
||||
tool_list.iter().filter_map(|t| {
|
||||
Some(McpToolInfo {
|
||||
name: t.get("name")?.as_str()?.to_string(),
|
||||
description: t.get("description").and_then(|v| v.as_str()).unwrap_or_else(|| {
|
||||
tracing::warn!("[mcp] tool {} missing description", t.get("name").and_then(|n| n.as_str()).unwrap_or("?"));
|
||||
""
|
||||
}).to_string(),
|
||||
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| {
|
||||
tracing::warn!("[mcp] tool {} missing inputSchema", t.get("name").and_then(|n| n.as_str()).unwrap_or("?"));
|
||||
serde_json::Value::Null
|
||||
}),
|
||||
tool_list
|
||||
.iter()
|
||||
.filter_map(|t| {
|
||||
Some(McpToolInfo {
|
||||
name: t.get("name")?.as_str()?.to_string(),
|
||||
description: t
|
||||
.get("description")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or_else(|| {
|
||||
tracing::warn!(
|
||||
"[mcp] tool {} missing description",
|
||||
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
|
||||
);
|
||||
""
|
||||
})
|
||||
.to_string(),
|
||||
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| {
|
||||
tracing::warn!(
|
||||
"[mcp] tool {} missing inputSchema",
|
||||
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
|
||||
);
|
||||
serde_json::Value::Null
|
||||
}),
|
||||
})
|
||||
})
|
||||
}).collect()
|
||||
.collect()
|
||||
} else {
|
||||
Vec::new()
|
||||
};
|
||||
@@ -458,12 +504,4 @@ impl McpManager {
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Removes a server by name. Returns `true` if a server was found and removed.
|
||||
#[allow(dead_code)]
|
||||
pub fn disconnect(&mut self, name: &str) -> bool {
|
||||
let len = self.servers.len();
|
||||
self.servers.retain(|s| s.name != name);
|
||||
self.servers.len() < len
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
//! Model Context Protocol (MCP) client: connects to external MCP servers
|
||||
//! (stdio or HTTP) and exposes their tools through the crate's `Tool` trait.
|
||||
|
||||
pub mod manager;
|
||||
|
||||
+5
-5
@@ -1,13 +1,13 @@
|
||||
//! Top-level application module: harness, modes, runtime loop, state,
|
||||
//! workflows, subagents, review, background bash, MCP integration, and
|
||||
//! native LSP client.
|
||||
pub mod bgbash;
|
||||
pub mod harness;
|
||||
pub mod lsp;
|
||||
pub mod mcp;
|
||||
pub mod mode;
|
||||
pub mod review;
|
||||
pub mod runtime;
|
||||
pub mod state;
|
||||
pub mod workflow;
|
||||
pub mod subagent;
|
||||
pub mod review;
|
||||
pub mod bgbash;
|
||||
pub mod mcp;
|
||||
pub mod lsp;
|
||||
pub mod workflow;
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Bash mode: handles submitting a shell command from the bash input panel.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
|
||||
/// Launch a background bash job for the submitted command.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Editor mode: a minimal in-TUI line editor for viewing/modifying a file,
|
||||
//! with bounded undo history.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
use crate::app::state::types::Overlay;
|
||||
|
||||
@@ -66,7 +65,9 @@ impl EditorState {
|
||||
self.cursor_line += 1;
|
||||
}
|
||||
self.cursor_col = self.cursor_col.min(
|
||||
self.content.get(self.cursor_line).map_or(0, std::string::String::len),
|
||||
self.content
|
||||
.get(self.cursor_line)
|
||||
.map_or(0, std::string::String::len),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -114,10 +115,9 @@ impl EditorState {
|
||||
/// the char directly → mark state dirty.
|
||||
pub fn handle_editor_input(state: &mut AppStateRest, text: &str) {
|
||||
let editor = &mut state.misc.editor;
|
||||
if editor.is_none() {
|
||||
let Some(ed) = editor.as_mut() else {
|
||||
return;
|
||||
}
|
||||
let ed = editor.as_mut().unwrap();
|
||||
};
|
||||
for c in text.chars() {
|
||||
match c {
|
||||
'\n' | '\r' => {
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
//! Effort mode: cycles the agent's reasoning effort level, which scales the
|
||||
//! LLM's temperature and `max_tokens` for subsequent turns.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
|
||||
pub const EFFORT_LEVELS: &[&str] = &["low", "medium", "high", "xhigh", "max"];
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Help mode: static help text and the action that opens/closes the help overlay.
|
||||
|
||||
use crate::app::runtime::actions::Action;
|
||||
use crate::app::state::types::Overlay;
|
||||
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Key input mode: raw text capture overlay used for one-off key/text prompts.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
|
||||
/// Replace the input buffer with the given text and mark state dirty.
|
||||
|
||||
@@ -33,14 +33,16 @@ pub fn get_learning_items(state: &AppStateRest) -> Vec<LearningItem> {
|
||||
let scope_str = match p.lesson.scope {
|
||||
crate::app::review::LessonScope::Project => "project",
|
||||
crate::app::review::LessonScope::Global => "global",
|
||||
}.to_string();
|
||||
}
|
||||
.to_string();
|
||||
|
||||
let conf_str = match p.lesson.confidence {
|
||||
crate::app::review::Confidence::Human => "human",
|
||||
crate::app::review::Confidence::Verified => "verified",
|
||||
crate::app::review::Confidence::Unverified => "unverified",
|
||||
crate::app::review::Confidence::Auto => "auto",
|
||||
}.to_string();
|
||||
}
|
||||
.to_string();
|
||||
|
||||
items.push(LearningItem::Pending {
|
||||
name: p.lesson.name,
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Loading mode: transient overlay shown while waiting on an async operation.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
|
||||
pub const LOADING_MESSAGES: &[&str] = &[
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! MCP mode: overlay for connecting to a configured MCP server.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
|
||||
/// Placeholder entry point for connecting to an MCP server by name.
|
||||
|
||||
+1
-2
@@ -1,14 +1,13 @@
|
||||
//! TUI mode definitions and per-mode input/action handlers, one submodule
|
||||
//! per overlay/mode (bash, editor, effort, mcp, quit confirm, rewind, etc.).
|
||||
|
||||
pub mod bash;
|
||||
pub mod editor;
|
||||
pub mod effort;
|
||||
pub mod key_input;
|
||||
pub mod mcp;
|
||||
|
||||
pub mod learning;
|
||||
pub mod quit_confirm;
|
||||
pub mod rewind;
|
||||
pub mod settings;
|
||||
pub mod todo;
|
||||
pub mod learning;
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Quit-confirm mode: the "are you sure?" overlay shown before exiting.
|
||||
|
||||
use crate::app::runtime::actions::Action;
|
||||
|
||||
/// Translate the user's yes/no answer on the quit-confirm overlay into an action.
|
||||
|
||||
+18
-7
@@ -1,13 +1,19 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
|
||||
//! session's `SQLite` blob store.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
use sha2::Digest;
|
||||
|
||||
/// Returns the number of stored pre-edit blobs (snapshots) for this session.
|
||||
pub fn rewind_count(state: &AppStateRest) -> usize {
|
||||
let Ok(conn) = open_session_db(&state.session_dir) else { return 0 };
|
||||
let Ok(conn) = open_session_db(&state.session_dir) else {
|
||||
return 0;
|
||||
};
|
||||
crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id)
|
||||
.ok()
|
||||
.map_or(0, |keys| keys.len())
|
||||
@@ -51,7 +57,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
|
||||
}
|
||||
|
||||
let blob_key = &keys[index];
|
||||
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key) {
|
||||
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key)
|
||||
{
|
||||
Ok(Some(b)) => b,
|
||||
Ok(None) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
@@ -74,8 +81,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
|
||||
// Look up the path from the edit log — the blob key is the tool_call_id.
|
||||
// The edit log doesn't store the tool_call_id directly, so fall back to the
|
||||
// path from the most recent write/edit entry.
|
||||
let restore_path = find_edit_path(state, blob_key)
|
||||
.unwrap_or_else(|| state.session_dir.join("snapshot.dat"));
|
||||
let restore_path =
|
||||
find_edit_path(state, blob_key).unwrap_or_else(|| state.session_dir.join("snapshot.dat"));
|
||||
|
||||
match std::fs::write(&restore_path, &bytes) {
|
||||
Ok(()) => {
|
||||
@@ -119,6 +126,10 @@ fn open_session_db(session_dir: &std::path::Path) -> anyhow::Result<rusqlite::Co
|
||||
|
||||
fn find_edit_path(state: &AppStateRest, _blob_key: &str) -> Option<std::path::PathBuf> {
|
||||
let el = crate::model::editlog::EditLog::new(&state.session_dir);
|
||||
let entry = el.entries.iter().rev().find(|e| e.tool == "write" || e.tool == "edit")?;
|
||||
let entry = el
|
||||
.entries
|
||||
.iter()
|
||||
.rev()
|
||||
.find(|e| e.tool == "write" || e.tool == "edit")?;
|
||||
Some(std::path::PathBuf::from(&entry.path))
|
||||
}
|
||||
|
||||
@@ -3,8 +3,7 @@
|
||||
//! Flow: exposes small mutation functions (currently just cycling the
|
||||
//! internet access mode) invoked by keybindings while the settings overlay
|
||||
//! is active.
|
||||
|
||||
use crate::model::settings::{Settings, InternetMode};
|
||||
use crate::model::settings::{InternetMode, Settings};
|
||||
|
||||
/// Advance the internet access mode to the next value in the cycle.
|
||||
///
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
//!
|
||||
//! Flow: exposes the toggle handler invoked by a keybinding to show/hide
|
||||
//! the todo overlay.
|
||||
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
use crate::app::state::types::Overlay;
|
||||
|
||||
|
||||
@@ -76,7 +76,7 @@ pub fn should_trigger_review(state: &AppStateRest, origin: Origin) -> bool {
|
||||
return false;
|
||||
}
|
||||
let Some(runtime) = &state.session_runtime else { return false };
|
||||
if !state.settings.review_enabled {
|
||||
if !state.settings.flags.review_enabled {
|
||||
return false;
|
||||
}
|
||||
if runtime.edit_count > 0 && runtime.edit_count % 5 == 0 {
|
||||
|
||||
@@ -556,7 +556,7 @@ pub fn apply_action(state: &mut AppStateRest, action: Action) {
|
||||
.map(str::len)
|
||||
.sum();
|
||||
let token_estimate = total_chars / 3;
|
||||
rt.messages = crate::app::runtime::shortsend::shape_messages(&rt.messages, token_estimate, max_wire_tokens, true, None);
|
||||
rt.messages = crate::app::runtime::context::shaping::shape_messages(&rt.messages, token_estimate, max_wire_tokens, true, None);
|
||||
state.push_toast(Toast::new(ToastKind::Success, "Conversation history compacted.".to_string()));
|
||||
state.dirty = true;
|
||||
}
|
||||
@@ -1154,10 +1154,10 @@ fn run_agent_turn(
|
||||
// Skip message compaction if abort was requested — the non-streaming
|
||||
// LLM call for summarization would block without checking abort_flag.
|
||||
let wire_msgs = if !tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
|
||||
&& crate::app::runtime::shortsend::should_shape(token_estimate, max_wire_tokens, prev_shaped)
|
||||
&& crate::app::runtime::context::shaping::should_shape(token_estimate, max_wire_tokens, prev_shaped)
|
||||
{
|
||||
prev_shaped = true;
|
||||
let compacted = crate::app::runtime::shortsend::shape_messages(&msgs, token_estimate, max_wire_tokens, false, Some(&tc.client));
|
||||
let compacted = crate::app::runtime::context::shaping::shape_messages(&msgs, token_estimate, max_wire_tokens, false, Some(&tc.client));
|
||||
|
||||
// Dispatch the compacted messages to the main thread so the local session history
|
||||
// is permanently compacted and doesn't trigger shaping again immediately on next turn.
|
||||
@@ -1612,7 +1612,7 @@ fn execute_one_tool(
|
||||
/// `should_trigger_review` on `Tick`), only informs the user that
|
||||
/// a review has material to examine.
|
||||
fn maybe_trigger_review(state: &mut AppStateRest) {
|
||||
if !state.settings.review_enabled {
|
||||
if !state.settings.flags.review_enabled {
|
||||
return;
|
||||
}
|
||||
let edit_count = state
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
//! Maps parsed `/` slash commands into one or more `Action` variants
|
||||
//! that `apply_action` can process.
|
||||
use crate::controller::command::Command;
|
||||
use crate::app::runtime::actions::Action;
|
||||
use crate::app::state::types::Overlay;
|
||||
use crate::controller::command::Command;
|
||||
|
||||
/// Convert a parsed `Command` into the corresponding sequence of `Action`s.
|
||||
///
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
#![allow(dead_code)]
|
||||
//! Cross-call tool-result deduplication: when a read-only tool is called
|
||||
//! again with identical arguments, the earlier result is replaced with a
|
||||
//! placeholder so only the latest copy occupies context.
|
||||
//!
|
||||
//! Flow: pair each `Role::Tool` message to its originating `ToolCall` via
|
||||
//! `tool_call_id` -> key on `(function.name, sha256(canonical_json(args)))`
|
||||
//! -> for read-only tools, keep only the last occurrence of each key in
|
||||
//! full, placeholder the rest.
|
||||
//!
|
||||
//! Why: reading the same file (or re-running the same grep) twice in a
|
||||
//! session otherwise keeps both full copies in context until compaction
|
||||
//! eventually drops the older one wholesale, along with everything else
|
||||
//! from that period. Mutating tools (`write`, `edit`, `bash`, `delete`,
|
||||
//! `git_operator`, ...) are never touched, even with identical
|
||||
//! arguments, because call order and repetition can be semantically
|
||||
//! meaningful (e.g. retrying a flaky `bash` command until it passes).
|
||||
use crate::app::subagent::division::tool_scope::READ_TOOLS;
|
||||
use crate::dto::chat::message::{ChatMessage, Role};
|
||||
use sha2::Digest;
|
||||
use std::collections::HashMap;
|
||||
|
||||
const DUPLICATE_PLACEHOLDER: &str =
|
||||
"[duplicate result — superseded by a later identical call, see below]";
|
||||
|
||||
/// Replace superseded read-only tool results with a placeholder.
|
||||
///
|
||||
/// Return: a `Vec<ChatMessage>` the same length as `messages`, and
|
||||
/// `true` iff at least one entry was replaced. The caller uses the
|
||||
/// `bool` to decide whether the result is worth persisting/announcing,
|
||||
/// without `ChatMessage` needing to implement `PartialEq`.
|
||||
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool) {
|
||||
// tool_call_id -> (tool name, canonical JSON of its arguments)
|
||||
let mut call_info: HashMap<String, (String, String)> = HashMap::new();
|
||||
for m in messages {
|
||||
if let Some(calls) = &m.tool_calls {
|
||||
for call in calls {
|
||||
let canonical = serde_json::to_string(&call.function.arguments).unwrap_or_default();
|
||||
call_info.insert(call.id.clone(), (call.function.name.clone(), canonical));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// For each (tool, args-hash) key among read-only tools, find the
|
||||
// index of its LAST occurrence — that's the one kept in full.
|
||||
let mut last_index_for_key: HashMap<String, usize> = HashMap::new();
|
||||
for (idx, m) in messages.iter().enumerate() {
|
||||
if m.role != Role::Tool {
|
||||
continue;
|
||||
}
|
||||
let Some(id) = &m.tool_call_id else { continue };
|
||||
let Some((name, args)) = call_info.get(id) else {
|
||||
continue;
|
||||
};
|
||||
if !READ_TOOLS.contains(&name.as_str()) {
|
||||
continue;
|
||||
}
|
||||
last_index_for_key.insert(dedup_key(name, args), idx);
|
||||
}
|
||||
|
||||
let mut changed = false;
|
||||
let result = messages
|
||||
.iter()
|
||||
.enumerate()
|
||||
.map(|(idx, m)| {
|
||||
if m.role != Role::Tool {
|
||||
return m.clone();
|
||||
}
|
||||
let Some(id) = &m.tool_call_id else {
|
||||
return m.clone();
|
||||
};
|
||||
let Some((name, args)) = call_info.get(id) else {
|
||||
return m.clone();
|
||||
};
|
||||
if !READ_TOOLS.contains(&name.as_str()) {
|
||||
return m.clone();
|
||||
}
|
||||
let key = dedup_key(name, args);
|
||||
if last_index_for_key.get(&key) == Some(&idx) {
|
||||
return m.clone();
|
||||
}
|
||||
changed = true;
|
||||
ChatMessage::tool_result(id.clone(), DUPLICATE_PLACEHOLDER.to_string())
|
||||
})
|
||||
.collect();
|
||||
|
||||
(result, changed)
|
||||
}
|
||||
|
||||
/// Build the dedup key for a tool call.
|
||||
///
|
||||
/// Why hash the arguments: keeps the key a fixed, short size regardless
|
||||
/// of argument payload size. `serde_json::to_string` is already
|
||||
/// canonical here — this codebase doesn't enable `serde_json`'s
|
||||
/// `preserve_order` feature, so `Value::Object` is backed by a
|
||||
/// `BTreeMap` and always serializes keys in sorted order.
|
||||
fn dedup_key(tool_name: &str, canonical_args: &str) -> String {
|
||||
let hash = hex::encode(sha2::Sha256::digest(canonical_args.as_bytes()));
|
||||
format!("{tool_name}:{hash}")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
use crate::dto::chat::tool::{ToolCall, ToolFunction};
|
||||
use serde_json::json;
|
||||
|
||||
fn assistant_with_call(id: &str, name: &str, args: serde_json::Value) -> ChatMessage {
|
||||
let mut m = ChatMessage::assistant(None);
|
||||
m.tool_calls = Some(vec![ToolCall {
|
||||
id: id.to_string(),
|
||||
type_: "function".to_string(),
|
||||
function: ToolFunction {
|
||||
name: name.to_string(),
|
||||
arguments: args,
|
||||
},
|
||||
}]);
|
||||
m
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn older_result_of_same_read_tool_and_args_is_replaced() {
|
||||
let messages = vec![
|
||||
assistant_with_call("call-1", "read", json!({"path": "a.rs"})),
|
||||
ChatMessage::tool_result("call-1".to_string(), "first read of a.rs".to_string()),
|
||||
assistant_with_call("call-2", "read", json!({"path": "a.rs"})),
|
||||
ChatMessage::tool_result("call-2".to_string(), "second read of a.rs".to_string()),
|
||||
];
|
||||
|
||||
let (result, changed) = collapse(&messages);
|
||||
|
||||
assert!(changed);
|
||||
assert_eq!(result[1].content.as_deref(), Some(DUPLICATE_PLACEHOLDER));
|
||||
assert_eq!(result[3].content.as_deref(), Some("second read of a.rs"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn different_arguments_are_not_deduplicated() {
|
||||
let messages = vec![
|
||||
assistant_with_call("call-1", "read", json!({"path": "a.rs"})),
|
||||
ChatMessage::tool_result("call-1".to_string(), "read of a.rs".to_string()),
|
||||
assistant_with_call("call-2", "read", json!({"path": "b.rs"})),
|
||||
ChatMessage::tool_result("call-2".to_string(), "read of b.rs".to_string()),
|
||||
];
|
||||
|
||||
let (result, changed) = collapse(&messages);
|
||||
|
||||
assert!(!changed);
|
||||
assert_eq!(result[1].content.as_deref(), Some("read of a.rs"));
|
||||
assert_eq!(result[3].content.as_deref(), Some("read of b.rs"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn key_order_in_arguments_does_not_prevent_dedup() {
|
||||
let messages = vec![
|
||||
assistant_with_call("call-1", "grep", json!({"pattern": "foo", "path": "."})),
|
||||
ChatMessage::tool_result("call-1".to_string(), "first grep".to_string()),
|
||||
assistant_with_call("call-2", "grep", json!({"path": ".", "pattern": "foo"})),
|
||||
ChatMessage::tool_result("call-2".to_string(), "second grep".to_string()),
|
||||
];
|
||||
|
||||
let (result, changed) = collapse(&messages);
|
||||
|
||||
assert!(changed);
|
||||
assert_eq!(result[1].content.as_deref(), Some(DUPLICATE_PLACEHOLDER));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mutating_tool_with_identical_args_is_never_deduplicated() {
|
||||
let messages = vec![
|
||||
assistant_with_call("call-1", "bash", json!({"command": "cargo test"})),
|
||||
ChatMessage::tool_result("call-1".to_string(), "first run: 3 failed".to_string()),
|
||||
assistant_with_call("call-2", "bash", json!({"command": "cargo test"})),
|
||||
ChatMessage::tool_result("call-2".to_string(), "second run: 0 failed".to_string()),
|
||||
];
|
||||
|
||||
let (result, changed) = collapse(&messages);
|
||||
|
||||
assert!(!changed);
|
||||
assert_eq!(result[1].content.as_deref(), Some("first run: 3 failed"));
|
||||
assert_eq!(result[3].content.as_deref(), Some("second run: 0 failed"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tool_result_with_no_matching_call_is_left_untouched() {
|
||||
let messages = vec![ChatMessage::tool_result(
|
||||
"orphan-id".to_string(),
|
||||
"some result".to_string(),
|
||||
)];
|
||||
|
||||
let (result, changed) = collapse(&messages);
|
||||
|
||||
assert!(!changed);
|
||||
assert_eq!(result[0].content.as_deref(), Some("some result"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
//! Context management: token counting, cross-call tool-result dedup,
|
||||
//! per-result compression, budget-based shaping, and shared
|
||||
//! context-window resolution — replaces `runtime::shortsend`.
|
||||
//!
|
||||
//! No facade function here: `dedup`, `shaping`, and `tokens` are called
|
||||
//! directly from each call site (the per-turn auto-compaction loop in
|
||||
//! `actions::run_agent_turn`, and `Action::Compact`), matching this
|
||||
//! codebase's "no DI, call modules directly" convention. An orchestration
|
||||
//! layer would only serve one of the two callers generically — the
|
||||
//! auto-loop already needs per-stage control to decide when to emit
|
||||
//! `TurnEvent::Compacted`.
|
||||
pub mod dedup;
|
||||
pub mod shaping;
|
||||
pub mod squash;
|
||||
pub mod tokens;
|
||||
pub mod window;
|
||||
@@ -0,0 +1,211 @@
|
||||
//! Budget-based message shaping: compacts long conversation histories so
|
||||
//! they fit within the provider's context window before being sent to
|
||||
//! the LLM API. Ported from the former `runtime::shortsend` — behavior
|
||||
//! is unchanged, only its token-counting now goes through
|
||||
//! `context::tokens` instead of an inline heuristic.
|
||||
use super::tokens::count_tokens;
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
|
||||
/// Decide whether the message list should be shaped (compacted) before
|
||||
/// sending to the LLM.
|
||||
///
|
||||
/// Flow: trigger based on token estimate. If `token_estimate` exceeds
|
||||
/// the threshold, we shape. When `prev_shaped` is true, the threshold is
|
||||
/// raised (95%) to avoid fluttering — compaction only re-triggers when
|
||||
/// the context is genuinely full again. When `prev_shaped` is false, the
|
||||
/// threshold is lower (85%) so compaction starts proactively.
|
||||
///
|
||||
/// Why: hysteresis prevents repeated compaction on every turn when the
|
||||
/// token count hovers near the boundary.
|
||||
///
|
||||
/// Return: `true` if shaping should be applied.
|
||||
pub fn should_shape(token_estimate: usize, max_wire_tokens: usize, prev_shaped: bool) -> bool {
|
||||
let threshold = if prev_shaped {
|
||||
(max_wire_tokens as f32 * 0.95) as usize
|
||||
} else {
|
||||
(max_wire_tokens as f32 * 0.85) as usize
|
||||
};
|
||||
token_estimate >= threshold
|
||||
}
|
||||
|
||||
/// Compact a long message list by dropping middle messages and inserting
|
||||
/// a summary placeholder.
|
||||
///
|
||||
/// Flow: if the estimated token count is within budget and not forced,
|
||||
/// return messages unchanged -> otherwise keep the system message and
|
||||
/// the most recent messages that fit a 70%-of-budget target, with a
|
||||
/// `[prior conversation compacted]` (or LLM-generated summary, if
|
||||
/// `client` is `Some`) system message in between.
|
||||
///
|
||||
/// Why: keeps context-size overhead roughly constant regardless of
|
||||
/// session length.
|
||||
///
|
||||
/// Return: the shaped message list, or `messages` unchanged if shaping
|
||||
/// wasn't needed.
|
||||
pub fn shape_messages(
|
||||
messages: &[ChatMessage],
|
||||
token_count: usize,
|
||||
max_wire_tokens: usize,
|
||||
force: bool,
|
||||
client: Option<&crate::service::provider::LlmClient>,
|
||||
) -> Vec<ChatMessage> {
|
||||
if !force && (token_count <= max_wire_tokens || messages.len() < 5) {
|
||||
return messages.to_vec();
|
||||
}
|
||||
|
||||
let target_tokens = (max_wire_tokens as f32 * 0.70) as usize;
|
||||
let mut current_tokens = 0;
|
||||
let mut keep_recent = Vec::new();
|
||||
let mut dropped_msgs = Vec::new();
|
||||
|
||||
let mut msgs_to_eval = messages.to_vec();
|
||||
let first = if msgs_to_eval.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(msgs_to_eval.remove(0))
|
||||
};
|
||||
|
||||
for m in msgs_to_eval.into_iter().rev() {
|
||||
let text = m.content.as_deref().unwrap_or("");
|
||||
let msg_tokens = count_tokens(text);
|
||||
|
||||
if current_tokens + msg_tokens <= target_tokens {
|
||||
current_tokens += msg_tokens;
|
||||
keep_recent.push(m);
|
||||
} else {
|
||||
dropped_msgs.push(m);
|
||||
}
|
||||
}
|
||||
|
||||
dropped_msgs.reverse();
|
||||
|
||||
let mut result = Vec::new();
|
||||
if let Some(f) = first {
|
||||
result.push(f);
|
||||
}
|
||||
|
||||
if !dropped_msgs.is_empty() {
|
||||
let mut summary_text = "[prior conversation compacted]".to_string();
|
||||
|
||||
if let Some(llm) = client {
|
||||
let prompt = format!(
|
||||
"Summarize the following dropped conversation history briefly. Focus on main goals, decisions made, and files modified, so the context is preserved for future turns. Keep it concise.\n\nHistory:\n{}",
|
||||
dropped_msgs.iter()
|
||||
.map(|m| format!("[{}]: {}", if m.role == crate::dto::chat::message::Role::User { "User" } else { "Assistant" }, m.content.as_deref().unwrap_or("")))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n\n")
|
||||
);
|
||||
|
||||
let req_msgs = vec![ChatMessage::user(prompt)];
|
||||
match llm.chat_with_tools_non_streaming(&req_msgs, None) {
|
||||
Ok(resp) => {
|
||||
if let Some(content) = resp.0.content {
|
||||
summary_text =
|
||||
format!("[Summary of compacted prior conversation:\n{content}\n]");
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"[context::shaping] LLM summarization failed: {}. \
|
||||
Prior conversation history is lost — no summary available. \
|
||||
This means the model will lose context about earlier parts of \
|
||||
the conversation.",
|
||||
e,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result.push(ChatMessage::system(summary_text));
|
||||
}
|
||||
|
||||
result.extend(keep_recent.into_iter().rev());
|
||||
result
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
|
||||
#[test]
|
||||
fn should_shape_triggers_at_85_percent_when_not_previously_shaped() {
|
||||
assert!(should_shape(850, 1000, false));
|
||||
assert!(!should_shape(849, 1000, false));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn should_shape_uses_95_percent_threshold_once_already_shaped() {
|
||||
assert!(
|
||||
!should_shape(900, 1000, true),
|
||||
"below 95% and already shaped: no re-trigger yet"
|
||||
);
|
||||
assert!(should_shape(950, 1000, true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_messages_is_a_noop_under_budget_and_not_forced() {
|
||||
let messages = vec![
|
||||
ChatMessage::system("sys"),
|
||||
ChatMessage::user("hi"),
|
||||
ChatMessage::assistant(Some("hello".to_string())),
|
||||
];
|
||||
let result = shape_messages(&messages, 10, 1000, false, None);
|
||||
assert_eq!(result.len(), messages.len());
|
||||
}
|
||||
|
||||
/// Build a message whose real BPE token count is large enough that 20
|
||||
/// of them (~49 tokens each, ~980 total — verified empirically with
|
||||
/// `context::tokens::count_tokens`) comfortably exceed
|
||||
/// `shape_messages`'s 70%-of-1000 = 700 token target, guaranteeing
|
||||
/// several get dropped. A short fixture like `format!("message {i}")`
|
||||
/// (~8 tokens each, ~160 total for 20) stays entirely under budget
|
||||
/// with real BPE counting and would make these tests pass vacuously
|
||||
/// (nothing ever gets dropped, so "must survive shaping" and "falls
|
||||
/// back to placeholder" hold trivially without exercising the actual
|
||||
/// drop logic) — this was a real bug caught during Task 5's first
|
||||
/// implementation attempt.
|
||||
fn padded_message(i: usize) -> String {
|
||||
format!(
|
||||
"message number {i} with some padding text {}",
|
||||
"additional padding content to increase token count substantially ".repeat(5),
|
||||
)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_messages_always_preserves_the_first_system_message() {
|
||||
let mut messages = vec![ChatMessage::system("system prompt")];
|
||||
for i in 0..20 {
|
||||
messages.push(ChatMessage::user(padded_message(i)));
|
||||
}
|
||||
let result = shape_messages(&messages, 100_000, 1000, true, None);
|
||||
assert_eq!(result[0].content.as_deref(), Some("system prompt"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_messages_without_a_client_falls_back_to_placeholder_summary() {
|
||||
let mut messages = vec![ChatMessage::system("system prompt")];
|
||||
for i in 0..20 {
|
||||
messages.push(ChatMessage::user(padded_message(i)));
|
||||
}
|
||||
let result = shape_messages(&messages, 100_000, 1000, true, None);
|
||||
let has_placeholder = result
|
||||
.iter()
|
||||
.any(|m| m.content.as_deref() == Some("[prior conversation compacted]"));
|
||||
assert!(has_placeholder);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shape_messages_keeps_most_recent_messages_over_older_ones() {
|
||||
let mut messages = vec![ChatMessage::system("system prompt")];
|
||||
for i in 0..20 {
|
||||
messages.push(ChatMessage::user(padded_message(i)));
|
||||
}
|
||||
let result = shape_messages(&messages, 100_000, 1000, true, None);
|
||||
let last_content = messages.last().unwrap().content.clone();
|
||||
assert!(
|
||||
result.iter().any(|m| m.content == last_content),
|
||||
"most recent message must survive shaping"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,464 @@
|
||||
#![allow(dead_code)]
|
||||
//! Per-tool-result compression: shrink large tool outputs before they
|
||||
//! ever enter conversation history, dispatching by content shape.
|
||||
//!
|
||||
//! Flow: `apply(tool_name, output)` -> `read` tool or under the size
|
||||
//! floor? pass through unchanged : valid JSON? `squash_json` : tool is
|
||||
//! `bash` and looks log-shaped? `squash_log` : `squash_generic`.
|
||||
//!
|
||||
//! Why: a single large `bash`/`grep` result can dominate a
|
||||
//! conversation's token budget even on its first occurrence, long
|
||||
//! before `dedup`/`shaping` ever get a chance to act on repeats or
|
||||
//! overall budget.
|
||||
use std::collections::HashSet;
|
||||
use std::fmt::Write;
|
||||
|
||||
/// Below this size, compression isn't worth the risk of losing detail —
|
||||
/// pass the output through unchanged.
|
||||
const SQUASH_FLOOR_BYTES: usize = 1500;
|
||||
|
||||
/// Byte budget for the generic fallback compressor — double the squash
|
||||
/// floor, so the fallback path still yields a real reduction on
|
||||
/// anything that triggered it.
|
||||
const GENERIC_BUDGET_BYTES: usize = SQUASH_FLOOR_BYTES * 2;
|
||||
|
||||
/// Tools whose output must never be altered. `read` is exempted because
|
||||
/// its output must stay byte-exact — the agent relies on it for
|
||||
/// exact-match edits afterward, and squashing a file that happens to
|
||||
/// parse as JSON (e.g. `package.json`) would silently corrupt the
|
||||
/// agent's view of real file content.
|
||||
const NEVER_SQUASH: &[&str] = &["read"];
|
||||
|
||||
/// Tools whose output the log classifier is allowed to run on.
|
||||
/// `looks_log_shaped` keys purely on content (>=3 error/warn/fail-shaped
|
||||
/// lines), which a `grep`/`search` result full of matches against
|
||||
/// error-handling code would trip just as easily as a real build log —
|
||||
/// but `squash_log` caps at 20 error + 10 warning lines with no byte
|
||||
/// budget, silently dropping legitimate matches past that cap. Only
|
||||
/// `bash` (the actual log-producing tool) is allowed to route through
|
||||
/// it; everything else that looks log-shaped falls through to the
|
||||
/// gentler, byte-budgeted `squash_generic` instead.
|
||||
const LOG_SHAPED_TOOLS: &[&str] = &["bash"];
|
||||
|
||||
/// Compress a tool's raw output before it's stored in conversation
|
||||
/// history.
|
||||
///
|
||||
/// Return: `output` unchanged if `tool_name` is in `NEVER_SQUASH` or at
|
||||
/// or under `SQUASH_FLOOR_BYTES`; otherwise the compressed form from
|
||||
/// whichever detector matches its content shape.
|
||||
pub fn apply(tool_name: &str, output: &str) -> String {
|
||||
if NEVER_SQUASH.contains(&tool_name) || output.len() <= SQUASH_FLOOR_BYTES {
|
||||
return output.to_string();
|
||||
}
|
||||
if serde_json::from_str::<serde_json::Value>(output).is_ok() {
|
||||
return squash_json(output);
|
||||
}
|
||||
if LOG_SHAPED_TOOLS.contains(&tool_name) && looks_log_shaped(output) {
|
||||
return squash_log(output);
|
||||
}
|
||||
squash_generic(output, GENERIC_BUDGET_BYTES)
|
||||
}
|
||||
|
||||
/// Compress a JSON tool result by keeping all structural content (keys,
|
||||
/// array/object shape) and eliding long, low-entropy string *values*,
|
||||
/// while keeping short values (<=20 chars) and high-entropy single-token
|
||||
/// ones (UUIDs, hashes, paths) intact. Array elements past the first 3
|
||||
/// are elided regardless of length/entropy.
|
||||
///
|
||||
/// Why walk a parsed `Value` instead of hand-rolling a JSON tokenizer:
|
||||
/// `serde_json` already handles escaping/nesting correctly (this
|
||||
/// codebase's own `dto::chat::tool::repair_json` exists specifically to
|
||||
/// work around how easy it is to get that wrong by hand) — reusing it
|
||||
/// is both simpler and more robust.
|
||||
///
|
||||
/// Return: re-serialized JSON with the same shape as the input.
|
||||
fn squash_json(text: &str) -> String {
|
||||
let Ok(mut value) = serde_json::from_str::<serde_json::Value>(text) else {
|
||||
return text.to_string();
|
||||
};
|
||||
squash_json_value(&mut value, false);
|
||||
serde_json::to_string(&value).unwrap_or_else(|_| text.to_string())
|
||||
}
|
||||
|
||||
/// Recursively elide long, low-entropy string values in place.
|
||||
/// `in_late_array` is true once past the first 3 elements of an
|
||||
/// enclosing array, tightening the elision rule for the rest of it.
|
||||
///
|
||||
/// Why the `!s.contains(' ')` gate before the entropy check: raw
|
||||
/// per-character Shannon entropy alone does NOT separate "meaningful
|
||||
/// prose" from "random-looking identifier" — verified empirically,
|
||||
/// repeated English prose scores ~3.89 bits/char, *higher* than a UUID's
|
||||
/// ~3.39 or a SHA-256 hex digest's ~3.66, because prose draws from a
|
||||
/// wide, fairly-balanced character set too. What actually distinguishes
|
||||
/// identifiers from prose is that identifiers are a single unbroken
|
||||
/// token — this mirrors headroom's own approach (its entropy check is
|
||||
/// "cheaply pre-filtered by 'no spaces'" before scoring). Multi-word
|
||||
/// values never reach the entropy branch at all; only whitespace-free
|
||||
/// tokens do, where entropy correctly separates "abc123" or "aaaaaaaa"
|
||||
/// (low, elided if long) from a UUID/hash/API-key-shaped string (high,
|
||||
/// kept).
|
||||
fn squash_json_value(value: &mut serde_json::Value, in_late_array: bool) {
|
||||
match value {
|
||||
serde_json::Value::String(s) => {
|
||||
let looks_like_identifier = !s.contains(' ') && shannon_entropy(s) >= 3.0;
|
||||
let keep = !in_late_array && (s.len() <= 20 || looks_like_identifier);
|
||||
if !keep {
|
||||
*s = "…".to_string();
|
||||
}
|
||||
}
|
||||
serde_json::Value::Array(items) => {
|
||||
for (i, item) in items.iter_mut().enumerate() {
|
||||
squash_json_value(item, i >= 3);
|
||||
}
|
||||
}
|
||||
serde_json::Value::Object(map) => {
|
||||
for v in map.values_mut() {
|
||||
squash_json_value(v, false);
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
/// Shannon entropy in bits per character — used, after the `squash_json`
|
||||
/// caller's own "no internal whitespace" pre-filter, to distinguish
|
||||
/// high-entropy single-token strings (UUIDs, hashes, random IDs, worth
|
||||
/// keeping) from low-entropy ones (e.g. `"aaaaaaaaaa"`, safe to elide).
|
||||
/// 3.0 sits comfortably below a UUID's ~3.39 and a SHA-256 hex digest's
|
||||
/// ~3.66 (both empirically measured with this exact formula) while
|
||||
/// staying well above a degenerate repeated-character string's 0.0.
|
||||
fn shannon_entropy(s: &str) -> f64 {
|
||||
if s.is_empty() {
|
||||
return 0.0;
|
||||
}
|
||||
let mut counts: std::collections::HashMap<char, usize> = std::collections::HashMap::new();
|
||||
for c in s.chars() {
|
||||
*counts.entry(c).or_insert(0) += 1;
|
||||
}
|
||||
let len = s.chars().count() as f64;
|
||||
counts
|
||||
.values()
|
||||
.map(|&count| {
|
||||
let p = f64::from(u32::try_from(count).unwrap_or(u32::MAX)) / len;
|
||||
-p * p.log2()
|
||||
})
|
||||
.sum()
|
||||
}
|
||||
|
||||
/// Coarse severity classification for a single log line, used by
|
||||
/// `squash_log` to rank which lines are most worth keeping.
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
enum LogLevel {
|
||||
Error,
|
||||
Warn,
|
||||
Info,
|
||||
Debug,
|
||||
}
|
||||
|
||||
/// Classify a single log line by scanning for level keywords.
|
||||
///
|
||||
/// Why substring matching on a lowercased copy instead of a real log
|
||||
/// parser: tool output comes from arbitrary external processes with no
|
||||
/// consistent log format, so keyword sniffing is the only detector that
|
||||
/// generalizes across all of them.
|
||||
fn classify_line(line: &str) -> LogLevel {
|
||||
let lower = line.to_lowercase();
|
||||
if lower.contains("error") || lower.contains("fail") || lower.contains("panic") {
|
||||
LogLevel::Error
|
||||
} else if lower.contains("warn") {
|
||||
LogLevel::Warn
|
||||
} else if lower.contains("debug") || lower.contains("trace") {
|
||||
LogLevel::Debug
|
||||
} else {
|
||||
LogLevel::Info
|
||||
}
|
||||
}
|
||||
|
||||
/// Heuristic gate for routing to `squash_log` vs `squash_generic`: at
|
||||
/// least 3 lines that look like error/warning/stack-trace output.
|
||||
fn looks_log_shaped(text: &str) -> bool {
|
||||
let hits = text
|
||||
.lines()
|
||||
.filter(|l| {
|
||||
let lower = l.to_lowercase();
|
||||
lower.contains("error")
|
||||
|| lower.contains("warn")
|
||||
|| lower.contains("fail")
|
||||
|| lower.contains("panic")
|
||||
|| l.trim_start().starts_with("at ")
|
||||
})
|
||||
.count();
|
||||
hits >= 3
|
||||
}
|
||||
|
||||
/// Compress log-shaped output: keep up to 20 highest-scored error lines
|
||||
/// and up to 10 highest-scored warning lines (score = level weight +
|
||||
/// 0.3 if the line looks like a stack-trace frame), each with a
|
||||
/// +/-2-line context window, replacing every gap with a `[N lines
|
||||
/// omitted]` marker.
|
||||
///
|
||||
/// Why not a comment-shaped marker (e.g. `// N lines omitted`): the
|
||||
/// `rtk` project's own regression tests found that shape gets parsed by
|
||||
/// the LLM as code and triggers a retry loop.
|
||||
fn squash_log(text: &str) -> String {
|
||||
let lines: Vec<&str> = text.lines().collect();
|
||||
let levels: Vec<LogLevel> = lines.iter().map(|l| classify_line(l)).collect();
|
||||
|
||||
let score = |i: usize| -> f32 {
|
||||
let level_score = match levels[i] {
|
||||
LogLevel::Error => 1.0,
|
||||
LogLevel::Warn => 0.5,
|
||||
LogLevel::Info => 0.1,
|
||||
LogLevel::Debug => 0.05,
|
||||
};
|
||||
let stack_boost = if lines[i].trim_start().starts_with("at ") {
|
||||
0.3
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
level_score + stack_boost
|
||||
};
|
||||
|
||||
let mut error_idxs: Vec<usize> = (0..lines.len())
|
||||
.filter(|&i| levels[i] == LogLevel::Error)
|
||||
.collect();
|
||||
error_idxs.sort_by(|&a, &b| {
|
||||
score(b)
|
||||
.partial_cmp(&score(a))
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
error_idxs.truncate(20);
|
||||
|
||||
let mut warn_idxs: Vec<usize> = (0..lines.len())
|
||||
.filter(|&i| levels[i] == LogLevel::Warn)
|
||||
.collect();
|
||||
warn_idxs.sort_by(|&a, &b| {
|
||||
score(b)
|
||||
.partial_cmp(&score(a))
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
warn_idxs.truncate(10);
|
||||
|
||||
let mut keep: HashSet<usize> = HashSet::new();
|
||||
for &i in error_idxs.iter().chain(warn_idxs.iter()) {
|
||||
let lo = i.saturating_sub(2);
|
||||
let hi = (i + 2).min(lines.len().saturating_sub(1));
|
||||
keep.extend(lo..=hi);
|
||||
}
|
||||
|
||||
if keep.is_empty() {
|
||||
return squash_generic(text, GENERIC_BUDGET_BYTES);
|
||||
}
|
||||
|
||||
render_kept_lines(&lines, &keep)
|
||||
}
|
||||
|
||||
/// Importance-ranked truncation for content that isn't JSON or
|
||||
/// log-shaped: keep the first 10 and last 10 lines, plus any
|
||||
/// non-blank line that isn't a repeat of the one before it, until
|
||||
/// `budget` bytes are used.
|
||||
fn squash_generic(text: &str, budget: usize) -> String {
|
||||
let lines: Vec<&str> = text.lines().collect();
|
||||
if lines.len() <= 20 {
|
||||
return text.chars().take(budget).collect();
|
||||
}
|
||||
|
||||
let head_end = 10;
|
||||
let tail_start = lines.len() - 10;
|
||||
let mut keep: HashSet<usize> = (0..head_end).chain(tail_start..lines.len()).collect();
|
||||
|
||||
let mut used: usize = lines[..head_end].iter().map(|l| l.len() + 1).sum::<usize>()
|
||||
+ lines[tail_start..]
|
||||
.iter()
|
||||
.map(|l| l.len() + 1)
|
||||
.sum::<usize>();
|
||||
let mut prev = "";
|
||||
for (i, &line) in lines.iter().enumerate().take(tail_start).skip(head_end) {
|
||||
let non_trivial = !line.trim().is_empty() && line != prev;
|
||||
if non_trivial && used + line.len() < budget {
|
||||
keep.insert(i);
|
||||
used += line.len() + 1;
|
||||
}
|
||||
prev = line;
|
||||
}
|
||||
|
||||
render_kept_lines(&lines, &keep)
|
||||
}
|
||||
|
||||
/// Render a subset of `lines` in order, inserting a `[N lines omitted]`
|
||||
/// marker at every gap between kept lines.
|
||||
fn render_kept_lines(lines: &[&str], keep: &HashSet<usize>) -> String {
|
||||
let mut kept_sorted: Vec<usize> = keep.iter().copied().collect();
|
||||
kept_sorted.sort_unstable();
|
||||
|
||||
let mut out = String::new();
|
||||
let mut cursor = 0usize;
|
||||
for &i in &kept_sorted {
|
||||
if i > cursor {
|
||||
let _ = writeln!(out, "[{} lines omitted]", i - cursor);
|
||||
}
|
||||
out.push_str(lines[i]);
|
||||
out.push('\n');
|
||||
cursor = i + 1;
|
||||
}
|
||||
if cursor < lines.len() {
|
||||
let _ = writeln!(out, "[{} lines omitted]", lines.len() - cursor);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn output_under_the_floor_passes_through_unchanged() {
|
||||
let small = "short output";
|
||||
assert_eq!(apply("bash", small), small);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn read_tool_output_is_never_squashed_even_when_huge_json() {
|
||||
let big_json = format!(
|
||||
"{{\"description\": \"{}\"}}",
|
||||
"a very long description value that repeats ".repeat(100),
|
||||
);
|
||||
assert!(big_json.len() > SQUASH_FLOOR_BYTES);
|
||||
assert_eq!(apply("read", &big_json), big_json);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_output_over_floor_keeps_structure_and_short_values() {
|
||||
let value = serde_json::json!({
|
||||
"id": "abc123",
|
||||
"note": "hi",
|
||||
"description": "a very long description value that repeats ".repeat(100),
|
||||
});
|
||||
let text = serde_json::to_string(&value).unwrap();
|
||||
assert!(text.len() > SQUASH_FLOOR_BYTES);
|
||||
|
||||
let result = apply("some_mcp_tool", &text);
|
||||
let parsed: serde_json::Value =
|
||||
serde_json::from_str(&result).expect("squashed JSON must still be valid JSON");
|
||||
|
||||
assert_eq!(parsed["id"], "abc123", "short values must survive");
|
||||
assert_eq!(parsed["note"], "hi", "short values must survive");
|
||||
assert_ne!(
|
||||
parsed["description"].as_str().unwrap().len(),
|
||||
value["description"].as_str().unwrap().len(),
|
||||
"long low-entropy value must be shrunk",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn json_array_elements_past_third_are_squashed_harder() {
|
||||
// A UUID-shaped value has no internal whitespace and clears the
|
||||
// entropy threshold, so under the *normal* per-value rule (which
|
||||
// still applies to array indices 0-2) it survives untouched.
|
||||
// Padding elsewhere in the object pushes total size over the
|
||||
// squash floor without affecting which array elements get kept.
|
||||
let identifier = "550e8400-e29b-41d4-a716-446655440000";
|
||||
let padding = "padding text to push this payload past the squash floor so apply() actually dispatches to squash_json ".repeat(20);
|
||||
let value = serde_json::json!({
|
||||
"padding": padding,
|
||||
"items": [identifier, identifier, identifier, identifier],
|
||||
});
|
||||
let text = serde_json::to_string(&value).unwrap();
|
||||
assert!(text.len() > SQUASH_FLOOR_BYTES);
|
||||
|
||||
let result = apply("some_mcp_tool", &text);
|
||||
let parsed: serde_json::Value = serde_json::from_str(&result).unwrap();
|
||||
let items = parsed["items"].as_array().unwrap();
|
||||
|
||||
assert_eq!(items[0].as_str().unwrap(), identifier, "index 0 is under the array cutoff and identifier-shaped, so it's kept under the normal rule");
|
||||
assert_eq!(
|
||||
items[2].as_str().unwrap(),
|
||||
identifier,
|
||||
"index 2 is still under the cutoff (past-third means index >= 3)"
|
||||
);
|
||||
assert_ne!(items[3].as_str().unwrap(), identifier, "index 3 must be force-elided even though it's identifier-shaped and would survive at any earlier index");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn log_like_output_keeps_error_lines_and_marks_omissions() {
|
||||
// `looks_log_shaped` requires >= 3 lines matching error/warn/fail/
|
||||
// panic/stack-frame patterns before routing to `squash_log` at
|
||||
// all — a single error line isn't enough and would silently fall
|
||||
// through to `squash_generic` instead, so this fixture needs at
|
||||
// least 3 such lines, spread apart, to actually exercise
|
||||
// squash_log's scoring/windowing logic (not just its fallback).
|
||||
let mut lines = vec!["build started".to_string()];
|
||||
for i in 0..200 {
|
||||
lines.push(format!("info: compiling module {i}"));
|
||||
}
|
||||
lines.push("error: something failed early in the build".to_string());
|
||||
for i in 0..200 {
|
||||
lines.push(format!("info: compiling module {}", i + 200));
|
||||
}
|
||||
lines.push("warning: deprecated api used somewhere".to_string());
|
||||
lines.push("error: something failed at the end".to_string());
|
||||
let text = lines.join("\n");
|
||||
assert!(text.len() > SQUASH_FLOOR_BYTES);
|
||||
|
||||
let result = apply("bash", &text);
|
||||
|
||||
assert!(result.contains("error: something failed early in the build"));
|
||||
assert!(result.contains("error: something failed at the end"));
|
||||
assert!(result.contains("lines omitted"));
|
||||
assert!(result.len() < text.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_bash_tool_with_log_shaped_content_is_not_log_compressed() {
|
||||
// A grep result whose matched lines all mention "error" would
|
||||
// trip `looks_log_shaped`'s >=3-line keyword threshold just like
|
||||
// a real build log — but `squash_log` caps at 20 highest-scored
|
||||
// error lines with no guaranteed tail retention, silently
|
||||
// dropping legitimate matches past that cap. Only `bash` is
|
||||
// treated as log-shaped; `grep` must fall through to
|
||||
// `squash_generic`, which always keeps the first and last 10
|
||||
// lines regardless of score. With every line tied at the same
|
||||
// score, a `squash_log` route would keep indices 0-19 (stable
|
||||
// sort preserves original order on ties) and drop index 49 —
|
||||
// so asserting the tail survives is a route-distinguishing
|
||||
// check, not just a content check.
|
||||
let lines: Vec<String> = (0..50)
|
||||
.map(|i| format!("src/file{i}.rs:{i}: error handling for case {i}"))
|
||||
.collect();
|
||||
let text = lines.join("\n");
|
||||
assert!(text.len() > SQUASH_FLOOR_BYTES);
|
||||
|
||||
let result = apply("grep", &text);
|
||||
|
||||
assert!(
|
||||
result.contains("src/file0.rs:0: error handling for case 0"),
|
||||
"generic keeps head"
|
||||
);
|
||||
assert!(
|
||||
result.contains("src/file49.rs:49: error handling for case 49"),
|
||||
"generic keeps tail — squash_log would have dropped this"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generic_large_text_is_truncated_with_omission_marker() {
|
||||
let lines: Vec<String> = (0..500)
|
||||
.map(|i| format!("line number {i} of plain output"))
|
||||
.collect();
|
||||
let text = lines.join("\n");
|
||||
assert!(text.len() > SQUASH_FLOOR_BYTES);
|
||||
|
||||
let result = apply("bash", &text);
|
||||
|
||||
assert!(
|
||||
result.contains("line number 0 of plain output"),
|
||||
"keeps head"
|
||||
);
|
||||
assert!(
|
||||
result.contains("line number 499 of plain output"),
|
||||
"keeps tail"
|
||||
);
|
||||
assert!(result.contains("lines omitted"));
|
||||
assert!(result.len() < text.len());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
//! Unified token-count estimation for context-window budgeting.
|
||||
//!
|
||||
//! Flow: text -> `tiktoken_rs::o200k_base_singleton()` (BPE vocab embedded
|
||||
//! in the binary via `include_str!`, no network access) -> `encode_ordinary`
|
||||
//! -> token count.
|
||||
//!
|
||||
//! Why: replaces three independent char-count heuristics that disagreed
|
||||
//! with each other (`/3` in the old `shortsend.rs`, `/4` in the turn
|
||||
//! loop, `/4` again in the status bar) with one real BPE tokenizer.
|
||||
//! `o200k_base` is an approximation for non-OpenAI providers but is far
|
||||
//! closer than a flat byte-per-token guess; it's only used for the
|
||||
//! 85%/95% budget thresholds, not for billing-accurate counts.
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
|
||||
/// Count tokens in a single string under `o200k_base`.
|
||||
///
|
||||
/// Return: the BPE token count for `text`. `encode_ordinary` (not
|
||||
/// `encode`/`encode_with_special_tokens`) is used deliberately — message
|
||||
/// content that happens to contain a special-token-shaped substring
|
||||
/// (e.g. literal text `<|endoftext|>` pasted by a user) must be counted
|
||||
/// as ordinary text, not interpreted as a control token.
|
||||
pub fn count_tokens(text: &str) -> usize {
|
||||
tiktoken_rs::o200k_base_singleton()
|
||||
.encode_ordinary(text)
|
||||
.len()
|
||||
}
|
||||
|
||||
/// Count tokens in a `ChatMessage`'s text content.
|
||||
///
|
||||
/// Return: 0 for a message with no `content` (e.g. an assistant message
|
||||
/// that only carries `tool_calls`).
|
||||
#[allow(dead_code)]
|
||||
pub fn count_message_tokens(msg: &ChatMessage) -> usize {
|
||||
msg.content.as_deref().map_or(0, count_tokens)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
|
||||
#[test]
|
||||
fn empty_string_has_zero_tokens() {
|
||||
assert_eq!(count_tokens(""), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn known_short_phrase_has_expected_token_count() {
|
||||
// Verified empirically against tiktoken-rs 0.12's o200k_base:
|
||||
// "hello world" -> [24912, 2375], i.e. 2 tokens.
|
||||
assert_eq!(count_tokens("hello world"), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn known_code_snippet_has_expected_token_count() {
|
||||
// Verified empirically: 9 tokens under o200k_base.
|
||||
assert_eq!(count_tokens("fn main() { println!(\"hi\"); }"), 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn message_with_no_content_counts_zero() {
|
||||
let msg = ChatMessage::assistant(None);
|
||||
assert_eq!(count_message_tokens(&msg), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn message_token_count_matches_count_tokens_on_its_content() {
|
||||
let msg = ChatMessage::user("hello world");
|
||||
assert_eq!(count_message_tokens(&msg), count_tokens("hello world"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
#![allow(dead_code)]
|
||||
//! Single source of truth for resolving the active model's context
|
||||
//! window size, replacing three copies of the same lookup that had
|
||||
//! drifted (`Action::Compact`, `spawn_turn`, and `view/status.rs` each
|
||||
//! had their own inline version — the status bar's copy additionally
|
||||
//! displayed "?" on no match instead of falling back like the other two,
|
||||
//! an inconsistency this unifies away).
|
||||
use crate::model::app_config::AppConfig;
|
||||
use crate::model::settings::Settings;
|
||||
|
||||
/// Resolve the context-window size (in tokens) for the currently
|
||||
/// configured provider/model.
|
||||
///
|
||||
/// Flow: find the `ModelRole` whose `provider`+`model` match
|
||||
/// `settings` -> use its `context_window` if set -> otherwise fall back
|
||||
/// to `app_config.default_context_window`.
|
||||
///
|
||||
/// Return: always a concrete token count, never "unknown".
|
||||
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize {
|
||||
app_config
|
||||
.model_roles
|
||||
.values()
|
||||
.find(|role| role.provider == settings.provider && role.model == settings.model)
|
||||
.and_then(|role| role.context_window)
|
||||
.unwrap_or(app_config.default_context_window) as usize
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::model::app_config::ModelRole;
|
||||
|
||||
#[test]
|
||||
fn resolves_context_window_from_matching_model_role() {
|
||||
let mut app_config = AppConfig::default();
|
||||
app_config.model_roles.insert(
|
||||
"default".to_string(),
|
||||
ModelRole {
|
||||
provider: "zen".to_string(),
|
||||
model: "deepseek-v4-flash-free".to_string(),
|
||||
max_tokens: None,
|
||||
context_window: Some(128_000),
|
||||
temperature: None,
|
||||
},
|
||||
);
|
||||
let mut settings = Settings::default();
|
||||
settings.provider = "zen".to_string();
|
||||
settings.model = "deepseek-v4-flash-free".to_string();
|
||||
|
||||
assert_eq!(resolve(&app_config, &settings), 128_000);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn falls_back_to_default_context_window_when_no_role_matches() {
|
||||
let app_config = AppConfig::default();
|
||||
let mut settings = Settings::default();
|
||||
settings.provider = "nonexistent".to_string();
|
||||
settings.model = "nonexistent-model".to_string();
|
||||
|
||||
assert_eq!(
|
||||
resolve(&app_config, &settings),
|
||||
app_config.default_context_window as usize
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn falls_back_to_default_when_matching_role_has_no_context_window_set() {
|
||||
let mut app_config = AppConfig::default();
|
||||
app_config.model_roles.insert(
|
||||
"default".to_string(),
|
||||
ModelRole {
|
||||
provider: "zen".to_string(),
|
||||
model: "deepseek-v4-flash-free".to_string(),
|
||||
max_tokens: None,
|
||||
context_window: None,
|
||||
temperature: None,
|
||||
},
|
||||
);
|
||||
let mut settings = Settings::default();
|
||||
settings.provider = "zen".to_string();
|
||||
settings.model = "deepseek-v4-flash-free".to_string();
|
||||
|
||||
assert_eq!(
|
||||
resolve(&app_config, &settings),
|
||||
app_config.default_context_window as usize
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2,5 +2,5 @@
|
||||
//! and the LLM streaming pipeline.
|
||||
pub mod actions;
|
||||
pub mod commands;
|
||||
pub mod shortsend;
|
||||
pub mod context;
|
||||
pub mod stream;
|
||||
|
||||
@@ -1,129 +0,0 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
//! Short-send / message shaping: compacts long conversation histories so
|
||||
//! they fit within the provider's context window before being sent to the
|
||||
//! LLM API.
|
||||
use crate::dto::chat::message::ChatMessage;
|
||||
|
||||
/// Decide whether the message list should be shaped (compacted) before
|
||||
/// sending to the LLM.
|
||||
///
|
||||
/// Flow: trigger based on token estimate. If `token_estimate` exceeds
|
||||
/// the threshold, we shape. When `prev_shaped` is true, the threshold is
|
||||
/// raised (95%) to avoid fluttering — compaction only re-triggers when
|
||||
/// the context is genuinely full again. When `prev_shaped` is false, the
|
||||
/// threshold is lower (85%) so compaction starts proactively.
|
||||
///
|
||||
/// Why: hysteresis prevents repeated compaction on every turn when the
|
||||
/// token count hovers near the boundary.
|
||||
///
|
||||
/// Return: `true` if shaping should be applied.
|
||||
pub fn should_shape(token_estimate: usize, max_wire_tokens: usize, prev_shaped: bool) -> bool {
|
||||
let threshold = if prev_shaped {
|
||||
// Higher threshold when already shaped — defer re-shaping until
|
||||
// the buffer is genuinely full again (95%).
|
||||
(max_wire_tokens as f32 * 0.95) as usize
|
||||
} else {
|
||||
// Lower threshold when not yet shaped — trigger shaping sooner
|
||||
// (85%) to avoid hitting the context window limit.
|
||||
(max_wire_tokens as f32 * 0.85) as usize
|
||||
};
|
||||
token_estimate >= threshold
|
||||
}
|
||||
|
||||
/// Compact a long message list by dropping middle messages and inserting
|
||||
/// a summary placeholder.
|
||||
///
|
||||
/// Flow: if the estimated token count is within budget and not forced, return messages
|
||||
/// unchanged → otherwise keep the system message and the most recent
|
||||
/// messages (up to `MAX_WIRE_TOKENS / 200` of them) with a `[prior
|
||||
/// conversation compacted]` system message in between.
|
||||
///
|
||||
/// Why: keeps context-size overhead roughly constant regardless of
|
||||
/// session length.
|
||||
///
|
||||
pub fn shape_messages(
|
||||
messages: &[ChatMessage],
|
||||
token_count: usize,
|
||||
max_wire_tokens: usize,
|
||||
force: bool,
|
||||
client: Option<&crate::service::provider::LlmClient>,
|
||||
) -> Vec<ChatMessage> {
|
||||
if !force && (token_count <= max_wire_tokens || messages.len() < 5) {
|
||||
return messages.to_vec();
|
||||
}
|
||||
|
||||
let target_tokens = (max_wire_tokens as f32 * 0.70) as usize;
|
||||
let mut current_tokens = 0;
|
||||
let mut keep_recent = Vec::new();
|
||||
let mut dropped_msgs = Vec::new();
|
||||
|
||||
// Always keep the very first message (System Prompt) which we don't count here
|
||||
// as we just blindly preserve it later.
|
||||
let mut msgs_to_eval = messages.to_vec();
|
||||
let first = if msgs_to_eval.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(msgs_to_eval.remove(0))
|
||||
};
|
||||
|
||||
// Iterate backwards from the most recent to oldest
|
||||
for m in msgs_to_eval.into_iter().rev() {
|
||||
let text = m.content.as_deref().unwrap_or("");
|
||||
// Estimate tokens: ~1 token per 3 bytes for mixed content (code,
|
||||
// prose, multi-byte). Conservative enough to stay under provider
|
||||
// limits while avoiding premature compaction.
|
||||
let msg_tokens = text.len() / 3;
|
||||
|
||||
if current_tokens + msg_tokens <= target_tokens {
|
||||
current_tokens += msg_tokens;
|
||||
keep_recent.push(m);
|
||||
} else {
|
||||
dropped_msgs.push(m); // These will end up in reverse chronological order
|
||||
}
|
||||
}
|
||||
|
||||
// Reverse dropped_msgs so they are back in chronological order
|
||||
dropped_msgs.reverse();
|
||||
|
||||
let mut result = Vec::new();
|
||||
if let Some(f) = first {
|
||||
result.push(f);
|
||||
}
|
||||
|
||||
if !dropped_msgs.is_empty() {
|
||||
let mut summary_text = "[prior conversation compacted]".to_string();
|
||||
|
||||
if let Some(llm) = client {
|
||||
let prompt = format!(
|
||||
"Summarize the following dropped conversation history briefly. Focus on main goals, decisions made, and files modified, so the context is preserved for future turns. Keep it concise.\n\nHistory:\n{}",
|
||||
dropped_msgs.iter()
|
||||
.map(|m| format!("[{}]: {}", if m.role == crate::dto::chat::message::Role::User { "User" } else { "Assistant" }, m.content.as_deref().unwrap_or("")))
|
||||
.collect::<Vec<_>>()
|
||||
.join("\n\n")
|
||||
);
|
||||
|
||||
let req_msgs = vec![ChatMessage::user(prompt)];
|
||||
match llm.chat_with_tools_non_streaming(&req_msgs, None) {
|
||||
Ok(resp) => {
|
||||
if let Some(content) = resp.0.content {
|
||||
summary_text = format!("[Summary of compacted prior conversation:\n{content}\n]");
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"[shortsend] LLM summarization failed: {}. \
|
||||
Prior conversation history is lost — no summary available. \
|
||||
This means the model will lose context about earlier parts of \
|
||||
the conversation.",
|
||||
e,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result.push(ChatMessage::system(summary_text));
|
||||
}
|
||||
|
||||
result.extend(keep_recent.into_iter().rev());
|
||||
result
|
||||
}
|
||||
@@ -1,8 +1,6 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
//! SSE stream parser: converts SSE- or JSON-chunked LLM responses into
|
||||
//! typed `StreamEvent` variants (tokens, reasoning, tool calls, usage, done).
|
||||
pub mod turn;
|
||||
pub mod tools;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
@@ -89,7 +87,7 @@ impl SseParser {
|
||||
/// provider-specific parsing layer.
|
||||
///
|
||||
/// Return: 0, 1, or more `StreamEvent`s from the flushed frame.
|
||||
#[allow(clippy::too_many_lines)]
|
||||
|
||||
fn flush_event(&mut self) -> Vec<StreamEvent> {
|
||||
let data = self.data_lines.join("\n");
|
||||
self.data_lines.clear();
|
||||
@@ -112,20 +110,32 @@ impl SseParser {
|
||||
|
||||
if let Some(usage) = value.get("usage") {
|
||||
if !usage.is_null() {
|
||||
let prompt_tokens = usage.get("prompt_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] prompt_tokens missing in usage chunk");
|
||||
0
|
||||
});
|
||||
let completion_tokens = usage.get("completion_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] completion_tokens missing in usage chunk");
|
||||
0
|
||||
});
|
||||
let total_tokens = usage.get("total_tokens").and_then(serde_json::Value::as_u64)
|
||||
let prompt_tokens = usage
|
||||
.get("prompt_tokens")
|
||||
.and_then(serde_json::Value::as_u64)
|
||||
.unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] prompt_tokens missing in usage chunk");
|
||||
0
|
||||
});
|
||||
let completion_tokens = usage
|
||||
.get("completion_tokens")
|
||||
.and_then(serde_json::Value::as_u64)
|
||||
.unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] completion_tokens missing in usage chunk");
|
||||
0
|
||||
});
|
||||
let total_tokens = usage
|
||||
.get("total_tokens")
|
||||
.and_then(serde_json::Value::as_u64)
|
||||
.unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] total_tokens missing in usage chunk");
|
||||
prompt_tokens + completion_tokens
|
||||
});
|
||||
events.push(StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens });
|
||||
events.push(StreamEvent::Usage {
|
||||
prompt_tokens,
|
||||
completion_tokens,
|
||||
total_tokens,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -143,23 +153,32 @@ impl SseParser {
|
||||
}
|
||||
|
||||
// Reasoning token
|
||||
if let Some(reasoning) = d.get("reasoning_content").and_then(|r| r.as_str()) {
|
||||
if let Some(reasoning) =
|
||||
d.get("reasoning_content").and_then(|r| r.as_str())
|
||||
{
|
||||
d_events.push(StreamEvent::Reasoning(reasoning.to_string()));
|
||||
}
|
||||
|
||||
// Tool calls — iterate ALL entries, not just first()
|
||||
if let Some(tool_calls) = d.get("tool_calls").and_then(|tc| tc.as_array()) {
|
||||
if let Some(tool_calls) =
|
||||
d.get("tool_calls").and_then(|tc| tc.as_array())
|
||||
{
|
||||
for tc in tool_calls {
|
||||
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] tool call delta missing index, defaulting to 0");
|
||||
0
|
||||
}) as usize;
|
||||
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string);
|
||||
let name = tc.get("function")
|
||||
let id = tc
|
||||
.get("id")
|
||||
.and_then(|i| i.as_str())
|
||||
.map(std::string::ToString::to_string);
|
||||
let name = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("name"))
|
||||
.and_then(|n| n.as_str())
|
||||
.map(std::string::ToString::to_string);
|
||||
let args_delta = tc.get("function")
|
||||
let args_delta = tc
|
||||
.get("function")
|
||||
.and_then(|f| f.get("arguments"))
|
||||
.and_then(|a| a.as_str())
|
||||
.unwrap_or("")
|
||||
@@ -174,7 +193,9 @@ impl SseParser {
|
||||
}
|
||||
|
||||
// Finish reason
|
||||
if let Some(reason) = choice.get("finish_reason").and_then(|r| r.as_str()) {
|
||||
if let Some(reason) =
|
||||
choice.get("finish_reason").and_then(|r| r.as_str())
|
||||
{
|
||||
if reason == "stop" || reason == "tool_calls" {
|
||||
d_events.push(StreamEvent::Done);
|
||||
}
|
||||
@@ -193,72 +214,6 @@ impl SseParser {
|
||||
events.append(&mut other_events);
|
||||
events
|
||||
}
|
||||
|
||||
/// Clears any partially-buffered SSE frame. Reserved for reconnect/retry flows that
|
||||
/// reuse a parser instance across requests rather than constructing a fresh one.
|
||||
#[allow(dead_code)]
|
||||
pub fn reset(&mut self) {
|
||||
self.buffer.clear();
|
||||
self.event_type = None;
|
||||
self.data_lines.clear();
|
||||
}
|
||||
}
|
||||
|
||||
/// Fallback parser for providers that send bare JSON chunks instead of SSE-framed
|
||||
/// `data: ...` lines. Not used by the `SseParser` streaming path (which handles
|
||||
/// standard SSE framing directly), kept for providers/tests that feed raw chunks.
|
||||
///
|
||||
/// Flow: parse `data` as JSON → extract first `choices[0].delta` →
|
||||
/// return a `Token`, `Reasoning`, `Done`, or `ToolCallDelta` event based
|
||||
/// on the fields present.
|
||||
///
|
||||
/// Return: `Some(StreamEvent)` if the chunk contained recognisable
|
||||
/// content, `None` otherwise.
|
||||
#[allow(dead_code)]
|
||||
pub fn parse_stream_chunk(data: &str) -> Option<StreamEvent> {
|
||||
let value: Value = serde_json::from_str(data).ok()?;
|
||||
if value == Value::Null {
|
||||
return None;
|
||||
}
|
||||
let choices = value.get("choices")?.as_array()?;
|
||||
let choice = choices.first()?;
|
||||
let delta = choice.get("delta")?;
|
||||
if let Some(content) = delta.get("content").and_then(|c| c.as_str()) {
|
||||
return Some(StreamEvent::Token(content.to_string()));
|
||||
}
|
||||
if let Some(reasoning) = delta.get("reasoning_content").and_then(|r| r.as_str()) {
|
||||
return Some(StreamEvent::Reasoning(reasoning.to_string()));
|
||||
}
|
||||
if let Some(finish) = choice.get("finish_reason").and_then(|r| r.as_str()) {
|
||||
if finish == "stop" || finish == "tool_calls" {
|
||||
return Some(StreamEvent::Done);
|
||||
}
|
||||
}
|
||||
if let Some(tool_calls) = delta.get("tool_calls").and_then(|tc| tc.as_array()) {
|
||||
if let Some(tc) = tool_calls.first() {
|
||||
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
|
||||
tracing::warn!("[stream] fallback parser: tool call missing index, defaulting to 0");
|
||||
0
|
||||
}) as usize;
|
||||
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string);
|
||||
let name = tc.get("function")
|
||||
.and_then(|f| f.get("name"))
|
||||
.and_then(|n| n.as_str())
|
||||
.map(std::string::ToString::to_string);
|
||||
let args = tc.get("function")
|
||||
.and_then(|f| f.get("arguments"))
|
||||
.and_then(|a| a.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
return Some(StreamEvent::ToolCallDelta {
|
||||
index,
|
||||
id,
|
||||
name,
|
||||
arguments_delta: args,
|
||||
});
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -280,7 +235,10 @@ mod tests {
|
||||
fn feed_handles_chunk_split_mid_line() {
|
||||
let mut p = SseParser::new();
|
||||
let e1 = p.feed("data: {\"choices\":[{\"delta\":{\"content\":\"partial");
|
||||
assert!(e1.is_empty(), "no event until the line and blank separator complete");
|
||||
assert!(
|
||||
e1.is_empty(),
|
||||
"no event until the line and blank separator complete"
|
||||
);
|
||||
let e2 = p.feed("\"}}]}\n\n");
|
||||
assert_eq!(e2.len(), 1);
|
||||
match &e2[0] {
|
||||
@@ -300,9 +258,7 @@ mod tests {
|
||||
#[test]
|
||||
fn feed_emits_done_on_finish_reason_stop() {
|
||||
let mut p = SseParser::new();
|
||||
let events = p.feed(
|
||||
"data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
|
||||
);
|
||||
let events = p.feed("data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n");
|
||||
assert_eq!(events.len(), 1);
|
||||
assert!(matches!(events[0], StreamEvent::Done));
|
||||
}
|
||||
@@ -315,7 +271,12 @@ mod tests {
|
||||
);
|
||||
assert_eq!(events.len(), 1);
|
||||
match &events[0] {
|
||||
StreamEvent::ToolCallDelta { index, id, name, arguments_delta } => {
|
||||
StreamEvent::ToolCallDelta {
|
||||
index,
|
||||
id,
|
||||
name,
|
||||
arguments_delta,
|
||||
} => {
|
||||
assert_eq!(*index, 0);
|
||||
assert_eq!(id.as_deref(), Some("call_1"));
|
||||
assert_eq!(name.as_deref(), Some("bash"));
|
||||
@@ -333,7 +294,11 @@ mod tests {
|
||||
);
|
||||
assert_eq!(events.len(), 1);
|
||||
match &events[0] {
|
||||
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens } => {
|
||||
StreamEvent::Usage {
|
||||
prompt_tokens,
|
||||
completion_tokens,
|
||||
total_tokens,
|
||||
} => {
|
||||
assert_eq!(*prompt_tokens, 10);
|
||||
assert_eq!(*completion_tokens, 5);
|
||||
assert_eq!(*total_tokens, 15);
|
||||
@@ -351,7 +316,11 @@ mod tests {
|
||||
assert_eq!(events.len(), 2);
|
||||
match (&events[0], &events[1]) {
|
||||
(
|
||||
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens },
|
||||
StreamEvent::Usage {
|
||||
prompt_tokens,
|
||||
completion_tokens,
|
||||
total_tokens,
|
||||
},
|
||||
StreamEvent::Token(t),
|
||||
) => {
|
||||
assert_eq!(*prompt_tokens, 10);
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
//! Standalone accumulator for streamed tool-call deltas.
|
||||
//!
|
||||
//! Flow: `ToolCallAccumulator::add_delta` is fed incremental `(index, id,
|
||||
//! name, arguments_delta)` chunks as they arrive over SSE → grows its
|
||||
//! internal `Vec<ParsedToolCall>` as needed → `is_complete` reports once
|
||||
//! every accumulated call has both a name and arguments.
|
||||
//!
|
||||
//! Why: mirrors the accumulation logic built into `StreamedTurn::apply_event`
|
||||
//! but as an independent, reusable type for callers that want to track
|
||||
//! tool-call deltas without a full `StreamedTurn` (e.g. a lighter-weight
|
||||
//! preview). Currently unused (`#[allow(dead_code)]`), kept for that future
|
||||
//! use case.
|
||||
|
||||
use super::turn::ParsedToolCall;
|
||||
use serde_json::{json, Value};
|
||||
|
||||
/// Standalone tool-call delta accumulator, functionally equivalent to the accumulation
|
||||
/// logic built into `StreamedTurn::apply_event`. Reserved for callers that want to track
|
||||
/// tool-call deltas independently of a full `StreamedTurn` (e.g. a lighter-weight preview).
|
||||
#[allow(dead_code)]
|
||||
pub struct ToolCallAccumulator {
|
||||
calls: Vec<ParsedToolCall>,
|
||||
}
|
||||
|
||||
#[allow(dead_code)]
|
||||
impl ToolCallAccumulator {
|
||||
/// Construct an empty accumulator with no tool calls tracked yet.
|
||||
///
|
||||
/// Return: a fresh `ToolCallAccumulator`.
|
||||
pub fn new() -> Self {
|
||||
ToolCallAccumulator { calls: Vec::new() }
|
||||
}
|
||||
|
||||
/// Append a delta to the tool call at the given index, growing the
|
||||
/// calls vector if needed.
|
||||
pub fn add_delta(
|
||||
&mut self,
|
||||
index: usize,
|
||||
id: Option<&str>,
|
||||
name: Option<&str>,
|
||||
arguments_delta: &str,
|
||||
) {
|
||||
while self.calls.len() <= index {
|
||||
self.calls.push(ParsedToolCall {
|
||||
id: String::new(),
|
||||
name: String::new(),
|
||||
arguments: String::new(),
|
||||
is_complete: false,
|
||||
});
|
||||
}
|
||||
let tc = &mut self.calls[index];
|
||||
if let Some(new_id) = id {
|
||||
if !new_id.is_empty() {
|
||||
tc.id = new_id.to_string();
|
||||
}
|
||||
}
|
||||
if let Some(new_name) = name {
|
||||
if !new_name.is_empty() {
|
||||
tc.name = new_name.to_string();
|
||||
}
|
||||
}
|
||||
tc.arguments.push_str(arguments_delta);
|
||||
}
|
||||
|
||||
/// Borrow the accumulated tool calls.
|
||||
pub fn calls(&self) -> &[ParsedToolCall] {
|
||||
&self.calls
|
||||
}
|
||||
|
||||
/// Return true once all tool calls have both a name and arguments.
|
||||
pub fn is_complete(&self) -> bool {
|
||||
!self.calls.is_empty() && self.calls.iter().all(|tc| !tc.name.is_empty() && !tc.arguments.is_empty())
|
||||
}
|
||||
|
||||
/// Clear all accumulated calls (starting a fresh turn).
|
||||
pub fn reset(&mut self) {
|
||||
self.calls.clear();
|
||||
}
|
||||
|
||||
/// Build a JSON-serialisable `Vec<Value>` of pending (non-empty-name)
|
||||
/// tool calls, suitable for downstream inspection or replay.
|
||||
pub fn pending_args(&self) -> Vec<Value> {
|
||||
self.calls
|
||||
.iter()
|
||||
.filter(|tc| !tc.name.is_empty())
|
||||
.map(|tc| {
|
||||
json!({
|
||||
"tool_call_id": tc.id,
|
||||
"name": tc.name,
|
||||
"arguments": tc.arguments,
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for ToolCallAccumulator {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
@@ -101,17 +101,7 @@ pub struct ParsedToolCall {
|
||||
pub is_complete: bool,
|
||||
}
|
||||
|
||||
impl ParsedToolCall {
|
||||
/// Attempt to parse the accumulated argument string as JSON before
|
||||
/// the tool call is marked complete — useful for a speculative preview.
|
||||
///
|
||||
/// Return: `Some(Value)` if the arguments are parsable JSON, `None`
|
||||
/// if still partial.
|
||||
#[allow(dead_code)]
|
||||
pub fn try_parse(&self) -> Option<Value> {
|
||||
serde_json::from_str(&self.arguments).ok()
|
||||
}
|
||||
}
|
||||
impl ParsedToolCall {}
|
||||
|
||||
impl StreamedTurn {
|
||||
/// Create an empty turn accumulator.
|
||||
@@ -186,12 +176,12 @@ impl StreamedTurn {
|
||||
let mut msg = if self.tool_calls.is_empty() {
|
||||
ChatMessage::assistant(None)
|
||||
} else {
|
||||
let tool_dtos: Vec<ToolCall> = self.tool_calls
|
||||
let tool_dtos: Vec<ToolCall> = self
|
||||
.tool_calls
|
||||
.iter()
|
||||
.filter(|tc| !tc.name.is_empty())
|
||||
.map(|tc| {
|
||||
let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments)
|
||||
{
|
||||
let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments) {
|
||||
Ok(v) => v,
|
||||
Err(e) => {
|
||||
let repaired = repair_incomplete_json(&tc.arguments);
|
||||
@@ -200,7 +190,8 @@ impl StreamedTurn {
|
||||
tracing::warn!(
|
||||
"[stream] tool call '{}' had truncated JSON \
|
||||
arguments — repaired successfully: {}",
|
||||
tc.name, e,
|
||||
tc.name,
|
||||
e,
|
||||
);
|
||||
v
|
||||
}
|
||||
@@ -209,7 +200,9 @@ impl StreamedTurn {
|
||||
"[stream] tool call '{}' has invalid JSON \
|
||||
arguments: {} (after repair: {}) — falling \
|
||||
back to raw string",
|
||||
tc.name, e, e2,
|
||||
tc.name,
|
||||
e,
|
||||
e2,
|
||||
);
|
||||
serde_json::Value::String(tc.arguments.clone())
|
||||
}
|
||||
@@ -235,7 +228,10 @@ impl StreamedTurn {
|
||||
let full_content = if self.accumulated_reasoning.is_empty() {
|
||||
self.accumulated_content.clone()
|
||||
} else {
|
||||
format!("<think>\n{}\n</think>\n\n{}", self.accumulated_reasoning, self.accumulated_content)
|
||||
format!(
|
||||
"<think>\n{}\n</think>\n\n{}",
|
||||
self.accumulated_reasoning, self.accumulated_content
|
||||
)
|
||||
};
|
||||
let content = if full_content.is_empty() {
|
||||
None
|
||||
@@ -259,7 +255,8 @@ impl StreamedTurn {
|
||||
/// Return: `Some((name, parse_error))` for the first bad tool call, or
|
||||
/// `None` if every tool call's arguments are complete, parsable JSON.
|
||||
pub fn incomplete_tool_call(&self) -> Option<(&str, String)> {
|
||||
self.tool_calls.iter()
|
||||
self.tool_calls
|
||||
.iter()
|
||||
.filter(|tc| !tc.name.is_empty())
|
||||
.find_map(|tc| {
|
||||
serde_json::from_str::<Value>(&tc.arguments)
|
||||
@@ -267,19 +264,6 @@ impl StreamedTurn {
|
||||
.map(|e| (tc.name.as_str(), e.to_string()))
|
||||
})
|
||||
}
|
||||
|
||||
/// Reserved accessor for callers that want to branch mid-stream before the turn
|
||||
/// completes; the current wiring only inspects the final `build_assistant_message()`.
|
||||
#[allow(dead_code)]
|
||||
pub fn has_tool_calls(&self) -> bool {
|
||||
self.tool_calls.iter().any(|tc| !tc.name.is_empty())
|
||||
}
|
||||
|
||||
/// Reserved accessor mirroring `has_tool_calls` for mid-stream content peeks.
|
||||
#[allow(dead_code)]
|
||||
pub fn content(&self) -> &str {
|
||||
&self.accumulated_content
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for StreamedTurn {
|
||||
@@ -353,7 +337,10 @@ mod tests {
|
||||
let tcs = msg.tool_calls.expect("should produce tool calls");
|
||||
assert_eq!(tcs.len(), 1);
|
||||
let args = &tcs[0].function.arguments;
|
||||
assert!(args.is_object(), "args should be an object after repair: {args:?}");
|
||||
assert!(
|
||||
args.is_object(),
|
||||
"args should be an object after repair: {args:?}"
|
||||
);
|
||||
assert_eq!(args.get("path").and_then(|v| v.as_str()), Some("a.txt"));
|
||||
assert_eq!(args.get("content").and_then(|v| v.as_str()), Some("short"));
|
||||
}
|
||||
@@ -361,7 +348,10 @@ mod tests {
|
||||
#[test]
|
||||
fn incomplete_tool_call_flags_truncated_json() {
|
||||
let mut turn = StreamedTurn::new();
|
||||
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm"));
|
||||
turn.tool_calls.push(tool_call(
|
||||
"write",
|
||||
"{\"path\": \"a.txt\", \"content\": \"unterm",
|
||||
));
|
||||
let bad = turn.incomplete_tool_call();
|
||||
assert_eq!(bad.map(|(name, _)| name), Some("write"));
|
||||
}
|
||||
@@ -369,7 +359,10 @@ mod tests {
|
||||
#[test]
|
||||
fn incomplete_tool_call_accepts_complete_json() {
|
||||
let mut turn = StreamedTurn::new();
|
||||
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"done\"}"));
|
||||
turn.tool_calls.push(tool_call(
|
||||
"write",
|
||||
"{\"path\": \"a.txt\", \"content\": \"done\"}",
|
||||
));
|
||||
assert!(turn.incomplete_tool_call().is_none());
|
||||
}
|
||||
|
||||
@@ -386,7 +379,10 @@ mod tests {
|
||||
// so it should still flag truncated JSON even though
|
||||
// `build_assistant_message` will later repair it.
|
||||
let mut turn = StreamedTurn::new();
|
||||
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm"));
|
||||
turn.tool_calls.push(tool_call(
|
||||
"write",
|
||||
"{\"path\": \"a.txt\", \"content\": \"unterm",
|
||||
));
|
||||
// Even though it's repairable, raw parse should still fail
|
||||
assert!(serde_json::from_str::<Value>(&turn.tool_calls[0].arguments).is_err());
|
||||
}
|
||||
|
||||
+212
-9
@@ -27,6 +27,51 @@ impl DirCache {
|
||||
}
|
||||
}
|
||||
|
||||
/// A shared, whole-workspace file-path index used for `@file` mention
|
||||
/// autocomplete. Built once by a background thread at startup (see
|
||||
/// `AppStateRest::new`) and incrementally appended to when tools create
|
||||
/// new files (see `tool/fs/write.rs`).
|
||||
#[derive(Clone)]
|
||||
pub struct MentionIndex {
|
||||
entries: Arc<std::sync::RwLock<Vec<String>>>,
|
||||
}
|
||||
|
||||
impl MentionIndex {
|
||||
/// Create an empty `MentionIndex`.
|
||||
pub fn new() -> Self {
|
||||
MentionIndex {
|
||||
entries: Arc::new(std::sync::RwLock::new(Vec::new())),
|
||||
}
|
||||
}
|
||||
|
||||
/// Replace the indexed paths (used by the startup background walk).
|
||||
pub fn set(&self, paths: Vec<String>) {
|
||||
if let Ok(mut w) = self.entries.write() {
|
||||
*w = paths;
|
||||
}
|
||||
}
|
||||
|
||||
/// Append a single newly created file's path (used by the `write` tool).
|
||||
pub fn push(&self, path: String) {
|
||||
if let Ok(mut w) = self.entries.write() {
|
||||
w.push(path);
|
||||
}
|
||||
}
|
||||
|
||||
/// Take a snapshot of the current indexed paths for fuzzy matching.
|
||||
pub fn snapshot(&self) -> Vec<String> {
|
||||
self.entries.read().map(|r| r.clone()).unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
/// Which source populated the autocomplete dropdown, since selecting a
|
||||
/// candidate is spliced into the buffer differently for each.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum AutocompleteKind {
|
||||
Command,
|
||||
FileMention,
|
||||
}
|
||||
|
||||
/// Manages the viewport scroll offset.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ScrollState {
|
||||
@@ -72,6 +117,8 @@ pub struct InputState {
|
||||
pub autocomplete_candidates: Vec<String>,
|
||||
pub autocomplete_idx: usize,
|
||||
pub autocomplete_visible: bool,
|
||||
pub autocomplete_kind: AutocompleteKind,
|
||||
pub mention_start: usize,
|
||||
pub history_file: Option<PathBuf>,
|
||||
}
|
||||
|
||||
@@ -106,6 +153,8 @@ impl InputState {
|
||||
autocomplete_candidates: Vec::new(),
|
||||
autocomplete_idx: 0,
|
||||
autocomplete_visible: false,
|
||||
autocomplete_kind: AutocompleteKind::Command,
|
||||
mention_start: 0,
|
||||
history_file: None,
|
||||
}
|
||||
}
|
||||
@@ -116,6 +165,8 @@ impl InputState {
|
||||
self.autocomplete_candidates.clear();
|
||||
self.autocomplete_prefix.clear();
|
||||
self.autocomplete_idx = 0;
|
||||
self.autocomplete_kind = AutocompleteKind::Command;
|
||||
self.mention_start = 0;
|
||||
}
|
||||
|
||||
/// Open or refresh the autocomplete dropdown by filtering `COMMANDS`
|
||||
@@ -138,6 +189,54 @@ impl InputState {
|
||||
.map(std::string::ToString::to_string)
|
||||
.collect();
|
||||
self.autocomplete_prefix = prefix;
|
||||
self.autocomplete_kind = AutocompleteKind::Command;
|
||||
self.autocomplete_idx = 0;
|
||||
self.autocomplete_visible = !self.autocomplete_candidates.is_empty();
|
||||
}
|
||||
|
||||
/// Find the `@mention` token (if any) immediately before the cursor.
|
||||
///
|
||||
/// Flow: find the nearest `@` before the cursor → if there's whitespace
|
||||
/// between that `@` and the cursor, no trigger → the `@` only counts as
|
||||
/// a trigger if it's at buffer start or immediately preceded by
|
||||
/// whitespace (so `foo@bar` mid-word never triggers).
|
||||
///
|
||||
/// Return: `Some((byte offset of '@', query text between '@' and cursor))`
|
||||
/// or `None` if the cursor isn't inside a mention token.
|
||||
pub fn mention_query_at_cursor(&self) -> Option<(usize, String)> {
|
||||
let before_cursor = &self.buffer[..self.cursor];
|
||||
let at_pos = before_cursor.rfind('@')?;
|
||||
let between = &before_cursor[at_pos + 1..];
|
||||
if between.chars().any(char::is_whitespace) {
|
||||
return None;
|
||||
}
|
||||
let boundary_ok = at_pos == 0
|
||||
|| before_cursor[..at_pos].chars().next_back().is_some_and(char::is_whitespace);
|
||||
if !boundary_ok {
|
||||
return None;
|
||||
}
|
||||
Some((at_pos, between.to_string()))
|
||||
}
|
||||
|
||||
/// Open or refresh the `@file` mention dropdown from `files`, fuzzy-matched
|
||||
/// against the mention query at the cursor.
|
||||
///
|
||||
/// Flow: `mention_query_at_cursor` finds the trigger `@` and query text →
|
||||
/// if none, close and return → otherwise fuzzy-match `query` against
|
||||
/// `files` via `nucleo-matcher`, keep the top 10 by score.
|
||||
pub fn open_mention_autocomplete(&mut self, files: &[String]) {
|
||||
use nucleo_matcher::{Config, Matcher};
|
||||
use nucleo_matcher::pattern::{CaseMatching, Normalization, Pattern};
|
||||
let Some((start, query)) = self.mention_query_at_cursor() else {
|
||||
self.close_autocomplete();
|
||||
return;
|
||||
};
|
||||
let mut matcher = Matcher::new(Config::DEFAULT.match_paths());
|
||||
let pattern = Pattern::parse(&query, CaseMatching::Smart, Normalization::Smart);
|
||||
let matched_files = pattern.match_list(files.iter(), &mut matcher);
|
||||
self.autocomplete_candidates = matched_files.into_iter().take(10).map(|(f, _)| f.clone()).collect();
|
||||
self.autocomplete_kind = AutocompleteKind::FileMention;
|
||||
self.mention_start = start;
|
||||
self.autocomplete_idx = 0;
|
||||
self.autocomplete_visible = !self.autocomplete_candidates.is_empty();
|
||||
}
|
||||
@@ -154,19 +253,41 @@ impl InputState {
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept the currently selected autocomplete candidate, placing it
|
||||
/// in the buffer and closing the dropdown.
|
||||
/// Accept the currently selected autocomplete candidate.
|
||||
///
|
||||
/// `Command` candidates replace the whole buffer; `FileMention`
|
||||
/// candidates splice `@path ` in at the mention's start position so the
|
||||
/// rest of the sentence around it is preserved.
|
||||
///
|
||||
/// Return: `true` if a candidate was selected, `false` if none existed.
|
||||
pub fn select_autocomplete(&mut self) -> bool {
|
||||
if let Some(candidate) = self.autocomplete_candidates.get(self.autocomplete_idx) {
|
||||
self.buffer = candidate.clone();
|
||||
self.cursor = self.buffer.len();
|
||||
self.close_autocomplete();
|
||||
true
|
||||
} else {
|
||||
false
|
||||
let Some(candidate) = self.autocomplete_candidates.get(self.autocomplete_idx).cloned() else {
|
||||
return false;
|
||||
};
|
||||
match self.autocomplete_kind {
|
||||
AutocompleteKind::Command => {
|
||||
self.buffer = candidate;
|
||||
self.cursor = self.buffer.len();
|
||||
}
|
||||
AutocompleteKind::FileMention => {
|
||||
// Cursor movement (Left/Right) does not close the dropdown, so
|
||||
// by the time Enter is pressed `mention_start` may no longer
|
||||
// describe a valid range against the current cursor/buffer
|
||||
// (e.g. the cursor moved left past the '@'). Splicing on a
|
||||
// stale range would panic (`start > end`) or, even when it
|
||||
// doesn't panic, produce a nonsensical replacement. Treat a
|
||||
// stale mention context the same as "nothing selected".
|
||||
if self.cursor < self.mention_start || self.mention_start > self.buffer.len() {
|
||||
self.close_autocomplete();
|
||||
return false;
|
||||
}
|
||||
let replacement = format!("@{candidate} ");
|
||||
self.buffer.replace_range(self.mention_start..self.cursor, &replacement);
|
||||
self.cursor = self.mention_start + replacement.len();
|
||||
}
|
||||
}
|
||||
self.close_autocomplete();
|
||||
true
|
||||
}
|
||||
|
||||
/// Legacy inline tab-complete — opens the dropdown on first Tab press,
|
||||
@@ -290,6 +411,7 @@ pub struct MiscState {
|
||||
pub tick_count: u64,
|
||||
pub todo_content: String,
|
||||
pub lesson_running: bool,
|
||||
pub pending_clipboard_copy: Option<String>,
|
||||
}
|
||||
|
||||
impl MiscState {
|
||||
@@ -309,6 +431,7 @@ impl MiscState {
|
||||
tick_count: 0,
|
||||
todo_content: String::new(),
|
||||
lesson_running: false,
|
||||
pending_clipboard_copy: None,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -325,3 +448,83 @@ impl MiscState {
|
||||
expired
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn input_with(buffer: &str, cursor: usize) -> InputState {
|
||||
let mut input = InputState::new();
|
||||
input.buffer = buffer.to_string();
|
||||
input.cursor = cursor;
|
||||
input
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mention_at_buffer_start_triggers() {
|
||||
let input = input_with("@mai", 4);
|
||||
assert_eq!(input.mention_query_at_cursor(), Some((0, "mai".to_string())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mention_after_space_mid_sentence_triggers() {
|
||||
let input = input_with("look at @read", 13);
|
||||
assert_eq!(input.mention_query_at_cursor(), Some((8, "read".to_string())));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mid_word_at_does_not_trigger() {
|
||||
let input = input_with("foo@bar", 7);
|
||||
assert_eq!(input.mention_query_at_cursor(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn whitespace_between_at_and_cursor_does_not_trigger() {
|
||||
let input = input_with("@foo bar", 8);
|
||||
assert_eq!(input.mention_query_at_cursor(), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_file_mention_splices_into_buffer() {
|
||||
let mut input = input_with("look at @rea and fix it", 12);
|
||||
input.autocomplete_candidates = vec!["src/main.rs".to_string()];
|
||||
input.autocomplete_idx = 0;
|
||||
input.autocomplete_kind = AutocompleteKind::FileMention;
|
||||
input.mention_start = 8;
|
||||
assert!(input.select_autocomplete());
|
||||
assert_eq!(input.buffer, "look at @src/main.rs and fix it");
|
||||
assert_eq!(input.cursor, 8 + "@src/main.rs ".len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_file_mention_with_stale_cursor_before_mention_start_does_not_panic() {
|
||||
// Simulates: user typed "foo @rea" (mention_start = 4, cursor = 8,
|
||||
// dropdown open), then pressed Left 5 times without closing the
|
||||
// dropdown, moving the cursor to byte 3 (before the '@'). Selecting
|
||||
// now must not panic on `replace_range(4..3, ...)`.
|
||||
let mut input = input_with("foo @rea", 3);
|
||||
input.autocomplete_candidates = vec!["src/main.rs".to_string()];
|
||||
input.autocomplete_idx = 0;
|
||||
input.autocomplete_kind = AutocompleteKind::FileMention;
|
||||
input.mention_start = 4;
|
||||
assert!(!input.select_autocomplete());
|
||||
assert!(!input.autocomplete_visible);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn select_command_still_replaces_whole_buffer() {
|
||||
let mut input = input_with("/mo", 3);
|
||||
input.autocomplete_candidates = vec!["/model".to_string()];
|
||||
input.autocomplete_idx = 0;
|
||||
input.autocomplete_kind = AutocompleteKind::Command;
|
||||
assert!(input.select_autocomplete());
|
||||
assert_eq!(input.buffer, "/model");
|
||||
assert_eq!(input.cursor, "/model".len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn misc_state_starts_with_no_pending_clipboard_copy() {
|
||||
let misc = MiscState::new();
|
||||
assert!(misc.pending_clipboard_copy.is_none());
|
||||
}
|
||||
}
|
||||
|
||||
+52
-2
@@ -9,7 +9,7 @@ use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use tokio::sync::RwLock;
|
||||
|
||||
use super::misc::{DirCache, InputState, MiscState, ScrollState};
|
||||
use super::misc::{DirCache, InputState, MentionIndex, MiscState, ScrollState};
|
||||
use super::runtime::{SessionRuntime, TurnEvent};
|
||||
use super::types::{Origin, Toast, TranscriptCache};
|
||||
use crate::app::lsp::LspManager;
|
||||
@@ -54,6 +54,7 @@ pub struct AppStateRest {
|
||||
pub memory_dir: PathBuf,
|
||||
pub worktrees_dir: PathBuf,
|
||||
pub dir_cache: Arc<RwLock<DirCache>>,
|
||||
pub mention_index: MentionIndex,
|
||||
pub edit_log: EditLog,
|
||||
pub session_runtime: Option<SessionRuntime>,
|
||||
pub sessions: Vec<crate::model::session::Session>,
|
||||
@@ -110,6 +111,7 @@ impl AppStateRest {
|
||||
turn_in_flight: Arc::new(Mutex::new(false)),
|
||||
abort_flag: Arc::new(std::sync::atomic::AtomicBool::new(false)),
|
||||
dir_cache: Arc::new(RwLock::new(dir_cache)),
|
||||
mention_index: MentionIndex::new(),
|
||||
edit_log: EditLog::new(session_dir),
|
||||
session_runtime: Some(SessionRuntime::new(session_dir.to_path_buf())),
|
||||
workflow_engine: WorkflowEngine::new(),
|
||||
@@ -165,7 +167,7 @@ impl AppStateRest {
|
||||
// async executor entirely. It is deliberately not joined -- startup
|
||||
// must not block on language server installation, and failures are
|
||||
// logged rather than surfaced, since editing still works without LSP.
|
||||
if state.settings.lsp_auto_provision {
|
||||
if state.settings.flags.lsp_auto_provision {
|
||||
let lsp_mgr = state.lsp_manager.clone();
|
||||
let msg_queue = state.lsp_provision_msgs.clone();
|
||||
std::thread::spawn(move || {
|
||||
@@ -207,6 +209,53 @@ impl AppStateRest {
|
||||
state
|
||||
}
|
||||
|
||||
/// Spawn the background thread that walks every workspace root and
|
||||
/// populates `mention_index` for `@file` mention autocomplete.
|
||||
///
|
||||
/// Why a separate method, not called from `new()`: the attach-only
|
||||
/// TUI client also constructs an `AppStateRest` (for local rendering
|
||||
/// state) but never runs tools or `handle_key` locally — it forwards
|
||||
/// keystrokes to the daemon over IPC, which has its own `AppStateRest`
|
||||
/// with its own index. Spawning this walk in the attach client would
|
||||
/// waste a full workspace scan for an index nothing there consumes.
|
||||
/// Callers that DO need the index (single-process mode, the daemon)
|
||||
/// call this explicitly after construction.
|
||||
///
|
||||
/// Flow: spawn OS thread -> `ignore::Walk` each workspace root,
|
||||
/// collecting file paths (workspace-index-prefixed for roots beyond
|
||||
/// the first, matching `resolve_path`'s `[N]path` convention) -> stop
|
||||
/// once 50,000 entries are collected -> store the result in
|
||||
/// `mention_index`.
|
||||
///
|
||||
/// Why a raw thread and not a background tokio task: there is no
|
||||
/// persistent async runtime driving the render loop, and this is
|
||||
/// blocking filesystem I/O -- a dedicated thread keeps startup
|
||||
/// non-blocking. Not joined, same rationale as the LSP provisioning
|
||||
/// thread above: a slow/huge repo must not delay the TUI appearing.
|
||||
pub fn spawn_mention_index_build(&self) {
|
||||
let mention_index = self.mention_index.clone();
|
||||
let roots = self.workspace_roots.clone();
|
||||
std::thread::spawn(move || {
|
||||
const MAX_MENTION_ENTRIES: usize = 50_000;
|
||||
let mut paths = Vec::new();
|
||||
'roots: for (i, root) in roots.iter().enumerate() {
|
||||
for entry in ignore::Walk::new(root).flatten() {
|
||||
if !entry.path().is_file() {
|
||||
continue;
|
||||
}
|
||||
let rel = entry.path().strip_prefix(root).unwrap_or(entry.path());
|
||||
let rel_str = rel.display().to_string();
|
||||
let formatted = if i == 0 { rel_str } else { format!("[{i}]{rel_str}") };
|
||||
paths.push(formatted);
|
||||
if paths.len() >= MAX_MENTION_ENTRIES {
|
||||
break 'roots;
|
||||
}
|
||||
}
|
||||
}
|
||||
mention_index.set(paths);
|
||||
});
|
||||
}
|
||||
|
||||
/// Whether an agent turn is currently running.
|
||||
///
|
||||
/// Return: `false` (and logs a warning) if the mutex is poisoned, rather
|
||||
@@ -278,6 +327,7 @@ impl AppStateRest {
|
||||
memory_dir: self.memory_dir.clone(),
|
||||
worktrees_dir: self.worktrees_dir.clone(),
|
||||
dir_cache: self.dir_cache.clone(),
|
||||
mention_index: self.mention_index.clone(),
|
||||
origin,
|
||||
graduated_checks: Vec::new(),
|
||||
lsp_manager: self.lsp_manager.clone(),
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
//! Per-session runtime state: message history, pending tool queue,
|
||||
//! background bash jobs, lesson/review counters, and the `TurnEvent`
|
||||
//! stream emitted while an agent turn is in flight.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// Cumulative token/latency counters for a session, persisted alongside it.
|
||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize, Default)]
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Opaque, serializable snapshot of application state used for
|
||||
//! attach/daemon IPC transfer.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A JSON-boxed snapshot of app state, opaque to the transport layer.
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
//! Shared small state types: toasts, overlays, the transcript cache,
|
||||
//! tool execution model, and call origin tags.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
|
||||
/// Severity/category of a toast notification, used to pick its color.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum ToastKind {
|
||||
|
||||
+72
-40
@@ -15,27 +15,30 @@
|
||||
//! wrote this file, let me check if it's correct before continuing").
|
||||
//! - Background reviews catch broader concerns (missing tests, architectural
|
||||
//! drift, security issues) without blocking the main agent's flow.
|
||||
|
||||
use std::path::Path;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::collections::VecDeque;
|
||||
use crate::app::state::runtime::TurnEvent;
|
||||
use crate::app::subagent::context::build_subagent_context;
|
||||
use crate::app::subagent::engine::run_subagent;
|
||||
use crate::app::subagent::spawn::AgentDefinition;
|
||||
use crate::app::subagent::event::SubagentEvent;
|
||||
use crate::app::subagent::spawn::AgentDefinition;
|
||||
use std::collections::VecDeque;
|
||||
use std::path::Path;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
/// File extensions that should not trigger auto-review (config, lock, data).
|
||||
const SKIP_REVIEW_EXTENSIONS: &[&str] = &[
|
||||
".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml",
|
||||
".svg", ".png", ".jpg", ".ico", ".woff", ".woff2",
|
||||
".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml", ".svg", ".png", ".jpg", ".ico",
|
||||
".woff", ".woff2",
|
||||
];
|
||||
|
||||
/// File names that should not trigger auto-review.
|
||||
const SKIP_REVIEW_FILES: &[&str] = &[
|
||||
"Cargo.lock", "yarn.lock", "package-lock.json",
|
||||
".gitignore", ".env", ".env.example",
|
||||
"Cargo.lock",
|
||||
"yarn.lock",
|
||||
"package-lock.json",
|
||||
".gitignore",
|
||||
".env",
|
||||
".env.example",
|
||||
];
|
||||
|
||||
/// Prevents a second background subagent of the same kind from spawning
|
||||
@@ -137,8 +140,19 @@ fn is_production_code(path: &str) -> bool {
|
||||
.is_some_and(|ext| {
|
||||
matches!(
|
||||
ext,
|
||||
"rs" | "ts" | "tsx" | "js" | "jsx" | "go" | "py" | "java" | "kt" | "swift"
|
||||
| "c" | "cpp" | "h" | "hpp"
|
||||
"rs" | "ts"
|
||||
| "tsx"
|
||||
| "js"
|
||||
| "jsx"
|
||||
| "go"
|
||||
| "py"
|
||||
| "java"
|
||||
| "kt"
|
||||
| "swift"
|
||||
| "c"
|
||||
| "cpp"
|
||||
| "h"
|
||||
| "hpp"
|
||||
)
|
||||
})
|
||||
}
|
||||
@@ -167,11 +181,8 @@ pub fn spawn_quick_review(
|
||||
file_path,
|
||||
);
|
||||
|
||||
let def = AgentDefinition::new(
|
||||
"quick-reviewer".to_string(),
|
||||
"reviewer".to_string(),
|
||||
)
|
||||
.with_system_prompt(prompt);
|
||||
let def = AgentDefinition::new("quick-reviewer".to_string(), "reviewer".to_string())
|
||||
.with_system_prompt(prompt);
|
||||
|
||||
let mut ctx = build_subagent_context(&def);
|
||||
ctx.session_dir = session_dir.to_path_buf();
|
||||
@@ -187,7 +198,7 @@ pub fn spawn_quick_review(
|
||||
SubagentEvent::ToolResult { tool, .. } => {
|
||||
tracing::debug!("[auto-review] tool result: {}", tool);
|
||||
}
|
||||
SubagentEvent::Completed { .. } => {
|
||||
SubagentEvent::Completed => {
|
||||
tracing::debug!("[auto-review] completed");
|
||||
}
|
||||
_ => {}
|
||||
@@ -277,7 +288,10 @@ pub fn spawn_background_test_gen(
|
||||
if file_paths.is_empty() {
|
||||
return;
|
||||
}
|
||||
if TEST_GEN_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
|
||||
if TEST_GEN_RUNNING
|
||||
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
|
||||
.is_err()
|
||||
{
|
||||
tracing::debug!("[bg-test-gen] skipped — a test-gen run is already in flight");
|
||||
return;
|
||||
}
|
||||
@@ -306,8 +320,7 @@ pub fn spawn_background_test_gen(
|
||||
"test-generator".to_string(),
|
||||
"coder".to_string(), // needs write access
|
||||
)
|
||||
.with_system_prompt(prompt)
|
||||
;
|
||||
.with_system_prompt(prompt);
|
||||
|
||||
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-test-gen", Some(&abort_flag));
|
||||
let message = match &result {
|
||||
@@ -347,7 +360,10 @@ pub fn spawn_background_arch_review(
|
||||
if file_paths.is_empty() {
|
||||
return;
|
||||
}
|
||||
if ARCH_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
|
||||
if ARCH_REVIEW_RUNNING
|
||||
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
|
||||
.is_err()
|
||||
{
|
||||
tracing::debug!("[bg-arch-review] skipped — an arch-review run is already in flight");
|
||||
return;
|
||||
}
|
||||
@@ -366,12 +382,8 @@ pub fn spawn_background_arch_review(
|
||||
file_list,
|
||||
);
|
||||
|
||||
let def = AgentDefinition::new(
|
||||
"arch-reviewer".to_string(),
|
||||
"reviewer".to_string(),
|
||||
)
|
||||
.with_system_prompt(prompt)
|
||||
;
|
||||
let def = AgentDefinition::new("arch-reviewer".to_string(), "reviewer".to_string())
|
||||
.with_system_prompt(prompt);
|
||||
|
||||
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-arch-review", Some(&abort_flag));
|
||||
let message = match &result {
|
||||
@@ -422,8 +434,13 @@ pub fn spawn_background_security_review(
|
||||
if prod_paths.is_empty() {
|
||||
return;
|
||||
}
|
||||
if SECURITY_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
|
||||
tracing::debug!("[bg-security-review] skipped — a security-review run is already in flight");
|
||||
if SECURITY_REVIEW_RUNNING
|
||||
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
|
||||
.is_err()
|
||||
{
|
||||
tracing::debug!(
|
||||
"[bg-security-review] skipped — a security-review run is already in flight"
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -441,14 +458,11 @@ pub fn spawn_background_security_review(
|
||||
file_list,
|
||||
);
|
||||
|
||||
let def = AgentDefinition::new(
|
||||
"security-reviewer".to_string(),
|
||||
"reviewer".to_string(),
|
||||
)
|
||||
.with_system_prompt(prompt)
|
||||
;
|
||||
let def = AgentDefinition::new("security-reviewer".to_string(), "reviewer".to_string())
|
||||
.with_system_prompt(prompt);
|
||||
|
||||
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag));
|
||||
let result =
|
||||
run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag));
|
||||
let message = match &result {
|
||||
Ok(output) => {
|
||||
let first = output.lines().next().unwrap_or(output);
|
||||
@@ -493,7 +507,13 @@ pub fn spawn_all_background(
|
||||
.filter(|p| is_production_code(p))
|
||||
.cloned()
|
||||
.collect();
|
||||
spawn_background_test_gen(&source_paths, session_dir, workspaces, turn_events, abort_flag.clone());
|
||||
spawn_background_test_gen(
|
||||
&source_paths,
|
||||
session_dir,
|
||||
workspaces,
|
||||
turn_events,
|
||||
abort_flag.clone(),
|
||||
);
|
||||
|
||||
// Background arch review: for all files that are reviewable
|
||||
let reviewable: Vec<String> = file_paths
|
||||
@@ -501,10 +521,22 @@ pub fn spawn_all_background(
|
||||
.filter(|p| is_reviewable_path(p))
|
||||
.cloned()
|
||||
.collect();
|
||||
spawn_background_arch_review(&reviewable, session_dir, workspaces, turn_events, abort_flag.clone());
|
||||
spawn_background_arch_review(
|
||||
&reviewable,
|
||||
session_dir,
|
||||
workspaces,
|
||||
turn_events,
|
||||
abort_flag.clone(),
|
||||
);
|
||||
|
||||
// Background security review: only production source files
|
||||
spawn_background_security_review(&source_paths, session_dir, workspaces, turn_events, abort_flag);
|
||||
spawn_background_security_review(
|
||||
&source_paths,
|
||||
session_dir,
|
||||
workspaces,
|
||||
turn_events,
|
||||
abort_flag,
|
||||
);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
//! Construction of a `SubagentContext` from an `AgentDefinition`,
|
||||
//! including the default read-only tool set for reviewer agents.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{Arc, Mutex, atomic::AtomicBool};
|
||||
use super::spawn::AgentDefinition;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::{atomic::AtomicBool, Arc, Mutex};
|
||||
|
||||
/// Default read-only tool names granted to `role == "reviewer"` agents.
|
||||
pub const REVIEWER_ALLOWED: &[&str] = &["read", "grep", "glob", "recall", "remember"];
|
||||
@@ -40,7 +39,10 @@ pub struct SubagentContext {
|
||||
pub fn build_subagent_context(def: &AgentDefinition) -> SubagentContext {
|
||||
let allowed_tools = def.allowed_tools.clone().unwrap_or_else(|| {
|
||||
if def.role == "reviewer" {
|
||||
REVIEWER_ALLOWED.iter().map(std::string::ToString::to_string).collect()
|
||||
REVIEWER_ALLOWED
|
||||
.iter()
|
||||
.map(std::string::ToString::to_string)
|
||||
.collect()
|
||||
} else {
|
||||
Vec::new()
|
||||
}
|
||||
|
||||
@@ -18,25 +18,68 @@ pub mod tool_scope {
|
||||
/// Write-tier plus delete, git, and the remaining LSP actions.
|
||||
pub const FULL: &str = "full";
|
||||
|
||||
const READ_TOOLS: &[&str] = &[
|
||||
"read", "grep", "glob", "search", "seqthink", "recall",
|
||||
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
|
||||
"lsp_references", "read_findings",
|
||||
/// The read-only tool set — reused by `context::dedup` as the
|
||||
/// authoritative "safe to deduplicate" classification, so there's a
|
||||
/// single list of read-only tool names in the codebase instead of two.
|
||||
pub const READ_TOOLS: &[&str] = &[
|
||||
"read",
|
||||
"grep",
|
||||
"glob",
|
||||
"search",
|
||||
"seqthink",
|
||||
"recall",
|
||||
"lsp_connect",
|
||||
"lsp_diagnostics",
|
||||
"lsp_hover",
|
||||
"lsp_definition",
|
||||
"lsp_references",
|
||||
"read_findings",
|
||||
];
|
||||
|
||||
const WRITE_TOOLS: &[&str] = &[
|
||||
"read", "grep", "glob", "search", "seqthink", "recall",
|
||||
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
|
||||
"lsp_references", "read_findings",
|
||||
"write", "edit", "bash", "todowrite", "todofinish", "remember",
|
||||
"read",
|
||||
"grep",
|
||||
"glob",
|
||||
"search",
|
||||
"seqthink",
|
||||
"recall",
|
||||
"lsp_connect",
|
||||
"lsp_diagnostics",
|
||||
"lsp_hover",
|
||||
"lsp_definition",
|
||||
"lsp_references",
|
||||
"read_findings",
|
||||
"write",
|
||||
"edit",
|
||||
"bash",
|
||||
"todowrite",
|
||||
"todofinish",
|
||||
"remember",
|
||||
];
|
||||
|
||||
const FULL_TOOLS: &[&str] = &[
|
||||
"read", "grep", "glob", "search", "seqthink", "recall",
|
||||
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
|
||||
"lsp_references", "read_findings",
|
||||
"write", "edit", "bash", "todowrite", "todofinish", "remember",
|
||||
"delete", "git_operator", "lsp_completion", "lsp_disconnect",
|
||||
"read",
|
||||
"grep",
|
||||
"glob",
|
||||
"search",
|
||||
"seqthink",
|
||||
"recall",
|
||||
"lsp_connect",
|
||||
"lsp_diagnostics",
|
||||
"lsp_hover",
|
||||
"lsp_definition",
|
||||
"lsp_references",
|
||||
"read_findings",
|
||||
"write",
|
||||
"edit",
|
||||
"bash",
|
||||
"todowrite",
|
||||
"todofinish",
|
||||
"remember",
|
||||
"delete",
|
||||
"git_operator",
|
||||
"lsp_completion",
|
||||
"lsp_disconnect",
|
||||
];
|
||||
|
||||
/// Resolve a tier name to its concrete tool allowlist.
|
||||
@@ -95,7 +138,13 @@ mod tests {
|
||||
let read: HashSet<_> = tools_for(READ).into_iter().collect();
|
||||
let write: HashSet<_> = tools_for(WRITE).into_iter().collect();
|
||||
let full: HashSet<_> = tools_for(FULL).into_iter().collect();
|
||||
assert!(read.is_subset(&write), "read tier must be a subset of write tier");
|
||||
assert!(write.is_subset(&full), "write tier must be a subset of full tier");
|
||||
assert!(
|
||||
read.is_subset(&write),
|
||||
"read tier must be a subset of write tier"
|
||||
);
|
||||
assert!(
|
||||
write.is_subset(&full),
|
||||
"write tier must be a subset of full tier"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -475,7 +475,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
// drain thread can show it as progress instead of just the tool name.
|
||||
if !content.is_empty() {
|
||||
let _ = tx.blocking_send(SubagentEvent::StepCompleted {
|
||||
step,
|
||||
output: content.clone(),
|
||||
});
|
||||
}
|
||||
@@ -612,7 +611,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
let _ = tx.blocking_send(SubagentEvent::ToolResult {
|
||||
tool: tool_name.clone(),
|
||||
args: args.clone(),
|
||||
output: output_text.clone(),
|
||||
});
|
||||
|
||||
let is_readonly = tool_name == "read"
|
||||
@@ -632,7 +630,7 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
shared_text.truncate(50_000);
|
||||
shared_text.push_str("\n...[truncated]");
|
||||
}
|
||||
f.push(format!("[Auto-Shared] Sibling drone executed '{}' with args {}:\n{}", tool_name, args_json, shared_text));
|
||||
f.push(format!("[Auto-Shared] Sibling drone executed '{tool_name}' with args {args_json}:\n{shared_text}"));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -651,7 +649,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
let _ = tx.blocking_send(SubagentEvent::ToolResult {
|
||||
tool: tool_name.clone(),
|
||||
args: args.clone(),
|
||||
output: msg,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -663,7 +660,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
output.push('\n');
|
||||
}
|
||||
let _ = tx.blocking_send(SubagentEvent::StepCompleted {
|
||||
step,
|
||||
output: content.clone(),
|
||||
});
|
||||
// Break only when we got real content; empty means something went wrong
|
||||
@@ -673,7 +669,7 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
|
||||
}
|
||||
}
|
||||
|
||||
let _ = tx.blocking_send(SubagentEvent::Completed { output: output.clone() });
|
||||
let _ = tx.blocking_send(SubagentEvent::Completed);
|
||||
Ok(output)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Event variants that a running subagent can emit to its parent via the
|
||||
//! shared mpsc channel.
|
||||
|
||||
use serde_json::Value;
|
||||
|
||||
/// Progress and outcome events emitted by `run_subagent` as it processes
|
||||
@@ -8,29 +7,20 @@ use serde_json::Value;
|
||||
#[derive(Debug, Clone)]
|
||||
pub enum SubagentEvent {
|
||||
StepCompleted {
|
||||
#[allow(dead_code)]
|
||||
step: usize,
|
||||
#[allow(dead_code)]
|
||||
output: String,
|
||||
},
|
||||
StepFailed {
|
||||
step: usize,
|
||||
error: String,
|
||||
},
|
||||
Completed {
|
||||
#[allow(dead_code)]
|
||||
output: String,
|
||||
},
|
||||
Completed,
|
||||
ToolCall {
|
||||
tool: String,
|
||||
#[allow(dead_code)]
|
||||
args: Value,
|
||||
},
|
||||
ToolResult {
|
||||
tool: String,
|
||||
args: Value,
|
||||
#[allow(dead_code)]
|
||||
output: String,
|
||||
},
|
||||
Progress(String),
|
||||
/// Token usage reported by the LLM after one streaming call inside the
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Subagent management: spawning, context building, engine loop, and
|
||||
//! progress events.
|
||||
|
||||
pub mod auto;
|
||||
pub mod context;
|
||||
pub mod division;
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! `AgentDefinition` -- declarative specification for instantiating a
|
||||
//! subagent from workflow scripts or programmatic calls.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Declarative specification for instantiating a subagent: name, role,
|
||||
@@ -29,13 +28,6 @@ impl AgentDefinition {
|
||||
}
|
||||
}
|
||||
|
||||
/// Builder method: limit this agent to at most `steps` LLM calls.
|
||||
#[allow(dead_code)]
|
||||
pub fn with_max_steps(mut self, steps: usize) -> Self {
|
||||
self.max_steps = Some(steps);
|
||||
self
|
||||
}
|
||||
|
||||
/// Builder method: set the system prompt for this agent.
|
||||
pub fn with_system_prompt(mut self, prompt: String) -> Self {
|
||||
self.system_prompt = Some(prompt);
|
||||
|
||||
+28
-16
@@ -6,11 +6,10 @@
|
||||
//! choice, so this step is plain Rust — not an LLM call, not a cycle the
|
||||
//! Core Intelligence can omit or reshape — and always runs after any
|
||||
//! hive-mind convergence completes.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::fmt::Write as _;
|
||||
use crate::app::workflow::hive_mind::NodeReport;
|
||||
use crate::model::memory::Memory;
|
||||
use std::fmt::Write as _;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Write a markdown report of one hive-mind convergence to
|
||||
/// `<workspace_root>/docs/runs/<timestamp>-<slug>.md`.
|
||||
@@ -42,22 +41,31 @@ pub fn write_hive_mind_convergence(
|
||||
}
|
||||
|
||||
/// Render a hive-mind convergence as a markdown document.
|
||||
fn render_report(user_request: &str, ts_millis: i64, reports: &[NodeReport], consensus: &str) -> String {
|
||||
fn render_report(
|
||||
user_request: &str,
|
||||
ts_millis: i64,
|
||||
reports: &[NodeReport],
|
||||
consensus: &str,
|
||||
) -> String {
|
||||
let mut out = String::new();
|
||||
writeln!(out, "# The Hive converges: {user_request}").unwrap();
|
||||
writeln!(out, "\nTimestamp (ms): {ts_millis}\n").unwrap();
|
||||
let _ = writeln!(out, "# The Hive converges: {user_request}");
|
||||
let _ = writeln!(out, "\nTimestamp (ms): {ts_millis}\n");
|
||||
|
||||
let cycle_count = reports.iter().map(|r| r.cycle_index).max().map_or(0, |m| m + 1);
|
||||
let cycle_count = reports
|
||||
.iter()
|
||||
.map(|r| r.cycle_index)
|
||||
.max()
|
||||
.map_or(0, |m| m + 1);
|
||||
for cycle_index in 0..cycle_count {
|
||||
writeln!(out, "## Cycle {cycle_index}\n").unwrap();
|
||||
let _ = writeln!(out, "## Cycle {cycle_index}\n");
|
||||
for r in reports.iter().filter(|r| r.cycle_index == cycle_index) {
|
||||
writeln!(out, "### {}\n", r.node_id).unwrap();
|
||||
writeln!(out, "{}\n", r.output).unwrap();
|
||||
let _ = writeln!(out, "### {}\n", r.node_id);
|
||||
let _ = writeln!(out, "{}\n", r.output);
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(out, "## The Hive's Verdict\n").unwrap();
|
||||
writeln!(out, "{consensus}\n").unwrap();
|
||||
let _ = writeln!(out, "## The Hive's Verdict\n");
|
||||
let _ = writeln!(out, "{consensus}\n");
|
||||
out
|
||||
}
|
||||
|
||||
@@ -70,10 +78,14 @@ mod tests {
|
||||
let tmp = std::env::temp_dir().join(format!("zesdex-docs-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&tmp).unwrap();
|
||||
|
||||
let reports = vec![
|
||||
NodeReport { node_id: "Node-0-0".to_string(), cycle_index: 0, output: "found the bug".to_string() },
|
||||
];
|
||||
let path = write_hive_mind_convergence(&tmp, "fix the bug", &reports, "the bug is a null check").unwrap();
|
||||
let reports = vec![NodeReport {
|
||||
node_id: "Node-0-0".to_string(),
|
||||
cycle_index: 0,
|
||||
output: "found the bug".to_string(),
|
||||
}];
|
||||
let path =
|
||||
write_hive_mind_convergence(&tmp, "fix the bug", &reports, "the bug is a null check")
|
||||
.unwrap();
|
||||
|
||||
assert!(path.starts_with(tmp.join("docs").join("runs")));
|
||||
let content = std::fs::read_to_string(&path).unwrap();
|
||||
|
||||
+156
-61
@@ -13,12 +13,14 @@
|
||||
//! `Arc<Mutex<Vec<String>>>` threaded through `execute_primitive` and
|
||||
//! `spawn_single_agent` rather than a global static, preventing data
|
||||
//! leaks between concurrent workflow runs.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex, atomic::{AtomicBool, Ordering}};
|
||||
use std::time::Duration;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use super::script::{ScriptPrimitive, WorkflowScript};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{
|
||||
atomic::{AtomicBool, Ordering},
|
||||
Arc, Mutex,
|
||||
};
|
||||
use std::time::Duration;
|
||||
|
||||
/// The lifecycle state of an agent within a workflow run.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
@@ -100,17 +102,31 @@ pub type LiveStateFn = Arc<dyn Fn(String, String, AgentStatus) + Send + Sync>;
|
||||
/// Return: the agent's text output, or an error on failure.
|
||||
fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value) -> String {
|
||||
let details = match tool {
|
||||
"read" | "view_file" | "write" | "write_to_file" | "edit" | "replace_file_content" | "multi_replace_file_content" | "delete" => {
|
||||
args.get("path")
|
||||
.or_else(|| args.get("TargetFile"))
|
||||
.or_else(|| args.get("AbsolutePath"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string()
|
||||
}
|
||||
"read"
|
||||
| "view_file"
|
||||
| "write"
|
||||
| "write_to_file"
|
||||
| "edit"
|
||||
| "replace_file_content"
|
||||
| "multi_replace_file_content"
|
||||
| "delete" => args
|
||||
.get("path")
|
||||
.or_else(|| args.get("TargetFile"))
|
||||
.or_else(|| args.get("AbsolutePath"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string(),
|
||||
"grep" | "grep_search" => {
|
||||
let pattern = args.get("pattern").or_else(|| args.get("Query")).and_then(|v| v.as_str()).unwrap_or("");
|
||||
let path = args.get("path").or_else(|| args.get("SearchPath")).and_then(|v| v.as_str()).unwrap_or("");
|
||||
let pattern = args
|
||||
.get("pattern")
|
||||
.or_else(|| args.get("Query"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
let path = args
|
||||
.get("path")
|
||||
.or_else(|| args.get("SearchPath"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
if path.is_empty() {
|
||||
format!("\"{pattern}\"")
|
||||
} else {
|
||||
@@ -127,31 +143,44 @@ fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value)
|
||||
}
|
||||
}
|
||||
"bash" | "run_command" => {
|
||||
let cmd = args.get("command").or_else(|| args.get("CommandLine")).and_then(|v| v.as_str()).unwrap_or("");
|
||||
let cmd = args
|
||||
.get("command")
|
||||
.or_else(|| args.get("CommandLine"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("");
|
||||
if cmd.len() > 60 {
|
||||
format!("\"{}...\"", &cmd[..57])
|
||||
} else {
|
||||
format!("\"{cmd}\"")
|
||||
}
|
||||
}
|
||||
"recall" => {
|
||||
args.get("query").and_then(|v| v.as_str()).unwrap_or("").to_string()
|
||||
}
|
||||
"remember" => {
|
||||
args.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string()
|
||||
}
|
||||
"dir_list" | "list_dir" => {
|
||||
args.get("DirectoryPath").or_else(|| args.get("path")).and_then(|v| v.as_str()).unwrap_or("").to_string()
|
||||
}
|
||||
"recall" => args
|
||||
.get("query")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string(),
|
||||
"remember" => args
|
||||
.get("name")
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string(),
|
||||
"dir_list" | "list_dir" => args
|
||||
.get("DirectoryPath")
|
||||
.or_else(|| args.get("path"))
|
||||
.and_then(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string(),
|
||||
_ => {
|
||||
if args.is_object() && !args.as_object().unwrap().is_empty() {
|
||||
args.as_object().unwrap().values()
|
||||
.find_map(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string()
|
||||
} else {
|
||||
String::new()
|
||||
if let Some(obj) = args.as_object() {
|
||||
if !obj.is_empty() {
|
||||
return obj
|
||||
.values()
|
||||
.find_map(|v| v.as_str())
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
};
|
||||
|
||||
@@ -181,7 +210,6 @@ fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value)
|
||||
/// a stuck stage from blocking the entire pipeline forever.
|
||||
///
|
||||
/// Return: the agent's text output, or an error on failure.
|
||||
#[allow(clippy::too_many_lines, clippy::too_many_arguments, clippy::ref_option)]
|
||||
fn spawn_single_agent(
|
||||
agent_id: &str,
|
||||
agent_name: &str,
|
||||
@@ -338,10 +366,13 @@ fn spawn_single_agent(
|
||||
);
|
||||
}
|
||||
}
|
||||
SubagentEvent::Completed { .. } => {
|
||||
SubagentEvent::Completed => {
|
||||
tracing::debug!("[subagent] completed");
|
||||
}
|
||||
SubagentEvent::Usage { tokens_in, tokens_out } => {
|
||||
SubagentEvent::Usage {
|
||||
tokens_in,
|
||||
tokens_out,
|
||||
} => {
|
||||
tracing::debug!("[subagent] usage: {} in, {} out", tokens_in, tokens_out);
|
||||
}
|
||||
}
|
||||
@@ -349,7 +380,10 @@ fn spawn_single_agent(
|
||||
});
|
||||
|
||||
// Check abort before even starting the subagent.
|
||||
if abort_flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
if abort_flag
|
||||
.as_ref()
|
||||
.is_some_and(|f| f.load(Ordering::SeqCst))
|
||||
{
|
||||
anyhow::bail!("subagent '{agent_name}' aborted before start");
|
||||
}
|
||||
|
||||
@@ -380,9 +414,7 @@ fn spawn_single_agent(
|
||||
));
|
||||
}
|
||||
if bg_abort.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
break Err(anyhow::anyhow!(
|
||||
"subagent '{bg_name}' aborted by user",
|
||||
));
|
||||
break Err(anyhow::anyhow!("subagent '{bg_name}' aborted by user"));
|
||||
}
|
||||
}
|
||||
} else {
|
||||
@@ -391,9 +423,7 @@ fn spawn_single_agent(
|
||||
break r;
|
||||
}
|
||||
if bg_abort.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
break Err(anyhow::anyhow!(
|
||||
"subagent '{bg_name}' aborted by user",
|
||||
));
|
||||
break Err(anyhow::anyhow!("subagent '{bg_name}' aborted by user"));
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -467,8 +497,6 @@ type ParallelResult = (usize, anyhow::Result<Vec<String>>);
|
||||
///
|
||||
/// Return: a `Vec<String>` of all agent outputs (or error strings) in
|
||||
/// the order they were submitted.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
#[allow(clippy::ref_option, clippy::too_many_lines)]
|
||||
pub fn execute_primitive(
|
||||
primitive: &ScriptPrimitive,
|
||||
args: &HashMap<String, String>,
|
||||
@@ -501,7 +529,20 @@ pub fn execute_primitive(
|
||||
let resolved = resolve_template(prompt, &resolved_args);
|
||||
let agent_id = uuid::Uuid::new_v4().to_string();
|
||||
let agent_name = resolved.chars().take(40).collect::<String>();
|
||||
match spawn_single_agent(&agent_id, &agent_name, &resolved, "coder", None, &findings_snapshot, findings, abort_flag, live, session_dir, workspaces, timeout_ms) {
|
||||
match spawn_single_agent(
|
||||
&agent_id,
|
||||
&agent_name,
|
||||
&resolved,
|
||||
"coder",
|
||||
None,
|
||||
&findings_snapshot,
|
||||
findings,
|
||||
abort_flag,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
timeout_ms,
|
||||
) {
|
||||
Ok(text) => Ok(vec![text]),
|
||||
Err(e) => {
|
||||
if continue_on_error {
|
||||
@@ -513,7 +554,11 @@ pub fn execute_primitive(
|
||||
}
|
||||
}
|
||||
|
||||
ScriptPrimitive::ScopedAgent { prompt, node_id, tool_scope } => {
|
||||
ScriptPrimitive::ScopedAgent {
|
||||
prompt,
|
||||
node_id,
|
||||
tool_scope,
|
||||
} => {
|
||||
let mut resolved_args = args.clone();
|
||||
let findings_snapshot = findings.lock().map(|f| f.clone()).unwrap_or_default();
|
||||
if !resolved_args.contains_key("findings") {
|
||||
@@ -535,9 +580,24 @@ pub fn execute_primitive(
|
||||
tracing::debug!("[hive] deploying drone {node_id}: {truncated}");
|
||||
let agent_name = format!("{node_id}: {truncated}");
|
||||
let allowed_tools = crate::app::subagent::division::tool_scope::tools_for(tool_scope);
|
||||
match spawn_single_agent(&agent_id, &agent_name, &resolved, node_id, Some(allowed_tools), &findings_snapshot, findings, abort_flag, live, session_dir, workspaces, timeout_ms) {
|
||||
match spawn_single_agent(
|
||||
&agent_id,
|
||||
&agent_name,
|
||||
&resolved,
|
||||
node_id,
|
||||
Some(allowed_tools),
|
||||
&findings_snapshot,
|
||||
findings,
|
||||
abort_flag,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
timeout_ms,
|
||||
) {
|
||||
Ok(text) => {
|
||||
tracing::debug!("[hive] drone {node_id} completed — merging into collective state");
|
||||
tracing::debug!(
|
||||
"[hive] drone {node_id} completed — merging into collective state"
|
||||
);
|
||||
// Merge this drone's complete output into the Hive's
|
||||
// collective state the instant it finishes — not after
|
||||
// the whole parallel cohort completes. Any sibling drone
|
||||
@@ -567,8 +627,7 @@ pub fn execute_primitive(
|
||||
// Each branch shares the same `findings` Arc so note_finding
|
||||
// calls within any branch are visible to all other branches.
|
||||
let semaphore = Arc::new(Semaphore::new(concurrency_cap.max(1)));
|
||||
let results: Arc<Mutex<Vec<ParallelResult>>> =
|
||||
Arc::new(Mutex::new(Vec::new()));
|
||||
let results: Arc<Mutex<Vec<ParallelResult>>> = Arc::new(Mutex::new(Vec::new()));
|
||||
|
||||
let handles: Vec<_> = scripts
|
||||
.iter()
|
||||
@@ -589,7 +648,10 @@ pub fn execute_primitive(
|
||||
std::thread::spawn(move || {
|
||||
let _permit = sem.acquire();
|
||||
let result = execute_primitive(
|
||||
&script, &args, cap, continue_on_error,
|
||||
&script,
|
||||
&args,
|
||||
cap,
|
||||
continue_on_error,
|
||||
&abort,
|
||||
live_clone.as_ref(),
|
||||
&session_dir,
|
||||
@@ -608,7 +670,9 @@ pub fn execute_primitive(
|
||||
let _ = handle.join();
|
||||
}
|
||||
|
||||
let mut locked = results.lock().map_err(|_| anyhow::anyhow!("parallel results lock poisoned"))?;
|
||||
let mut locked = results
|
||||
.lock()
|
||||
.map_err(|_| anyhow::anyhow!("parallel results lock poisoned"))?;
|
||||
locked.sort_by_key(|(idx, _)| *idx);
|
||||
let mut all = Vec::new();
|
||||
for (_, res) in locked.drain(..) {
|
||||
@@ -635,14 +699,28 @@ pub fn execute_primitive(
|
||||
for (idx, script) in scripts.iter().enumerate() {
|
||||
// Check abort before each pipeline stage so we don't
|
||||
// launch the next division after the user cancelled.
|
||||
if abort_flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
|
||||
if abort_flag
|
||||
.as_ref()
|
||||
.is_some_and(|f| f.load(Ordering::SeqCst))
|
||||
{
|
||||
if continue_on_error {
|
||||
all.push(format!("pipeline aborted at stage {idx}"));
|
||||
break;
|
||||
}
|
||||
anyhow::bail!("pipeline aborted by user at stage {idx}");
|
||||
}
|
||||
match execute_primitive(script, args, concurrency_cap, continue_on_error, abort_flag, live, session_dir, workspaces, findings, timeout_ms) {
|
||||
match execute_primitive(
|
||||
script,
|
||||
args,
|
||||
concurrency_cap,
|
||||
continue_on_error,
|
||||
abort_flag,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
findings,
|
||||
timeout_ms,
|
||||
) {
|
||||
Ok(outputs) => all.extend(outputs),
|
||||
Err(e) => {
|
||||
if continue_on_error {
|
||||
@@ -656,9 +734,21 @@ pub fn execute_primitive(
|
||||
Ok(all)
|
||||
}
|
||||
|
||||
ScriptPrimitive::Phase { name: _name, script } => {
|
||||
execute_primitive(script, args, concurrency_cap, continue_on_error, abort_flag, live, session_dir, workspaces, findings, timeout_ms)
|
||||
}
|
||||
ScriptPrimitive::Phase {
|
||||
name: _name,
|
||||
script,
|
||||
} => execute_primitive(
|
||||
script,
|
||||
args,
|
||||
concurrency_cap,
|
||||
continue_on_error,
|
||||
abort_flag,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
findings,
|
||||
timeout_ms,
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -687,7 +777,6 @@ pub fn run_workflow(
|
||||
/// `spawn_agents` invocations remain fully isolated.
|
||||
///
|
||||
/// Return: a human-readable summary string.
|
||||
#[allow(clippy::ref_option)]
|
||||
pub fn run_workflow_tracked(
|
||||
script: &WorkflowScript,
|
||||
args: &HashMap<String, String>,
|
||||
@@ -704,9 +793,15 @@ pub fn run_workflow_tracked(
|
||||
|
||||
let findings = Arc::new(Mutex::new(Vec::new()));
|
||||
let results = execute_primitive(
|
||||
&script.script, args, concurrency_cap,
|
||||
script.options.continue_on_error, abort_flag, live,
|
||||
session_dir, workspaces, &findings,
|
||||
&script.script,
|
||||
args,
|
||||
concurrency_cap,
|
||||
script.options.continue_on_error,
|
||||
abort_flag,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
&findings,
|
||||
script.options.timeout_ms,
|
||||
)?;
|
||||
|
||||
|
||||
+163
-82
@@ -25,12 +25,14 @@
|
||||
//! Synthesis node reads the complete collective state and converges it
|
||||
//! into one unified voice — returned to LO and persisted to docs/runs/*.md.
|
||||
//! ```
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{Arc, Mutex, atomic::{AtomicBool, Ordering}};
|
||||
use serde::Deserialize;
|
||||
use crate::app::workflow::engine::{execute_primitive, AgentStatus, LiveStateFn};
|
||||
use crate::app::workflow::script::ScriptPrimitive;
|
||||
use crate::app::workflow::engine::{execute_primitive, LiveStateFn, AgentStatus};
|
||||
use serde::Deserialize;
|
||||
use std::collections::HashMap;
|
||||
use std::sync::{
|
||||
atomic::{AtomicBool, Ordering},
|
||||
Arc, Mutex,
|
||||
};
|
||||
|
||||
/// One directive the Hive's Core Intelligence issues to a drone within a
|
||||
/// cognitive cycle. A drone's sole identity is its directive and access tier.
|
||||
@@ -85,60 +87,41 @@ pub const HIVE_MIND_CONSENSUS_TAG: &str = "[The Hive speaks]";
|
||||
/// Return: `true` if any prior system message begins with
|
||||
/// `HIVE_MIND_CONSENSUS_TAG`.
|
||||
pub fn hive_mind_already_ran<'a>(system_message_bodies: impl Iterator<Item = &'a str>) -> bool {
|
||||
system_message_bodies.into_iter().any(|body| body.starts_with(HIVE_MIND_CONSENSUS_TAG))
|
||||
system_message_bodies
|
||||
.into_iter()
|
||||
.any(|body| body.starts_with(HIVE_MIND_CONSENSUS_TAG))
|
||||
}
|
||||
|
||||
/// Build the live-state callback that forwards each drone's status to the
|
||||
/// TUI panel so LO can watch the Hive work.
|
||||
fn build_live(
|
||||
turn_events: Option<&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>>,
|
||||
turn_events: Option<
|
||||
&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>,
|
||||
>,
|
||||
) -> Option<LiveStateFn> {
|
||||
turn_events.map(|events| {
|
||||
let events = events.clone();
|
||||
let f: LiveStateFn = Arc::new(move |_agent_id: String, agent_name: String, status: AgentStatus| {
|
||||
let display_name = agent_name.chars().take(40).collect::<String>();
|
||||
if let Ok(mut q) = events.lock() {
|
||||
q.push_back(crate::app::state::runtime::TurnEvent::WorkflowAgentUpdate {
|
||||
agent_id: display_name.clone(),
|
||||
agent_name: display_name,
|
||||
status,
|
||||
});
|
||||
}
|
||||
});
|
||||
let f: LiveStateFn = Arc::new(
|
||||
move |_agent_id: String, agent_name: String, status: AgentStatus| {
|
||||
let display_name = agent_name.chars().take(40).collect::<String>();
|
||||
if let Ok(mut q) = events.lock() {
|
||||
q.push_back(crate::app::state::runtime::TurnEvent::WorkflowAgentUpdate {
|
||||
agent_id: display_name.clone(),
|
||||
agent_name: display_name,
|
||||
status,
|
||||
});
|
||||
}
|
||||
},
|
||||
);
|
||||
f
|
||||
})
|
||||
}
|
||||
|
||||
/// Deploy the Hive: execute a cognitive cycle plan authored by the Core
|
||||
/// Intelligence. Each cycle spawns drones (anonymous processing nodes) in
|
||||
/// parallel. Every drone's complete output merges into the Hive's
|
||||
/// collective state the instant it finishes, and a final synthesis node
|
||||
/// reconciles the entire collective state into one unified voice.
|
||||
/// Context struct threaded through all Hive cycle execution.
|
||||
///
|
||||
/// Flow: for each cycle (sequential) → spawn one `ScriptPrimitive::ScopedAgent`
|
||||
/// per directive, tagged with a system-assigned `node_id` (the Hive's
|
||||
/// coordinate system, never an LLM-chosen name) → run them as a `Parallel`
|
||||
/// block via `execute_primitive`, which merges each drone's output into the
|
||||
/// Hive's shared collective-state Arc the instant that drone completes, not
|
||||
/// after the whole cohort finishes → record `NodeReport`s → proceed to the
|
||||
/// next cycle. After all cycles: spawn one more read-only synthesis node
|
||||
/// whose directive is to converge the complete collective state into a
|
||||
/// single consensus — the Hive becoming one voice — not list what each
|
||||
/// drone said.
|
||||
///
|
||||
/// Concurrency per cycle and the per-drone timeout both come from
|
||||
/// `Settings::load()` (`workflow_max_concurrency`, `hive_mind_node_timeout_ms`)
|
||||
/// rather than a hardcoded cap/no-timeout — a stuck drone can no longer
|
||||
/// stall the entire Hive forever.
|
||||
///
|
||||
/// Return: `(consensus, all_node_reports)` on success. `consensus` is the
|
||||
/// synthesis node's converged output — what the Core Intelligence actually
|
||||
/// hears from the Hive. `all_node_reports` is the complete per-drone record.
|
||||
///
|
||||
/// The convergence doc under `docs/runs/*.md` is written unconditionally
|
||||
/// before this function returns — even when synthesis itself fails — so a
|
||||
/// synthesis error never discards the work already done by cycle drones.
|
||||
/// Callers must not write their own copy of this doc.
|
||||
/// Carries the user request, shared collective state, concurrency limits,
|
||||
/// abort flag, live-status callback, session/workspace paths, and per-drone
|
||||
/// timeout so individual cycle functions don't need long parameter lists.
|
||||
struct CycleCtx<'a> {
|
||||
user_request: &'a str,
|
||||
collective_state: &'a Arc<Mutex<Vec<String>>>,
|
||||
@@ -154,6 +137,8 @@ struct CycleCtx<'a> {
|
||||
///
|
||||
/// Flow: map cycle directives to `ScopedAgent` primitives -> group in a Parallel
|
||||
/// phase block -> run block via `execute_primitive` -> return reports.
|
||||
///
|
||||
/// Return: `Ok(Vec<NodeReport>)` with one report per directive in submission order.
|
||||
fn execute_cycle(
|
||||
cycle_index: usize,
|
||||
directives: &[NodeDirective],
|
||||
@@ -249,12 +234,44 @@ fn execute_cycle(
|
||||
Ok(reports)
|
||||
}
|
||||
|
||||
/// Deploy the Hive: execute a cognitive cycle plan authored by the Core
|
||||
/// Intelligence. Each cycle spawns drones (anonymous processing nodes) in
|
||||
/// parallel. Every drone's complete output merges into the Hive's
|
||||
/// collective state the instant it finishes, and a final synthesis node
|
||||
/// reconciles the entire collective state into one unified voice.
|
||||
///
|
||||
/// Flow: for each cycle (sequential) → spawn one `ScriptPrimitive::ScopedAgent`
|
||||
/// per directive, tagged with a system-assigned `node_id` (the Hive's
|
||||
/// coordinate system, never an LLM-chosen name) → run them as a `Parallel`
|
||||
/// block via `execute_primitive`, which merges each drone's output into the
|
||||
/// Hive's shared collective-state Arc the instant that drone completes, not
|
||||
/// after the whole cohort finishes → record `NodeReport`s → proceed to the
|
||||
/// next cycle. After all cycles: spawn one more read-only synthesis node
|
||||
/// whose directive is to converge the complete collective state into a
|
||||
/// single consensus — the Hive becoming one voice — not list what each
|
||||
/// drone said.
|
||||
///
|
||||
/// Concurrency per cycle and the per-drone timeout both come from
|
||||
/// `Settings::load()` (`workflow_max_concurrency`, `hive_mind_node_timeout_ms`)
|
||||
/// rather than a hardcoded cap/no-timeout — a stuck drone can no longer
|
||||
/// stall the entire Hive forever.
|
||||
///
|
||||
/// Return: `(consensus, all_node_reports)` on success. `consensus` is the
|
||||
/// synthesis node's converged output — what the Core Intelligence actually
|
||||
/// hears from the Hive. `all_node_reports` is the complete per-drone record.
|
||||
///
|
||||
/// The convergence doc under `docs/runs/*.md` is written unconditionally
|
||||
/// before this function returns — even when synthesis itself fails — so a
|
||||
/// synthesis error never discards the work already done by cycle drones.
|
||||
/// Callers must not write their own copy of this doc.
|
||||
pub fn run_hive_mind(
|
||||
user_request: &str,
|
||||
plan: &CognitiveCyclePlan,
|
||||
session_dir: &std::path::Path,
|
||||
workspaces: &[std::path::PathBuf],
|
||||
turn_events: Option<&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>>,
|
||||
turn_events: Option<
|
||||
&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>,
|
||||
>,
|
||||
abort_flag: Option<&Arc<AtomicBool>>,
|
||||
) -> anyhow::Result<(String, Vec<NodeReport>)> {
|
||||
if plan.cycles.is_empty() {
|
||||
@@ -288,20 +305,25 @@ pub fn run_hive_mind(
|
||||
anyhow::bail!("the Hive was recalled by LO before cycle {cycle_index}");
|
||||
}
|
||||
|
||||
tracing::info!("[hive-mind] cycle {cycle_index} deploying {} drone(s)", directives.len());
|
||||
tracing::info!(
|
||||
"[hive-mind] cycle {cycle_index} deploying {} drone(s)",
|
||||
directives.len()
|
||||
);
|
||||
|
||||
let mut cycle_reports = execute_cycle(
|
||||
cycle_index,
|
||||
directives,
|
||||
&ctx,
|
||||
)?;
|
||||
let mut cycle_reports = execute_cycle(cycle_index, directives, &ctx)?;
|
||||
reports.append(&mut cycle_reports);
|
||||
}
|
||||
|
||||
tracing::info!("[hive-mind] all cycles complete — the Hive begins convergence");
|
||||
|
||||
let consensus_result = synthesize_consensus(
|
||||
user_request, session_dir, workspaces, &collective_state, live.as_ref(), abort_flag, node_timeout_ms,
|
||||
user_request,
|
||||
session_dir,
|
||||
workspaces,
|
||||
&collective_state,
|
||||
live.as_ref(),
|
||||
abort_flag,
|
||||
node_timeout_ms,
|
||||
);
|
||||
|
||||
// Guaranteed documentation: write the convergence doc for whatever
|
||||
@@ -311,13 +333,19 @@ pub fn run_hive_mind(
|
||||
// CLAUDE.md promises for every convergence.
|
||||
let doc_consensus = match &consensus_result {
|
||||
Ok(c) => c.clone(),
|
||||
Err(e) => format!(
|
||||
"The Hive's convergence fractured: {e}. Partial node reports above.",
|
||||
),
|
||||
Err(e) => format!("The Hive's convergence fractured: {e}. Partial node reports above."),
|
||||
};
|
||||
if let Some(workspace_root) = workspaces.first() {
|
||||
match crate::app::workflow::docs::write_hive_mind_convergence(workspace_root, user_request, &reports, &doc_consensus) {
|
||||
Ok(path) => tracing::info!("[hive-mind] the Hive's convergence written to {}", path.display()),
|
||||
match crate::app::workflow::docs::write_hive_mind_convergence(
|
||||
workspace_root,
|
||||
user_request,
|
||||
&reports,
|
||||
&doc_consensus,
|
||||
) {
|
||||
Ok(path) => tracing::info!(
|
||||
"[hive-mind] the Hive's convergence written to {}",
|
||||
path.display()
|
||||
),
|
||||
Err(e) => tracing::warn!("[hive-mind] the Hive's convergence report failed: {e}"),
|
||||
}
|
||||
}
|
||||
@@ -373,7 +401,16 @@ fn synthesize_consensus(
|
||||
let args: HashMap<String, String> = HashMap::new();
|
||||
let abort_owned: Option<Arc<AtomicBool>> = abort_flag.cloned();
|
||||
let results = execute_primitive(
|
||||
&synthesis, &args, 1, false, &abort_owned, live, session_dir, workspaces, collective_state, node_timeout_ms,
|
||||
&synthesis,
|
||||
&args,
|
||||
1,
|
||||
false,
|
||||
&abort_owned,
|
||||
live,
|
||||
session_dir,
|
||||
workspaces,
|
||||
collective_state,
|
||||
node_timeout_ms,
|
||||
)?;
|
||||
Ok(results.into_iter().next().unwrap_or_default())
|
||||
}
|
||||
@@ -402,16 +439,31 @@ pub fn is_complex_request(request: &str) -> bool {
|
||||
// Single-line simple update patterns
|
||||
let lower = trimmed.to_lowercase();
|
||||
let negative_keywords = [
|
||||
"simple", "trivial", "typo", "just a", "only a", "minor",
|
||||
"quick", "tiny", "small fix", "rename", "nitpick",
|
||||
"cosmetic", "formatting", "spelling", "grammar",
|
||||
"bump", "version bump", "update comment",
|
||||
"simple",
|
||||
"trivial",
|
||||
"typo",
|
||||
"just a",
|
||||
"only a",
|
||||
"minor",
|
||||
"quick",
|
||||
"tiny",
|
||||
"small fix",
|
||||
"rename",
|
||||
"nitpick",
|
||||
"cosmetic",
|
||||
"formatting",
|
||||
"spelling",
|
||||
"grammar",
|
||||
"bump",
|
||||
"version bump",
|
||||
"update comment",
|
||||
];
|
||||
if negative_keywords.iter().any(|k| lower.contains(k)) {
|
||||
return false;
|
||||
}
|
||||
// Multi-line/multi-sentence → likely complex
|
||||
let sentences = trimmed.split(['.', '!', '?'])
|
||||
let sentences = trimmed
|
||||
.split(['.', '!', '?'])
|
||||
.filter(|s| !s.trim().is_empty())
|
||||
.count();
|
||||
if sentences >= 3 {
|
||||
@@ -419,11 +471,29 @@ pub fn is_complex_request(request: &str) -> bool {
|
||||
}
|
||||
// Positive complexity keywords
|
||||
let complexity_keywords = [
|
||||
"refactor", "redesign", "architecture", "feature", "implement",
|
||||
"migrate", "restructure", "rewrite", "new module", "new component",
|
||||
"scaffold", "multi", "multiple files", "api", "endpoint",
|
||||
"integration", "system", "workflow", "pipeline", "database",
|
||||
"authentication", "authorization", "full stack",
|
||||
"refactor",
|
||||
"redesign",
|
||||
"architecture",
|
||||
"feature",
|
||||
"implement",
|
||||
"migrate",
|
||||
"restructure",
|
||||
"rewrite",
|
||||
"new module",
|
||||
"new component",
|
||||
"scaffold",
|
||||
"multi",
|
||||
"multiple files",
|
||||
"api",
|
||||
"endpoint",
|
||||
"integration",
|
||||
"system",
|
||||
"workflow",
|
||||
"pipeline",
|
||||
"database",
|
||||
"authentication",
|
||||
"authorization",
|
||||
"full stack",
|
||||
];
|
||||
complexity_keywords.iter().any(|k| lower.contains(k))
|
||||
}
|
||||
@@ -445,7 +515,9 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn test_is_complex_request_multi_sentence() {
|
||||
assert!(is_complex_request("This is sentence one. This is sentence two. This is sentence three."));
|
||||
assert!(is_complex_request(
|
||||
"This is sentence one. This is sentence two. This is sentence three."
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -456,9 +528,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn test_default_access_is_read() {
|
||||
let d: NodeDirective = serde_json::from_str(
|
||||
r#"{"directive": "write tests"}"#
|
||||
).unwrap();
|
||||
let d: NodeDirective = serde_json::from_str(r#"{"directive": "write tests"}"#).unwrap();
|
||||
assert_eq!(d.access, crate::app::subagent::division::tool_scope::READ);
|
||||
}
|
||||
|
||||
@@ -468,14 +538,16 @@ mod tests {
|
||||
// "role" key, if an LLM emits one out of old habit, is simply
|
||||
// ignored rather than required or preserved.
|
||||
let d: NodeDirective = serde_json::from_str(
|
||||
r#"{"role": "Architect", "directive": "plan the migration", "access": "read"}"#
|
||||
).unwrap();
|
||||
r#"{"role": "Architect", "directive": "plan the migration", "access": "read"}"#,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(d.directive, "plan the migration");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_cognitive_cycle_plan_arbitrary_shape() {
|
||||
let plan: CognitiveCyclePlan = serde_json::from_str(r#"{
|
||||
let plan: CognitiveCyclePlan = serde_json::from_str(
|
||||
r#"{
|
||||
"cycles": [
|
||||
[{"directive": "scan the codebase topology", "access": "read"}],
|
||||
[
|
||||
@@ -484,7 +556,9 @@ mod tests {
|
||||
],
|
||||
[{"directive": "cut the release", "access": "full"}]
|
||||
]
|
||||
}"#).unwrap();
|
||||
}"#,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(plan.cycles.len(), 3);
|
||||
assert_eq!(plan.cycles[1].len(), 2);
|
||||
}
|
||||
@@ -502,9 +576,12 @@ mod tests {
|
||||
fn test_run_hive_mind_aborts_before_spawning_when_flag_preset() {
|
||||
// The abort check runs before execute_primitive for cycle 0, so a
|
||||
// pre-set abort flag must short-circuit without any LLM/network call.
|
||||
let plan: CognitiveCyclePlan = serde_json::from_str(r#"{
|
||||
let plan: CognitiveCyclePlan = serde_json::from_str(
|
||||
r#"{
|
||||
"cycles": [[{"directive": "whatever", "access": "read"}]]
|
||||
}"#).unwrap();
|
||||
}"#,
|
||||
)
|
||||
.unwrap();
|
||||
let tmp = std::env::temp_dir();
|
||||
let abort_flag = Arc::new(AtomicBool::new(true));
|
||||
let err = run_hive_mind("do something", &plan, &tmp, &[], None, Some(&abort_flag))
|
||||
@@ -526,12 +603,16 @@ mod tests {
|
||||
"you are a helpful assistant".to_string(),
|
||||
format!("{HIVE_MIND_CONSENSUS_TAG}\nthe bug is a null check"),
|
||||
];
|
||||
assert!(hive_mind_already_ran(bodies.iter().map(std::string::String::as_str)));
|
||||
assert!(hive_mind_already_ran(
|
||||
bodies.iter().map(std::string::String::as_str)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn hive_mind_already_ran_false_when_no_prior_convergence() {
|
||||
let bodies = ["you are a helpful assistant".to_string()];
|
||||
assert!(!hive_mind_already_ran(bodies.iter().map(std::string::String::as_str)));
|
||||
assert!(!hive_mind_already_ran(
|
||||
bodies.iter().map(std::string::String::as_str)
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
//! Workflow orchestration: a script interpreter that runs pipeline/parallel
|
||||
//! primitives across multiple subagent instances.
|
||||
|
||||
pub mod hive_mind;
|
||||
pub mod docs;
|
||||
pub mod engine;
|
||||
pub mod hive_mind;
|
||||
pub mod script;
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Script primitives for the workflow engine: agent invocation, parallel
|
||||
//! execution, pipelines, and phases.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A workflow script primitive — can be a single agent, a parallel fan-out,
|
||||
|
||||
+12
-10
@@ -11,10 +11,7 @@ pub enum Command {
|
||||
ClearConfirm,
|
||||
Login { provider: String },
|
||||
Edit(String),
|
||||
McpAdd {
|
||||
name: String,
|
||||
command: String,
|
||||
},
|
||||
McpAdd { name: String, command: String },
|
||||
ModelList,
|
||||
Compact,
|
||||
TodoOpen,
|
||||
@@ -44,13 +41,15 @@ pub fn parse_command(text: &str) -> Command {
|
||||
"/quit" => Command::Quit,
|
||||
"/clear" if arg1.is_empty() => Command::ClearConfirm,
|
||||
"/clear" => Command::Clear,
|
||||
"/login" if arg1.is_empty() => Command::Login { provider: String::new() },
|
||||
"/login" if !arg1.is_empty() => Command::Login { provider: arg1.to_string() },
|
||||
"/login" if arg1.is_empty() => Command::Login {
|
||||
provider: String::new(),
|
||||
},
|
||||
"/login" if !arg1.is_empty() => Command::Login {
|
||||
provider: arg1.to_string(),
|
||||
},
|
||||
"/edit" if !arg1.is_empty() => Command::Edit(arg1.to_string()),
|
||||
"/edit" => Command::Edit(".".to_string()),
|
||||
"/mcp" if arg1.is_empty() => {
|
||||
Command::McpOpen
|
||||
}
|
||||
"/mcp" if arg1.is_empty() => Command::McpOpen,
|
||||
"/mcp" if arg1 == "add" && !arg2.is_empty() => {
|
||||
let rest = arg2.trim();
|
||||
if let Some(space) = rest.find(' ') {
|
||||
@@ -58,7 +57,10 @@ pub fn parse_command(text: &str) -> Command {
|
||||
let command = rest[space + 1..].trim().to_string();
|
||||
Command::McpAdd { name, command }
|
||||
} else {
|
||||
Command::McpAdd { name: rest.to_string(), command: String::new() }
|
||||
Command::McpAdd {
|
||||
name: rest.to_string(),
|
||||
command: String::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
"/model" => Command::ModelList,
|
||||
|
||||
+134
-18
@@ -1,12 +1,12 @@
|
||||
//! Key event dispatcher: maps crossterm `KeyEvent` values into `Action`
|
||||
//! variants, with special handling for overlays, auto-complete, and the
|
||||
//! inline editor.
|
||||
|
||||
use crossterm::event::{KeyCode, KeyEvent, KeyModifiers};
|
||||
|
||||
use crate::app::mode;
|
||||
use crate::app::runtime::actions::Action;
|
||||
use crate::app::runtime::commands::apply_command;
|
||||
use crate::app::state::misc::AutocompleteKind;
|
||||
use crate::app::state::rest::AppStateRest;
|
||||
use crate::app::state::types::Overlay;
|
||||
use crate::controller::command::parse_command;
|
||||
@@ -21,7 +21,6 @@ use crate::controller::command::parse_command;
|
||||
/// Why: when Editor overlay is active, all key events are consumed by the
|
||||
/// editor handler and never reach the main action dispatch. Return `Vec`
|
||||
/// so that a single key press can trigger multiple actions.
|
||||
#[allow(clippy::too_many_lines)]
|
||||
pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
// While Editor overlay is active, route input directly to the editor handler
|
||||
if state.misc.overlay == Overlay::Editor {
|
||||
@@ -81,27 +80,39 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
KeyCode::Up => {
|
||||
let items = crate::app::mode::learning::get_learning_items(state);
|
||||
let n = items.len();
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 {
|
||||
n.saturating_sub(1)
|
||||
} else {
|
||||
state.misc.selected_index - 1
|
||||
};
|
||||
state.dirty = true;
|
||||
return vec![];
|
||||
}
|
||||
KeyCode::Down => {
|
||||
let items = crate::app::mode::learning::get_learning_items(state);
|
||||
let n = items.len();
|
||||
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
|
||||
state.misc.selected_index = if n == 0 {
|
||||
0
|
||||
} else {
|
||||
(state.misc.selected_index + 1) % n
|
||||
};
|
||||
state.dirty = true;
|
||||
return vec![];
|
||||
}
|
||||
KeyCode::Enter | KeyCode::Char('a') => {
|
||||
let items = crate::app::mode::learning::get_learning_items(state);
|
||||
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) = items.get(state.misc.selected_index) {
|
||||
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) =
|
||||
items.get(state.misc.selected_index)
|
||||
{
|
||||
return vec![Action::LessonAccept { name: name.clone() }];
|
||||
}
|
||||
return vec![];
|
||||
}
|
||||
KeyCode::Char('r') => {
|
||||
let items = crate::app::mode::learning::get_learning_items(state);
|
||||
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) = items.get(state.misc.selected_index) {
|
||||
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) =
|
||||
items.get(state.misc.selected_index)
|
||||
{
|
||||
return vec![Action::LessonReject { name: name.clone() }];
|
||||
}
|
||||
return vec![];
|
||||
@@ -131,6 +142,26 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL) => {
|
||||
vec![Action::CloseOverlay]
|
||||
}
|
||||
KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL) => {
|
||||
let last_assistant = state
|
||||
.transcript_cache
|
||||
.messages
|
||||
.iter()
|
||||
.rev()
|
||||
.find(|m| m.role == crate::dto::chat::message::Role::Assistant);
|
||||
match last_assistant {
|
||||
Some(msg) => {
|
||||
state.misc.pending_clipboard_copy = Some(msg.content.clone());
|
||||
}
|
||||
None => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Info,
|
||||
"No assistant message to copy yet".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
KeyCode::Enter => {
|
||||
if state.input.autocomplete_visible {
|
||||
state.input.select_autocomplete();
|
||||
@@ -176,18 +207,24 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
} else if state.misc.overlay == Overlay::Effort {
|
||||
mode::effort::cycle_effort(state);
|
||||
Vec::new()
|
||||
|
||||
} else if state.misc.overlay == Overlay::Rewind {
|
||||
let n = mode::rewind::rewind_count(state);
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 {
|
||||
n.saturating_sub(1)
|
||||
} else {
|
||||
state.misc.selected_index - 1
|
||||
};
|
||||
state.dirty = true;
|
||||
Vec::new()
|
||||
} else if state.misc.overlay == Overlay::ModelSelector {
|
||||
let n = state.app_config.providers.len();
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
|
||||
state.misc.selected_index = if state.misc.selected_index == 0 {
|
||||
n.saturating_sub(1)
|
||||
} else {
|
||||
state.misc.selected_index - 1
|
||||
};
|
||||
state.dirty = true;
|
||||
Vec::new()
|
||||
|
||||
} else if key.modifiers.contains(KeyModifiers::CONTROL) {
|
||||
vec![Action::ScrollUp]
|
||||
} else {
|
||||
@@ -202,18 +239,24 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
} else if state.misc.overlay == Overlay::Effort {
|
||||
mode::effort::cycle_effort(state);
|
||||
Vec::new()
|
||||
|
||||
} else if state.misc.overlay == Overlay::Rewind {
|
||||
let n = mode::rewind::rewind_count(state);
|
||||
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
|
||||
state.misc.selected_index = if n == 0 {
|
||||
0
|
||||
} else {
|
||||
(state.misc.selected_index + 1) % n
|
||||
};
|
||||
state.dirty = true;
|
||||
Vec::new()
|
||||
} else if state.misc.overlay == Overlay::ModelSelector {
|
||||
let n = state.app_config.providers.len();
|
||||
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
|
||||
state.misc.selected_index = if n == 0 {
|
||||
0
|
||||
} else {
|
||||
(state.misc.selected_index + 1) % n
|
||||
};
|
||||
state.dirty = true;
|
||||
Vec::new()
|
||||
|
||||
} else if key.modifiers.contains(KeyModifiers::CONTROL) {
|
||||
vec![Action::ScrollDown]
|
||||
} else {
|
||||
@@ -247,6 +290,11 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
state.input.tab_complete();
|
||||
}
|
||||
state.dirty = true;
|
||||
} else if state.input.autocomplete_kind == AutocompleteKind::FileMention
|
||||
&& state.input.autocomplete_visible
|
||||
{
|
||||
state.input.cycle_autocomplete(true);
|
||||
state.dirty = true;
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
@@ -263,6 +311,10 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
|
||||
// without requiring an extra Tab press.
|
||||
if state.input.buffer.starts_with('/') {
|
||||
state.input.open_autocomplete();
|
||||
} else if state.input.mention_query_at_cursor().is_some() {
|
||||
state
|
||||
.input
|
||||
.open_mention_autocomplete(&state.mention_index.snapshot());
|
||||
}
|
||||
Vec::new()
|
||||
}
|
||||
@@ -301,7 +353,10 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
|
||||
if text.is_empty() {
|
||||
state.settings.api_keys.remove(&state.settings.provider);
|
||||
} else {
|
||||
state.settings.api_keys.insert(state.settings.provider.clone(), text.clone());
|
||||
state
|
||||
.settings
|
||||
.api_keys
|
||||
.insert(state.settings.provider.clone(), text.clone());
|
||||
}
|
||||
let _ = state.settings.save();
|
||||
state.input.buffer.clear();
|
||||
@@ -329,14 +384,24 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
|
||||
if let Some(provider) = providers.get(state.misc.selected_index) {
|
||||
if let Some(cfg) = state.app_config.providers.get(provider) {
|
||||
let model = cfg.default_model.clone().unwrap_or_else(|| {
|
||||
tracing::warn!("[input] provider '{}' has no default_model, using 'claude-opus-4-8'", provider);
|
||||
tracing::warn!(
|
||||
"[input] provider '{}' has no default_model, using 'claude-opus-4-8'",
|
||||
provider
|
||||
);
|
||||
"claude-opus-4-8".to_string()
|
||||
});
|
||||
state.settings.provider.clone_from(provider);
|
||||
state.settings.model.clone_from(&model);
|
||||
if let Some(ref key) = cfg.default_api_key {
|
||||
state.settings.api_keys.insert(provider.clone(), key.clone());
|
||||
} else if let Some(env_key) = cfg.api_key_env.as_ref().and_then(|env| std::env::var(env).ok()) {
|
||||
state
|
||||
.settings
|
||||
.api_keys
|
||||
.insert(provider.clone(), key.clone());
|
||||
} else if let Some(env_key) = cfg
|
||||
.api_key_env
|
||||
.as_ref()
|
||||
.and_then(|env| std::env::var(env).ok())
|
||||
{
|
||||
state.settings.api_keys.insert(provider.clone(), env_key);
|
||||
}
|
||||
let _ = state.settings.save();
|
||||
@@ -363,3 +428,54 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
|
||||
_ => Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn test_state() -> AppStateRest {
|
||||
let tmp = std::env::temp_dir().join(format!("zesdex-input-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&tmp).unwrap();
|
||||
AppStateRest::new(vec![tmp.clone()], &tmp, tmp.join("memory"))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ctrl_y_sets_pending_clipboard_copy_to_last_assistant_message() {
|
||||
let mut state = test_state();
|
||||
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
|
||||
crate::dto::chat::message::Role::User,
|
||||
"hi".to_string(),
|
||||
));
|
||||
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
|
||||
crate::dto::chat::message::Role::Assistant,
|
||||
"first reply".to_string(),
|
||||
));
|
||||
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
|
||||
crate::dto::chat::message::Role::Tool,
|
||||
"tool output".to_string(),
|
||||
));
|
||||
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
|
||||
crate::dto::chat::message::Role::Assistant,
|
||||
"second reply".to_string(),
|
||||
));
|
||||
handle_key(
|
||||
KeyEvent::new(KeyCode::Char('y'), KeyModifiers::CONTROL),
|
||||
&mut state,
|
||||
);
|
||||
assert_eq!(
|
||||
state.misc.pending_clipboard_copy,
|
||||
Some("second reply".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ctrl_y_with_no_assistant_message_pushes_info_toast() {
|
||||
let mut state = test_state();
|
||||
handle_key(
|
||||
KeyEvent::new(KeyCode::Char('y'), KeyModifiers::CONTROL),
|
||||
&mut state,
|
||||
);
|
||||
assert!(state.misc.pending_clipboard_copy.is_none());
|
||||
assert_eq!(state.misc.toasts.len(), 1);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
//! Keyboard input handling and command parsing for the TUI.
|
||||
|
||||
pub mod command;
|
||||
pub mod input;
|
||||
|
||||
+15
-1
@@ -1,6 +1,5 @@
|
||||
//! Chat message types shared across the DTO layer: `Role` and `ChatMessage`
|
||||
//! with convenience constructors.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// The conversation participant who authored a message.
|
||||
@@ -17,6 +16,21 @@ pub enum Role {
|
||||
}
|
||||
|
||||
impl Role {
|
||||
/// Return the role as a lowercase string.
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
Role::User => "user",
|
||||
Role::Assistant => "assistant",
|
||||
Role::System => "system",
|
||||
Role::Tool => "tool",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl std::fmt::Display for Role {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
f.write_str(self.as_str())
|
||||
}
|
||||
}
|
||||
|
||||
/// A single message in a conversation, compatible with the OpenAI/Anthropic
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
//! Chat DTO submodules: message roles/content and tool-call structures.
|
||||
|
||||
pub mod message;
|
||||
pub mod tool;
|
||||
|
||||
+11
-11
@@ -7,7 +7,6 @@
|
||||
//!
|
||||
//! Why: kept separate from `dto::provider` because tool calls are a property
|
||||
//! of a chat *message*, not of the request/response envelope.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
@@ -41,10 +40,7 @@ mod tests {
|
||||
#[test]
|
||||
fn repair_json_bracket_then_brace() {
|
||||
// `[` opened first → `]` must close first, then `}`
|
||||
assert_eq!(
|
||||
repair_json("[[1, 2, {\"a\": 3"),
|
||||
"[[1, 2, {\"a\": 3}]]"
|
||||
);
|
||||
assert_eq!(repair_json("[[1, 2, {\"a\": 3"), "[[1, 2, {\"a\": 3}]]");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -212,7 +208,8 @@ pub fn sanitize_tool_arguments(args: &Value) -> Value {
|
||||
// Attempt 2: strip control chars (0x00-0x1F except \t, \n)
|
||||
// that some LLM providers emit as literal bytes in JSON strings
|
||||
// (e.g. multi-line commit messages), then retry.
|
||||
let cleaned: String = s.chars()
|
||||
let cleaned: String = s
|
||||
.chars()
|
||||
.filter(|&c| !c.is_control() || c == '\t' || c == '\n' || c == '\r')
|
||||
.collect();
|
||||
if cleaned.len() != s.len() {
|
||||
@@ -225,20 +222,23 @@ pub fn sanitize_tool_arguments(args: &Value) -> Value {
|
||||
}
|
||||
}
|
||||
// Attempt 3: repair truncated JSON and retry.
|
||||
let input = if cleaned.len() == s.len() { s } else { &cleaned };
|
||||
let input = if cleaned.len() == s.len() {
|
||||
s
|
||||
} else {
|
||||
&cleaned
|
||||
};
|
||||
let repaired = repair_json(input);
|
||||
match serde_json::from_str::<Value>(&repaired) {
|
||||
Ok(v) => {
|
||||
tracing::warn!(
|
||||
"tool argument string was truncated — repaired successfully",
|
||||
);
|
||||
tracing::warn!("tool argument string was truncated — repaired successfully",);
|
||||
v
|
||||
}
|
||||
Err(e2) => {
|
||||
tracing::error!(
|
||||
"tool argument is a JSON string but failed to parse. \
|
||||
Wrapping in object. Error: {}. Raw (first 200): {}",
|
||||
e2, s.chars().take(200).collect::<String>(),
|
||||
e2,
|
||||
s.chars().take(200).collect::<String>(),
|
||||
);
|
||||
serde_json::json!({"_raw": s, "_parse_error": e2.to_string()})
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Data transfer objects shared across the app: chat messages/tool calls
|
||||
//! and provider request/response/usage shapes.
|
||||
|
||||
pub mod chat;
|
||||
pub mod provider;
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Provider-facing DTOs: chat completion request, response, and usage/cost.
|
||||
|
||||
pub mod request;
|
||||
pub mod response;
|
||||
pub mod usage;
|
||||
|
||||
@@ -8,7 +8,6 @@
|
||||
//! for reserved words like `type`) so no manual (de)serialization glue is
|
||||
//! needed; optional fields use `skip_serializing_if` so unset knobs are
|
||||
//! omitted rather than sent as `null`, matching provider expectations.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
//!
|
||||
//! Why: separate from the streaming SSE path (see `app/runtime/stream/mod.rs`),
|
||||
//! which parses incremental deltas rather than a single complete payload.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Non-streaming chat completion response returned by the provider.
|
||||
|
||||
@@ -4,7 +4,6 @@
|
||||
//! chunk when `stream_options.include_usage` is set, or the `usage` field of
|
||||
//! a non-streaming `ChatResponse`) → surfaced to the TUI for cost/token
|
||||
//! display.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Token counts and optional cost breakdown for a single completion request.
|
||||
|
||||
+1
-2
@@ -4,9 +4,8 @@
|
||||
//! Flow: `IpcClient::connect_unix` opens a `Connection` (see `conn.rs`)
|
||||
//! to the daemon's socket path → `send`/`receive` exchange framed JSON
|
||||
//! messages (typically `ClientRequest`/`DaemonFrame` from `protocol.rs`).
|
||||
|
||||
use anyhow::Result;
|
||||
use super::conn::Connection;
|
||||
use anyhow::Result;
|
||||
|
||||
/// Client-side handle for the `--attach` process: wraps a `Connection`
|
||||
/// to a daemon's Unix socket.
|
||||
|
||||
+2
-3
@@ -6,10 +6,9 @@
|
||||
//! writes it as one length-prefixed frame (`frame::write_frame`) →
|
||||
//! `receive` reads one frame and deserializes it back to the caller's
|
||||
//! type, propagating a clean peer-close as `Ok(None)`.
|
||||
|
||||
use std::os::unix::net::UnixStream;
|
||||
use anyhow::Result;
|
||||
use super::frame;
|
||||
use anyhow::Result;
|
||||
use std::os::unix::net::UnixStream;
|
||||
|
||||
/// A framed Unix-socket connection shared by client and server sides of
|
||||
/// the IPC layer; each `send`/`receive` moves one length-prefixed JSON frame.
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
//! other mismatch is recorded wholesale → results accumulate into a
|
||||
//! `StateDiff`'s `Vec<Change>`, built via `StateDiff::new`/`add_change`
|
||||
//! and reset via `clear`.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
|
||||
+7
-3
@@ -1,4 +1,9 @@
|
||||
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
//! Length-prefixed binary framing and JSON (de)serialization helpers for
|
||||
//! the IPC wire protocol.
|
||||
//!
|
||||
@@ -10,9 +15,8 @@
|
||||
//! Why: a fixed-size length prefix lets the reader know exactly how many
|
||||
//! bytes to pull before attempting to parse, avoiding partial-JSON reads
|
||||
//! over a stream socket.
|
||||
|
||||
use std::io::{Read, Write};
|
||||
use anyhow::Result;
|
||||
use std::io::{Read, Write};
|
||||
|
||||
/// Upper bound on a single frame's byte size (64 MiB), enforced on both
|
||||
/// the write and read paths to bound memory use and reject malformed or
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
//! Unix-socket IPC layer used to connect a `--attach` TUI client to a
|
||||
//! `--daemon` process: length-prefixed framing, connection wrapper,
|
||||
//! client/server handles, and the wire protocol types.
|
||||
|
||||
pub mod client;
|
||||
pub mod conn;
|
||||
pub mod frame;
|
||||
|
||||
+1
-1
@@ -9,7 +9,6 @@
|
||||
//! Why: `StatePayload`/`MessageEntry`/`ToastEntry` are deliberately flat,
|
||||
//! serializable projections of daemon-side state so the client can
|
||||
//! redraw its TUI without sharing any in-process state with the daemon.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Wire-serializable subset of `crossterm::event::KeyCode`, sent from
|
||||
@@ -91,5 +90,6 @@ pub enum DaemonFrame {
|
||||
StateUpdate(Box<StatePayload>),
|
||||
StreamToken(String),
|
||||
SystemNote { kind: String, message: String },
|
||||
ClipboardCopy(String),
|
||||
Closed,
|
||||
}
|
||||
|
||||
+2
-3
@@ -4,10 +4,9 @@
|
||||
//! path (clearing any stale file left by a crashed prior daemon) →
|
||||
//! `accept` blocks for the next client and wraps it as a `Connection`
|
||||
//! (see `conn.rs`) for framed request/response traffic.
|
||||
|
||||
use std::os::unix::net::UnixListener;
|
||||
use anyhow::Result;
|
||||
use super::conn::Connection;
|
||||
use anyhow::Result;
|
||||
use std::os::unix::net::UnixListener;
|
||||
|
||||
/// Server-side handle for the `--daemon` process: listens on a Unix
|
||||
/// socket and hands out `Connection`s to accepted clients.
|
||||
|
||||
@@ -6,7 +6,6 @@
|
||||
//! callers populate/replace its fields as state changes →
|
||||
//! `serialize_snapshot`/`deserialize_snapshot` move it to/from JSON bytes
|
||||
//! for storage or IPC transport.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
|
||||
+123
-41
@@ -109,6 +109,7 @@ fn run_single_process() -> Result<()> {
|
||||
&session_dir,
|
||||
store.memory_dir,
|
||||
);
|
||||
state.spawn_mention_index_build();
|
||||
state.sessions = model::session::Session::list(&store.base_dir);
|
||||
|
||||
|
||||
@@ -119,6 +120,7 @@ fn run_single_process() -> Result<()> {
|
||||
let mut stdout = io::stdout();
|
||||
execute!(stdout, EnterAlternateScreen)?;
|
||||
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
|
||||
execute!(stdout, crossterm::event::EnableMouseCapture)?;
|
||||
let backend = CrosstermBackend::new(stdout);
|
||||
let mut terminal = Terminal::new(backend)?;
|
||||
terminal.clear()?;
|
||||
@@ -127,6 +129,7 @@ fn run_single_process() -> Result<()> {
|
||||
|
||||
let mut restore_stdout = io::stdout();
|
||||
let _ = execute!(restore_stdout, crossterm::event::DisableBracketedPaste);
|
||||
let _ = execute!(restore_stdout, crossterm::event::DisableMouseCapture);
|
||||
let _ = execute!(restore_stdout, LeaveAlternateScreen);
|
||||
let _ = disable_raw_mode();
|
||||
|
||||
@@ -394,6 +397,9 @@ fn handle_daemon_client(
|
||||
running = false;
|
||||
}
|
||||
}
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
conn.send(&ipc::protocol::DaemonFrame::ClipboardCopy(text))?;
|
||||
}
|
||||
send_daemon_update(&mut conn, state)?;
|
||||
}
|
||||
None => {
|
||||
@@ -437,6 +443,7 @@ fn run_daemon() -> Result<()> {
|
||||
&session_dir,
|
||||
store.memory_dir,
|
||||
);
|
||||
state.spawn_mention_index_build();
|
||||
state.sessions = model::session::Session::list(&store.base_dir);
|
||||
|
||||
let _rt = tokio::runtime::Runtime::new()?;
|
||||
@@ -473,6 +480,78 @@ fn run_daemon() -> Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Set up the IPC client connection, terminal, and initial state for attach mode.
|
||||
///
|
||||
/// Flow: resolve socket path → connect → enable raw/alt mode → create state.
|
||||
///
|
||||
/// Return: (client, terminal, `client_state`) on success.
|
||||
fn setup_attach_client(
|
||||
session_id: &str,
|
||||
) -> Result<(
|
||||
ipc::client::IpcClient,
|
||||
Terminal<CrosstermBackend<io::Stdout>>,
|
||||
app::state::rest::AppStateRest,
|
||||
)> {
|
||||
let store = model::store::Store::new();
|
||||
let socket_path = store.base_dir.join("run").join(format!("{session_id}.sock"));
|
||||
let addr = socket_path.to_string_lossy().to_string();
|
||||
let client = ipc::client::IpcClient::connect_unix(&addr)?;
|
||||
|
||||
enable_raw_mode()?;
|
||||
let mut stdout = io::stdout();
|
||||
execute!(stdout, EnterAlternateScreen)?;
|
||||
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
|
||||
execute!(stdout, crossterm::event::EnableMouseCapture)?;
|
||||
let backend = CrosstermBackend::new(stdout);
|
||||
let mut terminal = Terminal::new(backend)?;
|
||||
terminal.clear()?;
|
||||
|
||||
let workspace_roots = vec![std::env::current_dir()?];
|
||||
let session_dir = store.base_dir.join("sessions").join(session_id);
|
||||
std::fs::create_dir_all(&session_dir)?;
|
||||
let mut client_state = app::state::rest::AppStateRest::new(
|
||||
workspace_roots,
|
||||
&session_dir,
|
||||
store.memory_dir,
|
||||
);
|
||||
client_state.session_id = session_id.to_string();
|
||||
|
||||
Ok((client, terminal, client_state))
|
||||
}
|
||||
|
||||
/// Process a single daemon frame from the IPC channel, updating state accordingly.
|
||||
fn handle_daemon_frame(
|
||||
client_state: &mut app::state::rest::AppStateRest,
|
||||
frame: Option<ipc::protocol::DaemonFrame>,
|
||||
) {
|
||||
match frame {
|
||||
Some(ipc::protocol::DaemonFrame::StateUpdate(payload)) => {
|
||||
apply_client_update(client_state, *payload);
|
||||
}
|
||||
Some(ipc::protocol::DaemonFrame::StreamToken(_token)) => {}
|
||||
Some(ipc::protocol::DaemonFrame::SystemNote { kind: _, message }) => {
|
||||
client_state.push_toast(
|
||||
app::state::types::Toast::new(
|
||||
app::state::types::ToastKind::Info,
|
||||
message,
|
||||
),
|
||||
);
|
||||
}
|
||||
Some(ipc::protocol::DaemonFrame::ClipboardCopy(text)) => {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
client_state.push_toast(
|
||||
app::state::types::Toast::new(
|
||||
app::state::types::ToastKind::Success,
|
||||
"Copied to clipboard".to_string(),
|
||||
),
|
||||
);
|
||||
}
|
||||
Some(ipc::protocol::DaemonFrame::Closed) | None => {
|
||||
client_state.quit = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Run zesdex as a TUI-only client attached to an existing daemon session.
|
||||
///
|
||||
/// Flow: connect to the daemon's Unix socket → enter raw mode/alternate
|
||||
@@ -490,30 +569,7 @@ fn run_attach(session_id: &str) -> Result<()> {
|
||||
use crossterm::event::{Event, KeyCode, KeyEventKind, KeyModifiers, MouseEventKind};
|
||||
use ipc::protocol::ClientRequest;
|
||||
|
||||
let store = model::store::Store::new();
|
||||
|
||||
let socket_path = store.base_dir.join("run").join(format!("{session_id}.sock"));
|
||||
let addr = socket_path.to_string_lossy().to_string();
|
||||
let mut client = ipc::client::IpcClient::connect_unix(&addr)?;
|
||||
|
||||
enable_raw_mode()?;
|
||||
let mut stdout = io::stdout();
|
||||
execute!(stdout, EnterAlternateScreen)?;
|
||||
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
|
||||
let backend = CrosstermBackend::new(stdout);
|
||||
let mut terminal = Terminal::new(backend)?;
|
||||
terminal.clear()?;
|
||||
|
||||
let workspace_roots = vec![std::env::current_dir()?];
|
||||
let session_dir = store.base_dir.join("sessions").join(session_id);
|
||||
std::fs::create_dir_all(&session_dir)?;
|
||||
let mut client_state = app::state::rest::AppStateRest::new(
|
||||
workspace_roots,
|
||||
&session_dir,
|
||||
store.memory_dir,
|
||||
);
|
||||
client_state.session_id = session_id.to_string();
|
||||
|
||||
let (mut client, mut terminal, mut client_state) = setup_attach_client(session_id)?;
|
||||
let _rt = tokio::runtime::Runtime::new()?;
|
||||
|
||||
loop {
|
||||
@@ -567,23 +623,10 @@ fn run_attach(session_id: &str) -> Result<()> {
|
||||
client.send(&ClientRequest::Tick)?;
|
||||
}
|
||||
|
||||
match client.receive::<ipc::protocol::DaemonFrame>()? {
|
||||
Some(ipc::protocol::DaemonFrame::StateUpdate(payload)) => {
|
||||
apply_client_update(&mut client_state, *payload);
|
||||
}
|
||||
Some(ipc::protocol::DaemonFrame::StreamToken(_token)) => {}
|
||||
Some(ipc::protocol::DaemonFrame::SystemNote { kind: _, message }) => {
|
||||
client_state.push_toast(
|
||||
app::state::types::Toast::new(
|
||||
app::state::types::ToastKind::Info,
|
||||
message,
|
||||
),
|
||||
);
|
||||
}
|
||||
Some(ipc::protocol::DaemonFrame::Closed) | None => {
|
||||
client_state.quit = true;
|
||||
}
|
||||
}
|
||||
handle_daemon_frame(
|
||||
&mut client_state,
|
||||
client.receive::<ipc::protocol::DaemonFrame>()?,
|
||||
);
|
||||
|
||||
terminal.draw(|f| {
|
||||
view::draw(f, &client_state);
|
||||
@@ -591,6 +634,7 @@ fn run_attach(session_id: &str) -> Result<()> {
|
||||
}
|
||||
|
||||
let _ = execute!(io::stdout(), crossterm::event::DisableBracketedPaste);
|
||||
let _ = execute!(io::stdout(), crossterm::event::DisableMouseCapture);
|
||||
let _ = execute!(io::stdout(), LeaveAlternateScreen);
|
||||
let _ = disable_raw_mode();
|
||||
|
||||
@@ -617,11 +661,27 @@ fn run_loop(
|
||||
|
||||
let _ = disable_raw_mode();
|
||||
let _ = execute!(io::stdout(), crossterm::event::DisableBracketedPaste);
|
||||
let _ = execute!(io::stdout(), crossterm::event::DisableMouseCapture);
|
||||
let _ = execute!(io::stdout(), LeaveAlternateScreen);
|
||||
}
|
||||
result
|
||||
}
|
||||
|
||||
/// Write text to the system clipboard via an OSC52 terminal escape sequence.
|
||||
///
|
||||
/// Flow: base64-encode `text` -> wrap in `\x1b]52;c;<b64>\x07` -> write and
|
||||
/// flush to `stdout`.
|
||||
///
|
||||
/// Why: OSC52 asks the terminal emulator itself to set the clipboard, so no
|
||||
/// OS-level clipboard library (X11/Wayland/win32) is needed. Terminals that
|
||||
/// don't support it silently ignore the sequence.
|
||||
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
|
||||
use base64::Engine as _;
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
|
||||
write!(stdout, "\x1b]52;c;{b64}\x07")?;
|
||||
stdout.flush()
|
||||
}
|
||||
|
||||
/// The core single-process render/input loop.
|
||||
///
|
||||
/// Flow: until `state.quit` → drain expired toasts → draw the frame →
|
||||
@@ -660,6 +720,13 @@ fn run_loop_inner(
|
||||
for action in actions {
|
||||
apply_action(state, action);
|
||||
}
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
state.push_toast(app::state::types::Toast::new(
|
||||
app::state::types::ToastKind::Success,
|
||||
"Copied to clipboard".to_string(),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
Event::Paste(text) => {
|
||||
@@ -694,3 +761,18 @@ fn run_loop_inner(
|
||||
terminal.clear()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::write_osc52;
|
||||
|
||||
#[test]
|
||||
fn write_osc52_formats_the_escape_sequence() {
|
||||
let mut buf: Vec<u8> = Vec::new();
|
||||
write_osc52(&mut buf, "hello").unwrap();
|
||||
use base64::Engine as _;
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode("hello");
|
||||
let expected = format!("\x1b]52;c;{b64}\x07");
|
||||
assert_eq!(String::from_utf8(buf).unwrap(), expected);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
//! Hardcoded built-in subagent definitions (coder, reviewer, researcher, planner).
|
||||
|
||||
use crate::app::subagent::spawn::AgentDefinition;
|
||||
|
||||
/// Build the fixed list of built-in agent definitions shipped with zesdex.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Load, save, and remove user-defined agent definitions stored globally
|
||||
//! (under the store's `agents/` directory), independent of any session.
|
||||
|
||||
use crate::app::subagent::spawn::AgentDefinition;
|
||||
|
||||
/// Load all globally-registered agent definitions from disk.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Agent definition sources: built-in defaults, global (user-wide), and
|
||||
//! per-session overrides.
|
||||
|
||||
pub mod builtin;
|
||||
pub mod global;
|
||||
pub mod session;
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Load, save, add, and remove agent definitions scoped to a single
|
||||
//! session (`<session_dir>/agents.json`).
|
||||
|
||||
use std::path::Path;
|
||||
use crate::app::subagent::spawn::AgentDefinition;
|
||||
|
||||
|
||||
+43
-31
@@ -1,6 +1,5 @@
|
||||
//! Application-level configuration: LLM providers, model roles, and defaults,
|
||||
//! persisted to `app_config.json` in the store directory.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::collections::HashMap;
|
||||
|
||||
@@ -38,26 +37,35 @@ pub struct ModelRole {
|
||||
impl Default for AppConfig {
|
||||
fn default() -> Self {
|
||||
let mut providers = HashMap::new();
|
||||
providers.insert("zen".to_string(), ProviderConfig {
|
||||
api_base: "https://opencode.ai/zen/v1".to_string(),
|
||||
api_key_env: Some("API_KEY".to_string()),
|
||||
default_model: Some("deepseek-v4-flash-free".to_string()),
|
||||
default_api_key: None,
|
||||
});
|
||||
providers.insert("router".to_string(), ProviderConfig {
|
||||
api_base: "https://9router.asepharyana.my.id/v1".to_string(),
|
||||
api_key_env: Some("ROUTER_API_KEY".to_string()),
|
||||
default_model: Some("claude-opus-4-8".to_string()),
|
||||
default_api_key: None,
|
||||
});
|
||||
providers.insert(
|
||||
"zen".to_string(),
|
||||
ProviderConfig {
|
||||
api_base: "https://opencode.ai/zen/v1".to_string(),
|
||||
api_key_env: Some("API_KEY".to_string()),
|
||||
default_model: Some("deepseek-v4-flash-free".to_string()),
|
||||
default_api_key: None,
|
||||
},
|
||||
);
|
||||
providers.insert(
|
||||
"router".to_string(),
|
||||
ProviderConfig {
|
||||
api_base: "https://9router.asepharyana.my.id/v1".to_string(),
|
||||
api_key_env: Some("ROUTER_API_KEY".to_string()),
|
||||
default_model: Some("claude-opus-4-8".to_string()),
|
||||
default_api_key: None,
|
||||
},
|
||||
);
|
||||
let mut model_roles = HashMap::new();
|
||||
model_roles.insert("default".to_string(), ModelRole {
|
||||
provider: "zen".to_string(),
|
||||
model: "deepseek-v4-flash-free".to_string(),
|
||||
max_tokens: None,
|
||||
context_window: None,
|
||||
temperature: Some(0.7),
|
||||
});
|
||||
model_roles.insert(
|
||||
"default".to_string(),
|
||||
ModelRole {
|
||||
provider: "zen".to_string(),
|
||||
model: "deepseek-v4-flash-free".to_string(),
|
||||
max_tokens: None,
|
||||
context_window: None,
|
||||
temperature: Some(0.7),
|
||||
},
|
||||
);
|
||||
AppConfig {
|
||||
providers,
|
||||
model_roles,
|
||||
@@ -89,7 +97,8 @@ impl AppConfig {
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"warning: failed to parse config file '{}': {}. Loading defaults.",
|
||||
path.display(), e
|
||||
path.display(),
|
||||
e
|
||||
);
|
||||
Self::default()
|
||||
}
|
||||
@@ -103,7 +112,9 @@ impl AppConfig {
|
||||
}
|
||||
// Auto-detect provider from ~/.claude/settings.json
|
||||
if let Some(claude_provider) = detect_claude_settings_provider() {
|
||||
cfg.providers.entry("claude".to_string()).or_insert(claude_provider);
|
||||
cfg.providers
|
||||
.entry("claude".to_string())
|
||||
.or_insert(claude_provider);
|
||||
// Register known Claude models as named model roles
|
||||
let claude_models = [
|
||||
("claude-opus-4-8", "claude-opus-4-8"),
|
||||
@@ -111,13 +122,15 @@ impl AppConfig {
|
||||
("claude-haiku-4-5", "claude-haiku-4-5-20251001"),
|
||||
];
|
||||
for (role_name, model_name) in &claude_models {
|
||||
cfg.model_roles.entry(role_name.to_string()).or_insert(ModelRole {
|
||||
provider: "claude".to_string(),
|
||||
model: model_name.to_string(),
|
||||
max_tokens: Some(8192),
|
||||
context_window: Some(200_000),
|
||||
temperature: Some(0.7),
|
||||
});
|
||||
cfg.model_roles
|
||||
.entry(role_name.to_string())
|
||||
.or_insert(ModelRole {
|
||||
provider: "claude".to_string(),
|
||||
model: model_name.to_string(),
|
||||
max_tokens: Some(8192),
|
||||
context_window: Some(200_000),
|
||||
temperature: Some(0.7),
|
||||
});
|
||||
}
|
||||
// Set as default provider only if user hasn't picked a custom default
|
||||
if cfg.default_provider == defaults.default_provider {
|
||||
@@ -154,8 +167,7 @@ struct ClaudeSettings {
|
||||
/// than through its settings file, so reading only the file misses them.
|
||||
fn detect_claude_settings_provider() -> Option<ProviderConfig> {
|
||||
// Prefer the file, then fall back to env vars.
|
||||
let (base_url, key) = claude_credentials_from_file()
|
||||
.or_else(claude_credentials_from_env)?;
|
||||
let (base_url, key) = claude_credentials_from_file().or_else(claude_credentials_from_env)?;
|
||||
Some(ProviderConfig {
|
||||
api_base: base_url,
|
||||
// Keep the env-var name so runtime env overrides still work.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! In-memory conversation state: message history plus the system prompt and
|
||||
//! model parameters used to drive the LLM.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A single conversation's message history and generation settings.
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Append-only JSONL edit log recording every file mutation made by tools,
|
||||
//! for audit and undo/history purposes.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// A single recorded file edit: which tool made it, to which path, why,
|
||||
@@ -44,7 +43,9 @@ impl EditLog {
|
||||
/// regardless of the in-memory limit.
|
||||
fn load_from_disk(path: &std::path::Path) -> Vec<EditLogEntry> {
|
||||
use std::io::{BufRead, BufReader};
|
||||
let Ok(file) = std::fs::File::open(path) else { return Vec::new() };
|
||||
let Ok(file) = std::fs::File::open(path) else {
|
||||
return Vec::new();
|
||||
};
|
||||
let reader = BufReader::new(file);
|
||||
let mut entries: Vec<EditLogEntry> = Vec::new();
|
||||
for line in reader.lines() {
|
||||
@@ -151,7 +152,8 @@ mod tests {
|
||||
bytes_delta: 10 + i,
|
||||
origin: "main".to_string(),
|
||||
session_id: "sess-1".to_string(),
|
||||
}).unwrap();
|
||||
})
|
||||
.unwrap();
|
||||
}
|
||||
assert_eq!(log.len(), 5);
|
||||
assert_eq!(log.entries[0].reason, "reason 0");
|
||||
|
||||
+87
-26
@@ -1,8 +1,7 @@
|
||||
//! Long-term agent memory: markdown files with YAML-ish frontmatter storing
|
||||
//! lessons/references, plus slugified filenames and export/import helpers.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// A single memory entry (lesson, reference, etc.) with frontmatter
|
||||
/// metadata and free-form markdown content.
|
||||
@@ -72,15 +71,30 @@ impl Memory {
|
||||
///
|
||||
/// Return: `Ok(())` on success, or an `io::Error` from directory
|
||||
/// creation, the temp write, or the rename.
|
||||
#[allow(clippy::suspicious_open_options)]
|
||||
pub fn write(&self, memory_dir: &Path) -> std::io::Result<()> {
|
||||
let path = Self::path(memory_dir, &self.name);
|
||||
let parent = path.parent().unwrap();
|
||||
std::fs::create_dir_all(parent)?;
|
||||
let outcome_line = self.outcome.as_ref().map(|o| format!("outcome: {o}")).unwrap_or_default();
|
||||
let scope_line = self.scope.as_ref().map(|s| format!("scope: {s}")).unwrap_or_default();
|
||||
let before_line = self.before_snippet.as_ref().map(|s| format!("before: {s}")).unwrap_or_default();
|
||||
let after_line = self.after_snippet.as_ref().map(|s| format!("after: {s}")).unwrap_or_default();
|
||||
let outcome_line = self
|
||||
.outcome
|
||||
.as_ref()
|
||||
.map(|o| format!("outcome: {o}"))
|
||||
.unwrap_or_default();
|
||||
let scope_line = self
|
||||
.scope
|
||||
.as_ref()
|
||||
.map(|s| format!("scope: {s}"))
|
||||
.unwrap_or_default();
|
||||
let before_line = self
|
||||
.before_snippet
|
||||
.as_ref()
|
||||
.map(|s| format!("before: {s}"))
|
||||
.unwrap_or_default();
|
||||
let after_line = self
|
||||
.after_snippet
|
||||
.as_ref()
|
||||
.map(|s| format!("after: {s}"))
|
||||
.unwrap_or_default();
|
||||
let prov_line = if self.provenances.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
@@ -99,6 +113,7 @@ impl Memory {
|
||||
use std::io::Write;
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.truncate(true)
|
||||
.write(true)
|
||||
.open(&tmp)?;
|
||||
f.write_all(content.as_bytes())?;
|
||||
@@ -139,7 +154,10 @@ impl Memory {
|
||||
let content = content.strip_prefix("---\n").unwrap_or(content);
|
||||
let parts: Vec<&str> = content.splitn(2, "\n---\n").collect();
|
||||
if parts.len() < 2 {
|
||||
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, "missing frontmatter"));
|
||||
return Err(std::io::Error::new(
|
||||
std::io::ErrorKind::InvalidData,
|
||||
"missing frontmatter",
|
||||
));
|
||||
}
|
||||
let front: std::collections::HashMap<String, String> = parts[0]
|
||||
.lines()
|
||||
@@ -153,16 +171,34 @@ impl Memory {
|
||||
name: front.get("name").cloned().unwrap_or_default(),
|
||||
description: front.get("description").cloned().unwrap_or_default(),
|
||||
content: body,
|
||||
kind: front.get("kind").cloned().unwrap_or_else(|| "reference".to_string()),
|
||||
created_at: front.get("created_at").and_then(|v| v.parse().ok()).unwrap_or(0),
|
||||
updated_at: front.get("updated_at").and_then(|v| v.parse().ok()).unwrap_or(0),
|
||||
kind: front
|
||||
.get("kind")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| "reference".to_string()),
|
||||
created_at: front
|
||||
.get("created_at")
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(0),
|
||||
updated_at: front
|
||||
.get("updated_at")
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(0),
|
||||
outcome: front.get("outcome").cloned().filter(|s| !s.is_empty()),
|
||||
lifecycle: front.get("lifecycle").cloned().unwrap_or_else(|| "new".to_string()),
|
||||
lifecycle: front
|
||||
.get("lifecycle")
|
||||
.cloned()
|
||||
.unwrap_or_else(|| "new".to_string()),
|
||||
scope: front.get("scope").cloned().filter(|s| !s.is_empty()),
|
||||
before_snippet: front.get("before").cloned().filter(|s| !s.is_empty()),
|
||||
after_snippet: front.get("after").cloned().filter(|s| !s.is_empty()),
|
||||
provenances: front.get("provenances").cloned()
|
||||
.map(|s| s.split(", ").map(std::string::ToString::to_string).collect())
|
||||
provenances: front
|
||||
.get("provenances")
|
||||
.cloned()
|
||||
.map(|s| {
|
||||
s.split(", ")
|
||||
.map(std::string::ToString::to_string)
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default(),
|
||||
})
|
||||
}
|
||||
@@ -186,13 +222,17 @@ impl Memory {
|
||||
/// Return: slugs (without extension); empty `Vec` if the directory
|
||||
/// can't be read.
|
||||
pub fn list(memory_dir: &Path) -> Vec<String> {
|
||||
let Ok(entries) = std::fs::read_dir(memory_dir) else { return Vec::new() };
|
||||
let Ok(entries) = std::fs::read_dir(memory_dir) else {
|
||||
return Vec::new();
|
||||
};
|
||||
entries
|
||||
.filter_map(std::result::Result::ok)
|
||||
.filter(|e| e.path().extension().is_some_and(|x| x == "md"))
|
||||
.filter_map(|e| {
|
||||
let name = e.file_name().to_string_lossy().to_string();
|
||||
if name == "MEMORY.md" { return None; }
|
||||
if name == "MEMORY.md" {
|
||||
return None;
|
||||
}
|
||||
let slug = name.strip_suffix(".md")?.to_string();
|
||||
Some(slug)
|
||||
})
|
||||
@@ -209,11 +249,22 @@ impl Memory {
|
||||
/// Why: leading-dot stripping specifically blocks accidental hidden
|
||||
/// files and `..`-style traversal attempts embedded in `raw`.
|
||||
pub fn slug_path(memory_dir: &Path, raw: &str) -> PathBuf {
|
||||
let clean: String = raw.chars()
|
||||
.map(|c| if c.is_ascii_alphanumeric() || c == '.' || c == '-' { c } else { '-' })
|
||||
let clean: String = raw
|
||||
.chars()
|
||||
.map(|c| {
|
||||
if c.is_ascii_alphanumeric() || c == '.' || c == '-' {
|
||||
c
|
||||
} else {
|
||||
'-'
|
||||
}
|
||||
})
|
||||
.collect();
|
||||
let clean = clean.trim_start_matches('.').to_string();
|
||||
memory_dir.join(if clean.is_empty() { "memory.md" } else { &clean })
|
||||
memory_dir.join(if clean.is_empty() {
|
||||
"memory.md"
|
||||
} else {
|
||||
&clean
|
||||
})
|
||||
}
|
||||
|
||||
/// Export all memories in `memory_dir` to a single JSON file.
|
||||
@@ -227,11 +278,11 @@ pub fn slug_path(memory_dir: &Path, raw: &str) -> PathBuf {
|
||||
#[cfg(test)]
|
||||
pub fn export_lessons(memory_dir: &Path, output: &Path) -> std::io::Result<()> {
|
||||
let names = Memory::list(memory_dir);
|
||||
let lessons: Vec<Memory> = names.iter()
|
||||
let lessons: Vec<Memory> = names
|
||||
.iter()
|
||||
.filter_map(|n| Memory::read(memory_dir, n).ok())
|
||||
.collect();
|
||||
let data = serde_json::to_string_pretty(&lessons)
|
||||
.map_err(std::io::Error::other)?;
|
||||
let data = serde_json::to_string_pretty(&lessons).map_err(std::io::Error::other)?;
|
||||
// Write to temp, fsync, then rename for crash-safe export
|
||||
let tmp = output.with_extension("json.tmp");
|
||||
std::fs::write(&tmp, data)?;
|
||||
@@ -259,7 +310,8 @@ pub fn import_lessons(memory_dir: &Path, input: &Path) -> std::io::Result<usize>
|
||||
let data = std::fs::read_to_string(input)?;
|
||||
let lessons: Vec<Memory> = serde_json::from_str(&data)
|
||||
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
|
||||
let existing: std::collections::HashSet<String> = Memory::list(memory_dir).into_iter().collect();
|
||||
let existing: std::collections::HashSet<String> =
|
||||
Memory::list(memory_dir).into_iter().collect();
|
||||
let mut imported = 0;
|
||||
for lesson in &lessons {
|
||||
let slug = Memory::slugify(&lesson.name).unwrap_or_default();
|
||||
@@ -282,12 +334,18 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn test_slugify_basic() {
|
||||
assert_eq!(Memory::slugify("Hello World"), Some("hello-world".to_string()));
|
||||
assert_eq!(
|
||||
Memory::slugify("Hello World"),
|
||||
Some("hello-world".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_slugify_special_chars() {
|
||||
assert_eq!(Memory::slugify("Use & Avoid! @#$"), Some("use-avoid".to_string()));
|
||||
assert_eq!(
|
||||
Memory::slugify("Use & Avoid! @#$"),
|
||||
Some("use-avoid".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -375,7 +433,10 @@ mod tests {
|
||||
};
|
||||
mem.write(&dir).unwrap();
|
||||
let names = Memory::list(&dir);
|
||||
assert!(names.contains(&"alpha".to_string()), "list should contain 'alpha', got: {names:?}");
|
||||
assert!(
|
||||
names.contains(&"alpha".to_string()),
|
||||
"list should contain 'alpha', got: {names:?}"
|
||||
);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! Persistence and domain model layer: sessions, conversations, memory,
|
||||
//! message log (`SQLite`), edit log, and app/settings config.
|
||||
|
||||
pub mod app_config;
|
||||
pub mod editlog;
|
||||
pub mod memory;
|
||||
|
||||
+16
-23
@@ -1,8 +1,7 @@
|
||||
//! Binary blob storage in the message-log `SQLite` database (e.g. images,
|
||||
//! attachments), keyed by session id and an arbitrary blob key.
|
||||
|
||||
use rusqlite::{Connection, params};
|
||||
use anyhow::Result;
|
||||
use rusqlite::{params, Connection};
|
||||
|
||||
/// Insert or overwrite a blob for a session under `blob_key`.
|
||||
///
|
||||
@@ -10,7 +9,13 @@ use anyhow::Result;
|
||||
/// keyed on `(session_id, blob_key)`.
|
||||
///
|
||||
/// Return: `Ok(())` on success, or the underlying `SQLite` error.
|
||||
pub fn store_blob(conn: &Connection, session_id: &str, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()> {
|
||||
pub fn store_blob(
|
||||
conn: &Connection,
|
||||
session_id: &str,
|
||||
blob_key: &str,
|
||||
data: &[u8],
|
||||
mime_type: Option<&str>,
|
||||
) -> Result<()> {
|
||||
let created_at = chrono::Utc::now().timestamp_millis();
|
||||
conn.execute(
|
||||
"INSERT OR REPLACE INTO blobs (session_id, blob_key, data, mime_type, created_at) VALUES (?1, ?2, ?3, ?4, ?5)",
|
||||
@@ -23,7 +28,11 @@ pub fn store_blob(conn: &Connection, session_id: &str, blob_key: &str, data: &[u
|
||||
///
|
||||
/// Return: `Ok(Some(data))` if found, `Ok(None)` if no matching row
|
||||
/// exists, `Err` for any other `SQLite` failure.
|
||||
pub fn retrieve_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Result<Option<Vec<u8>>> {
|
||||
pub fn retrieve_blob(
|
||||
conn: &Connection,
|
||||
session_id: &str,
|
||||
blob_key: &str,
|
||||
) -> Result<Option<Vec<u8>>> {
|
||||
let result = conn.query_row(
|
||||
"SELECT data FROM blobs WHERE session_id = ?1 AND blob_key = ?2",
|
||||
params![session_id, blob_key],
|
||||
@@ -36,30 +45,14 @@ pub fn retrieve_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Res
|
||||
}
|
||||
}
|
||||
|
||||
/// Delete a blob for a session by key.
|
||||
///
|
||||
/// Return: `Ok(true)` if a row was deleted, `Ok(false)` if no matching
|
||||
/// row existed.
|
||||
#[allow(dead_code)]
|
||||
pub fn delete_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Result<bool> {
|
||||
let rows = conn.execute(
|
||||
"DELETE FROM blobs WHERE session_id = ?1 AND blob_key = ?2",
|
||||
params![session_id, blob_key],
|
||||
)?;
|
||||
Ok(rows > 0)
|
||||
}
|
||||
|
||||
/// List all blob keys stored for a session, oldest first.
|
||||
///
|
||||
/// Return: `Ok(Vec<String>)` of keys ordered by `created_at`, or the
|
||||
/// underlying `SQLite` error.
|
||||
pub fn list_blob_keys(conn: &Connection, session_id: &str) -> Result<Vec<String>> {
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT blob_key FROM blobs WHERE session_id = ?1 ORDER BY created_at ASC"
|
||||
)?;
|
||||
let rows = stmt.query_map(params![session_id], |row| {
|
||||
row.get::<_, String>(0)
|
||||
})?;
|
||||
let mut stmt =
|
||||
conn.prepare("SELECT blob_key FROM blobs WHERE session_id = ?1 ORDER BY created_at ASC")?;
|
||||
let rows = stmt.query_map(params![session_id], |row| row.get::<_, String>(0))?;
|
||||
let mut keys = Vec::new();
|
||||
for row in rows {
|
||||
keys.push(row?);
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
//! SQLite-backed message log: per-session `messages.sqlite` storing chat
|
||||
//! messages, blobs, and archive/summary metadata.
|
||||
|
||||
pub mod blobs;
|
||||
pub mod query;
|
||||
pub mod schema;
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
//! Insert queries against the message log's `messages` table.
|
||||
|
||||
use rusqlite::{Connection, params};
|
||||
use anyhow::Result;
|
||||
use crate::dto::chat::message::{ChatMessage, Role};
|
||||
use anyhow::Result;
|
||||
use rusqlite::{params, Connection};
|
||||
|
||||
/// Insert a chat message into the session's message log.
|
||||
///
|
||||
@@ -15,9 +14,10 @@ pub fn insert_message(conn: &Connection, session_id: &str, msg: &ChatMessage) ->
|
||||
let content = msg.content.as_deref();
|
||||
let tool_call_id = msg.tool_call_id.as_deref();
|
||||
let tool_name = msg.name.as_deref();
|
||||
let tool_arguments = msg.tool_calls.as_ref().map(|calls| {
|
||||
serde_json::to_string(calls).unwrap_or_default()
|
||||
});
|
||||
let tool_arguments = msg
|
||||
.tool_calls
|
||||
.as_ref()
|
||||
.map(|calls| serde_json::to_string(calls).unwrap_or_default());
|
||||
let created_at = chrono::Utc::now().timestamp_millis();
|
||||
let role_str = match msg.role {
|
||||
Role::User => "user",
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user