Compare commits

...
54 Commits
Author SHA1 Message Date
semantic-release-bot 86cc412395 chore(release): 1.14.0 [skip ci]
# [1.14.0](https://github.com/asepharyana/zesdex/compare/v1.13.0...v1.14.0) (2026-07-16)

### Bug Fixes

* **context:** batasi squash_log ke tool bash saja ([7d99cd6](https://github.com/asepharyana/zesdex/commit/7d99cd66187b3fafb2ddeb19d8e8aa7db139df64))
* **context:** perbaiki fixture test shaping agar men-drop pesan lama ([8bb697a](https://github.com/asepharyana/zesdex/commit/8bb697a53fe89c9d00f709df33ccbd7944482f07))
* **plan:** perbaiki bug entropy gate dan fixture test squash.rs ([ceb8479](https://github.com/asepharyana/zesdex/commit/ceb84790bb741352b2b3416cda4bdbc6b42767c9))
* **plan:** perbaiki fixture test array-cutoff squash_json ([7ffdf44](https://github.com/asepharyana/zesdex/commit/7ffdf441355cda0972b849bc9b475d3de8f39843))
* **plan:** perbaiki fixture test shaping agar benar-benar men-drop pesan ([3a6e32d](https://github.com/asepharyana/zesdex/commit/3a6e32d8f3f0dd47d2c5ff63ab0c184b0ee4a6bc))
* **plan:** perkuat fixture test log agar benar-benar uji squash_log ([682007a](https://github.com/asepharyana/zesdex/commit/682007a2507f209c2378796442d4bc879516191a))

### Features

* **context:** tambah context::dedup untuk hasil tool yang berulang ([12a03fd](https://github.com/asepharyana/zesdex/commit/12a03fd3d1123c7a88289e44064537a71b9f574b))
* **context:** tambah context::shaping (port dari shortsend) ([e080d9f](https://github.com/asepharyana/zesdex/commit/e080d9fc6b7daf2ab37fb242d03a3f64e8629acc))
* **context:** tambah context::squash untuk kompresi hasil tool ([ea0b498](https://github.com/asepharyana/zesdex/commit/ea0b4988299894e4588d851ebc91704a9e73bc73))
* **context:** tambah context::tokens dengan tiktoken-rs ([c219b6e](https://github.com/asepharyana/zesdex/commit/c219b6ec58ddad4770e491a48bbe1b6e3d0c8884))
* **context:** tambah context::window::resolve ([059ca8e](https://github.com/asepharyana/zesdex/commit/059ca8ea246768dbec06ae5336433853e4276cec))
* **runtime:** kompres hasil tool lewat squash sebelum masuk context ([96084f7](https://github.com/asepharyana/zesdex/commit/96084f74621636a72c209388580d0ebbb335d198))
* **settings:** tambah mode ringkas opsional (concise_output) ([d6de973](https://github.com/asepharyana/zesdex/commit/d6de9735aba7745f5f57a8e7a5e625613c1a2569))
2026-07-16 01:01:45 +00:00
asepharyana 1afca97c1b refactor(subagent): remove unused step parameter from SubagentEvent 2026-07-16 07:58:34 +07:00
asepharyana a00aa9bec8 Refactor view modules for improved readability and consistency
- Updated markdown rendering logic to use more concise methods for obtaining vector lengths.
- Changed review status display to use the correct flag from settings.
- Cleaned up sidebar rendering code for better formatting and readability.
- Enhanced status bar rendering with improved string formatting and consistent style application.
- Refined workflow panel rendering, ensuring consistent style usage and improved readability.
- Added architecture overview and detailed documentation for backend, data, dependencies, and frontend structures.
2026-07-16 07:56:11 +07:00
asepharyanaandClaude Sonnet 5 7d99cd6618 fix(context): batasi squash_log ke tool bash saja
Ditemukan reviewer whole-branch final: looks_log_shaped murni berbasis
konten (>=3 baris berpola error/warn/fail), jadi hasil grep/search yang
match ke kode error-handling ikut lolos ambang itu -- padahal
squash_log punya cap keras 20 error + 10 warning tanpa budget byte,
diam-diam membuang match yang sah di luar cap itu. Sekarang hanya tool
bash (penghasil log sungguhan) yang boleh lewat squash_log; tool lain
yang kebetulan konten-nya mirip log jatuh ke squash_generic yang lebih
longgar (head/tail + budget byte). Tambah test regresi yang membedakan
kedua jalur lewat retensi baris terakhir.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:56:11 +07:00
asepharyana e075eb7acc style(context): bersihkan warning clippy pedantic di squash.rs dan dedup.rs
Perbaikan gaya murni (format! di-append ke String -> write!, syntax
perbandingan yang lebih jelas, backtick di doc comment) tanpa
mengubah perilaku -- semua test tetap hijau.
2026-07-16 07:56:11 +07:00
asepharyana d6de9735ab feat(settings): tambah mode ringkas opsional (concise_output)
Off by default, diaktifkan lewat settings.json (belum ada UI toggle —
sama seperti review_enabled/session_archive_enabled/lsp_auto_provision
yang juga cuma bisa diedit manual hari ini). Saat aktif, system prompt
diberi instruksi menulis ringkas, dengan pengecualian eksplisit untuk
konfirmasi operasi destruktif dan peringatan keamanan yang tetap harus
detail penuh.
2026-07-16 07:56:11 +07:00
asepharyana 85d16ebe97 refactor(status): pakai context::tokens dan context::window
Status bar sekarang memakai penghitungan token yang sama persis dengan
compaction (bukan heuristik /4 terpisah), dan selalu menampilkan angka
context window nyata alih-alih '?' saat model role tidak override
context_window secara eksplisit — konsisten dengan fallback yang
sudah dipakai compaction sendiri.
2026-07-16 07:56:11 +07:00
asepharyana 96084f7462 feat(runtime): kompres hasil tool lewat squash sebelum masuk context
Salinan yang dikirim ke UI (TurnEvent::ToolResult) tetap utuh; hanya
salinan yang masuk riwayat percakapan (dikirim ke LLM) yang dikompres,
supaya user tetap melihat output tool apa adanya.
2026-07-16 07:56:11 +07:00
asepharyanaandClaude Sonnet 5 82c024a854 docs(runtime): perbarui doc comment run_agent_turn yang basi
Masih menyebut shortsend padahal modul itu sudah dihapus dan diganti
context::dedup/context::shaping di commit sebelumnya.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:56:11 +07:00
asepharyana 78e402cf14 refactor(runtime): pindah ke context::, perbaiki asimetri /compact manual
Loop auto-compaction sekarang selalu menjalankan dedup tiap iterasi
lalu shaping lewat context::, menggantikan shortsend:: yang dihapus.

Action::Compact tadinya berjalan sinkron dan selalu client: None,
sehingga hasil compact manual tidak pernah diringkas LLM (beda dengan
compaction otomatis di tengah turn). Sekarang /compact jalan di
thread background seperti spawn_turn, sehingga bisa memanggil LLM
untuk meringkas riwayat yang dibuang — perilaku manual dan otomatis
jadi setara.

Ekstrak resolve_llm_client_config() dari spawn_turn supaya logika
resolusi api_key/model/base_url tidak dua kali.
2026-07-16 07:56:11 +07:00
asepharyanaandClaude Sonnet 5 3a6e32d8f3 fix(plan): perbaiki fixture test shaping agar benar-benar men-drop pesan
Ditemukan implementer Task 5: fixture pesan pendek (~8 token nyata
lewat tiktoken untuk 20 pesan = ~160 token) tidak pernah melebihi
target 700 token (70% dari max_wire_tokens=1000), jadi force=true pun
tidak pernah men-drop satu pesan pun -- test placeholder-summary dan
keeps-most-recent lulus secara vakum tanpa benar-benar menguji jalur
drop. Diverifikasi ulang dengan context::tokens::count_tokens nyata:
fixture baru (20 pesan ~49 token = ~980 token total) melebihi target
700 dengan nyaman.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:55:48 +07:00
asepharyana e080d9fc6b feat(context): tambah context::shaping (port dari shortsend)
Perilaku should_shape/shape_messages tidak berubah, hanya sumber
penghitungan token yang sekarang lewat context::tokens (tiktoken-rs)
menggantikan heuristik char/3 bawaannya sendiri.
2026-07-16 07:55:48 +07:00
asepharyana ea0b498829 feat(context): tambah context::squash untuk kompresi hasil tool
Kompresi per-jenis-konten (JSON: pertahankan struktur & value pendek/
entropi tinggi, buang value panjang bertele-tele; log: simpan baris
error/warning berskor tertinggi + konteks sekitarnya; generic: potong
importance-ranked) untuk hasil tool di atas 1.5KB. Tool read dikecualikan
total karena isinya harus tetap byte-exact untuk edit selanjutnya.
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 682007a250 fix(plan): perkuat fixture test log agar benar-benar uji squash_log
Ditemukan lewat trace manual (bukan implementer): fixture lama cuma
punya 1 baris berpola error, di bawah ambang looks_log_shaped (>=3),
jadi tanpa sadar selalu jatuh ke squash_generic -- test tetap lulus
tapi tidak pernah menguji logika scoring/windowing squash_log sama
sekali. Tambah baris error/warning lagi supaya jalur squash_log
benar-benar terpakai.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 7ffdf44135 fix(plan): perbaiki fixture test array-cutoff squash_json
Ditemukan implementer Task 4 (percobaan kedua, lagi-lagi BLOCKED
sebelum commit apapun): test sebelumnya memakai string sama yang
panjang+entropi-rendah di semua 4 elemen array, jadi elemen index
0-2 pun ikut ter-elide oleh aturan panjang/entropi normal -- tidak
benar-benar menguji efek posisi array. Ganti fixture pakai string
berbentuk UUID (tanpa spasi, entropi tinggi) yang lolos aturan normal
di posisi manapun, supaya index 3 yang di-force-elide walau
identifier-shaped benar-benar membuktikan aturan "past index 3
regardless of length/entropy".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 ceb84790bb fix(plan): perbaiki bug entropy gate dan fixture test squash.rs
Ditemukan implementer Task 4 sebelum commit apapun (BLOCKED, bukan
kode salah): entropi Shannon mentah per-karakter tidak membedakan
prosa dari identifier acak — prosa berulang skor ~3.89 bit/char,
lebih tinggi dari UUID (~3.39). Tambah syarat "tanpa spasi" sebelum
cek entropi (meniru pre-filter headroom sendiri), turunkan ambang ke
3.0 pada skala mentah. Fixture test array JSON juga diperbesar
(repeat 5 -> 8) karena sebelumnya tidak pernah melewati SQUASH_FLOOR_BYTES
yang diasumsikan test itu sendiri.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 12a03fd3d1 feat(context): tambah context::dedup untuk hasil tool yang berulang
Panggilan tool read-only (read, grep, glob, dst) dengan argumen persis
sama menyisakan satu salinan penuh saja di context; entri lama diganti
placeholder tapi tool-call-nya sendiri tetap terlihat di riwayat. Tool
bersifat mutasi (write/edit/bash/dll) tidak pernah disentuh.

Jadikan tool_scope::READ_TOOLS pub supaya jadi satu-satunya sumber
klasifikasi read-only, dipakai ulang bukan didaftar dua kali.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 059ca8ea24 feat(context): tambah context::window::resolve
Satukan tiga salinan logika resolusi context_window (Action::Compact,
spawn_turn, status bar) yang sempat melenceng satu sama lain.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyana c219b6ec58 feat(context): tambah context::tokens dengan tiktoken-rs
Ganti tiga heuristik char-count (/3 di shortsend, /4 di loop turn, /4
di status bar) yang saling tidak konsisten dengan satu BPE tokenizer
nyata. tiktoken-rs membundel vocab lewat include_str! saat build, jadi
tidak ada akses jaringan saat runtime.
2026-07-16 07:49:43 +07:00
asepharyana 8bb697a53f fix(context): perbaiki fixture test shaping agar men-drop pesan lama
Fixture pendek sebelumnya (~8 token nyata per pesan lewat tiktoken)
tidak pernah melebihi target 700 token, jadi shape_messages tidak
pernah men-drop satu pesan pun -- satu test gagal, satu test lain
lulus secara vakum. Pakai fixture lebih panjang (~49 token/pesan)
yang diverifikasi melebihi target dengan nyaman.
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 75e9cadcd5 docs(plans): tambah plan implementasi rombak context & compaction
9 task: tokens.rs (tiktoken-rs), window.rs, dedup.rs, squash.rs,
shaping.rs (port shortsend), gabungan cutover+fix /compact manual,
wiring squash, status bar, dan mode ringkas opsional. Setiap task
diverifikasi dengan cargo build/test nyata, bukan -D warnings (yang
sudah merah di main karena warning pre-existing di file lain).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 c4e1d7c2b2 docs(specs): hapus facade prepare(), panggil modul context langsung
Facade cuma dipakai generically oleh satu caller (Action::Compact);
auto-loop tetap butuh kontrol per-stage sendiri. Selaras dengan prinsip
"No DI" di CLAUDE.md. dedup::collapse juga diubah mengembalikan
(Vec<ChatMessage>, bool) supaya caller tahu ada perubahan tanpa perlu
ChatMessage: PartialEq.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 da39033e81 docs(specs): kecualikan tool read dari squash
Ditemukan saat menulis plan: kompresi JSON pada hasil read akan merusak
byte-exactness yang dibutuhkan untuk edit selanjutnya jika file yang
dibaca kebetulan berformat JSON (package.json, tsconfig.json, dst).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
asepharyanaandClaude Sonnet 5 f62ac3f688 docs(specs): tambah desain rombak context & compaction
Rencana pengganti shortsend.rs dengan modul context/ (dedup, squash,
shaping, tokens, window) plus mode ringkas opsional, disusun dari studi
teknik rtk-ai/rtk, headroomlabs-ai/headroom, dan JuliusBrussee/caveman
(ide saja, tanpa menyalin kode).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 07:49:43 +07:00
semantic-release-bot aaab953d8a chore(release): 1.13.0 [skip ci]
# [1.13.0](https://github.com/asepharyana/zesdex/compare/v1.12.0...v1.13.0) (2026-07-14)

### Bug Fixes

* correct test assertion in dim_false_plain_text_has_no_color ([8d77e45](https://github.com/asepharyana/zesdex/commit/8d77e4565c2e3ca49058022c71e8277bdd790312))
* Remove orphaned span_text helper function from markdown test module ([199028f](https://github.com/asepharyana/zesdex/commit/199028fa2eb6056dad2bdf0753554939c153a163))
* **state:** cegah panic saat select mention dengan cursor stale ([dcfc5b9](https://github.com/asepharyana/zesdex/commit/dcfc5b9ec0d35f4c66b1648ded00f4f1279e0413))
* **state:** jangan bangun mention index di mode attach ([e0d0860](https://github.com/asepharyana/zesdex/commit/e0d0860d7ba0ab6a4c9b9f1562f6a767a6424d99))

### Features

* Deteksi trigger [@mention](https://github.com/mention) dan Tab-cycle di input handler ([2eebedf](https://github.com/asepharyana/zesdex/commit/2eebedfea56c7d4827f94a2739dbb5fa006e3069))
* **ipc:** dukung Ctrl+Y clipboard copy di mode daemon/attach ([472162d](https://github.com/asepharyana/zesdex/commit/472162d135478666913a888a68867e635e50d14a))
* Judul dropdown autocomplete mengikuti jenisnya (Commands vs Files) ([7f2509e](https://github.com/asepharyana/zesdex/commit/7f2509ecd2688495545333debee0c81016b28641))
* **state:** Alirkan mention_index lewat ToolCtx dan AppStateRest, bangun index di background thread ([93be92a](https://github.com/asepharyana/zesdex/commit/93be92a5fb8644c9020478c49421585ade2d0fd1))
* Tambah Ctrl+Y untuk menyalin pesan assistant terakhir ([d93bdba](https://github.com/asepharyana/zesdex/commit/d93bdba59b7dd14b4a76dc91c6b7f70a3c0f721c))
* Tambah field pending_clipboard_copy di MiscState ([3b6abd9](https://github.com/asepharyana/zesdex/commit/3b6abd920ea92329ebcf888c3bb50470675c4152))
* Tambah helper truncate_diff untuk membatasi panjang diff ([5dd835f](https://github.com/asepharyana/zesdex/commit/5dd835ff244a138bb5f1a8dbfd7cf252c2671e5e))
* Tambah MentionIndex, AutocompleteKind, dan deteksi [@mention](https://github.com/mention) di InputState ([95cae8f](https://github.com/asepharyana/zesdex/commit/95cae8fd8a8ce737ba60c3ac8bea77f4c2e14b1a))
* Tambah write_osc52 dan salin ke clipboard di mode single-process ([71b1613](https://github.com/asepharyana/zesdex/commit/71b1613d8467cfcb2383b3fce153a25c7883ac3d))
* Tambahkan file baru ke mention_index saat tool write membuatnya ([930961b](https://github.com/asepharyana/zesdex/commit/930961bd85701ab8906b8d81095e46d3635a951d))
* Tampilkan unified diff pada hasil tool edit ([c3c0ef6](https://github.com/asepharyana/zesdex/commit/c3c0ef632a605d8bf32712049035e4fc814af9d5))
* Tampilkan unified diff saat tool write menimpa file yang sudah ada ([2bb8e6f](https://github.com/asepharyana/zesdex/commit/2bb8e6f2555eac01baf2311d41811afad4aa3041))
* **view:** Tambah parameter dim dan pewarnaan baris diff di markdown renderer ([683715c](https://github.com/asepharyana/zesdex/commit/683715cd7af77d87a2ae3834360bb6f7a4388bd0))
2026-07-14 23:39:22 +00:00
asepharyana e0d0860d7b fix(state): jangan bangun mention index di mode attach
AppStateRest::new() men-spawn thread ignore::Walk untuk mention_index
tanpa syarat, padahal mode --attach cuma dipakai untuk render lokal —
handle_key dan logika mention berjalan di sisi daemon lewat IPC, jadi
index di client attach tidak pernah dipakai. Ini bikin setiap
--attach melakukan full workspace walk (sampai 50.000 entry) sia-sia.

Pindahkan thread-spawn itu ke method terpisah
spawn_mention_index_build(), dipanggil eksplisit dari
run_single_process() dan run_daemon() setelah AppStateRest::new(),
tapi sengaja tidak dipanggil dari run_attach().
2026-07-15 06:32:57 +07:00
asepharyana dcfc5b9ec0 fix(state): cegah panic saat select mention dengan cursor stale
Cursor bisa berpindah (Left/Right) tanpa menutup dropdown mention,
sehingga mention_start jadi stale relatif ke cursor saat Enter
ditekan. select_autocomplete() lalu memanggil replace_range dengan
start > end dan panic (crash seluruh TUI, termasuk daemon). Tambah
guard: jika cursor < mention_start atau mention_start > buffer.len(),
tutup dropdown dan kembalikan false alih-alih menyambung range yang
tidak valid. Tambah regression test yang mereproduksi skenario ini.
2026-07-15 06:32:57 +07:00
asepharyana 472162d135 feat(ipc): dukung Ctrl+Y clipboard copy di mode daemon/attach
Tambah DaemonFrame::ClipboardCopy(String) supaya daemon bisa
mengirim teks pending_clipboard_copy ke attach client, yang
kemudian menulis sekuens OSC52 ke stdout-nya sendiri (bukan
stdout daemon yang tidak dimiliki terminal user).
2026-07-15 06:32:57 +07:00
asepharyana 71b1613d84 feat: Tambah write_osc52 dan salin ke clipboard di mode single-process 2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 d93bdba59b feat: Tambah Ctrl+Y untuk menyalin pesan assistant terakhir
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyana 3b6abd920e feat: Tambah field pending_clipboard_copy di MiscState 2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 7f2509ecd2 feat: Judul dropdown autocomplete mengikuti jenisnya (Commands vs Files)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 2eebedfea5 feat: Deteksi trigger @mention dan Tab-cycle di input handler
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 930961bd85 feat: Tambahkan file baru ke mention_index saat tool write membuatnya
Ketika write tool membuat file baru (bukan overwrite), path file
sekarang di-push ke mention_index untuk autocomplete @file mention.

- Capture file existence status sebelum write
- Push ke mention_index hanya untuk genuinely new files
- Add 2 tests: one untuk new file push, one untuk overwrite non-duplication

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyana 93be92a5fb feat(state): Alirkan mention_index lewat ToolCtx dan AppStateRest, bangun index di background thread
Menambahkan field mention_index: MentionIndex ke ToolCtx/ToolCtxBuilder dan
AppStateRest, mengikuti pola dir_cache yang sudah ada. AppStateRest::new()
sekarang men-spawn thread background yang men-walk setiap workspace root
via ignore::Walk, membangun daftar path file (dengan prefix [N] untuk root
selain yang pertama) dan mengisi mention_index — data ini yang akan dipakai
fitur autocomplete @file-mention.
2026-07-15 06:32:57 +07:00
asepharyana 95cae8fd8a feat: Tambah MentionIndex, AutocompleteKind, dan deteksi @mention di InputState
Menambahkan MentionIndex (indeks path file thread-safe untuk fitur
autocomplete @file-mention), enum AutocompleteKind untuk membedakan
dropdown slash-command dan file-mention, serta method baru pada
InputState: mention_query_at_cursor untuk deteksi token @mention di
posisi cursor, dan open_mention_autocomplete untuk fuzzy-match file
via nucleo-matcher. select_autocomplete kini kind-aware: menyisipkan
path file ke posisi mention alih-alih mengganti seluruh buffer.
2026-07-15 06:32:57 +07:00
asepharyana 5c413cf9a3 chore: Tambah dependency nucleo-matcher untuk fuzzy file matching 2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 c386030573 refactor: Pakai parameter dim render_markdown, hapus override style manual di chat
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 199028fa2e fix: Remove orphaned span_text helper function from markdown test module
The span_text function in src/view/markdown.rs's test module was no longer
called after the dim_false_plain_text_has_no_color test was fixed. With the
crate's dead_code = "deny" lint enabled, this unused function caused a compile
error. Removed it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 8d77e4565c fix: correct test assertion in dim_false_plain_text_has_no_color
The first assertion was checking span_text() which concatenates all spans
including trailing paragraph-end newlines, making the test guaranteed to fail.
Changed to check spans[0].content directly to verify the text span itself
contains 'hello world' with no color applied when dim=false.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyana 683715cd7a feat(view): Tambah parameter dim dan pewarnaan baris diff di markdown renderer
Tambahkan helper apply_dim dan diff_line_style, ubah signature
render_markdown untuk menerima flag dim, serta deteksi fence bahasa
diff sehingga baris +/-/@@ tetap berwarna meskipun pesan sedang
dirender dim (tampilan tool-output).
2026-07-15 06:32:57 +07:00
asepharyana 2bb8e6f255 feat: Tampilkan unified diff saat tool write menimpa file yang sudah ada 2026-07-15 06:32:57 +07:00
asepharyana c3c0ef632a feat: Tampilkan unified diff pada hasil tool edit 2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 5dd835ff24 feat: Tambah helper truncate_diff untuk membatasi panjang diff
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyanaandClaude Sonnet 5 0a24903eb0 chore: Tambah dependency similar untuk diff computation
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-15 06:32:57 +07:00
asepharyana de8e3703f2 docs: Perbaiki perintah cargo test --lib jadi cargo test (crate ini tanpa lib target) 2026-07-15 06:32:57 +07:00
asepharyana b7fc335bf5 docs: Tambah implementation plan untuk diff view, @mention, dan clipboard copy 2026-07-15 06:28:49 +07:00
asepharyana 821622e80d docs: Tambah spec desain clipboard copy via OSC52 2026-07-15 06:28:49 +07:00
asepharyana 8fbc51534d docs: Tambah spec desain fuzzy @file-mention autocomplete 2026-07-15 06:28:49 +07:00
asepharyana 3f284cbb9a docs: Tambah spec desain diff view untuk tool edit/write 2026-07-15 06:28:49 +07:00
semantic-release-bot 54484ed137 chore(release): 1.12.0 [skip ci]
# [1.12.0](https://github.com/asepharyana/zesdex/compare/v1.11.0...v1.12.0) (2026-07-14)

### Features

* Add mouse capture functionality to terminal and enhance markdown rendering with table support ([4428e8b](https://github.com/asepharyana/zesdex/commit/4428e8bc01c196415ac408a42f57305227a79760))
* Improve markdown rendering with enhanced line wrapping and indentation for code blocks ([b2e848d](https://github.com/asepharyana/zesdex/commit/b2e848d124e726c4d8b644d473e518398fab1dea))
2026-07-14 21:40:13 +00:00
asepharyana 9bf5236680 refactor: Remove obsolete documentation files and unused test for mouse functionality 2026-07-15 04:36:00 +07:00
asepharyana 4428e8bc01 feat: Add mouse capture functionality to terminal and enhance markdown rendering with table support 2026-07-15 04:36:00 +07:00
asepharyana b2e848d124 feat: Improve markdown rendering with enhanced line wrapping and indentation for code blocks 2026-07-15 04:36:00 +07:00
152 changed files with 10129 additions and 2356 deletions
+58
View File
@@ -1,3 +1,61 @@
# [1.14.0](https://github.com/asepharyana/zesdex/compare/v1.13.0...v1.14.0) (2026-07-16)
### Bug Fixes
* **context:** batasi squash_log ke tool bash saja ([7d99cd6](https://github.com/asepharyana/zesdex/commit/7d99cd66187b3fafb2ddeb19d8e8aa7db139df64))
* **context:** perbaiki fixture test shaping agar men-drop pesan lama ([8bb697a](https://github.com/asepharyana/zesdex/commit/8bb697a53fe89c9d00f709df33ccbd7944482f07))
* **plan:** perbaiki bug entropy gate dan fixture test squash.rs ([ceb8479](https://github.com/asepharyana/zesdex/commit/ceb84790bb741352b2b3416cda4bdbc6b42767c9))
* **plan:** perbaiki fixture test array-cutoff squash_json ([7ffdf44](https://github.com/asepharyana/zesdex/commit/7ffdf441355cda0972b849bc9b475d3de8f39843))
* **plan:** perbaiki fixture test shaping agar benar-benar men-drop pesan ([3a6e32d](https://github.com/asepharyana/zesdex/commit/3a6e32d8f3f0dd47d2c5ff63ab0c184b0ee4a6bc))
* **plan:** perkuat fixture test log agar benar-benar uji squash_log ([682007a](https://github.com/asepharyana/zesdex/commit/682007a2507f209c2378796442d4bc879516191a))
### Features
* **context:** tambah context::dedup untuk hasil tool yang berulang ([12a03fd](https://github.com/asepharyana/zesdex/commit/12a03fd3d1123c7a88289e44064537a71b9f574b))
* **context:** tambah context::shaping (port dari shortsend) ([e080d9f](https://github.com/asepharyana/zesdex/commit/e080d9fc6b7daf2ab37fb242d03a3f64e8629acc))
* **context:** tambah context::squash untuk kompresi hasil tool ([ea0b498](https://github.com/asepharyana/zesdex/commit/ea0b4988299894e4588d851ebc91704a9e73bc73))
* **context:** tambah context::tokens dengan tiktoken-rs ([c219b6e](https://github.com/asepharyana/zesdex/commit/c219b6ec58ddad4770e491a48bbe1b6e3d0c8884))
* **context:** tambah context::window::resolve ([059ca8e](https://github.com/asepharyana/zesdex/commit/059ca8ea246768dbec06ae5336433853e4276cec))
* **runtime:** kompres hasil tool lewat squash sebelum masuk context ([96084f7](https://github.com/asepharyana/zesdex/commit/96084f74621636a72c209388580d0ebbb335d198))
* **settings:** tambah mode ringkas opsional (concise_output) ([d6de973](https://github.com/asepharyana/zesdex/commit/d6de9735aba7745f5f57a8e7a5e625613c1a2569))
# [1.13.0](https://github.com/asepharyana/zesdex/compare/v1.12.0...v1.13.0) (2026-07-14)
### Bug Fixes
* correct test assertion in dim_false_plain_text_has_no_color ([8d77e45](https://github.com/asepharyana/zesdex/commit/8d77e4565c2e3ca49058022c71e8277bdd790312))
* Remove orphaned span_text helper function from markdown test module ([199028f](https://github.com/asepharyana/zesdex/commit/199028fa2eb6056dad2bdf0753554939c153a163))
* **state:** cegah panic saat select mention dengan cursor stale ([dcfc5b9](https://github.com/asepharyana/zesdex/commit/dcfc5b9ec0d35f4c66b1648ded00f4f1279e0413))
* **state:** jangan bangun mention index di mode attach ([e0d0860](https://github.com/asepharyana/zesdex/commit/e0d0860d7ba0ab6a4c9b9f1562f6a767a6424d99))
### Features
* Deteksi trigger [@mention](https://github.com/mention) dan Tab-cycle di input handler ([2eebedf](https://github.com/asepharyana/zesdex/commit/2eebedfea56c7d4827f94a2739dbb5fa006e3069))
* **ipc:** dukung Ctrl+Y clipboard copy di mode daemon/attach ([472162d](https://github.com/asepharyana/zesdex/commit/472162d135478666913a888a68867e635e50d14a))
* Judul dropdown autocomplete mengikuti jenisnya (Commands vs Files) ([7f2509e](https://github.com/asepharyana/zesdex/commit/7f2509ecd2688495545333debee0c81016b28641))
* **state:** Alirkan mention_index lewat ToolCtx dan AppStateRest, bangun index di background thread ([93be92a](https://github.com/asepharyana/zesdex/commit/93be92a5fb8644c9020478c49421585ade2d0fd1))
* Tambah Ctrl+Y untuk menyalin pesan assistant terakhir ([d93bdba](https://github.com/asepharyana/zesdex/commit/d93bdba59b7dd14b4a76dc91c6b7f70a3c0f721c))
* Tambah field pending_clipboard_copy di MiscState ([3b6abd9](https://github.com/asepharyana/zesdex/commit/3b6abd920ea92329ebcf888c3bb50470675c4152))
* Tambah helper truncate_diff untuk membatasi panjang diff ([5dd835f](https://github.com/asepharyana/zesdex/commit/5dd835ff244a138bb5f1a8dbfd7cf252c2671e5e))
* Tambah MentionIndex, AutocompleteKind, dan deteksi [@mention](https://github.com/mention) di InputState ([95cae8f](https://github.com/asepharyana/zesdex/commit/95cae8fd8a8ce737ba60c3ac8bea77f4c2e14b1a))
* Tambah write_osc52 dan salin ke clipboard di mode single-process ([71b1613](https://github.com/asepharyana/zesdex/commit/71b1613d8467cfcb2383b3fce153a25c7883ac3d))
* Tambahkan file baru ke mention_index saat tool write membuatnya ([930961b](https://github.com/asepharyana/zesdex/commit/930961bd85701ab8906b8d81095e46d3635a951d))
* Tampilkan unified diff pada hasil tool edit ([c3c0ef6](https://github.com/asepharyana/zesdex/commit/c3c0ef632a605d8bf32712049035e4fc814af9d5))
* Tampilkan unified diff saat tool write menimpa file yang sudah ada ([2bb8e6f](https://github.com/asepharyana/zesdex/commit/2bb8e6f2555eac01baf2311d41811afad4aa3041))
* **view:** Tambah parameter dim dan pewarnaan baris diff di markdown renderer ([683715c](https://github.com/asepharyana/zesdex/commit/683715cd7af77d87a2ae3834360bb6f7a4388bd0))
# [1.12.0](https://github.com/asepharyana/zesdex/compare/v1.11.0...v1.12.0) (2026-07-14)
### Features
* Add mouse capture functionality to terminal and enhance markdown rendering with table support ([4428e8b](https://github.com/asepharyana/zesdex/commit/4428e8bc01c196415ac408a42f57305227a79760))
* Improve markdown rendering with enhanced line wrapping and indentation for code blocks ([b2e848d](https://github.com/asepharyana/zesdex/commit/b2e848d124e726c4d8b644d473e518398fab1dea))
# [1.11.0](https://github.com/asepharyana/zesdex/compare/v1.10.0...v1.11.0) (2026-07-14)
Generated
+51 -2
View File
@@ -196,6 +196,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cee35f73844aa3014bb606320a6c1f010249dbdf43342fe54b5a4f6a8ed4b79"
dependencies = [
"memchr",
"regex-automata",
"serde_core",
]
@@ -807,6 +808,17 @@ dependencies = [
"regex-syntax",
]
[[package]]
name = "fancy-regex"
version = "0.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72cf461f865c862bb7dc573f643dd6a2b6842f7c30b07882b56bd148cc2761b8"
dependencies = [
"bit-set 0.8.0",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "fast-srgb8"
version = "1.0.0"
@@ -1915,6 +1927,16 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "nucleo-matcher"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf33f538733d1a5a3494b836ba913207f14d9d4a1d3cd67030c5061bdd2cac85"
dependencies = [
"memchr",
"unicode-segmentation",
]
[[package]]
name = "num-conv"
version = "0.2.2"
@@ -3149,6 +3171,15 @@ version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3a9fe34e3e7a50316060351f37187a3f546bce95496156754b601a5fa71b76e"
[[package]]
name = "similar"
version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6505efef05804732ed8a3f2d4f279429eb485bd69d5b0cc6b19cc02005cda16"
dependencies = [
"bstr",
]
[[package]]
name = "siphasher"
version = "1.0.3"
@@ -3362,7 +3393,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -3502,6 +3533,21 @@ dependencies = [
"cfg-if",
]
[[package]]
name = "tiktoken-rs"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "027853bbf8c7763b77c5c595f1c271c7d536ced7d6f83452911b944621e57fc2"
dependencies = [
"anyhow",
"base64",
"bstr",
"fancy-regex 0.17.0",
"lazy_static",
"regex",
"rustc-hash",
]
[[package]]
name = "time"
version = "0.3.53"
@@ -4436,7 +4482,7 @@ dependencies = [
[[package]]
name = "zesdex"
version = "1.11.0"
version = "1.14.0"
dependencies = [
"anyhow",
"base64",
@@ -4453,6 +4499,7 @@ dependencies = [
"infer",
"libc",
"lsp-types",
"nucleo-matcher",
"percent-encoding",
"pulldown-cmark",
"ratatui",
@@ -4465,7 +4512,9 @@ dependencies = [
"serde_json",
"serde_yaml_ng",
"sha2 0.11.0",
"similar",
"syntect",
"tiktoken-rs",
"tokio",
"tracing",
"tracing-subscriber",
+4 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "zesdex"
version = "1.11.0"
version = "1.14.0"
edition = "2021"
authors = ["asepharyana <superaseph@gmail.com>"]
@@ -40,11 +40,13 @@ uuid = { version = "1", features = ["v4", "v5"] }
dirs = "6"
futures-util = "0.3"
pulldown-cmark = { version = "0.13", default-features = false }
similar = "3"
syntect = { version = "5", default-features = false, features = ["default-fancy"] }
rusqlite = { version = "0.40", features = ["bundled"] }
ignore = "0.4"
regex = "1"
globset = "0.4"
nucleo-matcher = "0.3"
infer = "0.19"
base64 = "0.22"
sha2 = "0.11"
@@ -56,6 +58,7 @@ chrono = { version = "0.4", features = ["serde"] }
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
webbrowser = "1"
lsp-types = "0.97"
tiktoken-rs = "0.12"
[[bin]]
name = "zesdex"
+53 -49
View File
@@ -1,61 +1,65 @@
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~750 -->
# Architecture
Zesdex is a single-process terminal AI coding agent with optional daemon/client split.
# Architecture Overview
## System Layout
```
┌──────────────────────────────────────────────────────┐
│ main.rs │
│ single-process ─┬── daemon ── Unix socket ── client │
│ └── attach <id> (TUI-only client) │
└──────────────────────┬───────────────────────────────┘
┌──────────────────────▼───────────────────────────────┐
│ Event Loop │
│ ┌────────┐ ┌───────────┐ ┌──────┐ ┌────────┐ │
│ │Input │──▶│ Actions │──▶│State │──▶│ TUI │ │
│ │Handler │ │ (dispatch)│ │ │ │ Render │ │
│ └────────┘ └─────┬─────┘ └──────┘ └────────┘ │
│ │ │
│ ┌──────▼──────┐ │
│ │ LLM Stream │ │
│ │ + Tool Exec │ │
│ └──────┬──────┘ │
│ ┌────┴────┐ │
│ │ │ │
│ ┌─────▼──┐ ┌───▼────┐ │
│ │ Tools │ │Sub- │ │
│ │ (37) │ │agents │ │
│ └────────┘ └────────┘ │
└───────────────────────────────────────────────────────┘
```
## Data Flow
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
```
User keystroke → Controller (KeyEvent → Action)
→ apply_action() mutates AppStateRest
→ TUI redraws (ratatui Frame)
→ On submit: LLM request → SSE stream → tool calls → tool results → more LLM
→ Session persisted to disk (editlog, msglog, memory)
┌─────────────────────────────────────────────────────────────┐
│ Process Mode │
│ Single-Process ─── Daemon (background) ─── Attach (client) │
└──────────────────────────┬──────────────────────────────────┘
│ IPC (Unix domain socket)
┌─────────────────────────────────────────────────────────────┐
│ src/main.rs │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Controller │──▶│ Runtime │──▶│ View │ │
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ Harness │ │
│ │ (tool dispatch)│ │
│ └───────┬────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
│ │ Tools │ │ Subagents │ │ Workflow │ │
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
│ └───────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
## Process Modes
| Mode | Impl | Process | IPC |
|------|------|---------|-----|
| Single | `run_single_process()` | One | No |
| Daemon | `run_daemon()` | Server | `ipc/server.rs` |
| Attach | `run_attach()` | Client | `ipc/client.rs` |
| Mode | Description |
|------|-------------|
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
## Data Flow
1. **Input**`controller/input.rs` handles key events and autocomplete
2. **Dispatch**`app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
3. **LLM Stream**`app/runtime/stream/mod.rs` parses SSE chunks into typed events
4. **Tool Execution**`app/harness.rs` gates and runs tool calls via the `Tool` trait
5. **Rendering**`view/` modules read `AppStateRest` and render via ratatui
## Key Files
| File | Lines | Role |
|------|-------|------|
| `src/main.rs` | 647 | Entry, TUI setup, daemon loop, attach loop |
| `src/app/runtime/actions/mod.rs` | 1815 | Action dispatch + LLM stream loop + tool execution |
| `src/controller/input.rs` | 365 | Key event → Action mapping |
| `src/view/mod.rs` | 975 | TUI rendering (ratatui) |
| File | Purpose |
|------|---------|
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
| `src/app/state/rest.rs` | Single source-of-truth state struct |
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
| `src/app/harness.rs` | Tool harness with safety gating |
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
| `src/view/mod.rs` | TUI render pipeline |
+47 -54
View File
@@ -1,75 +1,68 @@
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~850 -->
# Backend Architecture
# Backend / Service Layer
## Provider Layer
## AI Provider
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
`src/service/provider.rs` (310 lines)
- `LlmClient::new(api_key, model, base_url)` — constructs blocking reqwest client
- `chat_with_tools()` — non-streaming with tool definitions
- `chat_stream()` — SSE streaming, returns `SseParser` yielding `StreamEvent`
- Retry logic: up to 3 attempts on transient errors, exponential backoff
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
## OAuth
## IPC (Inter-Process Communication)
`src/service/oauth/manager.rs` (113 lines) + `loopback.rs` + `pkce.rs`
- PKCE flow: `CodeVerifier` → challenge → browser auth → loopback server → token exchange
- Configurable via `app_config.json` provider definitions (auth URL, token URL, scopes)
The daemon-client protocol in `src/ipc/`:
## IPC / Daemon
- **Transport**: Unix domain sockets
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
`src/ipc/` (7 files, ~350 lines total)
- Unix domain socket, length-prefixed JSON frames
- Daemon sends `DaemonFrame` (state payload, stream tokens, system notes)
- Clients send `ClientRequest` (key presses, resize, submit, scroll)
- State sync uses full-state push from daemon to client after each action
Flow:
```
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
└──StatePayload──▶ Client (render)
```
## Workflow Engine
`src/app/workflow/engine.rs` (648 lines) + `script.rs`
- Inline JS-style DSL executed by a lightweight runtime
- `agent()`, `parallel()`, `pipeline()`, `phase()`, `log()` — spawns sub-agents
- Max concurrency configurable via `workflow_max_concurrency` setting
- Hive-mind orchestrator in `hive_mind.rs`: Core Intelligence compiles a `CognitiveCyclePlan` per task — cycle count and nodes-per-cycle are decided fresh each time based on what the task actually needs
Located in `src/app/workflow/`:
## Sub-Agent System
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
`src/app/subagent/` (6 files: `spawn.rs`, `engine.rs`, `context.rs`, `event.rs`, `division.rs`, `auto.rs`, ~450 lines total)
- `run_subagent()` — spawns independent agent with its own tool set and context
- Communicates via `mpsc<SubagentEvent>` channel (tool calls, results, completion)
- Uses `LlmClient` (same as main agent) with tool-use API
- Auto-healing: on build/test failure, spawns auto-fix sub-agent
- Node access tiers (`division.rs`'s `tool_scope` module): `read`, `write`, `full` — granted per node by the Core Intelligence based on what its directive needs
## MCP (Model Context Protocol)
## MCP Client
`src/app/mcp/manager.rs` manages MCP client connections:
`src/app/mcp/manager.rs` (441+ lines)
- Stdio transport: spawns child process, JSON-RPC via stdin/stdout
- HTTP transport: streaming HTTP with JSON-RPC
- Dynamic tool list refresh and error recovery
- Persistent child handle for stdio (reuses connection across calls)
- Uses the `rmcp` crate for the MCP protocol
- Supports stdio-based transport (child process) and streamable HTTP
- Tool discovery via `list_tools()` and dynamic tool registration
## Self-Review
## LSP Integration
`src/app/review/mod.rs` (495 lines)
- Post-tool execution quality check against learned lessons
- Invokes `run_subagent()` with reviewer prompt
- Staleness detection: skips review after N consecutive empty results
- Three review types: code quality, architecture, security
`src/app/lsp/` provides Language Server Protocol support:
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
## Background Bash
`src/app/bgbash/` (2 files: `job.rs`, `control.rs`)
- `spawn_bash_job()` — runs `sh -c` in a thread, collects stdout line-by-line
- Channels: output via `mpsc<String>`, PID via `mpsc<u32>`
- Killable via PID (SIGTERM)
- Output buffering capped at 10,000 lines to prevent memory issues
`src/app/bgbash/` manages long-running shell jobs:
## Gate Guard / Harness
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
`src/app/harness.rs` (495 lines)
- `Harness::gate_tool_call()` — verdict-based tool gating (allow/block)
- Path traversal, credential read, and destructive command detection
- Pattern detection for stub code, denial language, and assumptions in write/edit content
- Reason validation for mutating tools (minimum 8 characters, rejects generic non-answers)
- Includes 8 unit tests for verdict parsing formats
## Review System
`src/app/subagent/auto.rs` spawns background reviews:
- Quick review after every edit
- Background test generation
- Architecture review
- Security review
- All retry once on failure, escalate to blocking error if retry also fails
+80 -42
View File
@@ -1,51 +1,89 @@
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~600 -->
# Data Architecture
# Data / Persistence Layer
## State Model
## Storage Overview
Base directory: `~/.config/zesdex/` (via `dirs::data_dir()`)
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
```
~/.config/zesdex/
├── settings.json # User preferences (provider, model, tokens)
├── app_config.json # Provider definitions (API base, auth, models)
├── agents/ # Global agent definitions
│ └── *.json
├── memory/ # Persistent lesson/reference store
│ └── *.md # Markdown with YAML frontmatter
├── sessions/ # Per-session data
│ └── <session-uuid>/
│ ├── edits.jsonl # Edit history (JSONL, append-only)
│ ├── msglog.db # SQLite message log
│ ├── transcript.json # Chat transcript
│ ├── session.json # Session metadata
│ ├── agents.json # Session-local agent defs
│ └── snapshot.dat # State snapshot (daemon mode)
├── run/ # Unix domain sockets
│ └── zesdex-*.sock
└── store.json # Legacy session index
AppStateRest
├── session: SessionRuntime (hive_mind state, convergence flag)
├── runtime: RuntimeState (mode, provider status)
├── chat: ChatState (messages, scroll)
├── input: InputState (text, cursor, autocomplete)
├── settings: Settings (provider, model, temperature, concise_output)
├── config: AppConfig (endpoints, credentials)
├── scroll: ScrollState (per-panel offset)
├── diff: DiffState (edit review)
├── tools: Vec with outputs
├── statusline, sidebar, etc.
└── toasts: pending notifications
```
## Key Files
**Mutation rules** (per CLAUDE.md):
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
- Read-only from every other module
- No generic update function — direct field mutation only
| File | Lines | Role |
|------|-------|------|
| `src/model/store.rs` | ~50 | File-system storage (ensure_dirs, base_dir resolution) |
| `src/model/settings.rs` | ~60 | `Settings` — load/save JSON, API keys map |
| `src/model/app_config.rs` | ~80 | `AppConfig` — provider definitions, model roles, auth |
| `src/model/memory.rs` | 440 | Memory CRUD — markdown files with frontmatter |
| `src/model/editlog.rs` | 161 | Edit log — append-only JSONL (not JSON array) |
| `src/model/msglog/` | 4 files | SQLite-backed message log (schema, query, blobs) |
| `src/model/session.rs` | ~60 | Session CRUD, listing, archival |
| `src/model/session_lock.rs` | ~50 | flock-based session lock |
| `src/model/agent_def/` | 3 files | Agent definitions (builtin, global, session-local) |
## Persistence
## Key Patterns
### SQLite Message Log (`src/model/msglog/`)
- **No ORM** — raw JSON files + SQLite via rusqlite
- **settings.json** — loaded at startup, saved on quit / mode switches
- **Memory format** — Markdown files with YAML frontmatter (`---\nname: ...\ndescription: ...\n---\ncontent`)
- **Edit log** — append-only, stores `(file, old, new, timestamp, tool)`
- **Session locking** — flock-based, prevents concurrent access to same session dir
- **Message log** — SQLite with attached blobs for tool arguments/outputs
| File | Purpose |
|------|---------|
| `schema.rs` | Table definitions (messages, sessions) |
| `mod.rs` | CRUD operations |
| `query.rs` | Query helpers (search, filter) |
| `blobs.rs` | Large message blob storage |
| `summary.rs` | Conversation summary cache |
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
### Memory System (`src/model/memory.rs`)
File-based memory stored under `~/.claude/projects/<project>/memory/`:
- Each memory is one markdown file with frontmatter (name, description, type)
- Types: `user`, `feedback`, `project`, `reference`
- Memory index in MEMORY.md
- Export/import for lesson sharing
- PID-file session lock prevents concurrent access
### Settings & Config (`src/model/`)
| File | Purpose |
|------|---------|
| `settings.rs` | Serialized user preferences (provider, model, theme) |
| `app_config.rs` | Provider endpoints, API key resolution from env |
| `session.rs` | Current session metadata |
| `conversation.rs` | In-memory conversation state |
| `editlog.rs` | Append-only JSONL edit audit trail |
### Edit Log
`src/model/editlog.rs` records every file mutation:
```json
{"ts": 123, "tool": "edit", "path": "src/main.rs",
"reason": "fix bug", "content_sha256": "abc123",
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
```
Max 5000 entries held in memory before pruning oldest.
## Context Management (`src/app/runtime/context/`)
| Module | Purpose |
|--------|---------|
| `tokens.rs` | Token counting via `tiktoken-rs` |
| `window.rs` | Token window resolution (fit within model context) |
| `dedup.rs` | Deduplication of repeated tool outputs |
| `squash.rs` | Compression of large JSON tool results |
| `shaping.rs` | Message dropping when context exceeds limits |
## IPC Data Flow
```
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
```
+91 -33
View File
@@ -1,41 +1,99 @@
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~400 -->
# Dependencies
## Rust Crates (Cargo.toml)
## Rust Crates (30+ direct)
### Core Framework
| Crate | Version | Purpose |
|-------|---------|---------|
| ratatui | 0.30 | TUI framework (tui-rs successor) |
| crossterm | 0.29 | Terminal manipulation (raw mode, alt screen) |
| tokio | 1 | Async runtime (daemon, OAuth loopback) |
| reqwest | 0.12 | HTTP client (blocking + streaming, vendored native-tls) |
| serde / serde_json | 1 | JSON serialization (state, DTOs, IPC, config) |
| serde_yaml_ng | 0.10 | YAML frontmatter parsing (memory files) |
| anyhow | 1 | Error handling (no custom error types) |
| tracing / tracing-subscriber | 0.1/0.3 | Structured logging → file |
| rusqlite | 0.40 | SQLite (bundled, for message log) |
| pulldown-cmark | 0.13 | Markdown → HTML (chat rendering) |
| syntect | 5 | Syntax highlighting (code blocks in chat) |
| sha2 | 0.10 | SHA-256 for PKCE challenge |
| base64 | 0.22 | URL-safe base64 for PKCE |
| libc | 0.2 | daemon PID file locking |
| rmcp | 2.2 | MCP client (stdio + HTTP transports) |
| uuid | 1 | Session IDs, job IDs |
| chrono | 0.4 | Timestamps (ISO 8601, millis) |
| dirs | 6 | Platform data directories |
| dom_smoothie | 0.18 | HTML → plain text (web scraping) |
| scraper | 0.27 | HTML parsing (web scraping) |
| ignore | 0.4 | .gitignore-aware file walking (glob tool) |
| regex / globset | 0.4 | Pattern matching (grep/glob tools) |
| url / percent-encoding | 2 | URL parsing + encoding (OAuth) |
| `ratatui` | 0.30.2 | TUI framework |
| `crossterm` | 0.29 | Terminal manipulation |
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
### HTTP & Networking
| Crate | Version | Purpose |
|-------|---------|---------|
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
| `webbrowser` | 1 | Open URLs in browser |
| `url` | 2 | URL parsing |
| `percent-encoding` | 2 | URL encoding |
### HTML/Markdown
| Crate | Version | Purpose |
|-------|---------|---------|
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
| `scraper` | 0.27.0 | HTML parsing/selecting |
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
### Serialization
| Crate | Version | Purpose |
|-------|---------|---------|
| `serde` | 1 | Serialization framework |
| `serde_json` | 1 | JSON serialization |
| `serde_yaml_ng` | 0.10 | YAML serialization |
### Storage & Files
| Crate | Version | Purpose |
|-------|---------|---------|
| `rusqlite` | 0.40 | SQLite (bundled) |
| `ignore` | 0.4 | `.gitignore`-aware file walking |
| `globset` | 0.4 | Glob pattern matching |
| `include_dir` | 0.7 | Embed directory contents in binary |
| `infer` | 0.19 | File type detection |
| `dirs` | 6 | Standard OS directories |
### Text & Search
| Crate | Version | Purpose |
|-------|---------|---------|
| `regex` | 1 | Regular expressions |
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
| `similar` | 3 | Diff computation |
| `syntect` | 5 | Syntax highlighting |
| `tiktoken-rs` | 0.12 | OpenAI token counting |
### Cryptography & Encoding
| Crate | Version | Purpose |
|-------|---------|---------|
| `base64` | 0.22 | Base64 encoding |
| `sha2` | 0.11 | SHA-256 hashing |
| `hex` | 0.4 | Hex encoding |
| `uuid` | 1 | UUID generation (v4, v5) |
| `libc` | 0.2 | Raw C FFI bindings |
### Error Handling & Logging
| Crate | Version | Purpose |
|-------|---------|---------|
| `anyhow` | 1 | Error handling |
| `tracing` | 0.1 | Structured logging |
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
| `chrono` | 0.4 | Date/time with serde |
### Other
| Crate | Version | Purpose |
|-------|---------|---------|
| `lsp-types` | 0.97 | LSP protocol types |
| `futures-util` | 0.3 | Async stream combinators |
## External Services
| Service | Integration | Notes |
|---------|-------------|-------|
| **LLM providers** | HTTP API (OpenAI-compatible) | Configurable via app_config.json |
| **MCP servers** | stdio or HTTP | Model Context Protocol |
| **git** | CLI (spawns `git`) | Via git_operator/git_worktree/git_cred tools |
| **sh** | CLI (spawns `sh`) | Via bash tool |
| **webbrowser** | opens URL | OAuth browser flow |
| Service | Purpose |
|---------|---------|
| **Anthropic API** | Primary LLM provider |
| **OpenAI API** | Alternative LLM provider (including OAuth) |
| **GitHub** | Release artifacts via semantic-release CI |
| **MCP Servers** | External tool servers (stdio or HTTP) |
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
## Build Configuration
### Compiler Lints (`.cargo/config.toml`)
All unused code, dead code, and deprecation warnings promoted to errors:
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
### Release Profile
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
### CI/CD
- **CI**: cargo build + test + clippy on every push
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
+64 -49
View File
@@ -1,64 +1,79 @@
<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~700 -->
# Frontend / TUI
# Frontend (TUI) Architecture
## Render Pipeline
```
ratatui::Terminal::draw(|frame|)
→ view::draw(frame, AppStateRest)
→ render_main_panel / render_overlay (based on overlay state)
→ render_input_bar
→ draw_status_bar
→ render_toasts (top-right floating notifications)
```
## Layout
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
```
┌──────────────────────────────────────────────┐
│ Chat Panel (main_area: Min 3)
│ ┌────────────────────────────────────────┐ │
│ │ User: Hello │ │
│ Agent: Hi there, how can I help? │
│ │ │ │
│ │ Toast notifications (top-right) │ │
│ └────────────────────────────────────────┘
├──────────────────────────────────────────────┤
│ Input Bar (3 lines)
│ > Some text... │
├──────────────────────────────────────────────┤
│ Status Bar (1 line) │
│ ┌ Provider │ Model │ Tokens │ Mode │ Quit ─┤
└──────────────────────────────────────────────┘
Timer tick
main.rs: fn tui_loop()
├── controller/input.rs: handle_key() → action
├── app/runtime/actions/mod.rs: apply_action()
│ └── state mutates (AppStateRest)
└── view/mod.rs: build TUI layout
├── view/chat.rs: Chat transcript
├── view/sidebar.rs: Usage dashboard
├── view/status.rs: Status bar
├── view/markdown.rs: Message renderer
├── view/workflow.rs: Hive-mind progress
└── view/theme.rs: Tokyo Night palette
```
## Key Files
## Overlay System
| File | Lines | Purpose |
|------|-------|---------|
| `src/view/mod.rs` | 623 | Frame draw, overlays (16 types), input bar, toasts |
| `src/view/chat.rs` | 155 | Chat transcript rendering with markdown |
| `src/view/markdown.rs` | 144 | Markdown → ratatui `Span` rendering (pulldown-cmark + syntect) |
| `src/view/status.rs` | ~50 | Status bar with provider/model/tokens |
| `src/view/workflow.rs` | 88 | Workflow progress visualization |
| `src/view/theme.rs` | 23 | Color palette (23 named colors) |
| `src/controller/input.rs` | 281 | Key event → Action mapping |
16 overlays managed by `app/mode/`:
## Overlays (16 types)
| Overlay | File | Purpose |
|---------|------|---------|
| Chat input | `mod.rs` | Main input bar with autocomplete |
| Bash | `bash.rs` | Interactive shell panel |
| Editor | `editor.rs` | Built-in file editor |
| Effort | `effort.rs` | LLM effort selector |
| Help | `help.rs` | Keybindings help |
| Key Input | `key_input.rs` | Custom key binding |
| Learning | `learning.rs` | Lesson viewer |
| Loading | `loading.rs` | Spinner overlay |
| MCP | `mcp.rs` | MCP server management |
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
| Rewind | `rewind.rs` | Message/history rewind |
| Settings | `settings.rs` | Settings panel |
| Todo | `todo.rs` | Task/TODO list |
| Workflow | (via view) | Workflow progress |
`Overlay::Help | Settings | Bash | QuitConfirm | Workflow | KeyInput | Editor | Effort | Mcp | Todo | Rewind | Learning | Usage | Loading | ModelSelector | ClearConfirm`
## Layout Structure
Each overlay renders a centered popup via `render_overlay()`.
```
┌─────────────────────────────────────────────┐
│ Status Bar (view/status.rs) │
├──────────────────────┬──────────────────────┤
│ │ │
│ Chat Transcript │ Sidebar │
│ (view/chat.rs) │ (view/sidebar.rs) │
│ scrollable, │ tokens, status, │
│ inline-log style │ agent info │
│ │ │
├──────────────────────┴──────────────────────┤
│ Input Bar + Autocomplete dropdown │
│ (view/mod.rs) │
└─────────────────────────────────────────────┘
```
## State Mutations
## Input Handling
State is mutated in-place from two locations:
- `src/controller/input.rs` — keyboard shortcuts and overlay interactions
- `src/app/runtime/actions/mod.rs``apply_action()` reducer for all programmatic actions
`controller/input.rs`:
## Toast Notifications
- Normal mode: keystrokes go to the active overlay
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
- Tab cycles autocomplete candidates
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
- Arrow keys scroll chat, sidebar, and other scrollable panels
`render_toasts()` — floating stack at top-right, color-coded by severity:
- Info: blue, Success: green, Warning: yellow, Error: red, Lesson: cyan
- Max 4 visible, auto-expire after 5s lifetime
## Theme
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,114 @@
# Clipboard Copy via OSC52 — Design
**Status:** Approved, pending implementation plan
**Date:** 2026-07-15
**Scope:** `src/app/state/misc.rs`, `src/controller/input.rs`, `src/main.rs`,
`src/ipc/protocol.rs`
## Context
There is no clipboard support anywhere in the TUI today, and mouse capture is enabled
(`EnableMouseCapture` in `main.rs`), which in most terminal emulators suppresses native
click-drag text selection unless the user holds a modifier — making an in-app copy action
more valuable than it would be in a plain scrollback. OSC52 is a terminal escape sequence
(`\x1b]52;c;<base64>\x07`) that asks the terminal emulator itself to set the system
clipboard; it needs no OS-level clipboard library (no X11/Wayland/win32 dependency) and
the `base64` crate is already a dependency (used in `service/oauth/pkce.rs`), so no new
crate is needed for this feature.
Key architectural constraint discovered while designing this: `controller::input::handle_key`
runs on the **daemon** process in `--daemon`/`--attach` mode (`main.rs:359`, inside
`handle_daemon_client`), not on the process that owns the user's actual terminal. A raw
`io::stdout()` write inside `handle_key` would go to the headless daemon's stdout in that
mode, not the user's terminal. The copy action therefore can't write the escape sequence
directly from `handle_key` — it has to signal intent via state, and the terminal-owning
process (single-process `run_loop_inner`, or the attach client's loop) performs the actual
write.
## Goals
- `Ctrl+Y` copies the most recent `Role::Assistant` message's raw text (not the rendered
markdown spans) to the system clipboard via OSC52.
- Works identically in single-process mode and in `--daemon`/`--attach` mode.
- No new dependency.
## Non-goals
- No native clipboard fallback (e.g. `arboard`) for terminals that don't honor OSC52 —
unsupported terminals silently swallow the escape sequence; no error surfaces to the
user beyond the optimistic "Copied to clipboard" toast (there's no ack mechanism in the
OSC52 protocol to verify the terminal actually did it).
- No copy-last-code-block variant — out of scope for this pass; the whole-message copy
covers the common case and is simple to extend later if needed.
- No mouse-drag text selection — unrelated, much larger feature; not being built here.
## State (`misc.rs`)
- `MiscState` gains `pub pending_clipboard_copy: Option<String>`, initialized to `None` in
`MiscState::new()`.
## `input.rs`
- New top-level arm alongside the existing `Ctrl+C`/`Ctrl+D` handlers:
`KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL)`. It finds the last
message in `state.transcript_cache.messages` with `role == Role::Assistant`:
- If found: `state.misc.pending_clipboard_copy = Some(msg.content.clone())`.
- If not found: push an `Info` toast ("No assistant message to copy yet") and leave
`pending_clipboard_copy` as `None`.
- Returns `Vec::new()` — this is a direct state mutation inside `handle_key`, matching
the existing `Ctrl+S` editor-save precedent (`main.rs`'s editor branch also mutates
state/does I/O directly rather than going through an `Action`).
## OSC52 write helper (`main.rs`)
```
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
write!(stdout, "\x1b]52;c;{b64}\x07")?;
stdout.flush()
}
```
Generic over `impl Write` so both the single-process loop (writing to `io::stdout()`) and
tests (writing to a `Vec<u8>` to assert the formatted sequence) can use it without a real
terminal.
## Single-process mode (`run_loop_inner`)
After the existing `for action in actions { apply_action(state, action); }` block, add:
```
if let Some(text) = state.misc.pending_clipboard_copy.take() {
let _ = write_osc52(&mut io::stdout(), &text);
state.push_toast(Toast::new(ToastKind::Success, "Copied to clipboard".into()));
}
```
## Daemon/attach mode
- `ipc/protocol.rs`: add `DaemonFrame::ClipboardCopy(String)` (alongside `StateUpdate`,
`StreamToken`, `SystemNote`, `Closed` — same `Serialize`/`Deserialize` derive).
- `handle_daemon_client` (`main.rs`): after each branch that calls `handle_key`/`apply_action`
(`KeyPress` and `Submit`, the only two that can reach the input handler), before the
existing `send_daemon_update(&mut conn, state)?;` call, add:
```
if let Some(text) = state.misc.pending_clipboard_copy.take() {
conn.send(&DaemonFrame::ClipboardCopy(text))?;
}
```
- Attach-client loop (`main.rs`, the function matching on `DaemonFrame::StateUpdate` /
`SystemNote` / `Closed` around line 573): add a `DaemonFrame::ClipboardCopy(text) => {
let _ = write_osc52(&mut io::stdout(), &text); client_state.push_toast(...); }` arm,
mirroring the existing `SystemNote` handling but performing the actual terminal write
since this process — not the daemon — owns the user's terminal.
## Testing
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
- `input.rs`: `Ctrl+Y` with a transcript containing multiple messages sets
`pending_clipboard_copy` to the *last* assistant message's content, ignoring later
user/tool messages that might follow it; with no assistant message present, it pushes
an info toast and leaves `pending_clipboard_copy` as `None`.
- `main.rs`: `write_osc52` writing into a `Vec<u8>` buffer produces the exact expected
`\x1b]52;c;<base64>\x07` byte sequence for a known input string.
@@ -0,0 +1,115 @@
# Diff View for edit/write Tools — Design
**Status:** Approved, pending implementation plan
**Date:** 2026-07-15
**Scope:** `src/tool/fs/edit.rs`, `src/tool/fs/write.rs`, `src/view/markdown.rs`, `src/view/chat.rs`
## Context
`edit` currently reports only a byte-delta (`"edited {rel} ({N} byte delta)"`), and `write`
reports only a byte count. Neither the model nor the user sees what actually changed —
just a number. This makes it hard for the model to self-verify an edit landed correctly,
and hard for the user to review a change without opening the file. No diff-computing
library exists in the dependency tree today.
## Goals
- `edit` returns a real unified diff (git-style, 3 lines of context) of the change it just
made, in place of the byte-delta note.
- `write` returns the same kind of diff when it overwrites a file that already existed
with valid UTF-8 content; falls back to the current "wrote N bytes" message for new
files or non-UTF-8 (binary) overwrites.
- Diffs render in the chat view with real color (green add / red remove / cyan hunk
header) instead of being flattened to dim/italic like other tool output.
- Large diffs are truncated with a trailing count, matching the existing pattern in
`read.rs` (`"... ({N} more lines, total {total})"`).
## Non-goals
- No diff view for any tool besides `edit`/`write` (e.g. no retroactive diffing of
`bash_tools.rs` shell edits).
- No side-by-side diff layout — unified format only, matching how every other tool
output already renders as a single text stream.
- No persistence of diff history; each diff is only the delta of the single tool call
that produced it, not a cumulative session diff.
- No changes to non-tool (assistant/user/system) message rendering or coloring.
## Dependency
Add `similar = "3"` (line/word diff crate; permissive MIT/Apache-2.0, no heavy
transitive deps). Use `TextDiff::from_lines(old, new).unified_diff().context_radius(3)`,
which produces standard `@@ -a,b +c,d @@` hunk headers and `-`/`+`/` `-prefixed lines —
no custom diff algorithm needed.
## Tool changes
### `edit.rs`
After computing `new_content` and writing it to disk:
1. Compute `similar::TextDiff::from_lines(&content, &new_content).unified_diff().context_radius(3).to_string()`.
2. Split into lines; if `> MAX_DIFF_LINES` (200), keep the first 200 and append
`"... ({N} more lines truncated)"`.
3. Wrap the (possibly truncated) diff text in a fenced ` ```diff ` block.
4. Replace the byte-delta note in the returned message with this block; keep the
existing "Graduated checks matched" / LSP note suffixes in their current position
(after the diff block).
### `write.rs`
Before overwriting:
1. If `path.exists()` and `fs::read_to_string(&path)` succeeds (valid UTF-8), capture it
as `old_content` and note `is_overwrite = true`.
2. If the file doesn't exist, or reading it fails (binary/non-UTF-8), `is_overwrite = false`
— no error, just skip the diff path silently.
3. After writing, if `is_overwrite`, compute and truncate the diff exactly as in `edit.rs`
and append the fenced block to the return message (in addition to the existing
"wrote N bytes" line, not instead of it — for `write`, unlike `edit`, the byte count is
still useful since it can be a full-file rewrite).
4. If not `is_overwrite`, return message is unchanged from today.
The truncation constant (`MAX_DIFF_LINES = 200`) and truncation message format are
shared — factor into a small helper in `tool/fs/helpers.rs` used by both tools.
## Rendering changes
### `markdown.rs`
- `render_markdown` gains a `dim: bool` parameter: `render_markdown(text, width, dim)`.
- Capture the fence language from `Tag::CodeBlock(CodeBlockKind::Fenced(lang))` (today
matched as `CodeBlock(_)`, discarding the language). Track `in_diff_block: bool` when
`lang == "diff"`.
- Inside a diff block, process text line-by-line instead of as one blob: a line starting
with `+` (not `+++`) is styled green, `-` (not `---`) red, `@@` cyan/muted, everything
else (context lines, `+++`/`---` file headers) uses the existing code-block teal.
- When `dim` is `true`: every span keeps its assigned color as computed above, but
non-diff spans (headings, links, plain text, non-diff code blocks, table cells) fall
back to `Theme::TEXT_DIM` + `Modifier::ITALIC` instead of their normal palette color —
this replicates today's "tool output is always dim" behavior for everything except
diff lines.
- When `dim` is `false`: behavior is unchanged from today (full color, used for
assistant/user/system messages).
### `chat.rs`
- `Role::Tool` branch: replace the two manual span-remapping loops (that force every
span to `dim_italic`) with a direct call to `render_markdown(&content, content_width, true)`
and use the returned spans as-is.
- All other roles: call `render_markdown(&content_str, content_width, false)` — same
call as today, just with the new explicit `false` argument.
## Testing
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
- `edit.rs`: a normal single-replace edit produces a diff block with matching
`-`/`+` lines; a `replace_all` across 250+ lines truncates at 200 with the correct
trailing count.
- `write.rs`: writing a brand-new file keeps the old "wrote N bytes" message with no
diff block; overwriting an existing UTF-8 file produces a diff block; overwriting
a path that reads as invalid UTF-8 (simulate via non-UTF-8 bytes) falls back to the
byte-count message without erroring.
- `markdown.rs`: a fenced ` ```diff ` block with `+`/`-`/`@@` lines produces spans with
the expected fg colors under `dim=true` (diff lines colored) and confirms non-diff
text in the same call falls back to `TEXT_DIM` + italic.
@@ -0,0 +1,126 @@
# Fuzzy @file-mention Autocomplete — Design
**Status:** Approved, pending implementation plan
**Date:** 2026-07-15
**Scope:** `src/app/state/misc.rs`, `src/app/state/rest.rs`, `src/controller/input.rs`,
`src/view/mod.rs`, `src/tool/mod.rs`, `src/tool/fs/write.rs`, `src/main.rs`
## Context
The chat input already has a dropdown autocomplete (`InputState` in `misc.rs`), but it
only covers slash commands: it requires the whole buffer to start with `/` and filters a
fixed `COMMANDS` list by prefix. There's no way to reference a project file from the chat
input without typing its exact path from memory. The existing `dir_cache` (used by the
`dir_cache_update` tool) looks like it could serve this but doesn't: it's a single,
non-recursive directory snapshot, overwritten on each LLM-driven `dir_cache_update` call —
not a standing, recursive, whole-workspace file index. `search.rs`'s `Grep`/`Glob` tools
already do the recursive, `.gitignore`-respecting walk this feature needs, via
`ignore::Walk`.
Also relevant: there is no persistent async runtime driving the TUI loop. `main.rs`
constructs a `tokio::runtime::Runtime` but never `.enter()`s or `block_on`s it in the
main loop — `run_loop` is fully synchronous. The one existing async-flavored pattern
(`dir_cache_update.rs`) spins up a throwaway one-shot runtime purely to satisfy
`tokio::sync::RwLock`'s API, then discards it. This feature does not need that ceremony:
a plain `std::sync::RwLock` is enough, since every reader/writer here is synchronous
(`handle_key`, `Tool::run`, and the index-build thread all being plain sync code).
## Goals
- Typing `@` at a word boundary (start of buffer or after whitespace) in the chat input,
followed by non-whitespace characters, opens a dropdown of fuzzy-matched project file
paths, live-updating as the query changes.
- Selecting a candidate splices `@relative/path ` into the buffer at the mention's
position (not a whole-buffer replace) and the user keeps typing.
- Candidates come from a background-built, whole-workspace file index — not the
LLM-facing `dir_cache`.
## Non-goals
- No auto-reading of the selected file's content into the conversation — the inserted
`@path` is plain text; the model reads it via the `read` tool if it wants to, same as
any other path reference.
- No live re-filter on Backspace/Delete while a mention dropdown is open — mirrors the
slash-command dropdown's existing behavior (closes on Backspace/Delete rather than
refiltering). Not fixing that for commands here; file mentions just inherit it for
consistency.
- No periodic re-walk of the index after startup — only single-file incremental updates
on file creation (see below). A deleted or renamed file may show a stale entry until
restart; acceptable since selecting it just inserts text, it doesn't touch the
filesystem.
- No fuzzy matching over directories, only files.
## Dependency
Add `nucleo-matcher = "0.3"` (the fuzzy-matching engine from the Helix editor project;
small, actively maintained, no heavy transitive deps).
## Index storage & construction
- New type in `misc.rs`: `MentionIndex { entries: Arc<std::sync::RwLock<Vec<String>>> }`, with `MentionIndex::new()`, `set(&self, paths: Vec<String>)`, and `snapshot(&self) -> Vec<String>` (both plain sync `.write()`/`.read()`, no `try_`/async — a std `RwLock` doesn't block indefinitely here since every hold is a quick vec swap or clone).
- `AppStateRest` gets a `pub mention_index: MentionIndex` field, initialized in `AppStateRest::new()`, threaded into `ToolCtx`/`ToolCtxBuilder` the same way `dir_cache` is (new `mention_index` field on both, wired through `tool_ctx()`/`tool_ctx_for()`/`build()`).
- In `main.rs`, right after `AppStateRest::new(...)` in the single-process TUI path and the daemon path (not the attach-only client path, which has no local `ToolCtx`), spawn `std::thread::spawn` that:
1. For each workspace root (index `i`, path `w`): `ignore::Walk::new(w)`, keep only files, strip `w` as prefix, format as `rel` for `i == 0` or `[i]rel` for `i > 0` (matching `resolve_path`'s existing workspace-index convention).
2. Stop collecting once the total across all workspaces hits 50,000 entries (repos larger than that are rare here; this is a soft cap to bound memory/scan time, not a hard requirement).
3. Call `mention_index.set(all_paths)`.
- `write.rs`: after a successful write, if the target path did **not** exist before the write (i.e. this created a new file, not an overwrite), compute its relative/workspace-prefixed form and push it onto `ctx.mention_index`'s vec directly (read-modify-write under the same lock) rather than re-walking.
## `InputState` changes (`misc.rs`)
- New `pub enum AutocompleteKind { Command, FileMention }`.
- `InputState` gains `pub autocomplete_kind: AutocompleteKind` (default `Command`) and
`pub mention_start: usize` (byte offset of the triggering `@`).
- New `fn mention_query_at_cursor(&self) -> Option<(usize, String)>`: scans backward from
`self.cursor` for an `@`; the scan stops (returns `None`) if it hits whitespace before
finding `@`. The `@` only counts as a trigger if it's at buffer start or immediately
preceded by whitespace. Returns `(byte offset of '@', query text between '@' and cursor)`.
- New `fn open_mention_autocomplete(&mut self, files: &[String])`: calls
`mention_query_at_cursor()`; if `None`, calls `close_autocomplete()` and returns. If
`Some((start, query))`, fuzzy-matches `query` against `files` via `nucleo-matcher`,
keeps the top 10 by score, sets `autocomplete_candidates`, `autocomplete_kind =
FileMention`, `mention_start = start`, `autocomplete_visible = !candidates.is_empty()`.
- `select_autocomplete()` becomes kind-aware:
- `Command` (today's behavior, unchanged): `buffer = candidate.clone()`, `cursor =
buffer.len()`.
- `FileMention`: `buffer.replace_range(mention_start..cursor, &format!("@{candidate} "))`,
`cursor = mention_start + candidate.len() + 2` (the `@` plus the candidate plus the
trailing space).
- Both paths end with `close_autocomplete()`, same as today.
## `input.rs` wiring
- `KeyCode::Char(c)` handler: after `state.input.insert(c)`, keep the existing
`if buffer.starts_with('/') { open_autocomplete() }` check, and add an `else if let
Some(_) = state.input.mention_query_at_cursor() { state.input.open_mention_autocomplete(&state.mention_index.snapshot()) }` branch. These are mutually exclusive in practice (a
buffer starting with `/` is a slash command, not a sentence with an `@mention` in it).
- `KeyCode::Backspace` / `KeyCode::Delete`: unchanged — both already just call
`close_autocomplete()` when a dropdown is visible, regardless of kind. No new branching
needed since `close_autocomplete()` already resets `autocomplete_kind` isn't touched but
becomes irrelevant once `autocomplete_visible` is false.
- `KeyCode::Tab`: currently gated on `buffer.starts_with('/')`. Extend the condition to
also fire when `autocomplete_kind == FileMention && autocomplete_visible` so Tab cycles
file-mention candidates too.
- `KeyCode::Enter`: unchanged — already calls `select_autocomplete()` whenever
`autocomplete_visible`, which is now kind-aware internally.
## Rendering (`view/mod.rs`)
- `render_input_bar`'s dropdown block reuses the exact same list-rendering code (already
generic over `autocomplete_candidates`/`autocomplete_idx`); only the title changes based
on `state.input.autocomplete_kind`: `" ⌘ Commands "` (unchanged) vs `" 📁 Files "`.
## Testing
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
- `misc.rs`: `mention_query_at_cursor` returns the right `(start, query)` for `@` at
buffer start, `@` after a space mid-sentence, and correctly returns `None` when the `@`
is mid-word (e.g. `foo@bar`) or when whitespace exists between the `@` and the cursor.
`select_autocomplete` for `FileMention` splices correctly into a buffer with text before
and after the mention span; `Command` selection still replaces the whole buffer as
before.
- `write.rs`: creating a new file appends its path to the shared `mention_index`;
overwriting an existing file does not add a duplicate entry.
- Index construction: not unit-tested directly (it's a `std::thread::spawn` walking the
real filesystem at startup) — covered implicitly by exercising the app manually per the
`verify` skill during implementation.
@@ -0,0 +1,281 @@
# Context & Compaction Overhaul — Design
**Status:** Approved, pending implementation plan
**Date:** 2026-07-16
**Scope:** replaces `src/app/runtime/shortsend.rs`; touches `src/app/runtime/actions/mod.rs`,
`src/view/status.rs`, `src/model/settings.rs`, `src/app/subagent/division.rs`, `Cargo.toml`
## Context
The existing conversation-compaction system (`shortsend.rs`, 129 lines) only acts once the
context is already close to the model's window limit, and has accumulated inconsistencies
found during a codebase audit:
1. Three different token-count heuristics for the same job: `/3` inside
`shortsend::shape_messages`, `/4` in the auto-compact loop
(`actions/mod.rs` ~line 1146), `/4` again in the live status bar (`view/status.rs:68`).
2. Manual `/compact` (`Action::Compact`, `actions/mod.rs:547-563`) passes `client: None`
because `apply_action` is synchronous, so it never gets LLM summarization — it always
falls back to the bare `"[prior conversation compacted]"` placeholder, unlike automatic
mid-turn compaction (`Some(&tc.client)`, line 1160). Undocumented asymmetry between the
two trigger paths.
3. `context_window` resolution (`model_roles.values().find(...).and_then(...).unwrap_or(...)`)
duplicated three times (`Action::Compact`, `spawn_turn`, `view/status.rs` twice).
4. No repeated-tool-call dedup: reading the same file (or running the same grep) twice in a
session keeps both full copies in context forever, until compaction eventually drops the
older one wholesale along with everything else from that period.
5. No per-result compression: a single large tool output (a big `bash` log, a large `grep`
result) is stored verbatim even when most of it is redundant or low-value.
6. Zero test coverage on `shortsend.rs`.
Separately, research into three real, permissively-licensed open-source projects
(`rtk-ai/rtk`, Apache-2.0; `headroomlabs-ai/headroom`, Apache-2.0; `JuliusBrussee/caveman`,
MIT — verified via `gh api` for authenticity/license, and by cloning and reading source, not
taken from marketing blog posts) surfaced techniques worth reimplementing natively:
- **rtk**: generic line-scan compression (strip comment/blank runs, brace-depth collapse of
function bodies, importance-ranked truncation ending in an unambiguous `[N more lines]`
marker — their own regression tests show a comment-shaped marker confuses the LLM into
retry-looping) plus structured per-toolchain parsing (e.g. `cargo --message-format=json`
bucketed into errors/warnings, boilerplate lines dropped).
- **headroom**: per-content-type compressors — logs (classify lines by level/stack-trace/
summary, score, keep highest-value lines + surrounding context, adaptive cap), grep
results (group by file, score matches, cap globally and per-file), JSON (keep all
structural tokens — keys, brackets, colons — drop or shrink long low-entropy string
values, keep short values and UUID/hash-shaped high-entropy ones).
- **caveman**: a pure prompt/persona instruction (no algorithm) that tells the model to
write tersely — drop articles/filler/hedging, keep code/commands/errors verbatim — with an
explicit carve-out that disables terseness for destructive-op confirmations and security
warnings. This compresses *output* tokens, a different axis from everything else in this
design, which compresses *input* context.
This is a from-scratch reimplementation of the underlying ideas, not a port — no code is
copied from any of the three projects.
## Goals
- One unified, always-on pipeline that keeps context lean from turn 1, not just once near
the limit.
- Deduplicate repeated tool calls: an older copy of a tool result superseded by an identical
later call (same tool name + same arguments) is replaced with a placeholder, for read-only
tools only.
- Compress large individual tool results (logs, JSON, generic text) at capture time, above a
size floor.
- Fix the three known inconsistencies (token heuristic, manual/auto asymmetry,
`context_window` duplication).
- Optional, off-by-default "concise mode" system-prompt toggle for terser model output.
- Full inline test coverage per repo convention.
## Non-goals
- Not adding a runtime dependency on `rtk`, `headroom`, or `caveman` themselves (as a binary,
proxy, or crate) — everything is implemented natively in Rust inside zesdex.
- Not building rtk's per-toolchain structured parsers (`cargo --message-format=json`
re-invocation, etc.) — too invasive for a general-purpose `bash` tool that runs arbitrary
commands zesdex doesn't control the flags of. Only the generic line-scan/log/JSON layer is
built.
- Not switching to an exact per-provider tokenizer — `tiktoken-rs` (BPE, cl100k_base/
o200k_base) is an approximation good enough for the 85%/95% budget thresholds; it is not
used for billing-accurate counts.
- `caveman-compress`-style memory-file rewriting (the LLM-round-trip variant of caveman) is
out of scope — only the pure-prompt persona mechanism is adopted.
## Architecture
Replace `src/app/runtime/shortsend.rs` with `src/app/runtime/context/`:
```
context/
mod.rs — module registration only, no facade (see below)
tokens.rs — unified token counting (tiktoken-rs)
dedup.rs — cross-call tool-result deduplication
squash.rs — per-result compression (log/json/generic), applied at
tool-result construction time, upstream of prepare()
shaping.rs — budget-based drop + LLM summarize (renamed shortsend logic)
window.rs — shared context_window resolution
```
### `tokens.rs`
```rust
pub fn count_tokens(text: &str) -> usize
pub fn count_message_tokens(msg: &ChatMessage) -> usize
```
Backed by `tiktoken-rs` (new dependency, pure Rust, embedded BPE vocab, no network calls at
runtime), using `o200k_base`. Replaces all three existing heuristic call sites: `shortsend`'s
internal `/3`, the auto-loop's `/4` (`actions/mod.rs` ~1146), and `status.rs:68`'s `/4`.
### `dedup.rs`
```rust
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool)
```
The `bool` is `true` iff at least one message was replaced with a placeholder — callers use
it to decide whether the result is worth persisting/announcing, without needing `ChatMessage`
to implement `PartialEq` (it doesn't today, and adding it purely to diff whole message lists
would be needless surface area for what `collapse` already knows precisely mid-walk).
Flow: walk messages, pair each `Role::Tool` message to its originating `ToolCall` via
`tool_call_id`. Key = `(function.name, sha256(canonical_json(function.arguments)))` (`sha2`
is already a dependency). Track the last index seen per key. For any earlier occurrence of a
key whose tool name is in the read-only set, replace that earlier `Tool` message's `content`
with a short placeholder (`"[duplicate result — superseded by a later identical call, see
below]"`); the assistant's tool-call entry (name + arguments) is left untouched, so the
action/audit trail stays intact. Mutating tools are never touched, even with identical
arguments, because call order and repetition can be semantically meaningful (e.g. retrying a
flaky `bash` command).
Read-only classification reuses `subagent::division::tool_scope::READ_TOOLS`
(`src/app/subagent/division.rs:21`) rather than a new list — that `const` is made `pub` for
this purpose. It already enumerates exactly the read-only tool set (`read`, `grep`, `glob`,
`search`, `seqthink`, `recall`, `lsp_*`, `read_findings`).
Runs every turn, unconditionally, before token counting — not gated on `should_shape`.
### `squash.rs`
```rust
pub fn apply(tool_name: &str, output: &str) -> String
```
`read` is exempted entirely, always passed through unchanged regardless of size: its output
must stay byte-exact because the agent relies on it for exact-match edits afterward, and a
squashed view of a JSON config file (or any file whose content happens to parse as JSON)
would otherwise be silently altered. Size floor for every other tool: outputs under 1500
bytes pass through unchanged (compression only pays off on large output, and touching small
results risks losing detail with no token benefit). Above the floor, dispatch by content
shape:
- `squash_json(&str) -> String` — walks a parsed `serde_json::Value` (not a hand-rolled
tokenizer — `serde_json` already handles escaping/nesting correctly, reusing it is simpler
and more robust); structural tokens (keys, brackets, colons, commas, booleans, null) always
kept; string values kept if ≤20 chars or "identifier-shaped" (no internal whitespace *and*
Shannon entropy ≥3.0 bits/char — catches UUIDs/hashes/paths), otherwise replaced with `"…"`
in place; array elements past the first 3 compressed harder (values elided regardless of
length/entropy). Applied when `serde_json::from_str` on the output succeeds. The
no-whitespace pre-filter matters: raw per-character entropy alone doesn't separate prose
from identifiers — repeated English prose measures ~3.89 bits/char, higher than a UUID's
~3.39 — because prose also draws from a wide character set. headroom's own entropy gate is
"cheaply pre-filtered by 'no spaces'" before scoring for the same reason; multi-word values
never reach the entropy check at all under this rule.
- `squash_log(&str) -> String` — line classifier (error/fail/warn/info/debug/trace by
keyword + stack-trace-frame detection) → score
(`level_score {1.0 error/fail, 0.5 warn, 0.1 info, 0.05 debug/trace} + 0.3 if
stack-trace-frame + 0.4 if summary-shaped line`) → keep up to 20 highest-scored error
lines, up to 10 highest-scored warning lines, all summary lines, plus a ±2-line context
window around each kept line → single `[N lines omitted]` marker for drops (not
comment-shaped, per rtk's own finding on LLM confusion). Applied when the output isn't
valid JSON, the tool is `bash`, and the output has ≥3 lines matching error/warn/stack-trace
patterns. The tool restriction (added after the final whole-branch review) matters: a `grep`
result full of matches against error-handling code trips the same ≥3-line keyword threshold
as a real build log, but `squash_log`'s hard 20-error/10-warning cap has no byte budget and
would silently drop legitimate matches past it — the wrong compressor for search results.
Only `bash` (the actual log-producing tool) routes through `squash_log`; every other tool
whose output happens to look log-shaped falls through to the gentler, byte-budgeted
`squash_generic` instead.
- `squash_generic(&str, budget) -> String` — importance-ranked truncation: keeps the first 10
and last 10 lines plus any line matching a small "looks important" heuristic (non-blank,
not a byte-for-byte repeat of the immediately preceding line), single `[N lines omitted]`
marker for the rest, capped to `budget` bytes overall (`budget` = the 1500-byte squash
floor doubled, i.e. 3000 bytes, chosen so the fallback path still yields a real reduction
on anything that triggered it). Fallback for anything that isn't JSON or log-shaped.
Called once, at the single tool-result construction site
(`actions/mod.rs:1420`, `let tool_msg = ChatMessage::tool_result(tool_call.id.clone(),
output);`) — `output` is passed through `squash::apply(&tool_name, &output)` before being
wrapped. Runs before the result is ever archived or pushed into `msgs`, so compression is
permanent and applies uniformly whether or not compaction ever triggers.
### `shaping.rs`
Unchanged behavior from today's `shortsend.rs` (hysteresis `should_shape`, 70%-budget
newest-first retention, LLM summarization of dropped messages), moved as-is into this file
and updated to source token counts from `tokens.rs` instead of its own heuristic.
### `window.rs`
```rust
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize
```
Replaces the three duplicated `model_roles.values().find(...).and_then(...).unwrap_or(...)`
blocks in `Action::Compact`, `spawn_turn`, and `view/status.rs` (×2).
### `mod.rs`
No facade function — just `pub mod dedup; pub mod shaping; pub mod squash; pub mod tokens;
pub mod window;`. `dedup`, `shaping`, and `tokens` are called directly from each call site
(the auto-loop and `Action::Compact`), matching CLAUDE.md's "No DI — modules call ...
directly" convention rather than introducing an orchestration layer that only one of the two
callers would use generically (the auto-loop already needs per-stage control today — it
inspects `should_shape` itself to decide whether to emit `TurnEvent::Compacted` — and would
have to unpack a facade's result anyway).
## Data flow (per turn)
1. Tool executes → raw `output: String`.
2. `squash::apply(tool_name, &output)` — compress if over the size floor (`read` exempted).
3. Wrapped into `ChatMessage::tool_result(...)`, archived, pushed to `msgs`.
4. Once per loop iteration: `dedup::collapse(&msgs)` (always) → sum
`tokens::count_message_tokens` over the result → `shaping::should_shape` → conditionally
`shaping::shape_messages`.
5. Result pushed as `TurnEvent::Compacted` if dedup changed anything or shaping triggered,
consumed on the main thread to update `SessionRuntime.messages`.
## Fixing the manual/auto asymmetry
`Action::Compact` (`actions/mod.rs:547`) currently runs synchronously inside `apply_action`
and can't block on an LLM call. Fix: make it spawn a background `std::thread::spawn` — the
same pattern `spawn_turn` already uses (`actions/mod.rs:694`) — that runs `dedup::collapse`
then unconditionally `shaping::shape_messages(.., force=true, Some(&client))` and reports back
via `TurnEvent::Compacted`, identical to the automatic path. The toast sequence becomes
"Compacting…" immediately (optimistic, non-blocking) then "History compacted" when the
`TurnEvent` arrives. This gives manual `/compact` real LLM summarization instead of always
falling back to the placeholder.
## Concise mode (separate from the `context/` module)
- `Settings` (`src/model/settings.rs`) gains `pub concise_output: bool`, default `false`,
with `#[serde(default)]` for backward-compatible deserialization of existing
`settings.json` files (matching the existing `hive_mind_node_timeout_ms` precedent in the
same file).
- When `true`, `run_agent_turn`'s system-prompt assembly (`actions/mod.rs:930-936`) appends a
fourth section to `system_text`: a terse-writing instruction (persona-prompt only, no
algorithm — drop articles/filler/hedging/pleasantries, keep code/commands/error text
byte-exact) with an explicit carve-out disabling terseness for destructive-operation
confirmations and security-relevant warnings, mirroring caveman's own "Auto-Clarity"
safety exception.
- No UI toggle is in scope for this pass — confirmed no such mechanism exists today for any
boolean `Settings` field (`review_enabled`, `session_archive_enabled`,
`lsp_auto_provision` are all hand-edited in `settings.json`, same as this one will be).
## New dependency
`tiktoken-rs` — pure Rust, embedded BPE vocab (`cl100k_base`/`o200k_base`), no network calls
at runtime, MIT/Apache-2.0 dual-licensed. Added to `Cargo.toml`.
## Testing
Inline `#[cfg(test)] mod tests` per repo convention, one per new file:
- `dedup.rs`: same tool+args → older result replaced; different args → no-op; mutating tool
with identical args → both kept in full; unmatched `tool_call_id` (malformed history) →
no panic, treated as unpaired.
- `squash.rs`: JSON input under/over the size floor; JSON with long low-entropy string values
gets them elided while short/UUID-shaped values survive; log input with error/warn lines
keeps highest-scored lines and emits exactly one `[N lines omitted]` marker; generic text
keeps first/last N lines.
- `tokens.rs`: known-string token counts against fixed expected values; empty string → 0.
- `shaping.rs`: port the behavioral cases implied by today's hysteresis logic (85% trigger
when not previously shaped, 95% once shaped) plus budget-drop ordering.
- `window.rs`: role match resolves to the role's `context_window`; no match falls back to
`default_context_window`.
## Migration
- Delete `src/app/runtime/shortsend.rs`; all three call sites (`actions/mod.rs` auto-loop,
`Action::Compact`, and the module path itself) updated to `context::`.
- `view/status.rs` switches its live token display to `tokens::count_tokens`, so the status
bar finally matches what compaction measures internally.
+14 -4
View File
@@ -1,4 +1,9 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
#![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Global registry of running background bash jobs, and control operations
//! (output polling, kill) exposed to the rest of the app.
//!
@@ -10,7 +15,6 @@
//! Why: a single static map (rather than storing jobs in `AppStateRest`)
//! lets background jobs outlive the borrow of any particular state mutation
//! and be looked up by id from tool calls issued at arbitrary points.
use std::collections::HashMap;
use std::sync::Mutex;
use std::sync::OnceLock;
@@ -44,7 +48,11 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
while let Some(line) = job.try_read_line() {
lines.push(line);
}
if lines.is_empty() { None } else { Some(lines) }
if lines.is_empty() {
None
} else {
Some(lines)
}
}
/// Terminate a running background bash job and remove it from the registry.
@@ -58,7 +66,9 @@ pub fn bash_output(id: &str) -> Option<Vec<String>> {
/// Return: `Ok(())` on success, `Err` if the lock is poisoned or no job
/// with that id exists.
pub fn bash_kill(id: &str) -> anyhow::Result<()> {
let mut map = bash_jobs_map().lock().map_err(|e| anyhow::anyhow!("lock error: {e}"))?;
let mut map = bash_jobs_map()
.lock()
.map_err(|e| anyhow::anyhow!("lock error: {e}"))?;
let job = map.remove(id);
match job {
Some(job) => {
+19 -13
View File
@@ -9,11 +9,10 @@
//! Why: running bash commands on a detached thread with a channel (rather
//! than synchronously) lets the TUI stay responsive while long-running
//! shell commands execute in the background.
use std::io::BufRead;
use std::process::{Command, Stdio};
use std::sync::mpsc;
use std::thread;
use std::io::BufRead;
/// Maximum number of output lines buffered in memory per background job.
/// Beyond this limit, old output is dropped to prevent OOM (CWE-770).
@@ -59,17 +58,23 @@ pub fn spawn_bash_job(command: String) -> BashJob {
// Spawn a named thread for easier debugging. If Builder::spawn fails
// (e.g. OS resource limit), fall back to unnameable thread::spawn.
let thread_name = format!("bgbash-{}", &thread_id[..8.min(thread_id.len())]);
if thread::Builder::new().name(thread_name).spawn({
// Clone everything the closure captures so we can also pass it
// to the fallback thread without moving.
let cmd = cmd.clone();
let output_tx = output_tx.clone();
let pid_tx = pid_tx.clone();
let id_for_log = id_for_log.clone();
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log)
}).is_err()
if thread::Builder::new()
.name(thread_name)
.spawn({
// Clone everything the closure captures so we can also pass it
// to the fallback thread without moving.
let cmd = cmd.clone();
let output_tx = output_tx.clone();
let pid_tx = pid_tx.clone();
let id_for_log = id_for_log.clone();
move || spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log)
})
.is_err()
{
tracing::warn!("[bgbash:{}] failed to spawn named thread, using unnamed fallback", id_for_log);
tracing::warn!(
"[bgbash:{}] failed to spawn named thread, using unnamed fallback",
id_for_log
);
thread::spawn(move || {
spawn_bash_thread_body(&cmd, &output_tx, &pid_tx, &id_for_log);
});
@@ -139,7 +144,8 @@ fn spawn_bash_thread_body(
if output_tx.try_send(line).is_err() {
tracing::debug!(
"[bgbash:{}] output buffer full ({} lines), discarding remaining output",
id_for_log, MAX_OUTPUT_LINES,
id_for_log,
MAX_OUTPUT_LINES,
);
break;
}
-1
View File
@@ -1,5 +1,4 @@
//! Background bash: run shell commands off the main thread, poll their
//! output non-blockingly, and terminate them on demand.
pub mod control;
pub mod job;
+231 -158
View File
@@ -84,10 +84,17 @@ const ASSUMPTION_PATTERNS: &[&str] = &[
/// Network-exfiltration and credential-disclosure patterns for bash.
const EXFIL_PATTERNS: &[&str] = &[
"curl ", "wget ", "nc -e ", "ncat ", "/dev/tcp/",
"base64 -d |", "base64 --decode |",
"openssl s_client", "ssh -R ",
"scp /", "rsync /",
"curl ",
"wget ",
"nc -e ",
"ncat ",
"/dev/tcp/",
"base64 -d |",
"base64 --decode |",
"openssl s_client",
"ssh -R ",
"scp /",
"rsync /",
];
/// Substrings of well-known credential / secret files that bash must not read.
@@ -115,169 +122,63 @@ impl Harness {
/// Decide whether a tool call is allowed to execute.
///
/// Flow: ALL tools are gated (not just risky ones), closing the bypass
/// for MCP tools (which are never in the risky list). Basic path
/// traversal and reason validation applies to any tool with a `path`
/// argument. Heavy content scanning (stub/denial/assumption/exfiltration)
/// only applies to risky tools. MCP tools (mcp__ prefix) are treated
/// as risky because their behaviour is unknown.
/// for MCP tools (which are never in the risky list). Delegates to
/// smaller helper methods for each concern: path traversal, output
/// path validation, content scanning, bash safety, and reason checks.
///
/// Return: `Verdict::Allow` or `Verdict::Block(reason)`.
#[allow(clippy::too_many_lines, clippy::unnecessary_debug_formatting)]
pub fn gate_tool_call(
tool_name: &str,
args: &serde_json::Value,
workspace_roots: &[&std::path::Path],
) -> Verdict {
let is_risky = crate::tool::tool_is_risky(tool_name);
let is_mcp = tool_name.starts_with("mcp__");
// ── Universal checks applied to EVERY tool ──
// Path traversal: check ANY tool that accepts a path argument,
// not just write/edit/delete, so tools like read, MCP tools,
// and future tools are also protected.
if let Some(path) = args.get("path").and_then(|v| v.as_str()) {
if path.contains("..") {
return Verdict::Block(
"path traversal detected in 'path' argument".to_string(),
);
}
if !workspace_roots.is_empty() {
let abs_check = std::path::PathBuf::from(path);
if abs_check.is_absolute()
&& !workspace_roots.iter().any(|r| abs_check.starts_with(r))
{
return Verdict::Block(format!(
"absolute path '{path}' is outside all workspace roots"
));
}
}
// Universal checks applied to EVERY tool.
if let Some(v) = Self::check_path_traversal(args, workspace_roots) {
return v;
}
if let Some(v) = Self::check_output_path(tool_name, args, workspace_roots) {
return v;
}
// Workspace-root validation for output path.
if let Some(out_path) = Self::find_output_path(tool_name, args) {
if !workspace_roots.is_empty()
&& !out_path.starts_with("/tmp")
&& !out_path.is_absolute()
{
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
if !allowed {
return Verdict::Block(format!(
"output path '{out_path:?}' is outside all workspace roots"
));
}
}
}
// ── Risky / MCP tool checks ──
// Non-risky, non-MCP tools (read, grep, glob, recall, etc.) are
// allowed after universal checks above.
// Non-risky, non-MCP tools pass after universal checks.
if !is_risky && !is_mcp {
return Verdict::Allow;
}
// File-mutating tools: write / edit / delete
// File-mutating tools: require a meaningful reason.
if matches!(tool_name, "write" | "edit" | "delete") {
match Self::validate_reason(tool_name, args) {
Ok(()) => {}
Err(msg) => return Verdict::Block(msg),
if let Err(msg) = Self::validate_reason(tool_name, args) {
return Verdict::Block(msg);
}
}
// write / edit content must not contain stubs, denial language, or
// assumption language.
if matches!(tool_name, "write" | "edit") {
if let Some(content) = Self::extract_content(tool_name, args) {
if let Some(pat) = Self::first_match(&content, STUB_PATTERNS) {
return Verdict::Block(format!(
"content contains stub/placeholder pattern '{pat}'; \
production code must be fully implemented \
replace the stub with a real implementation"
));
}
if let Some(pat) = Self::first_match(&content, DENIAL_PATTERNS) {
return Verdict::Block(format!(
"content contains denial/punt pattern '{pat}'; \
implement the change properly instead of skipping"
));
}
if let Some(pat) = Self::first_match(&content, ASSUMPTION_PATTERNS) {
return Verdict::Block(format!(
"content contains assumption pattern '{pat}'; \
verify against data/tests instead of guessing"
));
}
}
// write / edit content scanning for stub/denial/assumption patterns.
if let Some(v) = Self::check_content_safety(tool_name, args) {
return v;
}
// Bash: destructive patterns, exfiltration (ALL commands checked,
// no safe-command whitelist), sensitive-path reads.
if tool_name == "bash" {
let cmd = args.get("command").and_then(|v| v.as_str()).unwrap_or("");
if cmd.contains("..") {
return Verdict::Block(
"path traversal detected in bash command".to_string(),
);
}
// Exfiltration patterns are checked on EVERY bash command,
// regardless of prefix. The safe-command whitelist was removed
// because it could be bypassed with command chaining.
for pat in EXFIL_PATTERNS {
if cmd.contains(pat) {
return Verdict::Block(format!(
"potential data-exfiltration command blocked (matched '{pat}')"
));
}
}
for pat in SENSITIVE_PATH_PATTERNS {
if cmd.contains(pat) {
return Verdict::Block(format!(
"refused to read/write sensitive path '{pat}'"
));
}
}
let dangerous_patterns = [
"rm -rf /", "rm -rf --no-preserve-root",
"rm -rf ~", "rm -fr /", "mkfs.", "dd if=",
":(){", "> /dev/sda", "chmod -R 000 /",
"shutdown ", "poweroff ", "reboot ", "halt ",
];
for pat in &dangerous_patterns {
if cmd.contains(pat) {
return Verdict::Block(format!(
"destructive command pattern blocked: {pat}"
));
}
}
// Also scan heredocs / -c / inline content for stub/denial
// language (e.g. `bash -c 'echo todo!()'`)
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
// Bash-specific destructive / exfiltration checks.
if let Some(v) = Self::check_bash_safety(args) {
return v;
}
// git_operator: require a non-trivial reason.
if tool_name == "git_operator" && !Self::has_valid_reason(args, MIN_REASON_LEN) {
if args.get("reason").and_then(|v| v.as_str()).is_some() {
return Verdict::Block(format!(
"bash command contains stub pattern '{pat}'"
"git_operator requires a non-trivial 'reason' \
(>= {MIN_REASON_LEN} chars) explaining the operation"
));
}
return Verdict::Block(
"git_operator requires a 'reason' argument explaining the operation".to_string(),
);
}
// git_operator: require a non-trivial reason as well.
if tool_name == "git_operator" {
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
if reason.trim().len() < MIN_REASON_LEN {
return Verdict::Block(format!(
"git_operator requires a non-trivial 'reason' \
(>= {MIN_REASON_LEN} chars) explaining the operation"
));
}
} else {
return Verdict::Block(
"git_operator requires a 'reason' argument explaining the operation"
.to_string(),
);
}
}
// MCP tools: unknown behaviour — require a reason if they take
// arguments, to discourage lazy invocations.
// MCP tools: require a reason when they take meaningful arguments.
if is_mcp {
if let Some(reason) = args.get("reason").and_then(|v| v.as_str()) {
if reason.trim().len() < MIN_REASON_LEN {
@@ -287,7 +188,6 @@ impl Harness {
));
}
} else if args.as_object().is_some_and(|m| !m.is_empty()) {
// Only require reason when there are meaningful arguments
return Verdict::Block(format!(
"MCP tool '{tool_name}' requires a 'reason' argument \
explaining the operation"
@@ -298,6 +198,163 @@ impl Harness {
Verdict::Allow
}
/// Check for path traversal in the `path` argument and verify it stays
/// within workspace roots.
///
/// Flow: reject any path containing `..` → if workspace roots are set,
/// reject absolute paths outside every root.
///
/// Return: `Some(Verdict::Block)` on violation, `None` if the check
/// passes or the tool has no `path` argument.
fn check_path_traversal(
args: &serde_json::Value,
workspace_roots: &[&std::path::Path],
) -> Option<Verdict> {
let path = args.get("path")?.as_str()?;
if path.contains("..") {
return Some(Verdict::Block(
"path traversal detected in 'path' argument".to_string(),
));
}
if !workspace_roots.is_empty() {
let abs_check = std::path::PathBuf::from(path);
if abs_check.is_absolute() && !workspace_roots.iter().any(|r| abs_check.starts_with(r))
{
return Some(Verdict::Block(format!(
"absolute path '{path}' is outside all workspace roots"
)));
}
}
None
}
/// Verify that a tool's output path (if any) stays within workspace roots.
///
/// Flow: if `find_output_path` yields a path, reject it unless it's
/// under `/tmp`, already absolute, or within a workspace root.
///
/// Return: `Some(Verdict::Block)` on violation, `None` otherwise.
fn check_output_path(
tool_name: &str,
args: &serde_json::Value,
workspace_roots: &[&std::path::Path],
) -> Option<Verdict> {
let out_path = Self::find_output_path(tool_name, args)?;
if !workspace_roots.is_empty() && !out_path.starts_with("/tmp") && !out_path.is_absolute() {
let allowed = workspace_roots.iter().any(|r| out_path.starts_with(r));
if !allowed {
return Some(Verdict::Block(format!(
"output path '{}' is outside all workspace roots",
out_path.display(),
)));
}
}
None
}
/// Check write/edit content for stub, denial, and assumption patterns.
///
/// Return: `Some(Verdict::Block)` with a description of the first
/// matched pattern, `None` if the content is clean or not applicable.
fn check_content_safety(tool_name: &str, args: &serde_json::Value) -> Option<Verdict> {
if !matches!(tool_name, "write" | "edit") {
return None;
}
let content = Self::extract_content(tool_name, args)?;
for (patterns, msg_prefix) in [
(&STUB_PATTERNS, "stub/placeholder"),
(&DENIAL_PATTERNS, "denial/punt"),
(&ASSUMPTION_PATTERNS, "assumption"),
] {
if let Some(pat) = Self::first_match(&content, patterns) {
let msg = match msg_prefix {
"stub/placeholder" => format!(
"content contains stub/placeholder pattern '{pat}'; \
production code must be fully implemented \
replace the stub with a real implementation"
),
"denial/punt" => format!(
"content contains denial/punt pattern '{pat}'; \
implement the change properly instead of skipping"
),
_ => format!(
"content contains assumption pattern '{pat}'; \
verify against data/tests instead of guessing"
),
};
return Some(Verdict::Block(msg));
}
}
None
}
/// Check bash commands for path traversal, exfiltration, sensitive
/// path reads, destructive patterns, and stub language.
///
/// Flow: extract the `command` argument → check each category in
/// sequence, returning the first violation found.
///
/// Return: `Some(Verdict::Block)` on any violation, `None` if the
/// tool is not bash or the command is safe.
fn check_bash_safety(args: &serde_json::Value) -> Option<Verdict> {
let cmd = args.get("command")?.as_str()?;
if cmd.contains("..") {
return Some(Verdict::Block(
"path traversal detected in bash command".to_string(),
));
}
for pat in EXFIL_PATTERNS {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"potential data-exfiltration command blocked (matched '{pat}')"
)));
}
}
for pat in SENSITIVE_PATH_PATTERNS {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"refused to read/write sensitive path '{pat}'"
)));
}
}
let dangerous_patterns = [
"rm -rf /",
"rm -rf --no-preserve-root",
"rm -rf ~",
"rm -fr /",
"mkfs.",
"dd if=",
":(){",
"> /dev/sda",
"chmod -R 000 /",
"shutdown ",
"poweroff ",
"reboot ",
"halt ",
];
for pat in &dangerous_patterns {
if cmd.contains(pat) {
return Some(Verdict::Block(format!(
"destructive command pattern blocked: {pat}"
)));
}
}
if let Some(pat) = Self::first_match(cmd, STUB_PATTERNS) {
return Some(Verdict::Block(format!(
"bash command contains stub pattern '{pat}'"
)));
}
None
}
/// Check whether the given `args` contain a non-trivial `reason`
/// argument meeting the minimum length requirement.
fn has_valid_reason(args: &serde_json::Value, min_len: usize) -> bool {
args.get("reason")
.and_then(|v| v.as_str())
.is_some_and(|r| r.trim().len() >= min_len)
}
/// Validate the `reason` argument for a mutating tool.
///
/// Flow: require the field to exist and be a non-empty string ≥
@@ -317,17 +374,13 @@ impl Harness {
Some(v) => match v.as_str() {
Some(s) => s,
None => {
return Err(format!(
"{tool_name} 'reason' must be a string"
));
return Err(format!("{tool_name} 'reason' must be a string"));
}
},
};
let trimmed = reason.trim();
if trimmed.is_empty() {
return Err(format!(
"{tool_name} 'reason' must not be empty"
));
return Err(format!("{tool_name} 'reason' must not be empty"));
}
if trimmed.len() < MIN_REASON_LEN {
return Err(format!(
@@ -339,9 +392,20 @@ impl Harness {
// Reject generic non-answers
let lower = trimmed.to_lowercase();
let non_answers = [
"fix", "update", "change", "edit", "modify",
"implement", "add", "remove", "delete",
"make it work", "make work", "test", "wip", "tbd",
"fix",
"update",
"change",
"edit",
"modify",
"implement",
"add",
"remove",
"delete",
"make it work",
"make work",
"test",
"wip",
"tbd",
];
if non_answers.iter().any(|n| lower == *n) {
return Err(format!(
@@ -356,7 +420,10 @@ impl Harness {
/// Extract the textual content of a write/edit call, if any.
fn extract_content(tool_name: &str, args: &serde_json::Value) -> Option<String> {
match tool_name {
"write" => args.get("content").and_then(|v| v.as_str()).map(String::from),
"write" => args
.get("content")
.and_then(|v| v.as_str())
.map(String::from),
"edit" => {
let old = args.get("old").and_then(|v| v.as_str()).unwrap_or("");
let new = args.get("new").and_then(|v| v.as_str()).unwrap_or("");
@@ -377,9 +444,10 @@ impl Harness {
/// Extract a candidate output path from a tool call, if one exists.
fn find_output_path(tool_name: &str, args: &serde_json::Value) -> Option<std::path::PathBuf> {
match tool_name {
"write" | "edit" | "delete" | "read" => {
args.get("path").and_then(|v| v.as_str()).map(std::path::PathBuf::from)
}
"write" | "edit" | "delete" | "read" => args
.get("path")
.and_then(|v| v.as_str())
.map(std::path::PathBuf::from),
"bash" => {
let cmd = args.get("command").and_then(|v| v.as_str())?;
let lower = cmd.to_lowercase();
@@ -397,7 +465,6 @@ impl Harness {
_ => None,
}
}
}
impl Default for Harness {
@@ -418,7 +485,10 @@ mod tests {
return match verdict.to_lowercase().as_str() {
"allow" => Some(Verdict::Allow),
"block" => Some(Verdict::Block(
v.get("reason").and_then(|r| r.as_str()).unwrap_or("blocked").to_string()
v.get("reason")
.and_then(|r| r.as_str())
.unwrap_or("blocked")
.to_string(),
)),
_ => None,
};
@@ -430,7 +500,11 @@ mod tests {
return Some(Verdict::Allow);
}
if l.starts_with("verdict: block") {
let reason = line.split_once(':').map_or("blocked", |x| x.1).trim().to_string();
let reason = line
.split_once(':')
.map_or("blocked", |x| x.1)
.trim()
.to_string();
return Some(Verdict::Block(reason));
}
}
@@ -450,7 +524,6 @@ mod tests {
assert_eq!(result, Verdict::Allow);
}
#[test]
fn test_parse_verdict_json_allow() {
let v = parse_verdict(r#"{"verdict": "allow"}"#);
+120 -119
View File
@@ -47,13 +47,20 @@ impl LspClient {
cmd.stdout(Stdio::piped());
cmd.stderr(Stdio::piped());
let mut child = cmd.spawn()
let mut child = cmd
.spawn()
.map_err(|e| anyhow::anyhow!("failed to spawn LSP server '{command}': {e}"))?;
let stdin = child.stdin.take()
let stdin = child
.stdin
.take()
.ok_or_else(|| anyhow::anyhow!("failed to capture stdin for LSP server"))?;
let stdout = BufReader::new(child.stdout.take()
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?);
let stdout = BufReader::new(
child
.stdout
.take()
.ok_or_else(|| anyhow::anyhow!("failed to capture stdout for LSP server"))?,
);
let mut client = LspClient {
stdin,
@@ -106,7 +113,11 @@ impl LspClient {
}
});
let result = client.call_with_timeout("initialize", &init_params, Duration::from_millis(LSP_INIT_TIMEOUT_MS))?;
let result = client.call_with_timeout(
"initialize",
&init_params,
Duration::from_millis(LSP_INIT_TIMEOUT_MS),
)?;
client.server_capabilities = result.get("capabilities").cloned().unwrap_or_default();
client.notify("initialized", &json!({}))?;
@@ -122,7 +133,12 @@ impl LspClient {
self.call_with_timeout(method, params, Duration::from_millis(LSP_CALL_TIMEOUT_MS))
}
fn call_with_timeout(&mut self, method: &str, params: &Value, timeout: Duration) -> anyhow::Result<Value> {
fn call_with_timeout(
&mut self,
method: &str,
params: &Value,
timeout: Duration,
) -> anyhow::Result<Value> {
self.next_id += 1;
let id = self.next_id;
let req = json!({
@@ -148,11 +164,14 @@ impl LspClient {
let body = serde_json::to_string(msg)
.map_err(|e| anyhow::anyhow!("failed to serialize LSP message: {e}"))?;
let header = format!("Content-Length: {}\r\n\r\n", body.len());
self.stdin.write_all(header.as_bytes())
self.stdin
.write_all(header.as_bytes())
.map_err(|e| anyhow::anyhow!("failed to write LSP frame header: {e}"))?;
self.stdin.write_all(body.as_bytes())
self.stdin
.write_all(body.as_bytes())
.map_err(|e| anyhow::anyhow!("failed to write LSP frame body: {e}"))?;
self.stdin.flush()
self.stdin
.flush()
.map_err(|e| anyhow::anyhow!("failed to flush LSP stdin: {e}"))?;
Ok(())
}
@@ -166,8 +185,14 @@ impl LspClient {
let frame = self.read_frame()?;
if frame.get("id") == Some(&json!(expected_id)) {
if let Some(err) = frame.get("error") {
let code = err.get("code").and_then(serde_json::Value::as_i64).unwrap_or(0);
let msg = err.get("message").and_then(|m| m.as_str()).unwrap_or("unknown error");
let code = err
.get("code")
.and_then(serde_json::Value::as_i64)
.unwrap_or(0);
let msg = err
.get("message")
.and_then(|m| m.as_str())
.unwrap_or("unknown error");
anyhow::bail!("LSP error {code}: {msg}");
}
return Ok(frame.get("result").cloned().unwrap_or(Value::Null));
@@ -205,8 +230,9 @@ impl LspClient {
// Cap Content-Length at 64 MiB to prevent OOM from a
// malicious or misconfigured LSP server (CWE-400).
const MAX_CONTENT_LENGTH: usize = 64 * 1024 * 1024;
let length: usize = len_str.trim().parse::<usize>()
.map_err(|e| anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e))?;
let length: usize = len_str.trim().parse::<usize>().map_err(|e| {
anyhow::anyhow!("invalid Content-Length '{}': {}", len_str.trim(), e)
})?;
if length > MAX_CONTENT_LENGTH {
anyhow::bail!(
"Content-Length {length} exceeds maximum allowed size of {MAX_CONTENT_LENGTH} bytes",
@@ -220,7 +246,8 @@ impl LspClient {
.ok_or_else(|| anyhow::anyhow!("missing Content-Length header in LSP response"))?;
let mut body = vec![0u8; length];
self.stdout.read_exact(&mut body)
self.stdout
.read_exact(&mut body)
.map_err(|e| anyhow::anyhow!("failed to read LSP body ({length} bytes): {e}"))?;
let json_str = String::from_utf8(body)
@@ -230,74 +257,98 @@ impl LspClient {
.map_err(|e| anyhow::anyhow!("invalid JSON in LSP response: {e}"))
}
pub fn did_open(&mut self, uri: &str, language_id: &str, version: i32, text: &str) -> anyhow::Result<()> {
self.notify("textDocument/didOpen", &json!({
"textDocument": {
"uri": uri,
"languageId": language_id,
"version": version,
"text": text
}
}))
pub fn did_open(
&mut self,
uri: &str,
language_id: &str,
version: i32,
text: &str,
) -> anyhow::Result<()> {
self.notify(
"textDocument/didOpen",
&json!({
"textDocument": {
"uri": uri,
"languageId": language_id,
"version": version,
"text": text
}
}),
)
}
#[allow(dead_code)]
pub fn did_change(&mut self, uri: &str, version: i32, text: &str) -> anyhow::Result<()> {
self.notify("textDocument/didChange", &json!({
"textDocument": {
"uri": uri,
"version": version
},
"contentChanges": [{
"text": text
}]
}))
self.notify(
"textDocument/didChange",
&json!({
"textDocument": {
"uri": uri,
"version": version
},
"contentChanges": [{
"text": text
}]
}),
)
}
pub fn did_close(&mut self, uri: &str) -> anyhow::Result<()> {
self.notify("textDocument/didClose", &json!({
"textDocument": {
"uri": uri
}
}))
self.notify(
"textDocument/didClose",
&json!({
"textDocument": {
"uri": uri
}
}),
)
}
pub fn hover(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/hover", &json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}))
self.call(
"textDocument/hover",
&json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
}
pub fn completion(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/completion", &json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}))
self.call(
"textDocument/completion",
&json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
}
pub fn goto_definition(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/definition", &json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}))
pub fn goto_definition(
&mut self,
uri: &str,
line: u32,
character: u32,
) -> anyhow::Result<Value> {
self.call(
"textDocument/definition",
&json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character }
}),
)
}
pub fn references(&mut self, uri: &str, line: u32, character: u32) -> anyhow::Result<Value> {
self.call("textDocument/references", &json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character },
"context": {
"includeDeclaration": true
}
}))
}
#[allow(dead_code)]
pub fn document_symbols(&mut self, uri: &str) -> anyhow::Result<Value> {
self.call("textDocument/documentSymbol", &json!({
"textDocument": { "uri": uri }
}))
self.call(
"textDocument/references",
&json!({
"textDocument": { "uri": uri },
"position": { "line": line, "character": character },
"context": {
"includeDeclaration": true
}
}),
)
}
pub fn collect_diagnostics(
@@ -313,64 +364,14 @@ impl LspClient {
);
self.did_close(uri)?;
match result {
Ok(params) => Ok(params.get("diagnostics").cloned().unwrap_or_else(|| json!([]))),
Ok(params) => Ok(params
.get("diagnostics")
.cloned()
.unwrap_or_else(|| json!([]))),
Err(e) => Err(e),
}
}
/// Health-check the LSP server.
///
/// Sends a `textDocument/documentSymbol` request on a dummy URI with a
/// 2-second timeout. Returns `true` if the server responds at all —
/// including with an error response such as "file not found", which
/// still proves the process is up and the JSON-RPC channel is live.
/// Returns `false` on timeout, EOF, or any read/write error.
///
/// Flow: build request → `send_frame` → poll frames until id matches
/// (alive) or deadline/read error fires (dead).
#[allow(dead_code)]
pub fn is_alive(&mut self) -> bool {
self.next_id += 1;
let id = self.next_id;
let req = json!({
"jsonrpc": "2.0",
"id": id,
"method": "textDocument/documentSymbol",
"params": {
"textDocument": { "uri": "file:///__zesdex_lsp_health_check__.txt" }
}
});
if self.send_frame(&req).is_err() {
return false;
}
let timeout = Duration::from_secs(2);
let deadline = Instant::now() + timeout;
loop {
if Instant::now() > deadline {
return false;
}
match self.read_frame() {
Ok(frame) => {
if frame.get("id") == Some(&json!(id)) {
return true;
}
// Skip unrelated notifications/responses on the same channel.
}
Err(_) => return false,
}
}
}
/// Send the LSP `exit` notification to request graceful shutdown.
///
/// Per the LSP spec, `exit` is a notification — the server is expected
/// to terminate after receiving it without sending a response. We do
/// not block on any reply.
#[allow(dead_code)]
pub fn exit(&mut self) -> anyhow::Result<()> {
self.notify("exit", &json!({}))
}
pub fn shutdown(&mut self) {
let _ = self.call_with_timeout("shutdown", &json!({}), Duration::from_secs(5));
let _ = self.notify("exit", &json!({}));
+54 -113
View File
@@ -13,12 +13,6 @@ pub use client::{path_to_lsp_uri, LspClient};
/// to issue LSP requests from threads or async tasks.
#[derive(Clone)]
pub struct LspServer {
#[allow(dead_code)]
pub name: String,
#[allow(dead_code)]
pub command: String,
#[allow(dead_code)]
pub args: Vec<String>,
pub language_id: String,
pub client: Arc<Mutex<LspClient>>,
}
@@ -38,12 +32,12 @@ pub struct OpenDoc {
///
/// Flow: caller calls `connect*` -> client spawned -> entry pushed to
/// `servers` -> `extension_registry` is populated by `register_extensions`.
/// File edits route through `find_server_for_path` / `find_server_for_extension`
/// and are dispatched as `didOpen` / `didChange` notifications.
/// File edits route through `extension_registry` and are dispatched as
/// `didOpen` / `didChange` notifications.
#[derive(Clone)]
pub struct LspManager {
pub servers: Vec<LspServer>,
/// Maps file extension (".rs", ".ts", ...) -> server name.
/// Maps file extension (".rs", ".ts", ...) -> language id.
pub extension_registry: HashMap<String, String>,
/// Maps document URI -> tracked open document state.
pub open_files: HashMap<String, OpenDoc>,
@@ -59,112 +53,73 @@ impl LspManager {
}
}
/// Spawn an LSP server and register it under `name`.
/// Spawn an LSP server and register it under `language_id`.
///
/// Fails if a server with the same name is already connected.
/// Fails if a server with the same `language_id` is already connected.
pub fn connect(
&mut self,
name: &str,
command: &str,
args: &[String],
language_id: &str,
) -> anyhow::Result<()> {
if self.servers.iter().any(|s| s.name == name) {
anyhow::bail!("LSP server '{name}' is already connected");
if self.servers.iter().any(|s| s.language_id == language_id) {
anyhow::bail!("LSP server for language '{language_id}' is already connected");
}
let client = LspClient::spawn(command, args)?;
self.servers.push(LspServer {
name: name.to_string(),
command: command.to_string(),
args: args.to_vec(),
language_id: language_id.to_string(),
client: Arc::new(Mutex::new(client)),
});
Ok(())
}
/// Look up a connected server by name and return a reference to its entry.
#[allow(dead_code)]
pub fn find_server(&self, name: &str) -> Option<&LspServer> {
self.servers.iter().find(|s| s.name == name)
}
/// Return a clone of the `Arc<Mutex<LspClient>>` for a connected server.
///
/// Cloning the `Arc` lets callers issue requests without holding a
/// borrow on the manager.
pub fn get_client(&self, name: &str) -> Option<Arc<Mutex<LspClient>>> {
self.servers.iter().find(|s| s.name == name).map(|s| s.client.clone())
pub fn get_client(&self, language_id: &str) -> Option<Arc<Mutex<LspClient>>> {
self.servers
.iter()
.find(|s| s.language_id == language_id)
.map(|s| s.client.clone())
}
/// Shut down and remove a server by name. Returns true if it existed.
pub fn disconnect(&mut self, name: &str) -> bool {
if let Some(server) = self.servers.iter().find(|s| s.name == name) {
/// Shut down and remove a server by language. Returns true if it existed.
pub fn disconnect(&mut self, language_id: &str) -> bool {
if let Some(server) = self.servers.iter().find(|s| s.language_id == language_id) {
if let Ok(mut client) = server.client.lock() {
client.shutdown();
}
}
let len = self.servers.len();
self.servers.retain(|s| s.name != name);
self.servers.retain(|s| s.language_id != language_id);
self.servers.len() < len
}
/// Return the language id (e.g. "rust") registered for `name`.
pub fn get_language_id(&self, name: &str) -> Option<String> {
self.servers.iter().find(|s| s.name == name).map(|s| s.language_id.clone())
}
/// Resolve an extension (".rs", ".ts", ...) to its server's client.
///
/// Flow: lookup `extension_registry` -> resolve server name -> clone client.
/// Returns `None` if no server has been registered for `ext`.
#[allow(dead_code)]
pub fn find_server_for_extension(&self, ext: &str) -> Option<Arc<Mutex<LspClient>>> {
self.extension_registry
.get(ext)
.and_then(|name| self.get_client(name))
}
/// Resolve a file path to its server's client by extension.
///
/// Flow: extract the extension from `path` -> delegate to
/// `find_server_for_extension`. Files without an extension or with
/// an unmapped extension return `None`.
#[allow(dead_code)]
pub fn find_server_for_path(&self, path: &Path) -> Option<Arc<Mutex<LspClient>>> {
path.extension()
.and_then(|e| e.to_str())
.map(|s| format!(".{s}"))
.and_then(|ext| self.find_server_for_extension(&ext))
/// Return the language id (e.g. "rust") registered for `language_id`.
pub fn get_language_id(&self, language_id: &str) -> Option<String> {
self.servers
.iter()
.find(|s| s.language_id == language_id)
.map(|s| s.language_id.clone())
}
/// Register a set of file extensions for an already-connected server.
///
/// Flow: for each `ext`, write `server_name` into `extension_registry`.
/// Re-registration overwrites the previous target. Unknown server
/// names are accepted at this layer — caller must ensure `server_name`
/// is connected or will be connected later.
pub fn register_extensions(&mut self, server_name: &str, extensions: &[&str]) {
/// Flow: for each `ext`, write `language_id` into `extension_registry`.
/// Re-registration overwrites the previous target. Unknown language IDs
/// are accepted at this layer — caller must ensure a server for
/// `language_id` is connected or will be connected later.
pub fn register_extensions(&mut self, language_id: &str, extensions: &[&str]) {
for ext in extensions {
self.extension_registry.insert(ext.to_string(), server_name.to_string());
self.extension_registry
.insert(ext.to_string(), language_id.to_string());
}
}
/// Return the registered server name for a given language id.
///
/// Flow: scan `servers` for the first entry whose `language_id` matches.
/// Used when callers have a language hint rather than a file path.
#[allow(dead_code)]
pub fn get_server_name(&self, language: &str) -> Option<String> {
self.servers
.iter()
.find(|s| s.language_id == language)
.map(|s| s.name.clone())
}
/// Notify the relevant LSP server that a file's contents have changed.
///
/// Flow: resolve server by extension -> read file contents ->
/// Flow: resolve language by extension from the registry -> read file contents ->
/// either send `didOpen` (first time) or `didChange` (already tracked)
/// -> update `open_files` with the new version.
///
@@ -172,13 +127,20 @@ impl LspManager {
/// error) are logged with `tracing::warn!` rather than propagated,
/// so a stale notification cannot abort the calling flow.
pub fn did_change_file(&mut self, path: &Path) {
let Some(ext) = path.extension().and_then(|e| e.to_str()).map(|s| format!(".{s}")) else {
let Some(ext) = path
.extension()
.and_then(|e| e.to_str())
.map(|s| format!(".{s}"))
else {
tracing::warn!("did_change_file: path has no extension: {:?}", path);
return;
};
let server_name = if let Some(name) = self.extension_registry.get(&ext) { name.clone() } else {
tracing::warn!("did_change_file: no LSP server registered for extension '{}'", ext);
let Some(language_id) = self.extension_registry.get(&ext).cloned() else {
tracing::warn!(
"did_change_file: no LSP server registered for extension '{}'",
ext
);
return;
};
@@ -192,12 +154,8 @@ impl LspManager {
}
};
let language_id = self
.get_language_id(&server_name)
.unwrap_or_else(|| "plaintext".to_string());
let Some(client) = self.get_client(&server_name) else {
tracing::warn!("did_change_file: server '{}' has no client", server_name);
let Some(client) = self.get_client(&language_id) else {
tracing::warn!("did_change_file: no client for language '{}'", language_id);
return;
};
@@ -210,7 +168,11 @@ impl LspManager {
let mut client = match client.lock() {
Ok(c) => c,
Err(e) => {
tracing::warn!("did_change_file: client mutex poisoned for '{}': {}", server_name, e);
tracing::warn!(
"did_change_file: client mutex poisoned for '{}': {}",
language_id,
e
);
return;
}
};
@@ -224,7 +186,7 @@ impl LspManager {
if let Err(e) = send_result {
tracing::warn!(
"did_change_file: failed to notify '{}' for {}: {}",
server_name,
language_id,
uri,
e
);
@@ -238,24 +200,6 @@ impl LspManager {
version: next_version,
},
);
}
/// Record that `server_name` has an open document at `uri`.
///
/// Flow: insert/overwrite the `OpenDoc` entry in `open_files`.
/// Does not contact the LSP server — pure local bookkeeping.
#[allow(dead_code)]
pub fn track_open_doc(&mut self, server_name: &str, uri: &str, language: &str, version: i32) {
// server_name retained for future routing extensions; not stored today.
let _ = server_name;
self.open_files.insert(
uri.to_string(),
OpenDoc {
language: language.to_string(),
version,
},
);
}
/// Shut down every connected server and clear the server list.
@@ -272,21 +216,20 @@ impl LspManager {
self.servers.clear();
}
/// Snapshot the connected servers as `(name, language_id, has_open_docs)` triples.
/// Snapshot the connected servers as `(language_id, has_open_docs)` pairs.
///
/// `has_open_docs` is true if any tracked `OpenDoc` was registered
/// against this server's clients. Useful for status displays.
pub fn list_servers(&self) -> Vec<(String, String, bool)> {
pub fn list_servers(&self) -> Vec<(String, bool)> {
self.servers
.iter()
.map(|s| {
let name = s.name.clone();
let lang = s.language_id.clone();
let has_open = self
.open_files
.values()
.any(|d| d.language == s.language_id);
(name, lang, has_open)
(lang, has_open)
})
.collect()
}
@@ -294,18 +237,17 @@ impl LspManager {
/// Connect an LSP server and register its default extensions in one call.
///
/// Flow: invoke `connect` -> on success, register `extensions` against
/// `name` in `extension_registry`. If `connect` fails, the registries
/// `language_id` in `extension_registry`. If `connect` fails, the registries
/// are left untouched and the error is propagated.
pub fn connect_with_extensions(
&mut self,
name: &str,
command: &str,
args: &[String],
language_id: &str,
extensions: &[&str],
) -> anyhow::Result<()> {
self.connect(name, command, args, language_id)?;
self.register_extensions(name, extensions);
self.connect(command, args, language_id)?;
self.register_extensions(language_id, extensions);
Ok(())
}
}
@@ -315,4 +257,3 @@ impl Default for LspManager {
Self::new()
}
}
+205 -161
View File
@@ -11,7 +11,6 @@
//! Each tier is a fallback for the previous, so we try the most
//! user-friendly path first (rustup component, npm global, etc.) and
//! only fall back to package managers or manual download if those fail.
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::sync::{Arc, Mutex};
@@ -45,14 +44,11 @@ pub enum ProvisionResult {
language: String,
binary_path: String,
},
/// Every install tier failed — `manual_instructions` tells the user how
/// to install by hand.
/// Every install tier failed. Tells the user how to install by hand.
Failed {
language: String,
server_name: String,
reason: String,
#[allow(dead_code)]
manual_instructions: String,
},
}
@@ -97,28 +93,55 @@ pub struct InstallTier {
pub args: Vec<String>,
}
/// Rust toolchain availability on the host PATH.
#[derive(Debug, Clone)]
pub struct RustToolchain {
pub has_rustup: bool,
pub has_cargo: bool,
}
/// Web / scripting language toolchain availability.
#[derive(Debug, Clone)]
pub struct WebToolchain {
pub has_npm: bool,
pub has_go: bool,
pub has_java: bool,
}
/// General-purpose platform utilities.
#[derive(Debug, Clone)]
pub struct PlatformUtils {
pub has_curl: bool,
pub has_tar: bool,
}
/// Pacman and Brew package managers (Arch / macOS).
#[derive(Debug, Clone)]
pub struct PacmanBrew {
pub has_pacman: bool,
pub has_brew: bool,
}
/// Apt and DNF package managers (Debian / Fedora).
#[derive(Debug, Clone)]
pub struct AptDnf {
pub has_apt: bool,
pub has_dnf: bool,
}
/// Snapshot of the host environment used to decide which install tiers are viable.
///
/// Populated by `detect_env()` once per `provision_all()` call so we
/// Populated by `detect_env()` once per `provision_all_with_progress()` call so we
/// don't re-shell out for every server. `is_linux` / `is_macos` are
/// computed at startup (compile time would also work, but keeping the
/// shape uniform with the rest of the struct makes the call sites tidy).
#[derive(Debug, Clone)]
#[allow(dead_code)]
#[allow(clippy::struct_excessive_bools)]
pub struct EnvInfo {
pub has_rustup: bool,
pub has_npm: bool,
pub has_go: bool,
pub has_java: bool,
pub has_cargo: bool,
pub has_curl: bool,
pub has_wget: bool,
pub has_tar: bool,
pub has_pacman: bool,
pub has_apt: bool,
pub has_brew: bool,
pub has_dnf: bool,
pub rust: RustToolchain,
pub web: WebToolchain,
pub platform: PlatformUtils,
pub pacman_brew: PacmanBrew,
pub apt_dnf: AptDnf,
pub is_linux: bool,
pub is_macos: bool,
}
@@ -159,18 +182,27 @@ pub fn which(binary: &str) -> Option<PathBuf> {
/// this only ever runs on Unix-like targets.
pub fn detect_env() -> EnvInfo {
EnvInfo {
has_rustup: which("rustup").is_some(),
has_npm: which("npm").is_some(),
has_go: which("go").is_some(),
has_java: which("java").is_some(),
has_cargo: which("cargo").is_some(),
has_curl: which("curl").is_some(),
has_wget: which("wget").is_some(),
has_tar: which("tar").is_some(),
has_pacman: which("pacman").is_some(),
has_apt: which("apt").is_some() || which("apt-get").is_some(),
has_brew: which("brew").is_some(),
has_dnf: which("dnf").is_some(),
rust: RustToolchain {
has_rustup: which("rustup").is_some(),
has_cargo: which("cargo").is_some(),
},
web: WebToolchain {
has_npm: which("npm").is_some(),
has_go: which("go").is_some(),
has_java: which("java").is_some(),
},
platform: PlatformUtils {
has_curl: which("curl").is_some(),
has_tar: which("tar").is_some(),
},
pacman_brew: PacmanBrew {
has_pacman: which("pacman").is_some(),
has_brew: which("brew").is_some(),
},
apt_dnf: AptDnf {
has_apt: which("apt").is_some() || which("apt-get").is_some(),
has_dnf: which("dnf").is_some(),
},
is_linux: cfg!(target_os = "linux"),
is_macos: cfg!(target_os = "macos"),
}
@@ -179,14 +211,13 @@ pub fn detect_env() -> EnvInfo {
/// Return the static set of supported language servers.
///
/// The order is significant: it determines provisioning order and
/// the order results appear in `provision_all()`. Tier 1 paths are
/// the order results appear in `provision_all_with_progress()`. Tier 1 paths are
/// the canonical/idiomatic install for each ecosystem; later tiers
/// are fallbacks for hosts that lack the primary tooling.
///
/// Why hard-coded rather than loaded from settings: the set is small,
/// changes rarely, and bundling it lets the provisioner run before any
/// user config has been read (e.g. on first launch).
#[allow(clippy::too_many_lines)]
pub fn supported_servers() -> Vec<LanguageServerDef> {
vec![
LanguageServerDef {
@@ -199,13 +230,22 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
label: "rustup component".to_string(),
requires: vec!["rustup".to_string()],
command: "rustup".to_string(),
args: vec!["component".to_string(), "add".to_string(), "rust-analyzer".to_string()],
args: vec![
"component".to_string(),
"add".to_string(),
"rust-analyzer".to_string(),
],
},
InstallTier {
label: "pacman".to_string(),
requires: vec!["pacman".to_string()],
command: "pacman".to_string(),
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "rust-analyzer".to_string()],
args: vec![
"-S".to_string(),
"--noconfirm".to_string(),
"--needed".to_string(),
"rust-analyzer".to_string(),
],
},
InstallTier {
label: "brew".to_string(),
@@ -217,7 +257,11 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
label: "cargo install".to_string(),
requires: vec!["cargo".to_string()],
command: "cargo".to_string(),
args: vec!["install".to_string(), "--locked".to_string(), "rust-analyzer".to_string()],
args: vec![
"install".to_string(),
"--locked".to_string(),
"rust-analyzer".to_string(),
],
},
InstallTier {
label: "download prebuilt".to_string(),
@@ -268,19 +312,33 @@ pub fn supported_servers() -> Vec<LanguageServerDef> {
name: "jdtls".to_string(),
language: "java".to_string(),
extensions: vec![".java".to_string()],
binary_names: vec!["jdtls".to_string(), "eclipse-jdt-ls".to_string(), "jdtls-launcher".to_string()],
binary_names: vec![
"jdtls".to_string(),
"eclipse-jdt-ls".to_string(),
"jdtls-launcher".to_string(),
],
install_tiers: vec![
InstallTier {
label: "pacman".to_string(),
requires: vec!["java".to_string(), "pacman".to_string()],
command: "pacman".to_string(),
args: vec!["-S".to_string(), "--noconfirm".to_string(), "--needed".to_string(), "eclipse-jdt-ls".to_string()],
args: vec![
"-S".to_string(),
"--noconfirm".to_string(),
"--needed".to_string(),
"eclipse-jdt-ls".to_string(),
],
},
InstallTier {
label: "apt".to_string(),
requires: vec!["java".to_string(), "apt".to_string()],
command: "sudo".to_string(),
args: vec!["apt".to_string(), "install".to_string(), "-y".to_string(), "eclipse-jdt-ls".to_string()],
args: vec![
"apt".to_string(),
"install".to_string(),
"-y".to_string(),
"eclipse-jdt-ls".to_string(),
],
},
InstallTier {
label: "brew".to_string(),
@@ -339,7 +397,9 @@ pub fn run_command(cmd: &str, args: &[&str]) -> std::io::Result<(bool, String)>
let timeout = Duration::from_mins(3);
let start = Instant::now();
let status = loop {
if let Some(status) = child.try_wait()? { break Ok(status) }
if let Some(status) = child.try_wait()? {
break Ok(status);
}
if start.elapsed() > timeout {
let _ = child.kill();
let _ = child.wait();
@@ -405,9 +465,12 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
info!(url = url, dest = %path_str, "downloading");
let args = [
"-fsSL",
"--connect-timeout", "15",
"--max-time", &max_secs.to_string(),
"-o", &path_str,
"--connect-timeout",
"15",
"--max-time",
&max_secs.to_string(),
"-o",
&path_str,
url,
];
let (ok, out) = run_command("curl", &args).map_err(|e| format!("curl spawn: {e}"))?;
@@ -419,7 +482,10 @@ fn download_url(url: &str, dest: &Path, max_secs: u64) -> Result<(), String> {
/// Download rust-analyzer from GitHub releases and install into
/// `~/.local/share/zesdex/lsp/rust-analyzer/bin/rust-analyzer`.
fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> {
fn install_rust_analyzer_binary(
env: &EnvInfo,
progress: ProgressFn<'_>,
) -> Result<PathBuf, String> {
let base = lsp_install_dir("rust-analyzer")?;
std::fs::create_dir_all(&base).map_err(|e| format!("mkdir: {e}"))?;
@@ -434,9 +500,13 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
let gz = base.join("rust-analyzer.gz");
let target = base.join("rust-analyzer");
if let Some(cb) = progress { cb("Rust: downloading prebuilt binary..."); }
if let Some(cb) = progress {
cb("Rust: downloading prebuilt binary...");
}
download_url(url, &gz, 120)?;
if let Some(cb) = progress { cb("Rust: decompressing..."); }
if let Some(cb) = progress {
cb("Rust: decompressing...");
}
let (ok, out) = run_command("gunzip", &["-f", &gz.to_string_lossy()])
.map_err(|e| format!("gunzip spawn: {e}"))?;
if !ok {
@@ -452,7 +522,9 @@ fn install_rust_analyzer_binary(env: &EnvInfo, progress: ProgressFn<'_>) -> Resu
std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o755))
.map_err(|e| format!("chmod: {e}"))?;
}
if let Some(cb) = progress { cb("Rust: installed ✓"); }
if let Some(cb) = progress {
cb("Rust: installed ✓");
}
Ok(target)
}
@@ -464,14 +536,24 @@ fn install_jdtls_from_eclipse(progress: ProgressFn) -> Result<PathBuf, String> {
let url = "https://download.eclipse.org/jdtls/snapshots/jdt-language-server-latest.tar.gz";
let tarball = base.join("jdtls.tar.gz");
if let Some(cb) = progress { cb("Java: downloading JDT-LS (~150MB)..."); }
if let Some(cb) = progress {
cb("Java: downloading JDT-LS (~150MB)...");
}
download_url(url, &tarball, 300)?;
if let Some(cb) = progress { cb("Java: extracting..."); }
if let Some(cb) = progress {
cb("Java: extracting...");
}
let (ok, out) = run_command("tar", &[
"-xzf", tarball.to_str().unwrap_or(""),
"-C", base.to_str().unwrap_or("."),
]).map_err(|e| format!("tar spawn: {e}"))?;
let (ok, out) = run_command(
"tar",
&[
"-xzf",
tarball.to_str().unwrap_or(""),
"-C",
base.to_str().unwrap_or("."),
],
)
.map_err(|e| format!("tar spawn: {e}"))?;
if !ok {
return Err(format!("tar: {}", out.trim()));
}
@@ -510,12 +592,18 @@ exec java \
std::fs::set_permissions(&launcher, std::fs::Permissions::from_mode(0o755))
.map_err(|e| format!("chmod launcher: {e}"))?;
}
if let Some(cb) = progress { cb("Java: JDT-LS installed ✓"); }
if let Some(cb) = progress {
cb("Java: JDT-LS installed ✓");
}
Ok(launcher)
}
/// Dispatch a sentinel download tier to the correct helper.
fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Result<PathBuf, String> {
fn run_download_tier(
name: &str,
env: &EnvInfo,
progress: ProgressFn<'_>,
) -> Result<PathBuf, String> {
match name {
DOWNLOAD_RUST_BIN => install_rust_analyzer_binary(env, progress),
DOWNLOAD_JDTLS => install_jdtls_from_eclipse(progress),
@@ -523,59 +611,17 @@ fn run_download_tier(name: &str, env: &EnvInfo, progress: ProgressFn<'_>) -> Res
}
}
/// Render the "install by hand" message shown to the user when every
/// automated tier fails.
fn manual_instructions(def: &LanguageServerDef) -> String {
match def.language.as_str() {
"rust" => "Install rust-analyzer:\n \
Arch: sudo pacman -S rust-analyzer\n \
macOS: brew install rust-analyzer\n \
Any: cargo install --locked rust-analyzer\n \
Rustup: rustup component add rust-analyzer"
.to_string(),
"typescript" => "Install typescript-language-server:\n \
npm install -g typescript typescript-language-server\n \
Arch: sudo pacman -S typescript-language-server"
.to_string(),
"go" => "Install gopls:\n \
go install golang.org/x/tools/gopls@latest\n \
Arch: sudo pacman -S gopls"
.to_string(),
"java" => "Install Eclipse JDT-LS:\n \
Arch: sudo pacman -S eclipse-jdt-ls\n \
Debian: sudo apt install eclipse-jdt-ls\n \
macOS: brew install jdtls\n \
Other: see https://.eclipse.org/jdtls/#download"
.to_string(),
_ => format!("No automated install available for '{}'.", def.language),
}
}
/// Try to provision a single language server.
///
/// Flow: check whether any `binary_names` candidate is already on PATH
/// → if yes, return `AlreadyAvailable` → otherwise walk
/// `install_tiers` in order, skipping tiers whose `requires`
/// binaries are missing → for each viable tier, run the install
/// command (120s timeout) → if it succeeds AND the binary now
/// appears on PATH (or the tier is jdtls-manual returning a
/// launcher path), return Installed → if every tier fails, return
/// Failed with the last error and manual install instructions.
///
/// Why we re-check `which` after the install: `rustup component add`
/// can exit 0 even if the binary wasn't actually placed on PATH (rare,
/// but happens with broken rustup installs). Re-checking gives us a
/// real signal rather than trusting the exit code alone.
#[allow(dead_code)]
pub fn provision_single(def: &LanguageServerDef, env: &EnvInfo) -> ProvisionResult {
provision_single_with_progress(def, env, None)
}
fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progress: ProgressFn<'_>) -> ProvisionResult {
fn provision_single_with_progress(
def: &LanguageServerDef,
env: &EnvInfo,
progress: ProgressFn<'_>,
) -> ProvisionResult {
// 1. Check PATH.
for bin in &def.binary_names {
if let Some(path) = which(bin) {
if let Some(cb) = progress { cb(&format!("{}: already installed (PATH)", def.language)); }
if let Some(cb) = progress {
cb(&format!("{}: already installed (PATH)", def.language));
}
return ProvisionResult::AlreadyAvailable {
server_name: def.name.clone(),
language: def.language.clone(),
@@ -586,7 +632,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
// 2. Check download-install directory (~/.local/share/zesdex/lsp/<name>/...).
if let Some(path) = previous_download_install(def) {
if let Some(cb) = progress { cb(&format!("{}: found previous install", def.language)); }
if let Some(cb) = progress {
cb(&format!("{}: found previous install", def.language));
}
return ProvisionResult::AlreadyAvailable {
server_name: def.name.clone(),
language: def.language.clone(),
@@ -594,30 +642,42 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
};
}
if let Some(cb) = progress { cb(&format!("{}: checking install options...", def.language)); }
if let Some(cb) = progress {
cb(&format!("{}: checking install options...", def.language));
}
let mut last_reason = String::from("no install tiers succeeded");
for tier in &def.install_tiers {
// Prerequisite gating
let prereqs_met = tier.requires.iter().all(|req| match req.as_str() {
"rustup" => env.has_rustup, "npm" => env.has_npm,
"go" => env.has_go, "java" => env.has_java,
"cargo" => env.has_cargo, "curl" => env.has_curl,
"tar" => env.has_tar, "pacman" => env.has_pacman,
"apt" => env.has_apt, "brew" => env.has_brew,
"dnf" => env.has_dnf, _ => which(req).is_some(),
"rustup" => env.rust.has_rustup,
"npm" => env.web.has_npm,
"go" => env.web.has_go,
"java" => env.web.has_java,
"cargo" => env.rust.has_cargo,
"curl" => env.platform.has_curl,
"tar" => env.platform.has_tar,
"pacman" => env.pacman_brew.has_pacman,
"apt" => env.apt_dnf.has_apt,
"brew" => env.pacman_brew.has_brew,
"dnf" => env.apt_dnf.has_dnf,
_ => which(req).is_some(),
});
if !prereqs_met {
let skip = format!("{}: {} — missing prerequisite", def.language, tier.label);
if let Some(cb) = progress { cb(&skip); }
if let Some(cb) = progress {
cb(&skip);
}
last_reason = format!("tier '{}' skipped: missing prerequisite", tier.label);
warn!(server = %def.name, tier = %tier.label, "skipped — missing prerequisites");
continue;
}
let trying = format!("{}: {}...", def.language, tier.label);
if let Some(cb) = progress { cb(&trying); }
if let Some(cb) = progress {
cb(&trying);
}
// Download sentinel → helper.
if tier.command.starts_with("__download_") && tier.command.ends_with("__") {
@@ -647,7 +707,9 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
.iter()
.find_map(|b| which(b).map(|p| p.to_string_lossy().to_string()));
if let Some(path) = located {
if let Some(cb) = progress { cb(&format!("{}: installed ✓", def.language)); }
if let Some(cb) = progress {
cb(&format!("{}: installed ✓", def.language));
}
info!(server = %def.name, tier = %tier.label, binary = %path, "installed");
return ProvisionResult::Installed {
server_name: def.name.clone(),
@@ -671,58 +733,36 @@ fn provision_single_with_progress(def: &LanguageServerDef, env: &EnvInfo, progre
}
}
let manual = manual_instructions(def);
ProvisionResult::Failed {
language: def.language.clone(), server_name: def.name.clone(),
reason: last_reason, manual_instructions: manual,
language: def.language.clone(),
server_name: def.name.clone(),
reason: last_reason,
}
}
/// Provision every supported server in order, returning one
/// `ProvisionResult` per server.
///
/// Flow: `detect_env()` once → for each server in `supported_servers()`
/// call `provision_single()` → collect results. Order matches
/// `supported_servers()` (rust, typescript, go, java).
#[allow(dead_code)]
pub fn provision_all() -> Vec<ProvisionResult> {
let env = detect_env();
info!(
linux = env.is_linux,
macos = env.is_macos,
rustup = env.has_rustup,
cargo = env.has_cargo,
npm = env.has_npm,
go = env.has_go,
java = env.has_java,
curl = env.has_curl,
tar = env.has_tar,
pacman = env.has_pacman,
apt = env.has_apt,
brew = env.has_brew,
dnf = env.has_dnf,
"starting LSP provisioning"
);
supported_servers()
.iter()
.map(|def| provision_single(def, &env))
.collect()
}
/// Like `provision_all` but calls `progress` with a human-readable status
/// Provision every supported server with progress callbacks with a human-readable status
/// string at each stage of each server's install attempt.
pub fn provision_all_with_progress(progress: ProgressFn) -> Vec<ProvisionResult> {
let env = detect_env();
if let Some(cb) = progress {
let flags = [
("rustup", env.has_rustup), ("cargo", env.has_cargo),
("npm", env.has_npm), ("go", env.has_go), ("java", env.has_java),
("curl", env.has_curl), ("tar", env.has_tar),
("pacman", env.has_pacman), ("apt", env.has_apt), ("brew", env.has_brew),
("rustup", env.rust.has_rustup),
("cargo", env.rust.has_cargo),
("npm", env.web.has_npm),
("go", env.web.has_go),
("java", env.web.has_java),
("curl", env.platform.has_curl),
("tar", env.platform.has_tar),
("pacman", env.pacman_brew.has_pacman),
("apt", env.apt_dnf.has_apt),
("brew", env.pacman_brew.has_brew),
];
let avail: String = flags.iter()
.filter(|(_, v)| *v).map(|(k, _)| *k)
.collect::<Vec<_>>().join(", ");
let avail: String = flags
.iter()
.filter(|(_, v)| *v)
.map(|(k, _)| *k)
.collect::<Vec<_>>()
.join(", ");
cb(&format!("LSP: environment ready — {avail}"));
}
supported_servers()
@@ -779,9 +819,13 @@ pub fn auto_connect(manager: &Arc<Mutex<LspManager>>, results: &[ProvisionResult
};
// Build extension slice for connect_with_extensions.
let ext_refs: Vec<&str> = def.extensions.iter().map(std::string::String::as_str).collect();
let ext_refs: Vec<&str> = def
.extensions
.iter()
.map(std::string::String::as_str)
.collect();
match guard.connect_with_extensions(&name, &binary, &[], &language, &ext_refs) {
match guard.connect_with_extensions(&binary, &[], &language, &ext_refs) {
Ok(()) => {
info!(
name = %name,
+129 -91
View File
@@ -1,13 +1,11 @@
//! MCP server connection management: spawning/talking to stdio child
//! processes and HTTP endpoints, and adapting their advertised tools to
//! the crate's `Tool` trait.
use serde_json::{json, Value};
use serde::{Deserialize, Serialize};
use serde_json::{json, Value};
use std::io::{BufRead, BufReader, Write};
use std::sync::{Arc, Mutex, OnceLock};
const MCP_CONNECT_TIMEOUT_MS: u64 = 20_000;
const MCP_CALL_TIMEOUT_MS: u64 = 60_000;
@@ -35,13 +33,8 @@ fn mcp_static_str(s: &str) -> &'static str {
/// newline-delimited JSON-RPC over stdio, or a remote HTTP endpoint.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum McpTransport {
Stdio {
command: String,
args: Vec<String>,
},
StreamableHttp {
url: String,
},
Stdio { command: String, args: Vec<String> },
StreamableHttp { url: String },
}
/// A single tool advertised by an MCP server, as returned by `tools/list`.
@@ -104,8 +97,8 @@ impl StdioChild {
self.stdin.flush()?;
let mut response_line = String::new();
let deadline = std::time::Instant::now()
+ std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS);
let deadline =
std::time::Instant::now() + std::time::Duration::from_millis(MCP_CALL_TIMEOUT_MS);
loop {
if std::time::Instant::now() > deadline {
anyhow::bail!("MCP call timed out after {MCP_CALL_TIMEOUT_MS}ms");
@@ -136,7 +129,9 @@ impl StdioChild {
line_truncated = true;
// Consume rest of line to keep stream in sync
loop {
let buf = self.stdout.fill_buf()
let buf = self
.stdout
.fill_buf()
.map_err(|e| anyhow::anyhow!("MCP stdio read error: {e}"))?;
if buf.is_empty() {
anyhow::bail!("MCP stdio child closed mid-line");
@@ -152,9 +147,7 @@ impl StdioChild {
response_line.push(byte as char);
}
if line_truncated {
anyhow::bail!(
"MCP response line exceeded {MAX_LINE_LENGTH} byte limit",
);
anyhow::bail!("MCP response line exceeded {MAX_LINE_LENGTH} byte limit");
}
let trimmed = response_line.trim();
if trimmed.is_empty() {
@@ -172,12 +165,16 @@ impl StdioChild {
}));
}
}
} // close fn call
} // close impl StdioChild
} // close fn call
} // close impl StdioChild
pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow::Result<StdioChild> {
pub(crate) fn spawn_stdio_child(
command: &str,
extra_args: &[String],
) -> anyhow::Result<StdioChild> {
let parts: Vec<&str> = command.split_whitespace().collect();
let (prog, prog_args) = parts.split_first()
let (prog, prog_args) = parts
.split_first()
.ok_or_else(|| anyhow::anyhow!("MCP stdio command is empty"))?;
let mut cmd = std::process::Command::new(prog);
@@ -189,12 +186,17 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
// rather than discarded silently, making connectivity issues debugable.
cmd.stderr(std::process::Stdio::piped());
let mut child = cmd.spawn()
let mut child = cmd
.spawn()
.map_err(|e| anyhow::anyhow!("failed to spawn MCP stdio server '{command}': {e}"))?;
let stdin = child.stdin.take()
let stdin = child
.stdin
.take()
.ok_or_else(|| anyhow::anyhow!("failed to get stdin for MCP server"))?;
let stdout = child.stdout.take()
let stdout = child
.stdout
.take()
.ok_or_else(|| anyhow::anyhow!("failed to get stdout for MCP server"))?;
let mut mcp = StdioChild {
@@ -203,17 +205,20 @@ pub(crate) fn spawn_stdio_child(command: &str, extra_args: &[String]) -> anyhow:
next_id: 0,
};
let deadline = std::time::Instant::now()
+ std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS);
let deadline =
std::time::Instant::now() + std::time::Duration::from_millis(MCP_CONNECT_TIMEOUT_MS);
let init_result = mcp.call("initialize", &json!({
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": {
"name": "zesdex",
"version": "0.1.0"
}
}));
let init_result = mcp.call(
"initialize",
&json!({
"protocolVersion": "2024-11-05",
"capabilities": {},
"clientInfo": {
"name": "zesdex",
"version": "0.1.0"
}
}),
);
if std::time::Instant::now() > deadline {
anyhow::bail!("MCP initialize timed out");
@@ -236,21 +241,29 @@ fn call_via_stdio(
// Reuse the persistent child handle if available; otherwise spawn a new one.
let mut guard;
let child: &mut StdioChild = if let Some(mtx) = existing_handle {
guard = mtx.lock().map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?;
guard = mtx
.lock()
.map_err(|e| anyhow::anyhow!("MCP handle lock: {e}"))?;
&mut guard
} else {
let mut fresh = spawn_stdio_child(command, extra_args)?;
let result = fresh.call("tools/call", &json!({
"name": tool_name,
"arguments": tool_args
}))?;
let result = fresh.call(
"tools/call",
&json!({
"name": tool_name,
"arguments": tool_args
}),
)?;
return Ok(extract_text_content(&result));
};
let result = child.call("tools/call", &json!({
"name": tool_name,
"arguments": tool_args
}))?;
let result = child.call(
"tools/call",
&json!({
"name": tool_name,
"arguments": tool_args
}),
)?;
Ok(extract_text_content(&result))
}
@@ -289,7 +302,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
}
});
let resp = client.post(url)
let resp = client
.post(url)
.header("Content-Type", "application/json")
.json(&body)
.send()
@@ -304,7 +318,8 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
anyhow::bail!("MCP HTTP server returned {status}: {text}");
}
let response: Value = resp.json()
let response: Value = resp
.json()
.map_err(|e| anyhow::anyhow!("invalid JSON from MCP HTTP server: {e}"))?;
if let Some(err) = response.get("error") {
@@ -321,13 +336,18 @@ fn call_via_http(url: &str, tool_name: &str, tool_args: &Value) -> anyhow::Resul
fn extract_text_content(result: &Value) -> String {
if let Some(content) = result.get("content") {
if let Some(arr) = content.as_array() {
let text: Vec<String> = arr.iter().filter_map(|item| {
if item.get("type").and_then(|t| t.as_str()) == Some("text") {
item.get("text").and_then(|t| t.as_str()).map(std::string::ToString::to_string)
} else {
None
}
}).collect();
let text: Vec<String> = arr
.iter()
.filter_map(|item| {
if item.get("type").and_then(|t| t.as_str()) == Some("text") {
item.get("text")
.and_then(|t| t.as_str())
.map(std::string::ToString::to_string)
} else {
None
}
})
.collect();
if !text.is_empty() {
return text.join("\n");
}
@@ -372,12 +392,17 @@ impl crate::tool::Tool for McpToolAdapter {
fn run(&self, _ctx: &crate::tool::ToolCtx, args: &Value) -> anyhow::Result<String> {
match &self.transport {
McpTransport::Stdio { command, args: extra_args } => {
call_via_stdio(self.child_handle.as_ref().map(std::convert::AsRef::as_ref), command, extra_args, &self.tool_name, args)
}
McpTransport::StreamableHttp { url } => {
call_via_http(url, &self.tool_name, args)
}
McpTransport::Stdio {
command,
args: extra_args,
} => call_via_stdio(
self.child_handle.as_ref().map(std::convert::AsRef::as_ref),
command,
extra_args,
&self.tool_name,
args,
),
McpTransport::StreamableHttp { url } => call_via_http(url, &self.tool_name, args),
}
}
}
@@ -400,27 +425,35 @@ impl McpManager {
///
/// Return: boxed `Tool` trait objects ready to merge into the harness's tool list.
pub fn as_tools(&self) -> Vec<Box<dyn crate::tool::Tool>> {
self.servers.iter().flat_map(|server| {
let handle = server.child_handle.clone();
server.tools.iter().map(move |info| {
let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter {
tool_name: info.name.clone(),
server_name: server.name.clone(),
transport: server.transport.clone(),
description: info.description.clone(),
parameters: info.input_schema.clone(),
child_handle: handle.clone(),
});
adapter
self.servers
.iter()
.flat_map(|server| {
let handle = server.child_handle.clone();
server.tools.iter().map(move |info| {
let adapter: Box<dyn crate::tool::Tool> = Box::new(McpToolAdapter {
tool_name: info.name.clone(),
server_name: server.name.clone(),
transport: server.transport.clone(),
description: info.description.clone(),
parameters: info.input_schema.clone(),
child_handle: handle.clone(),
});
adapter
})
})
}).collect()
.collect()
}
/// Connects to an MCP server via stdio by spawning the child process, running
/// the `initialize` handshake, calling `tools/list`, and registering the server
/// with its advertised tools in `self.servers`. The child process stays alive
/// for subsequent `tools/call` invocations via the stored `McpServer.tools`.
pub fn connect_stdio(&mut self, name: &str, command: &str, extra_args: &[String]) -> anyhow::Result<()> {
pub fn connect_stdio(
&mut self,
name: &str,
command: &str,
extra_args: &[String],
) -> anyhow::Result<()> {
let transport = McpTransport::Stdio {
command: command.to_string(),
args: extra_args.to_vec(),
@@ -430,19 +463,32 @@ impl McpManager {
let result = child.call("tools/list", &json!({}))?;
let tools = if let Some(tool_list) = result.get("tools").and_then(|v| v.as_array()) {
tool_list.iter().filter_map(|t| {
Some(McpToolInfo {
name: t.get("name")?.as_str()?.to_string(),
description: t.get("description").and_then(|v| v.as_str()).unwrap_or_else(|| {
tracing::warn!("[mcp] tool {} missing description", t.get("name").and_then(|n| n.as_str()).unwrap_or("?"));
""
}).to_string(),
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| {
tracing::warn!("[mcp] tool {} missing inputSchema", t.get("name").and_then(|n| n.as_str()).unwrap_or("?"));
serde_json::Value::Null
}),
tool_list
.iter()
.filter_map(|t| {
Some(McpToolInfo {
name: t.get("name")?.as_str()?.to_string(),
description: t
.get("description")
.and_then(|v| v.as_str())
.unwrap_or_else(|| {
tracing::warn!(
"[mcp] tool {} missing description",
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
);
""
})
.to_string(),
input_schema: t.get("inputSchema").cloned().unwrap_or_else(|| {
tracing::warn!(
"[mcp] tool {} missing inputSchema",
t.get("name").and_then(|n| n.as_str()).unwrap_or("?")
);
serde_json::Value::Null
}),
})
})
}).collect()
.collect()
} else {
Vec::new()
};
@@ -458,12 +504,4 @@ impl McpManager {
Ok(())
}
/// Removes a server by name. Returns `true` if a server was found and removed.
#[allow(dead_code)]
pub fn disconnect(&mut self, name: &str) -> bool {
let len = self.servers.len();
self.servers.retain(|s| s.name != name);
self.servers.len() < len
}
}
-1
View File
@@ -1,4 +1,3 @@
//! Model Context Protocol (MCP) client: connects to external MCP servers
//! (stdio or HTTP) and exposes their tools through the crate's `Tool` trait.
pub mod manager;
+5 -5
View File
@@ -1,13 +1,13 @@
//! Top-level application module: harness, modes, runtime loop, state,
//! workflows, subagents, review, background bash, MCP integration, and
//! native LSP client.
pub mod bgbash;
pub mod harness;
pub mod lsp;
pub mod mcp;
pub mod mode;
pub mod review;
pub mod runtime;
pub mod state;
pub mod workflow;
pub mod subagent;
pub mod review;
pub mod bgbash;
pub mod mcp;
pub mod lsp;
pub mod workflow;
-1
View File
@@ -1,5 +1,4 @@
//! Bash mode: handles submitting a shell command from the bash input panel.
use crate::app::state::rest::AppStateRest;
/// Launch a background bash job for the submitted command.
+5 -5
View File
@@ -1,6 +1,5 @@
//! Editor mode: a minimal in-TUI line editor for viewing/modifying a file,
//! with bounded undo history.
use crate::app::state::rest::AppStateRest;
use crate::app::state::types::Overlay;
@@ -66,7 +65,9 @@ impl EditorState {
self.cursor_line += 1;
}
self.cursor_col = self.cursor_col.min(
self.content.get(self.cursor_line).map_or(0, std::string::String::len),
self.content
.get(self.cursor_line)
.map_or(0, std::string::String::len),
);
}
@@ -114,10 +115,9 @@ impl EditorState {
/// the char directly → mark state dirty.
pub fn handle_editor_input(state: &mut AppStateRest, text: &str) {
let editor = &mut state.misc.editor;
if editor.is_none() {
let Some(ed) = editor.as_mut() else {
return;
}
let ed = editor.as_mut().unwrap();
};
for c in text.chars() {
match c {
'\n' | '\r' => {
+6 -2
View File
@@ -1,7 +1,11 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
#![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Effort mode: cycles the agent's reasoning effort level, which scales the
//! LLM's temperature and `max_tokens` for subsequent turns.
use crate::app::state::rest::AppStateRest;
pub const EFFORT_LEVELS: &[&str] = &["low", "medium", "high", "xhigh", "max"];
-1
View File
@@ -1,5 +1,4 @@
//! Help mode: static help text and the action that opens/closes the help overlay.
use crate::app::runtime::actions::Action;
use crate::app::state::types::Overlay;
-1
View File
@@ -1,5 +1,4 @@
//! Key input mode: raw text capture overlay used for one-off key/text prompts.
use crate::app::state::rest::AppStateRest;
/// Replace the input buffer with the given text and mark state dirty.
+4 -2
View File
@@ -33,14 +33,16 @@ pub fn get_learning_items(state: &AppStateRest) -> Vec<LearningItem> {
let scope_str = match p.lesson.scope {
crate::app::review::LessonScope::Project => "project",
crate::app::review::LessonScope::Global => "global",
}.to_string();
}
.to_string();
let conf_str = match p.lesson.confidence {
crate::app::review::Confidence::Human => "human",
crate::app::review::Confidence::Verified => "verified",
crate::app::review::Confidence::Unverified => "unverified",
crate::app::review::Confidence::Auto => "auto",
}.to_string();
}
.to_string();
items.push(LearningItem::Pending {
name: p.lesson.name,
-1
View File
@@ -1,5 +1,4 @@
//! Loading mode: transient overlay shown while waiting on an async operation.
use crate::app::state::rest::AppStateRest;
pub const LOADING_MESSAGES: &[&str] = &[
-1
View File
@@ -1,5 +1,4 @@
//! MCP mode: overlay for connecting to a configured MCP server.
use crate::app::state::rest::AppStateRest;
/// Placeholder entry point for connecting to an MCP server by name.
+1 -2
View File
@@ -1,14 +1,13 @@
//! TUI mode definitions and per-mode input/action handlers, one submodule
//! per overlay/mode (bash, editor, effort, mcp, quit confirm, rewind, etc.).
pub mod bash;
pub mod editor;
pub mod effort;
pub mod key_input;
pub mod mcp;
pub mod learning;
pub mod quit_confirm;
pub mod rewind;
pub mod settings;
pub mod todo;
pub mod learning;
-1
View File
@@ -1,5 +1,4 @@
//! Quit-confirm mode: the "are you sure?" overlay shown before exiting.
use crate::app::runtime::actions::Action;
/// Translate the user's yes/no answer on the quit-confirm overlay into an action.
+18 -7
View File
@@ -1,13 +1,19 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
#![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
//! session's `SQLite` blob store.
use crate::app::state::rest::AppStateRest;
use sha2::Digest;
/// Returns the number of stored pre-edit blobs (snapshots) for this session.
pub fn rewind_count(state: &AppStateRest) -> usize {
let Ok(conn) = open_session_db(&state.session_dir) else { return 0 };
let Ok(conn) = open_session_db(&state.session_dir) else {
return 0;
};
crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id)
.ok()
.map_or(0, |keys| keys.len())
@@ -51,7 +57,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
}
let blob_key = &keys[index];
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key) {
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key)
{
Ok(Some(b)) => b,
Ok(None) => {
state.push_toast(crate::app::state::types::Toast::new(
@@ -74,8 +81,8 @@ pub fn rewind_to(state: &mut AppStateRest, index: usize) {
// Look up the path from the edit log — the blob key is the tool_call_id.
// The edit log doesn't store the tool_call_id directly, so fall back to the
// path from the most recent write/edit entry.
let restore_path = find_edit_path(state, blob_key)
.unwrap_or_else(|| state.session_dir.join("snapshot.dat"));
let restore_path =
find_edit_path(state, blob_key).unwrap_or_else(|| state.session_dir.join("snapshot.dat"));
match std::fs::write(&restore_path, &bytes) {
Ok(()) => {
@@ -119,6 +126,10 @@ fn open_session_db(session_dir: &std::path::Path) -> anyhow::Result<rusqlite::Co
fn find_edit_path(state: &AppStateRest, _blob_key: &str) -> Option<std::path::PathBuf> {
let el = crate::model::editlog::EditLog::new(&state.session_dir);
let entry = el.entries.iter().rev().find(|e| e.tool == "write" || e.tool == "edit")?;
let entry = el
.entries
.iter()
.rev()
.find(|e| e.tool == "write" || e.tool == "edit")?;
Some(std::path::PathBuf::from(&entry.path))
}
+1 -2
View File
@@ -3,8 +3,7 @@
//! Flow: exposes small mutation functions (currently just cycling the
//! internet access mode) invoked by keybindings while the settings overlay
//! is active.
use crate::model::settings::{Settings, InternetMode};
use crate::model::settings::{InternetMode, Settings};
/// Advance the internet access mode to the next value in the cycle.
///
-1
View File
@@ -2,7 +2,6 @@
//!
//! Flow: exposes the toggle handler invoked by a keybinding to show/hide
//! the todo overlay.
use crate::app::state::rest::AppStateRest;
use crate::app::state::types::Overlay;
+1 -1
View File
@@ -76,7 +76,7 @@ pub fn should_trigger_review(state: &AppStateRest, origin: Origin) -> bool {
return false;
}
let Some(runtime) = &state.session_runtime else { return false };
if !state.settings.review_enabled {
if !state.settings.flags.review_enabled {
return false;
}
if runtime.edit_count > 0 && runtime.edit_count % 5 == 0 {
+4 -4
View File
@@ -556,7 +556,7 @@ pub fn apply_action(state: &mut AppStateRest, action: Action) {
.map(str::len)
.sum();
let token_estimate = total_chars / 3;
rt.messages = crate::app::runtime::shortsend::shape_messages(&rt.messages, token_estimate, max_wire_tokens, true, None);
rt.messages = crate::app::runtime::context::shaping::shape_messages(&rt.messages, token_estimate, max_wire_tokens, true, None);
state.push_toast(Toast::new(ToastKind::Success, "Conversation history compacted.".to_string()));
state.dirty = true;
}
@@ -1154,10 +1154,10 @@ fn run_agent_turn(
// Skip message compaction if abort was requested — the non-streaming
// LLM call for summarization would block without checking abort_flag.
let wire_msgs = if !tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
&& crate::app::runtime::shortsend::should_shape(token_estimate, max_wire_tokens, prev_shaped)
&& crate::app::runtime::context::shaping::should_shape(token_estimate, max_wire_tokens, prev_shaped)
{
prev_shaped = true;
let compacted = crate::app::runtime::shortsend::shape_messages(&msgs, token_estimate, max_wire_tokens, false, Some(&tc.client));
let compacted = crate::app::runtime::context::shaping::shape_messages(&msgs, token_estimate, max_wire_tokens, false, Some(&tc.client));
// Dispatch the compacted messages to the main thread so the local session history
// is permanently compacted and doesn't trigger shaping again immediately on next turn.
@@ -1612,7 +1612,7 @@ fn execute_one_tool(
/// `should_trigger_review` on `Tick`), only informs the user that
/// a review has material to examine.
fn maybe_trigger_review(state: &mut AppStateRest) {
if !state.settings.review_enabled {
if !state.settings.flags.review_enabled {
return;
}
let edit_count = state
+1 -1
View File
@@ -1,8 +1,8 @@
//! Maps parsed `/` slash commands into one or more `Action` variants
//! that `apply_action` can process.
use crate::controller::command::Command;
use crate::app::runtime::actions::Action;
use crate::app::state::types::Overlay;
use crate::controller::command::Command;
/// Convert a parsed `Command` into the corresponding sequence of `Action`s.
///
+197
View File
@@ -0,0 +1,197 @@
#![allow(dead_code)]
//! Cross-call tool-result deduplication: when a read-only tool is called
//! again with identical arguments, the earlier result is replaced with a
//! placeholder so only the latest copy occupies context.
//!
//! Flow: pair each `Role::Tool` message to its originating `ToolCall` via
//! `tool_call_id` -> key on `(function.name, sha256(canonical_json(args)))`
//! -> for read-only tools, keep only the last occurrence of each key in
//! full, placeholder the rest.
//!
//! Why: reading the same file (or re-running the same grep) twice in a
//! session otherwise keeps both full copies in context until compaction
//! eventually drops the older one wholesale, along with everything else
//! from that period. Mutating tools (`write`, `edit`, `bash`, `delete`,
//! `git_operator`, ...) are never touched, even with identical
//! arguments, because call order and repetition can be semantically
//! meaningful (e.g. retrying a flaky `bash` command until it passes).
use crate::app::subagent::division::tool_scope::READ_TOOLS;
use crate::dto::chat::message::{ChatMessage, Role};
use sha2::Digest;
use std::collections::HashMap;
const DUPLICATE_PLACEHOLDER: &str =
"[duplicate result — superseded by a later identical call, see below]";
/// Replace superseded read-only tool results with a placeholder.
///
/// Return: a `Vec<ChatMessage>` the same length as `messages`, and
/// `true` iff at least one entry was replaced. The caller uses the
/// `bool` to decide whether the result is worth persisting/announcing,
/// without `ChatMessage` needing to implement `PartialEq`.
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool) {
// tool_call_id -> (tool name, canonical JSON of its arguments)
let mut call_info: HashMap<String, (String, String)> = HashMap::new();
for m in messages {
if let Some(calls) = &m.tool_calls {
for call in calls {
let canonical = serde_json::to_string(&call.function.arguments).unwrap_or_default();
call_info.insert(call.id.clone(), (call.function.name.clone(), canonical));
}
}
}
// For each (tool, args-hash) key among read-only tools, find the
// index of its LAST occurrence — that's the one kept in full.
let mut last_index_for_key: HashMap<String, usize> = HashMap::new();
for (idx, m) in messages.iter().enumerate() {
if m.role != Role::Tool {
continue;
}
let Some(id) = &m.tool_call_id else { continue };
let Some((name, args)) = call_info.get(id) else {
continue;
};
if !READ_TOOLS.contains(&name.as_str()) {
continue;
}
last_index_for_key.insert(dedup_key(name, args), idx);
}
let mut changed = false;
let result = messages
.iter()
.enumerate()
.map(|(idx, m)| {
if m.role != Role::Tool {
return m.clone();
}
let Some(id) = &m.tool_call_id else {
return m.clone();
};
let Some((name, args)) = call_info.get(id) else {
return m.clone();
};
if !READ_TOOLS.contains(&name.as_str()) {
return m.clone();
}
let key = dedup_key(name, args);
if last_index_for_key.get(&key) == Some(&idx) {
return m.clone();
}
changed = true;
ChatMessage::tool_result(id.clone(), DUPLICATE_PLACEHOLDER.to_string())
})
.collect();
(result, changed)
}
/// Build the dedup key for a tool call.
///
/// Why hash the arguments: keeps the key a fixed, short size regardless
/// of argument payload size. `serde_json::to_string` is already
/// canonical here — this codebase doesn't enable `serde_json`'s
/// `preserve_order` feature, so `Value::Object` is backed by a
/// `BTreeMap` and always serializes keys in sorted order.
fn dedup_key(tool_name: &str, canonical_args: &str) -> String {
let hash = hex::encode(sha2::Sha256::digest(canonical_args.as_bytes()));
format!("{tool_name}:{hash}")
}
#[cfg(test)]
mod tests {
use super::*;
use crate::dto::chat::message::ChatMessage;
use crate::dto::chat::tool::{ToolCall, ToolFunction};
use serde_json::json;
fn assistant_with_call(id: &str, name: &str, args: serde_json::Value) -> ChatMessage {
let mut m = ChatMessage::assistant(None);
m.tool_calls = Some(vec![ToolCall {
id: id.to_string(),
type_: "function".to_string(),
function: ToolFunction {
name: name.to_string(),
arguments: args,
},
}]);
m
}
#[test]
fn older_result_of_same_read_tool_and_args_is_replaced() {
let messages = vec![
assistant_with_call("call-1", "read", json!({"path": "a.rs"})),
ChatMessage::tool_result("call-1".to_string(), "first read of a.rs".to_string()),
assistant_with_call("call-2", "read", json!({"path": "a.rs"})),
ChatMessage::tool_result("call-2".to_string(), "second read of a.rs".to_string()),
];
let (result, changed) = collapse(&messages);
assert!(changed);
assert_eq!(result[1].content.as_deref(), Some(DUPLICATE_PLACEHOLDER));
assert_eq!(result[3].content.as_deref(), Some("second read of a.rs"));
}
#[test]
fn different_arguments_are_not_deduplicated() {
let messages = vec![
assistant_with_call("call-1", "read", json!({"path": "a.rs"})),
ChatMessage::tool_result("call-1".to_string(), "read of a.rs".to_string()),
assistant_with_call("call-2", "read", json!({"path": "b.rs"})),
ChatMessage::tool_result("call-2".to_string(), "read of b.rs".to_string()),
];
let (result, changed) = collapse(&messages);
assert!(!changed);
assert_eq!(result[1].content.as_deref(), Some("read of a.rs"));
assert_eq!(result[3].content.as_deref(), Some("read of b.rs"));
}
#[test]
fn key_order_in_arguments_does_not_prevent_dedup() {
let messages = vec![
assistant_with_call("call-1", "grep", json!({"pattern": "foo", "path": "."})),
ChatMessage::tool_result("call-1".to_string(), "first grep".to_string()),
assistant_with_call("call-2", "grep", json!({"path": ".", "pattern": "foo"})),
ChatMessage::tool_result("call-2".to_string(), "second grep".to_string()),
];
let (result, changed) = collapse(&messages);
assert!(changed);
assert_eq!(result[1].content.as_deref(), Some(DUPLICATE_PLACEHOLDER));
}
#[test]
fn mutating_tool_with_identical_args_is_never_deduplicated() {
let messages = vec![
assistant_with_call("call-1", "bash", json!({"command": "cargo test"})),
ChatMessage::tool_result("call-1".to_string(), "first run: 3 failed".to_string()),
assistant_with_call("call-2", "bash", json!({"command": "cargo test"})),
ChatMessage::tool_result("call-2".to_string(), "second run: 0 failed".to_string()),
];
let (result, changed) = collapse(&messages);
assert!(!changed);
assert_eq!(result[1].content.as_deref(), Some("first run: 3 failed"));
assert_eq!(result[3].content.as_deref(), Some("second run: 0 failed"));
}
#[test]
fn tool_result_with_no_matching_call_is_left_untouched() {
let messages = vec![ChatMessage::tool_result(
"orphan-id".to_string(),
"some result".to_string(),
)];
let (result, changed) = collapse(&messages);
assert!(!changed);
assert_eq!(result[0].content.as_deref(), Some("some result"));
}
}
+16
View File
@@ -0,0 +1,16 @@
//! Context management: token counting, cross-call tool-result dedup,
//! per-result compression, budget-based shaping, and shared
//! context-window resolution — replaces `runtime::shortsend`.
//!
//! No facade function here: `dedup`, `shaping`, and `tokens` are called
//! directly from each call site (the per-turn auto-compaction loop in
//! `actions::run_agent_turn`, and `Action::Compact`), matching this
//! codebase's "no DI, call modules directly" convention. An orchestration
//! layer would only serve one of the two callers generically — the
//! auto-loop already needs per-stage control to decide when to emit
//! `TurnEvent::Compacted`.
pub mod dedup;
pub mod shaping;
pub mod squash;
pub mod tokens;
pub mod window;
+211
View File
@@ -0,0 +1,211 @@
//! Budget-based message shaping: compacts long conversation histories so
//! they fit within the provider's context window before being sent to
//! the LLM API. Ported from the former `runtime::shortsend` — behavior
//! is unchanged, only its token-counting now goes through
//! `context::tokens` instead of an inline heuristic.
use super::tokens::count_tokens;
use crate::dto::chat::message::ChatMessage;
/// Decide whether the message list should be shaped (compacted) before
/// sending to the LLM.
///
/// Flow: trigger based on token estimate. If `token_estimate` exceeds
/// the threshold, we shape. When `prev_shaped` is true, the threshold is
/// raised (95%) to avoid fluttering — compaction only re-triggers when
/// the context is genuinely full again. When `prev_shaped` is false, the
/// threshold is lower (85%) so compaction starts proactively.
///
/// Why: hysteresis prevents repeated compaction on every turn when the
/// token count hovers near the boundary.
///
/// Return: `true` if shaping should be applied.
pub fn should_shape(token_estimate: usize, max_wire_tokens: usize, prev_shaped: bool) -> bool {
let threshold = if prev_shaped {
(max_wire_tokens as f32 * 0.95) as usize
} else {
(max_wire_tokens as f32 * 0.85) as usize
};
token_estimate >= threshold
}
/// Compact a long message list by dropping middle messages and inserting
/// a summary placeholder.
///
/// Flow: if the estimated token count is within budget and not forced,
/// return messages unchanged -> otherwise keep the system message and
/// the most recent messages that fit a 70%-of-budget target, with a
/// `[prior conversation compacted]` (or LLM-generated summary, if
/// `client` is `Some`) system message in between.
///
/// Why: keeps context-size overhead roughly constant regardless of
/// session length.
///
/// Return: the shaped message list, or `messages` unchanged if shaping
/// wasn't needed.
pub fn shape_messages(
messages: &[ChatMessage],
token_count: usize,
max_wire_tokens: usize,
force: bool,
client: Option<&crate::service::provider::LlmClient>,
) -> Vec<ChatMessage> {
if !force && (token_count <= max_wire_tokens || messages.len() < 5) {
return messages.to_vec();
}
let target_tokens = (max_wire_tokens as f32 * 0.70) as usize;
let mut current_tokens = 0;
let mut keep_recent = Vec::new();
let mut dropped_msgs = Vec::new();
let mut msgs_to_eval = messages.to_vec();
let first = if msgs_to_eval.is_empty() {
None
} else {
Some(msgs_to_eval.remove(0))
};
for m in msgs_to_eval.into_iter().rev() {
let text = m.content.as_deref().unwrap_or("");
let msg_tokens = count_tokens(text);
if current_tokens + msg_tokens <= target_tokens {
current_tokens += msg_tokens;
keep_recent.push(m);
} else {
dropped_msgs.push(m);
}
}
dropped_msgs.reverse();
let mut result = Vec::new();
if let Some(f) = first {
result.push(f);
}
if !dropped_msgs.is_empty() {
let mut summary_text = "[prior conversation compacted]".to_string();
if let Some(llm) = client {
let prompt = format!(
"Summarize the following dropped conversation history briefly. Focus on main goals, decisions made, and files modified, so the context is preserved for future turns. Keep it concise.\n\nHistory:\n{}",
dropped_msgs.iter()
.map(|m| format!("[{}]: {}", if m.role == crate::dto::chat::message::Role::User { "User" } else { "Assistant" }, m.content.as_deref().unwrap_or("")))
.collect::<Vec<_>>()
.join("\n\n")
);
let req_msgs = vec![ChatMessage::user(prompt)];
match llm.chat_with_tools_non_streaming(&req_msgs, None) {
Ok(resp) => {
if let Some(content) = resp.0.content {
summary_text =
format!("[Summary of compacted prior conversation:\n{content}\n]");
}
}
Err(e) => {
tracing::warn!(
"[context::shaping] LLM summarization failed: {}. \
Prior conversation history is lost no summary available. \
This means the model will lose context about earlier parts of \
the conversation.",
e,
);
}
}
}
result.push(ChatMessage::system(summary_text));
}
result.extend(keep_recent.into_iter().rev());
result
}
#[cfg(test)]
mod tests {
use super::*;
use crate::dto::chat::message::ChatMessage;
#[test]
fn should_shape_triggers_at_85_percent_when_not_previously_shaped() {
assert!(should_shape(850, 1000, false));
assert!(!should_shape(849, 1000, false));
}
#[test]
fn should_shape_uses_95_percent_threshold_once_already_shaped() {
assert!(
!should_shape(900, 1000, true),
"below 95% and already shaped: no re-trigger yet"
);
assert!(should_shape(950, 1000, true));
}
#[test]
fn shape_messages_is_a_noop_under_budget_and_not_forced() {
let messages = vec![
ChatMessage::system("sys"),
ChatMessage::user("hi"),
ChatMessage::assistant(Some("hello".to_string())),
];
let result = shape_messages(&messages, 10, 1000, false, None);
assert_eq!(result.len(), messages.len());
}
/// Build a message whose real BPE token count is large enough that 20
/// of them (~49 tokens each, ~980 total — verified empirically with
/// `context::tokens::count_tokens`) comfortably exceed
/// `shape_messages`'s 70%-of-1000 = 700 token target, guaranteeing
/// several get dropped. A short fixture like `format!("message {i}")`
/// (~8 tokens each, ~160 total for 20) stays entirely under budget
/// with real BPE counting and would make these tests pass vacuously
/// (nothing ever gets dropped, so "must survive shaping" and "falls
/// back to placeholder" hold trivially without exercising the actual
/// drop logic) — this was a real bug caught during Task 5's first
/// implementation attempt.
fn padded_message(i: usize) -> String {
format!(
"message number {i} with some padding text {}",
"additional padding content to increase token count substantially ".repeat(5),
)
}
#[test]
fn shape_messages_always_preserves_the_first_system_message() {
let mut messages = vec![ChatMessage::system("system prompt")];
for i in 0..20 {
messages.push(ChatMessage::user(padded_message(i)));
}
let result = shape_messages(&messages, 100_000, 1000, true, None);
assert_eq!(result[0].content.as_deref(), Some("system prompt"));
}
#[test]
fn shape_messages_without_a_client_falls_back_to_placeholder_summary() {
let mut messages = vec![ChatMessage::system("system prompt")];
for i in 0..20 {
messages.push(ChatMessage::user(padded_message(i)));
}
let result = shape_messages(&messages, 100_000, 1000, true, None);
let has_placeholder = result
.iter()
.any(|m| m.content.as_deref() == Some("[prior conversation compacted]"));
assert!(has_placeholder);
}
#[test]
fn shape_messages_keeps_most_recent_messages_over_older_ones() {
let mut messages = vec![ChatMessage::system("system prompt")];
for i in 0..20 {
messages.push(ChatMessage::user(padded_message(i)));
}
let result = shape_messages(&messages, 100_000, 1000, true, None);
let last_content = messages.last().unwrap().content.clone();
assert!(
result.iter().any(|m| m.content == last_content),
"most recent message must survive shaping"
);
}
}
+464
View File
@@ -0,0 +1,464 @@
#![allow(dead_code)]
//! Per-tool-result compression: shrink large tool outputs before they
//! ever enter conversation history, dispatching by content shape.
//!
//! Flow: `apply(tool_name, output)` -> `read` tool or under the size
//! floor? pass through unchanged : valid JSON? `squash_json` : tool is
//! `bash` and looks log-shaped? `squash_log` : `squash_generic`.
//!
//! Why: a single large `bash`/`grep` result can dominate a
//! conversation's token budget even on its first occurrence, long
//! before `dedup`/`shaping` ever get a chance to act on repeats or
//! overall budget.
use std::collections::HashSet;
use std::fmt::Write;
/// Below this size, compression isn't worth the risk of losing detail —
/// pass the output through unchanged.
const SQUASH_FLOOR_BYTES: usize = 1500;
/// Byte budget for the generic fallback compressor — double the squash
/// floor, so the fallback path still yields a real reduction on
/// anything that triggered it.
const GENERIC_BUDGET_BYTES: usize = SQUASH_FLOOR_BYTES * 2;
/// Tools whose output must never be altered. `read` is exempted because
/// its output must stay byte-exact — the agent relies on it for
/// exact-match edits afterward, and squashing a file that happens to
/// parse as JSON (e.g. `package.json`) would silently corrupt the
/// agent's view of real file content.
const NEVER_SQUASH: &[&str] = &["read"];
/// Tools whose output the log classifier is allowed to run on.
/// `looks_log_shaped` keys purely on content (>=3 error/warn/fail-shaped
/// lines), which a `grep`/`search` result full of matches against
/// error-handling code would trip just as easily as a real build log —
/// but `squash_log` caps at 20 error + 10 warning lines with no byte
/// budget, silently dropping legitimate matches past that cap. Only
/// `bash` (the actual log-producing tool) is allowed to route through
/// it; everything else that looks log-shaped falls through to the
/// gentler, byte-budgeted `squash_generic` instead.
const LOG_SHAPED_TOOLS: &[&str] = &["bash"];
/// Compress a tool's raw output before it's stored in conversation
/// history.
///
/// Return: `output` unchanged if `tool_name` is in `NEVER_SQUASH` or at
/// or under `SQUASH_FLOOR_BYTES`; otherwise the compressed form from
/// whichever detector matches its content shape.
pub fn apply(tool_name: &str, output: &str) -> String {
if NEVER_SQUASH.contains(&tool_name) || output.len() <= SQUASH_FLOOR_BYTES {
return output.to_string();
}
if serde_json::from_str::<serde_json::Value>(output).is_ok() {
return squash_json(output);
}
if LOG_SHAPED_TOOLS.contains(&tool_name) && looks_log_shaped(output) {
return squash_log(output);
}
squash_generic(output, GENERIC_BUDGET_BYTES)
}
/// Compress a JSON tool result by keeping all structural content (keys,
/// array/object shape) and eliding long, low-entropy string *values*,
/// while keeping short values (<=20 chars) and high-entropy single-token
/// ones (UUIDs, hashes, paths) intact. Array elements past the first 3
/// are elided regardless of length/entropy.
///
/// Why walk a parsed `Value` instead of hand-rolling a JSON tokenizer:
/// `serde_json` already handles escaping/nesting correctly (this
/// codebase's own `dto::chat::tool::repair_json` exists specifically to
/// work around how easy it is to get that wrong by hand) — reusing it
/// is both simpler and more robust.
///
/// Return: re-serialized JSON with the same shape as the input.
fn squash_json(text: &str) -> String {
let Ok(mut value) = serde_json::from_str::<serde_json::Value>(text) else {
return text.to_string();
};
squash_json_value(&mut value, false);
serde_json::to_string(&value).unwrap_or_else(|_| text.to_string())
}
/// Recursively elide long, low-entropy string values in place.
/// `in_late_array` is true once past the first 3 elements of an
/// enclosing array, tightening the elision rule for the rest of it.
///
/// Why the `!s.contains(' ')` gate before the entropy check: raw
/// per-character Shannon entropy alone does NOT separate "meaningful
/// prose" from "random-looking identifier" — verified empirically,
/// repeated English prose scores ~3.89 bits/char, *higher* than a UUID's
/// ~3.39 or a SHA-256 hex digest's ~3.66, because prose draws from a
/// wide, fairly-balanced character set too. What actually distinguishes
/// identifiers from prose is that identifiers are a single unbroken
/// token — this mirrors headroom's own approach (its entropy check is
/// "cheaply pre-filtered by 'no spaces'" before scoring). Multi-word
/// values never reach the entropy branch at all; only whitespace-free
/// tokens do, where entropy correctly separates "abc123" or "aaaaaaaa"
/// (low, elided if long) from a UUID/hash/API-key-shaped string (high,
/// kept).
fn squash_json_value(value: &mut serde_json::Value, in_late_array: bool) {
match value {
serde_json::Value::String(s) => {
let looks_like_identifier = !s.contains(' ') && shannon_entropy(s) >= 3.0;
let keep = !in_late_array && (s.len() <= 20 || looks_like_identifier);
if !keep {
*s = "".to_string();
}
}
serde_json::Value::Array(items) => {
for (i, item) in items.iter_mut().enumerate() {
squash_json_value(item, i >= 3);
}
}
serde_json::Value::Object(map) => {
for v in map.values_mut() {
squash_json_value(v, false);
}
}
_ => {}
}
}
/// Shannon entropy in bits per character — used, after the `squash_json`
/// caller's own "no internal whitespace" pre-filter, to distinguish
/// high-entropy single-token strings (UUIDs, hashes, random IDs, worth
/// keeping) from low-entropy ones (e.g. `"aaaaaaaaaa"`, safe to elide).
/// 3.0 sits comfortably below a UUID's ~3.39 and a SHA-256 hex digest's
/// ~3.66 (both empirically measured with this exact formula) while
/// staying well above a degenerate repeated-character string's 0.0.
fn shannon_entropy(s: &str) -> f64 {
if s.is_empty() {
return 0.0;
}
let mut counts: std::collections::HashMap<char, usize> = std::collections::HashMap::new();
for c in s.chars() {
*counts.entry(c).or_insert(0) += 1;
}
let len = s.chars().count() as f64;
counts
.values()
.map(|&count| {
let p = f64::from(u32::try_from(count).unwrap_or(u32::MAX)) / len;
-p * p.log2()
})
.sum()
}
/// Coarse severity classification for a single log line, used by
/// `squash_log` to rank which lines are most worth keeping.
#[derive(Clone, Copy, PartialEq, Eq)]
enum LogLevel {
Error,
Warn,
Info,
Debug,
}
/// Classify a single log line by scanning for level keywords.
///
/// Why substring matching on a lowercased copy instead of a real log
/// parser: tool output comes from arbitrary external processes with no
/// consistent log format, so keyword sniffing is the only detector that
/// generalizes across all of them.
fn classify_line(line: &str) -> LogLevel {
let lower = line.to_lowercase();
if lower.contains("error") || lower.contains("fail") || lower.contains("panic") {
LogLevel::Error
} else if lower.contains("warn") {
LogLevel::Warn
} else if lower.contains("debug") || lower.contains("trace") {
LogLevel::Debug
} else {
LogLevel::Info
}
}
/// Heuristic gate for routing to `squash_log` vs `squash_generic`: at
/// least 3 lines that look like error/warning/stack-trace output.
fn looks_log_shaped(text: &str) -> bool {
let hits = text
.lines()
.filter(|l| {
let lower = l.to_lowercase();
lower.contains("error")
|| lower.contains("warn")
|| lower.contains("fail")
|| lower.contains("panic")
|| l.trim_start().starts_with("at ")
})
.count();
hits >= 3
}
/// Compress log-shaped output: keep up to 20 highest-scored error lines
/// and up to 10 highest-scored warning lines (score = level weight +
/// 0.3 if the line looks like a stack-trace frame), each with a
/// +/-2-line context window, replacing every gap with a `[N lines
/// omitted]` marker.
///
/// Why not a comment-shaped marker (e.g. `// N lines omitted`): the
/// `rtk` project's own regression tests found that shape gets parsed by
/// the LLM as code and triggers a retry loop.
fn squash_log(text: &str) -> String {
let lines: Vec<&str> = text.lines().collect();
let levels: Vec<LogLevel> = lines.iter().map(|l| classify_line(l)).collect();
let score = |i: usize| -> f32 {
let level_score = match levels[i] {
LogLevel::Error => 1.0,
LogLevel::Warn => 0.5,
LogLevel::Info => 0.1,
LogLevel::Debug => 0.05,
};
let stack_boost = if lines[i].trim_start().starts_with("at ") {
0.3
} else {
0.0
};
level_score + stack_boost
};
let mut error_idxs: Vec<usize> = (0..lines.len())
.filter(|&i| levels[i] == LogLevel::Error)
.collect();
error_idxs.sort_by(|&a, &b| {
score(b)
.partial_cmp(&score(a))
.unwrap_or(std::cmp::Ordering::Equal)
});
error_idxs.truncate(20);
let mut warn_idxs: Vec<usize> = (0..lines.len())
.filter(|&i| levels[i] == LogLevel::Warn)
.collect();
warn_idxs.sort_by(|&a, &b| {
score(b)
.partial_cmp(&score(a))
.unwrap_or(std::cmp::Ordering::Equal)
});
warn_idxs.truncate(10);
let mut keep: HashSet<usize> = HashSet::new();
for &i in error_idxs.iter().chain(warn_idxs.iter()) {
let lo = i.saturating_sub(2);
let hi = (i + 2).min(lines.len().saturating_sub(1));
keep.extend(lo..=hi);
}
if keep.is_empty() {
return squash_generic(text, GENERIC_BUDGET_BYTES);
}
render_kept_lines(&lines, &keep)
}
/// Importance-ranked truncation for content that isn't JSON or
/// log-shaped: keep the first 10 and last 10 lines, plus any
/// non-blank line that isn't a repeat of the one before it, until
/// `budget` bytes are used.
fn squash_generic(text: &str, budget: usize) -> String {
let lines: Vec<&str> = text.lines().collect();
if lines.len() <= 20 {
return text.chars().take(budget).collect();
}
let head_end = 10;
let tail_start = lines.len() - 10;
let mut keep: HashSet<usize> = (0..head_end).chain(tail_start..lines.len()).collect();
let mut used: usize = lines[..head_end].iter().map(|l| l.len() + 1).sum::<usize>()
+ lines[tail_start..]
.iter()
.map(|l| l.len() + 1)
.sum::<usize>();
let mut prev = "";
for (i, &line) in lines.iter().enumerate().take(tail_start).skip(head_end) {
let non_trivial = !line.trim().is_empty() && line != prev;
if non_trivial && used + line.len() < budget {
keep.insert(i);
used += line.len() + 1;
}
prev = line;
}
render_kept_lines(&lines, &keep)
}
/// Render a subset of `lines` in order, inserting a `[N lines omitted]`
/// marker at every gap between kept lines.
fn render_kept_lines(lines: &[&str], keep: &HashSet<usize>) -> String {
let mut kept_sorted: Vec<usize> = keep.iter().copied().collect();
kept_sorted.sort_unstable();
let mut out = String::new();
let mut cursor = 0usize;
for &i in &kept_sorted {
if i > cursor {
let _ = writeln!(out, "[{} lines omitted]", i - cursor);
}
out.push_str(lines[i]);
out.push('\n');
cursor = i + 1;
}
if cursor < lines.len() {
let _ = writeln!(out, "[{} lines omitted]", lines.len() - cursor);
}
out
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn output_under_the_floor_passes_through_unchanged() {
let small = "short output";
assert_eq!(apply("bash", small), small);
}
#[test]
fn read_tool_output_is_never_squashed_even_when_huge_json() {
let big_json = format!(
"{{\"description\": \"{}\"}}",
"a very long description value that repeats ".repeat(100),
);
assert!(big_json.len() > SQUASH_FLOOR_BYTES);
assert_eq!(apply("read", &big_json), big_json);
}
#[test]
fn json_output_over_floor_keeps_structure_and_short_values() {
let value = serde_json::json!({
"id": "abc123",
"note": "hi",
"description": "a very long description value that repeats ".repeat(100),
});
let text = serde_json::to_string(&value).unwrap();
assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("some_mcp_tool", &text);
let parsed: serde_json::Value =
serde_json::from_str(&result).expect("squashed JSON must still be valid JSON");
assert_eq!(parsed["id"], "abc123", "short values must survive");
assert_eq!(parsed["note"], "hi", "short values must survive");
assert_ne!(
parsed["description"].as_str().unwrap().len(),
value["description"].as_str().unwrap().len(),
"long low-entropy value must be shrunk",
);
}
#[test]
fn json_array_elements_past_third_are_squashed_harder() {
// A UUID-shaped value has no internal whitespace and clears the
// entropy threshold, so under the *normal* per-value rule (which
// still applies to array indices 0-2) it survives untouched.
// Padding elsewhere in the object pushes total size over the
// squash floor without affecting which array elements get kept.
let identifier = "550e8400-e29b-41d4-a716-446655440000";
let padding = "padding text to push this payload past the squash floor so apply() actually dispatches to squash_json ".repeat(20);
let value = serde_json::json!({
"padding": padding,
"items": [identifier, identifier, identifier, identifier],
});
let text = serde_json::to_string(&value).unwrap();
assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("some_mcp_tool", &text);
let parsed: serde_json::Value = serde_json::from_str(&result).unwrap();
let items = parsed["items"].as_array().unwrap();
assert_eq!(items[0].as_str().unwrap(), identifier, "index 0 is under the array cutoff and identifier-shaped, so it's kept under the normal rule");
assert_eq!(
items[2].as_str().unwrap(),
identifier,
"index 2 is still under the cutoff (past-third means index >= 3)"
);
assert_ne!(items[3].as_str().unwrap(), identifier, "index 3 must be force-elided even though it's identifier-shaped and would survive at any earlier index");
}
#[test]
fn log_like_output_keeps_error_lines_and_marks_omissions() {
// `looks_log_shaped` requires >= 3 lines matching error/warn/fail/
// panic/stack-frame patterns before routing to `squash_log` at
// all — a single error line isn't enough and would silently fall
// through to `squash_generic` instead, so this fixture needs at
// least 3 such lines, spread apart, to actually exercise
// squash_log's scoring/windowing logic (not just its fallback).
let mut lines = vec!["build started".to_string()];
for i in 0..200 {
lines.push(format!("info: compiling module {i}"));
}
lines.push("error: something failed early in the build".to_string());
for i in 0..200 {
lines.push(format!("info: compiling module {}", i + 200));
}
lines.push("warning: deprecated api used somewhere".to_string());
lines.push("error: something failed at the end".to_string());
let text = lines.join("\n");
assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("bash", &text);
assert!(result.contains("error: something failed early in the build"));
assert!(result.contains("error: something failed at the end"));
assert!(result.contains("lines omitted"));
assert!(result.len() < text.len());
}
#[test]
fn non_bash_tool_with_log_shaped_content_is_not_log_compressed() {
// A grep result whose matched lines all mention "error" would
// trip `looks_log_shaped`'s >=3-line keyword threshold just like
// a real build log — but `squash_log` caps at 20 highest-scored
// error lines with no guaranteed tail retention, silently
// dropping legitimate matches past that cap. Only `bash` is
// treated as log-shaped; `grep` must fall through to
// `squash_generic`, which always keeps the first and last 10
// lines regardless of score. With every line tied at the same
// score, a `squash_log` route would keep indices 0-19 (stable
// sort preserves original order on ties) and drop index 49 —
// so asserting the tail survives is a route-distinguishing
// check, not just a content check.
let lines: Vec<String> = (0..50)
.map(|i| format!("src/file{i}.rs:{i}: error handling for case {i}"))
.collect();
let text = lines.join("\n");
assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("grep", &text);
assert!(
result.contains("src/file0.rs:0: error handling for case 0"),
"generic keeps head"
);
assert!(
result.contains("src/file49.rs:49: error handling for case 49"),
"generic keeps tail — squash_log would have dropped this"
);
}
#[test]
fn generic_large_text_is_truncated_with_omission_marker() {
let lines: Vec<String> = (0..500)
.map(|i| format!("line number {i} of plain output"))
.collect();
let text = lines.join("\n");
assert!(text.len() > SQUASH_FLOOR_BYTES);
let result = apply("bash", &text);
assert!(
result.contains("line number 0 of plain output"),
"keeps head"
);
assert!(
result.contains("line number 499 of plain output"),
"keeps tail"
);
assert!(result.contains("lines omitted"));
assert!(result.len() < text.len());
}
}
+71
View File
@@ -0,0 +1,71 @@
//! Unified token-count estimation for context-window budgeting.
//!
//! Flow: text -> `tiktoken_rs::o200k_base_singleton()` (BPE vocab embedded
//! in the binary via `include_str!`, no network access) -> `encode_ordinary`
//! -> token count.
//!
//! Why: replaces three independent char-count heuristics that disagreed
//! with each other (`/3` in the old `shortsend.rs`, `/4` in the turn
//! loop, `/4` again in the status bar) with one real BPE tokenizer.
//! `o200k_base` is an approximation for non-OpenAI providers but is far
//! closer than a flat byte-per-token guess; it's only used for the
//! 85%/95% budget thresholds, not for billing-accurate counts.
use crate::dto::chat::message::ChatMessage;
/// Count tokens in a single string under `o200k_base`.
///
/// Return: the BPE token count for `text`. `encode_ordinary` (not
/// `encode`/`encode_with_special_tokens`) is used deliberately — message
/// content that happens to contain a special-token-shaped substring
/// (e.g. literal text `<|endoftext|>` pasted by a user) must be counted
/// as ordinary text, not interpreted as a control token.
pub fn count_tokens(text: &str) -> usize {
tiktoken_rs::o200k_base_singleton()
.encode_ordinary(text)
.len()
}
/// Count tokens in a `ChatMessage`'s text content.
///
/// Return: 0 for a message with no `content` (e.g. an assistant message
/// that only carries `tool_calls`).
#[allow(dead_code)]
pub fn count_message_tokens(msg: &ChatMessage) -> usize {
msg.content.as_deref().map_or(0, count_tokens)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::dto::chat::message::ChatMessage;
#[test]
fn empty_string_has_zero_tokens() {
assert_eq!(count_tokens(""), 0);
}
#[test]
fn known_short_phrase_has_expected_token_count() {
// Verified empirically against tiktoken-rs 0.12's o200k_base:
// "hello world" -> [24912, 2375], i.e. 2 tokens.
assert_eq!(count_tokens("hello world"), 2);
}
#[test]
fn known_code_snippet_has_expected_token_count() {
// Verified empirically: 9 tokens under o200k_base.
assert_eq!(count_tokens("fn main() { println!(\"hi\"); }"), 9);
}
#[test]
fn message_with_no_content_counts_zero() {
let msg = ChatMessage::assistant(None);
assert_eq!(count_message_tokens(&msg), 0);
}
#[test]
fn message_token_count_matches_count_tokens_on_its_content() {
let msg = ChatMessage::user("hello world");
assert_eq!(count_message_tokens(&msg), count_tokens("hello world"));
}
}
+88
View File
@@ -0,0 +1,88 @@
#![allow(dead_code)]
//! Single source of truth for resolving the active model's context
//! window size, replacing three copies of the same lookup that had
//! drifted (`Action::Compact`, `spawn_turn`, and `view/status.rs` each
//! had their own inline version — the status bar's copy additionally
//! displayed "?" on no match instead of falling back like the other two,
//! an inconsistency this unifies away).
use crate::model::app_config::AppConfig;
use crate::model::settings::Settings;
/// Resolve the context-window size (in tokens) for the currently
/// configured provider/model.
///
/// Flow: find the `ModelRole` whose `provider`+`model` match
/// `settings` -> use its `context_window` if set -> otherwise fall back
/// to `app_config.default_context_window`.
///
/// Return: always a concrete token count, never "unknown".
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize {
app_config
.model_roles
.values()
.find(|role| role.provider == settings.provider && role.model == settings.model)
.and_then(|role| role.context_window)
.unwrap_or(app_config.default_context_window) as usize
}
#[cfg(test)]
mod tests {
use super::*;
use crate::model::app_config::ModelRole;
#[test]
fn resolves_context_window_from_matching_model_role() {
let mut app_config = AppConfig::default();
app_config.model_roles.insert(
"default".to_string(),
ModelRole {
provider: "zen".to_string(),
model: "deepseek-v4-flash-free".to_string(),
max_tokens: None,
context_window: Some(128_000),
temperature: None,
},
);
let mut settings = Settings::default();
settings.provider = "zen".to_string();
settings.model = "deepseek-v4-flash-free".to_string();
assert_eq!(resolve(&app_config, &settings), 128_000);
}
#[test]
fn falls_back_to_default_context_window_when_no_role_matches() {
let app_config = AppConfig::default();
let mut settings = Settings::default();
settings.provider = "nonexistent".to_string();
settings.model = "nonexistent-model".to_string();
assert_eq!(
resolve(&app_config, &settings),
app_config.default_context_window as usize
);
}
#[test]
fn falls_back_to_default_when_matching_role_has_no_context_window_set() {
let mut app_config = AppConfig::default();
app_config.model_roles.insert(
"default".to_string(),
ModelRole {
provider: "zen".to_string(),
model: "deepseek-v4-flash-free".to_string(),
max_tokens: None,
context_window: None,
temperature: None,
},
);
let mut settings = Settings::default();
settings.provider = "zen".to_string();
settings.model = "deepseek-v4-flash-free".to_string();
assert_eq!(
resolve(&app_config, &settings),
app_config.default_context_window as usize
);
}
}
+1 -1
View File
@@ -2,5 +2,5 @@
//! and the LLM streaming pipeline.
pub mod actions;
pub mod commands;
pub mod shortsend;
pub mod context;
pub mod stream;
-129
View File
@@ -1,129 +0,0 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
//! Short-send / message shaping: compacts long conversation histories so
//! they fit within the provider's context window before being sent to the
//! LLM API.
use crate::dto::chat::message::ChatMessage;
/// Decide whether the message list should be shaped (compacted) before
/// sending to the LLM.
///
/// Flow: trigger based on token estimate. If `token_estimate` exceeds
/// the threshold, we shape. When `prev_shaped` is true, the threshold is
/// raised (95%) to avoid fluttering — compaction only re-triggers when
/// the context is genuinely full again. When `prev_shaped` is false, the
/// threshold is lower (85%) so compaction starts proactively.
///
/// Why: hysteresis prevents repeated compaction on every turn when the
/// token count hovers near the boundary.
///
/// Return: `true` if shaping should be applied.
pub fn should_shape(token_estimate: usize, max_wire_tokens: usize, prev_shaped: bool) -> bool {
let threshold = if prev_shaped {
// Higher threshold when already shaped — defer re-shaping until
// the buffer is genuinely full again (95%).
(max_wire_tokens as f32 * 0.95) as usize
} else {
// Lower threshold when not yet shaped — trigger shaping sooner
// (85%) to avoid hitting the context window limit.
(max_wire_tokens as f32 * 0.85) as usize
};
token_estimate >= threshold
}
/// Compact a long message list by dropping middle messages and inserting
/// a summary placeholder.
///
/// Flow: if the estimated token count is within budget and not forced, return messages
/// unchanged → otherwise keep the system message and the most recent
/// messages (up to `MAX_WIRE_TOKENS / 200` of them) with a `[prior
/// conversation compacted]` system message in between.
///
/// Why: keeps context-size overhead roughly constant regardless of
/// session length.
///
pub fn shape_messages(
messages: &[ChatMessage],
token_count: usize,
max_wire_tokens: usize,
force: bool,
client: Option<&crate::service::provider::LlmClient>,
) -> Vec<ChatMessage> {
if !force && (token_count <= max_wire_tokens || messages.len() < 5) {
return messages.to_vec();
}
let target_tokens = (max_wire_tokens as f32 * 0.70) as usize;
let mut current_tokens = 0;
let mut keep_recent = Vec::new();
let mut dropped_msgs = Vec::new();
// Always keep the very first message (System Prompt) which we don't count here
// as we just blindly preserve it later.
let mut msgs_to_eval = messages.to_vec();
let first = if msgs_to_eval.is_empty() {
None
} else {
Some(msgs_to_eval.remove(0))
};
// Iterate backwards from the most recent to oldest
for m in msgs_to_eval.into_iter().rev() {
let text = m.content.as_deref().unwrap_or("");
// Estimate tokens: ~1 token per 3 bytes for mixed content (code,
// prose, multi-byte). Conservative enough to stay under provider
// limits while avoiding premature compaction.
let msg_tokens = text.len() / 3;
if current_tokens + msg_tokens <= target_tokens {
current_tokens += msg_tokens;
keep_recent.push(m);
} else {
dropped_msgs.push(m); // These will end up in reverse chronological order
}
}
// Reverse dropped_msgs so they are back in chronological order
dropped_msgs.reverse();
let mut result = Vec::new();
if let Some(f) = first {
result.push(f);
}
if !dropped_msgs.is_empty() {
let mut summary_text = "[prior conversation compacted]".to_string();
if let Some(llm) = client {
let prompt = format!(
"Summarize the following dropped conversation history briefly. Focus on main goals, decisions made, and files modified, so the context is preserved for future turns. Keep it concise.\n\nHistory:\n{}",
dropped_msgs.iter()
.map(|m| format!("[{}]: {}", if m.role == crate::dto::chat::message::Role::User { "User" } else { "Assistant" }, m.content.as_deref().unwrap_or("")))
.collect::<Vec<_>>()
.join("\n\n")
);
let req_msgs = vec![ChatMessage::user(prompt)];
match llm.chat_with_tools_non_streaming(&req_msgs, None) {
Ok(resp) => {
if let Some(content) = resp.0.content {
summary_text = format!("[Summary of compacted prior conversation:\n{content}\n]");
}
}
Err(e) => {
tracing::warn!(
"[shortsend] LLM summarization failed: {}. \
Prior conversation history is lost no summary available. \
This means the model will lose context about earlier parts of \
the conversation.",
e,
);
}
}
}
result.push(ChatMessage::system(summary_text));
}
result.extend(keep_recent.into_iter().rev());
result
}
+61 -92
View File
@@ -1,8 +1,6 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
//! SSE stream parser: converts SSE- or JSON-chunked LLM responses into
//! typed `StreamEvent` variants (tokens, reasoning, tool calls, usage, done).
pub mod turn;
pub mod tools;
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -89,7 +87,7 @@ impl SseParser {
/// provider-specific parsing layer.
///
/// Return: 0, 1, or more `StreamEvent`s from the flushed frame.
#[allow(clippy::too_many_lines)]
fn flush_event(&mut self) -> Vec<StreamEvent> {
let data = self.data_lines.join("\n");
self.data_lines.clear();
@@ -112,20 +110,32 @@ impl SseParser {
if let Some(usage) = value.get("usage") {
if !usage.is_null() {
let prompt_tokens = usage.get("prompt_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] prompt_tokens missing in usage chunk");
0
});
let completion_tokens = usage.get("completion_tokens").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] completion_tokens missing in usage chunk");
0
});
let total_tokens = usage.get("total_tokens").and_then(serde_json::Value::as_u64)
let prompt_tokens = usage
.get("prompt_tokens")
.and_then(serde_json::Value::as_u64)
.unwrap_or_else(|| {
tracing::warn!("[stream] prompt_tokens missing in usage chunk");
0
});
let completion_tokens = usage
.get("completion_tokens")
.and_then(serde_json::Value::as_u64)
.unwrap_or_else(|| {
tracing::warn!("[stream] completion_tokens missing in usage chunk");
0
});
let total_tokens = usage
.get("total_tokens")
.and_then(serde_json::Value::as_u64)
.unwrap_or_else(|| {
tracing::warn!("[stream] total_tokens missing in usage chunk");
prompt_tokens + completion_tokens
});
events.push(StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens });
events.push(StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
});
}
}
@@ -143,23 +153,32 @@ impl SseParser {
}
// Reasoning token
if let Some(reasoning) = d.get("reasoning_content").and_then(|r| r.as_str()) {
if let Some(reasoning) =
d.get("reasoning_content").and_then(|r| r.as_str())
{
d_events.push(StreamEvent::Reasoning(reasoning.to_string()));
}
// Tool calls — iterate ALL entries, not just first()
if let Some(tool_calls) = d.get("tool_calls").and_then(|tc| tc.as_array()) {
if let Some(tool_calls) =
d.get("tool_calls").and_then(|tc| tc.as_array())
{
for tc in tool_calls {
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] tool call delta missing index, defaulting to 0");
0
}) as usize;
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string);
let name = tc.get("function")
let id = tc
.get("id")
.and_then(|i| i.as_str())
.map(std::string::ToString::to_string);
let name = tc
.get("function")
.and_then(|f| f.get("name"))
.and_then(|n| n.as_str())
.map(std::string::ToString::to_string);
let args_delta = tc.get("function")
let args_delta = tc
.get("function")
.and_then(|f| f.get("arguments"))
.and_then(|a| a.as_str())
.unwrap_or("")
@@ -174,7 +193,9 @@ impl SseParser {
}
// Finish reason
if let Some(reason) = choice.get("finish_reason").and_then(|r| r.as_str()) {
if let Some(reason) =
choice.get("finish_reason").and_then(|r| r.as_str())
{
if reason == "stop" || reason == "tool_calls" {
d_events.push(StreamEvent::Done);
}
@@ -193,72 +214,6 @@ impl SseParser {
events.append(&mut other_events);
events
}
/// Clears any partially-buffered SSE frame. Reserved for reconnect/retry flows that
/// reuse a parser instance across requests rather than constructing a fresh one.
#[allow(dead_code)]
pub fn reset(&mut self) {
self.buffer.clear();
self.event_type = None;
self.data_lines.clear();
}
}
/// Fallback parser for providers that send bare JSON chunks instead of SSE-framed
/// `data: ...` lines. Not used by the `SseParser` streaming path (which handles
/// standard SSE framing directly), kept for providers/tests that feed raw chunks.
///
/// Flow: parse `data` as JSON → extract first `choices[0].delta` →
/// return a `Token`, `Reasoning`, `Done`, or `ToolCallDelta` event based
/// on the fields present.
///
/// Return: `Some(StreamEvent)` if the chunk contained recognisable
/// content, `None` otherwise.
#[allow(dead_code)]
pub fn parse_stream_chunk(data: &str) -> Option<StreamEvent> {
let value: Value = serde_json::from_str(data).ok()?;
if value == Value::Null {
return None;
}
let choices = value.get("choices")?.as_array()?;
let choice = choices.first()?;
let delta = choice.get("delta")?;
if let Some(content) = delta.get("content").and_then(|c| c.as_str()) {
return Some(StreamEvent::Token(content.to_string()));
}
if let Some(reasoning) = delta.get("reasoning_content").and_then(|r| r.as_str()) {
return Some(StreamEvent::Reasoning(reasoning.to_string()));
}
if let Some(finish) = choice.get("finish_reason").and_then(|r| r.as_str()) {
if finish == "stop" || finish == "tool_calls" {
return Some(StreamEvent::Done);
}
}
if let Some(tool_calls) = delta.get("tool_calls").and_then(|tc| tc.as_array()) {
if let Some(tc) = tool_calls.first() {
let index = tc.get("index").and_then(serde_json::Value::as_u64).unwrap_or_else(|| {
tracing::warn!("[stream] fallback parser: tool call missing index, defaulting to 0");
0
}) as usize;
let id = tc.get("id").and_then(|i| i.as_str()).map(std::string::ToString::to_string);
let name = tc.get("function")
.and_then(|f| f.get("name"))
.and_then(|n| n.as_str())
.map(std::string::ToString::to_string);
let args = tc.get("function")
.and_then(|f| f.get("arguments"))
.and_then(|a| a.as_str())
.unwrap_or("")
.to_string();
return Some(StreamEvent::ToolCallDelta {
index,
id,
name,
arguments_delta: args,
});
}
}
None
}
#[cfg(test)]
@@ -280,7 +235,10 @@ mod tests {
fn feed_handles_chunk_split_mid_line() {
let mut p = SseParser::new();
let e1 = p.feed("data: {\"choices\":[{\"delta\":{\"content\":\"partial");
assert!(e1.is_empty(), "no event until the line and blank separator complete");
assert!(
e1.is_empty(),
"no event until the line and blank separator complete"
);
let e2 = p.feed("\"}}]}\n\n");
assert_eq!(e2.len(), 1);
match &e2[0] {
@@ -300,9 +258,7 @@ mod tests {
#[test]
fn feed_emits_done_on_finish_reason_stop() {
let mut p = SseParser::new();
let events = p.feed(
"data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n",
);
let events = p.feed("data: {\"choices\":[{\"delta\":{},\"finish_reason\":\"stop\"}]}\n\n");
assert_eq!(events.len(), 1);
assert!(matches!(events[0], StreamEvent::Done));
}
@@ -315,7 +271,12 @@ mod tests {
);
assert_eq!(events.len(), 1);
match &events[0] {
StreamEvent::ToolCallDelta { index, id, name, arguments_delta } => {
StreamEvent::ToolCallDelta {
index,
id,
name,
arguments_delta,
} => {
assert_eq!(*index, 0);
assert_eq!(id.as_deref(), Some("call_1"));
assert_eq!(name.as_deref(), Some("bash"));
@@ -333,7 +294,11 @@ mod tests {
);
assert_eq!(events.len(), 1);
match &events[0] {
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens } => {
StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
} => {
assert_eq!(*prompt_tokens, 10);
assert_eq!(*completion_tokens, 5);
assert_eq!(*total_tokens, 15);
@@ -351,7 +316,11 @@ mod tests {
assert_eq!(events.len(), 2);
match (&events[0], &events[1]) {
(
StreamEvent::Usage { prompt_tokens, completion_tokens, total_tokens },
StreamEvent::Usage {
prompt_tokens,
completion_tokens,
total_tokens,
},
StreamEvent::Token(t),
) => {
assert_eq!(*prompt_tokens, 10);
-101
View File
@@ -1,101 +0,0 @@
//! Standalone accumulator for streamed tool-call deltas.
//!
//! Flow: `ToolCallAccumulator::add_delta` is fed incremental `(index, id,
//! name, arguments_delta)` chunks as they arrive over SSE → grows its
//! internal `Vec<ParsedToolCall>` as needed → `is_complete` reports once
//! every accumulated call has both a name and arguments.
//!
//! Why: mirrors the accumulation logic built into `StreamedTurn::apply_event`
//! but as an independent, reusable type for callers that want to track
//! tool-call deltas without a full `StreamedTurn` (e.g. a lighter-weight
//! preview). Currently unused (`#[allow(dead_code)]`), kept for that future
//! use case.
use super::turn::ParsedToolCall;
use serde_json::{json, Value};
/// Standalone tool-call delta accumulator, functionally equivalent to the accumulation
/// logic built into `StreamedTurn::apply_event`. Reserved for callers that want to track
/// tool-call deltas independently of a full `StreamedTurn` (e.g. a lighter-weight preview).
#[allow(dead_code)]
pub struct ToolCallAccumulator {
calls: Vec<ParsedToolCall>,
}
#[allow(dead_code)]
impl ToolCallAccumulator {
/// Construct an empty accumulator with no tool calls tracked yet.
///
/// Return: a fresh `ToolCallAccumulator`.
pub fn new() -> Self {
ToolCallAccumulator { calls: Vec::new() }
}
/// Append a delta to the tool call at the given index, growing the
/// calls vector if needed.
pub fn add_delta(
&mut self,
index: usize,
id: Option<&str>,
name: Option<&str>,
arguments_delta: &str,
) {
while self.calls.len() <= index {
self.calls.push(ParsedToolCall {
id: String::new(),
name: String::new(),
arguments: String::new(),
is_complete: false,
});
}
let tc = &mut self.calls[index];
if let Some(new_id) = id {
if !new_id.is_empty() {
tc.id = new_id.to_string();
}
}
if let Some(new_name) = name {
if !new_name.is_empty() {
tc.name = new_name.to_string();
}
}
tc.arguments.push_str(arguments_delta);
}
/// Borrow the accumulated tool calls.
pub fn calls(&self) -> &[ParsedToolCall] {
&self.calls
}
/// Return true once all tool calls have both a name and arguments.
pub fn is_complete(&self) -> bool {
!self.calls.is_empty() && self.calls.iter().all(|tc| !tc.name.is_empty() && !tc.arguments.is_empty())
}
/// Clear all accumulated calls (starting a fresh turn).
pub fn reset(&mut self) {
self.calls.clear();
}
/// Build a JSON-serialisable `Vec<Value>` of pending (non-empty-name)
/// tool calls, suitable for downstream inspection or replay.
pub fn pending_args(&self) -> Vec<Value> {
self.calls
.iter()
.filter(|tc| !tc.name.is_empty())
.map(|tc| {
json!({
"tool_call_id": tc.id,
"name": tc.name,
"arguments": tc.arguments,
})
})
.collect()
}
}
impl Default for ToolCallAccumulator {
fn default() -> Self {
Self::new()
}
}
+31 -35
View File
@@ -101,17 +101,7 @@ pub struct ParsedToolCall {
pub is_complete: bool,
}
impl ParsedToolCall {
/// Attempt to parse the accumulated argument string as JSON before
/// the tool call is marked complete — useful for a speculative preview.
///
/// Return: `Some(Value)` if the arguments are parsable JSON, `None`
/// if still partial.
#[allow(dead_code)]
pub fn try_parse(&self) -> Option<Value> {
serde_json::from_str(&self.arguments).ok()
}
}
impl ParsedToolCall {}
impl StreamedTurn {
/// Create an empty turn accumulator.
@@ -186,12 +176,12 @@ impl StreamedTurn {
let mut msg = if self.tool_calls.is_empty() {
ChatMessage::assistant(None)
} else {
let tool_dtos: Vec<ToolCall> = self.tool_calls
let tool_dtos: Vec<ToolCall> = self
.tool_calls
.iter()
.filter(|tc| !tc.name.is_empty())
.map(|tc| {
let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments)
{
let args_value: serde_json::Value = match serde_json::from_str(&tc.arguments) {
Ok(v) => v,
Err(e) => {
let repaired = repair_incomplete_json(&tc.arguments);
@@ -200,7 +190,8 @@ impl StreamedTurn {
tracing::warn!(
"[stream] tool call '{}' had truncated JSON \
arguments repaired successfully: {}",
tc.name, e,
tc.name,
e,
);
v
}
@@ -209,7 +200,9 @@ impl StreamedTurn {
"[stream] tool call '{}' has invalid JSON \
arguments: {} (after repair: {}) falling \
back to raw string",
tc.name, e, e2,
tc.name,
e,
e2,
);
serde_json::Value::String(tc.arguments.clone())
}
@@ -235,7 +228,10 @@ impl StreamedTurn {
let full_content = if self.accumulated_reasoning.is_empty() {
self.accumulated_content.clone()
} else {
format!("<think>\n{}\n</think>\n\n{}", self.accumulated_reasoning, self.accumulated_content)
format!(
"<think>\n{}\n</think>\n\n{}",
self.accumulated_reasoning, self.accumulated_content
)
};
let content = if full_content.is_empty() {
None
@@ -259,7 +255,8 @@ impl StreamedTurn {
/// Return: `Some((name, parse_error))` for the first bad tool call, or
/// `None` if every tool call's arguments are complete, parsable JSON.
pub fn incomplete_tool_call(&self) -> Option<(&str, String)> {
self.tool_calls.iter()
self.tool_calls
.iter()
.filter(|tc| !tc.name.is_empty())
.find_map(|tc| {
serde_json::from_str::<Value>(&tc.arguments)
@@ -267,19 +264,6 @@ impl StreamedTurn {
.map(|e| (tc.name.as_str(), e.to_string()))
})
}
/// Reserved accessor for callers that want to branch mid-stream before the turn
/// completes; the current wiring only inspects the final `build_assistant_message()`.
#[allow(dead_code)]
pub fn has_tool_calls(&self) -> bool {
self.tool_calls.iter().any(|tc| !tc.name.is_empty())
}
/// Reserved accessor mirroring `has_tool_calls` for mid-stream content peeks.
#[allow(dead_code)]
pub fn content(&self) -> &str {
&self.accumulated_content
}
}
impl Default for StreamedTurn {
@@ -353,7 +337,10 @@ mod tests {
let tcs = msg.tool_calls.expect("should produce tool calls");
assert_eq!(tcs.len(), 1);
let args = &tcs[0].function.arguments;
assert!(args.is_object(), "args should be an object after repair: {args:?}");
assert!(
args.is_object(),
"args should be an object after repair: {args:?}"
);
assert_eq!(args.get("path").and_then(|v| v.as_str()), Some("a.txt"));
assert_eq!(args.get("content").and_then(|v| v.as_str()), Some("short"));
}
@@ -361,7 +348,10 @@ mod tests {
#[test]
fn incomplete_tool_call_flags_truncated_json() {
let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm"));
turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"unterm",
));
let bad = turn.incomplete_tool_call();
assert_eq!(bad.map(|(name, _)| name), Some("write"));
}
@@ -369,7 +359,10 @@ mod tests {
#[test]
fn incomplete_tool_call_accepts_complete_json() {
let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"done\"}"));
turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"done\"}",
));
assert!(turn.incomplete_tool_call().is_none());
}
@@ -386,7 +379,10 @@ mod tests {
// so it should still flag truncated JSON even though
// `build_assistant_message` will later repair it.
let mut turn = StreamedTurn::new();
turn.tool_calls.push(tool_call("write", "{\"path\": \"a.txt\", \"content\": \"unterm"));
turn.tool_calls.push(tool_call(
"write",
"{\"path\": \"a.txt\", \"content\": \"unterm",
));
// Even though it's repairable, raw parse should still fail
assert!(serde_json::from_str::<Value>(&turn.tool_calls[0].arguments).is_err());
}
+212 -9
View File
@@ -27,6 +27,51 @@ impl DirCache {
}
}
/// A shared, whole-workspace file-path index used for `@file` mention
/// autocomplete. Built once by a background thread at startup (see
/// `AppStateRest::new`) and incrementally appended to when tools create
/// new files (see `tool/fs/write.rs`).
#[derive(Clone)]
pub struct MentionIndex {
entries: Arc<std::sync::RwLock<Vec<String>>>,
}
impl MentionIndex {
/// Create an empty `MentionIndex`.
pub fn new() -> Self {
MentionIndex {
entries: Arc::new(std::sync::RwLock::new(Vec::new())),
}
}
/// Replace the indexed paths (used by the startup background walk).
pub fn set(&self, paths: Vec<String>) {
if let Ok(mut w) = self.entries.write() {
*w = paths;
}
}
/// Append a single newly created file's path (used by the `write` tool).
pub fn push(&self, path: String) {
if let Ok(mut w) = self.entries.write() {
w.push(path);
}
}
/// Take a snapshot of the current indexed paths for fuzzy matching.
pub fn snapshot(&self) -> Vec<String> {
self.entries.read().map(|r| r.clone()).unwrap_or_default()
}
}
/// Which source populated the autocomplete dropdown, since selecting a
/// candidate is spliced into the buffer differently for each.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AutocompleteKind {
Command,
FileMention,
}
/// Manages the viewport scroll offset.
#[derive(Debug, Clone)]
pub struct ScrollState {
@@ -72,6 +117,8 @@ pub struct InputState {
pub autocomplete_candidates: Vec<String>,
pub autocomplete_idx: usize,
pub autocomplete_visible: bool,
pub autocomplete_kind: AutocompleteKind,
pub mention_start: usize,
pub history_file: Option<PathBuf>,
}
@@ -106,6 +153,8 @@ impl InputState {
autocomplete_candidates: Vec::new(),
autocomplete_idx: 0,
autocomplete_visible: false,
autocomplete_kind: AutocompleteKind::Command,
mention_start: 0,
history_file: None,
}
}
@@ -116,6 +165,8 @@ impl InputState {
self.autocomplete_candidates.clear();
self.autocomplete_prefix.clear();
self.autocomplete_idx = 0;
self.autocomplete_kind = AutocompleteKind::Command;
self.mention_start = 0;
}
/// Open or refresh the autocomplete dropdown by filtering `COMMANDS`
@@ -138,6 +189,54 @@ impl InputState {
.map(std::string::ToString::to_string)
.collect();
self.autocomplete_prefix = prefix;
self.autocomplete_kind = AutocompleteKind::Command;
self.autocomplete_idx = 0;
self.autocomplete_visible = !self.autocomplete_candidates.is_empty();
}
/// Find the `@mention` token (if any) immediately before the cursor.
///
/// Flow: find the nearest `@` before the cursor → if there's whitespace
/// between that `@` and the cursor, no trigger → the `@` only counts as
/// a trigger if it's at buffer start or immediately preceded by
/// whitespace (so `foo@bar` mid-word never triggers).
///
/// Return: `Some((byte offset of '@', query text between '@' and cursor))`
/// or `None` if the cursor isn't inside a mention token.
pub fn mention_query_at_cursor(&self) -> Option<(usize, String)> {
let before_cursor = &self.buffer[..self.cursor];
let at_pos = before_cursor.rfind('@')?;
let between = &before_cursor[at_pos + 1..];
if between.chars().any(char::is_whitespace) {
return None;
}
let boundary_ok = at_pos == 0
|| before_cursor[..at_pos].chars().next_back().is_some_and(char::is_whitespace);
if !boundary_ok {
return None;
}
Some((at_pos, between.to_string()))
}
/// Open or refresh the `@file` mention dropdown from `files`, fuzzy-matched
/// against the mention query at the cursor.
///
/// Flow: `mention_query_at_cursor` finds the trigger `@` and query text →
/// if none, close and return → otherwise fuzzy-match `query` against
/// `files` via `nucleo-matcher`, keep the top 10 by score.
pub fn open_mention_autocomplete(&mut self, files: &[String]) {
use nucleo_matcher::{Config, Matcher};
use nucleo_matcher::pattern::{CaseMatching, Normalization, Pattern};
let Some((start, query)) = self.mention_query_at_cursor() else {
self.close_autocomplete();
return;
};
let mut matcher = Matcher::new(Config::DEFAULT.match_paths());
let pattern = Pattern::parse(&query, CaseMatching::Smart, Normalization::Smart);
let matched_files = pattern.match_list(files.iter(), &mut matcher);
self.autocomplete_candidates = matched_files.into_iter().take(10).map(|(f, _)| f.clone()).collect();
self.autocomplete_kind = AutocompleteKind::FileMention;
self.mention_start = start;
self.autocomplete_idx = 0;
self.autocomplete_visible = !self.autocomplete_candidates.is_empty();
}
@@ -154,19 +253,41 @@ impl InputState {
}
}
/// Accept the currently selected autocomplete candidate, placing it
/// in the buffer and closing the dropdown.
/// Accept the currently selected autocomplete candidate.
///
/// `Command` candidates replace the whole buffer; `FileMention`
/// candidates splice `@path ` in at the mention's start position so the
/// rest of the sentence around it is preserved.
///
/// Return: `true` if a candidate was selected, `false` if none existed.
pub fn select_autocomplete(&mut self) -> bool {
if let Some(candidate) = self.autocomplete_candidates.get(self.autocomplete_idx) {
self.buffer = candidate.clone();
self.cursor = self.buffer.len();
self.close_autocomplete();
true
} else {
false
let Some(candidate) = self.autocomplete_candidates.get(self.autocomplete_idx).cloned() else {
return false;
};
match self.autocomplete_kind {
AutocompleteKind::Command => {
self.buffer = candidate;
self.cursor = self.buffer.len();
}
AutocompleteKind::FileMention => {
// Cursor movement (Left/Right) does not close the dropdown, so
// by the time Enter is pressed `mention_start` may no longer
// describe a valid range against the current cursor/buffer
// (e.g. the cursor moved left past the '@'). Splicing on a
// stale range would panic (`start > end`) or, even when it
// doesn't panic, produce a nonsensical replacement. Treat a
// stale mention context the same as "nothing selected".
if self.cursor < self.mention_start || self.mention_start > self.buffer.len() {
self.close_autocomplete();
return false;
}
let replacement = format!("@{candidate} ");
self.buffer.replace_range(self.mention_start..self.cursor, &replacement);
self.cursor = self.mention_start + replacement.len();
}
}
self.close_autocomplete();
true
}
/// Legacy inline tab-complete — opens the dropdown on first Tab press,
@@ -290,6 +411,7 @@ pub struct MiscState {
pub tick_count: u64,
pub todo_content: String,
pub lesson_running: bool,
pub pending_clipboard_copy: Option<String>,
}
impl MiscState {
@@ -309,6 +431,7 @@ impl MiscState {
tick_count: 0,
todo_content: String::new(),
lesson_running: false,
pending_clipboard_copy: None,
}
}
@@ -325,3 +448,83 @@ impl MiscState {
expired
}
}
#[cfg(test)]
mod tests {
use super::*;
fn input_with(buffer: &str, cursor: usize) -> InputState {
let mut input = InputState::new();
input.buffer = buffer.to_string();
input.cursor = cursor;
input
}
#[test]
fn mention_at_buffer_start_triggers() {
let input = input_with("@mai", 4);
assert_eq!(input.mention_query_at_cursor(), Some((0, "mai".to_string())));
}
#[test]
fn mention_after_space_mid_sentence_triggers() {
let input = input_with("look at @read", 13);
assert_eq!(input.mention_query_at_cursor(), Some((8, "read".to_string())));
}
#[test]
fn mid_word_at_does_not_trigger() {
let input = input_with("foo@bar", 7);
assert_eq!(input.mention_query_at_cursor(), None);
}
#[test]
fn whitespace_between_at_and_cursor_does_not_trigger() {
let input = input_with("@foo bar", 8);
assert_eq!(input.mention_query_at_cursor(), None);
}
#[test]
fn select_file_mention_splices_into_buffer() {
let mut input = input_with("look at @rea and fix it", 12);
input.autocomplete_candidates = vec!["src/main.rs".to_string()];
input.autocomplete_idx = 0;
input.autocomplete_kind = AutocompleteKind::FileMention;
input.mention_start = 8;
assert!(input.select_autocomplete());
assert_eq!(input.buffer, "look at @src/main.rs and fix it");
assert_eq!(input.cursor, 8 + "@src/main.rs ".len());
}
#[test]
fn select_file_mention_with_stale_cursor_before_mention_start_does_not_panic() {
// Simulates: user typed "foo @rea" (mention_start = 4, cursor = 8,
// dropdown open), then pressed Left 5 times without closing the
// dropdown, moving the cursor to byte 3 (before the '@'). Selecting
// now must not panic on `replace_range(4..3, ...)`.
let mut input = input_with("foo @rea", 3);
input.autocomplete_candidates = vec!["src/main.rs".to_string()];
input.autocomplete_idx = 0;
input.autocomplete_kind = AutocompleteKind::FileMention;
input.mention_start = 4;
assert!(!input.select_autocomplete());
assert!(!input.autocomplete_visible);
}
#[test]
fn select_command_still_replaces_whole_buffer() {
let mut input = input_with("/mo", 3);
input.autocomplete_candidates = vec!["/model".to_string()];
input.autocomplete_idx = 0;
input.autocomplete_kind = AutocompleteKind::Command;
assert!(input.select_autocomplete());
assert_eq!(input.buffer, "/model");
assert_eq!(input.cursor, "/model".len());
}
#[test]
fn misc_state_starts_with_no_pending_clipboard_copy() {
let misc = MiscState::new();
assert!(misc.pending_clipboard_copy.is_none());
}
}
+52 -2
View File
@@ -9,7 +9,7 @@ use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use tokio::sync::RwLock;
use super::misc::{DirCache, InputState, MiscState, ScrollState};
use super::misc::{DirCache, InputState, MentionIndex, MiscState, ScrollState};
use super::runtime::{SessionRuntime, TurnEvent};
use super::types::{Origin, Toast, TranscriptCache};
use crate::app::lsp::LspManager;
@@ -54,6 +54,7 @@ pub struct AppStateRest {
pub memory_dir: PathBuf,
pub worktrees_dir: PathBuf,
pub dir_cache: Arc<RwLock<DirCache>>,
pub mention_index: MentionIndex,
pub edit_log: EditLog,
pub session_runtime: Option<SessionRuntime>,
pub sessions: Vec<crate::model::session::Session>,
@@ -110,6 +111,7 @@ impl AppStateRest {
turn_in_flight: Arc::new(Mutex::new(false)),
abort_flag: Arc::new(std::sync::atomic::AtomicBool::new(false)),
dir_cache: Arc::new(RwLock::new(dir_cache)),
mention_index: MentionIndex::new(),
edit_log: EditLog::new(session_dir),
session_runtime: Some(SessionRuntime::new(session_dir.to_path_buf())),
workflow_engine: WorkflowEngine::new(),
@@ -165,7 +167,7 @@ impl AppStateRest {
// async executor entirely. It is deliberately not joined -- startup
// must not block on language server installation, and failures are
// logged rather than surfaced, since editing still works without LSP.
if state.settings.lsp_auto_provision {
if state.settings.flags.lsp_auto_provision {
let lsp_mgr = state.lsp_manager.clone();
let msg_queue = state.lsp_provision_msgs.clone();
std::thread::spawn(move || {
@@ -207,6 +209,53 @@ impl AppStateRest {
state
}
/// Spawn the background thread that walks every workspace root and
/// populates `mention_index` for `@file` mention autocomplete.
///
/// Why a separate method, not called from `new()`: the attach-only
/// TUI client also constructs an `AppStateRest` (for local rendering
/// state) but never runs tools or `handle_key` locally — it forwards
/// keystrokes to the daemon over IPC, which has its own `AppStateRest`
/// with its own index. Spawning this walk in the attach client would
/// waste a full workspace scan for an index nothing there consumes.
/// Callers that DO need the index (single-process mode, the daemon)
/// call this explicitly after construction.
///
/// Flow: spawn OS thread -> `ignore::Walk` each workspace root,
/// collecting file paths (workspace-index-prefixed for roots beyond
/// the first, matching `resolve_path`'s `[N]path` convention) -> stop
/// once 50,000 entries are collected -> store the result in
/// `mention_index`.
///
/// Why a raw thread and not a background tokio task: there is no
/// persistent async runtime driving the render loop, and this is
/// blocking filesystem I/O -- a dedicated thread keeps startup
/// non-blocking. Not joined, same rationale as the LSP provisioning
/// thread above: a slow/huge repo must not delay the TUI appearing.
pub fn spawn_mention_index_build(&self) {
let mention_index = self.mention_index.clone();
let roots = self.workspace_roots.clone();
std::thread::spawn(move || {
const MAX_MENTION_ENTRIES: usize = 50_000;
let mut paths = Vec::new();
'roots: for (i, root) in roots.iter().enumerate() {
for entry in ignore::Walk::new(root).flatten() {
if !entry.path().is_file() {
continue;
}
let rel = entry.path().strip_prefix(root).unwrap_or(entry.path());
let rel_str = rel.display().to_string();
let formatted = if i == 0 { rel_str } else { format!("[{i}]{rel_str}") };
paths.push(formatted);
if paths.len() >= MAX_MENTION_ENTRIES {
break 'roots;
}
}
}
mention_index.set(paths);
});
}
/// Whether an agent turn is currently running.
///
/// Return: `false` (and logs a warning) if the mutex is poisoned, rather
@@ -278,6 +327,7 @@ impl AppStateRest {
memory_dir: self.memory_dir.clone(),
worktrees_dir: self.worktrees_dir.clone(),
dir_cache: self.dir_cache.clone(),
mention_index: self.mention_index.clone(),
origin,
graduated_checks: Vec::new(),
lsp_manager: self.lsp_manager.clone(),
+1 -2
View File
@@ -1,9 +1,8 @@
//! Per-session runtime state: message history, pending tool queue,
//! background bash jobs, lesson/review counters, and the `TurnEvent`
//! stream emitted while an agent turn is in flight.
use std::path::PathBuf;
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
/// Cumulative token/latency counters for a session, persisted alongside it.
#[derive(Debug, Clone, Copy, Serialize, Deserialize, Default)]
-1
View File
@@ -1,6 +1,5 @@
//! Opaque, serializable snapshot of application state used for
//! attach/daemon IPC transfer.
use serde::{Deserialize, Serialize};
/// A JSON-boxed snapshot of app state, opaque to the transport layer.
+6 -3
View File
@@ -1,10 +1,13 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
#![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Shared small state types: toasts, overlays, the transcript cache,
//! tool execution model, and call origin tags.
use serde::{Deserialize, Serialize};
/// Severity/category of a toast notification, used to pick its color.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum ToastKind {
+72 -40
View File
@@ -15,27 +15,30 @@
//! wrote this file, let me check if it's correct before continuing").
//! - Background reviews catch broader concerns (missing tests, architectural
//! drift, security issues) without blocking the main agent's flow.
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicBool, Ordering};
use std::collections::VecDeque;
use crate::app::state::runtime::TurnEvent;
use crate::app::subagent::context::build_subagent_context;
use crate::app::subagent::engine::run_subagent;
use crate::app::subagent::spawn::AgentDefinition;
use crate::app::subagent::event::SubagentEvent;
use crate::app::subagent::spawn::AgentDefinition;
use std::collections::VecDeque;
use std::path::Path;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
/// File extensions that should not trigger auto-review (config, lock, data).
const SKIP_REVIEW_EXTENSIONS: &[&str] = &[
".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml",
".svg", ".png", ".jpg", ".ico", ".woff", ".woff2",
".lock", ".md", ".txt", ".json", ".toml", ".yaml", ".yml", ".svg", ".png", ".jpg", ".ico",
".woff", ".woff2",
];
/// File names that should not trigger auto-review.
const SKIP_REVIEW_FILES: &[&str] = &[
"Cargo.lock", "yarn.lock", "package-lock.json",
".gitignore", ".env", ".env.example",
"Cargo.lock",
"yarn.lock",
"package-lock.json",
".gitignore",
".env",
".env.example",
];
/// Prevents a second background subagent of the same kind from spawning
@@ -137,8 +140,19 @@ fn is_production_code(path: &str) -> bool {
.is_some_and(|ext| {
matches!(
ext,
"rs" | "ts" | "tsx" | "js" | "jsx" | "go" | "py" | "java" | "kt" | "swift"
| "c" | "cpp" | "h" | "hpp"
"rs" | "ts"
| "tsx"
| "js"
| "jsx"
| "go"
| "py"
| "java"
| "kt"
| "swift"
| "c"
| "cpp"
| "h"
| "hpp"
)
})
}
@@ -167,11 +181,8 @@ pub fn spawn_quick_review(
file_path,
);
let def = AgentDefinition::new(
"quick-reviewer".to_string(),
"reviewer".to_string(),
)
.with_system_prompt(prompt);
let def = AgentDefinition::new("quick-reviewer".to_string(), "reviewer".to_string())
.with_system_prompt(prompt);
let mut ctx = build_subagent_context(&def);
ctx.session_dir = session_dir.to_path_buf();
@@ -187,7 +198,7 @@ pub fn spawn_quick_review(
SubagentEvent::ToolResult { tool, .. } => {
tracing::debug!("[auto-review] tool result: {}", tool);
}
SubagentEvent::Completed { .. } => {
SubagentEvent::Completed => {
tracing::debug!("[auto-review] completed");
}
_ => {}
@@ -277,7 +288,10 @@ pub fn spawn_background_test_gen(
if file_paths.is_empty() {
return;
}
if TEST_GEN_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
if TEST_GEN_RUNNING
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!("[bg-test-gen] skipped — a test-gen run is already in flight");
return;
}
@@ -306,8 +320,7 @@ pub fn spawn_background_test_gen(
"test-generator".to_string(),
"coder".to_string(), // needs write access
)
.with_system_prompt(prompt)
;
.with_system_prompt(prompt);
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-test-gen", Some(&abort_flag));
let message = match &result {
@@ -347,7 +360,10 @@ pub fn spawn_background_arch_review(
if file_paths.is_empty() {
return;
}
if ARCH_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
if ARCH_REVIEW_RUNNING
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!("[bg-arch-review] skipped — an arch-review run is already in flight");
return;
}
@@ -366,12 +382,8 @@ pub fn spawn_background_arch_review(
file_list,
);
let def = AgentDefinition::new(
"arch-reviewer".to_string(),
"reviewer".to_string(),
)
.with_system_prompt(prompt)
;
let def = AgentDefinition::new("arch-reviewer".to_string(), "reviewer".to_string())
.with_system_prompt(prompt);
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-arch-review", Some(&abort_flag));
let message = match &result {
@@ -422,8 +434,13 @@ pub fn spawn_background_security_review(
if prod_paths.is_empty() {
return;
}
if SECURITY_REVIEW_RUNNING.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst).is_err() {
tracing::debug!("[bg-security-review] skipped — a security-review run is already in flight");
if SECURITY_REVIEW_RUNNING
.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
.is_err()
{
tracing::debug!(
"[bg-security-review] skipped — a security-review run is already in flight"
);
return;
}
@@ -441,14 +458,11 @@ pub fn spawn_background_security_review(
file_list,
);
let def = AgentDefinition::new(
"security-reviewer".to_string(),
"reviewer".to_string(),
)
.with_system_prompt(prompt)
;
let def = AgentDefinition::new("security-reviewer".to_string(), "reviewer".to_string())
.with_system_prompt(prompt);
let result = run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag));
let result =
run_subagent_with_retry(&def, &sd, &ws, "bg-security-review", Some(&abort_flag));
let message = match &result {
Ok(output) => {
let first = output.lines().next().unwrap_or(output);
@@ -493,7 +507,13 @@ pub fn spawn_all_background(
.filter(|p| is_production_code(p))
.cloned()
.collect();
spawn_background_test_gen(&source_paths, session_dir, workspaces, turn_events, abort_flag.clone());
spawn_background_test_gen(
&source_paths,
session_dir,
workspaces,
turn_events,
abort_flag.clone(),
);
// Background arch review: for all files that are reviewable
let reviewable: Vec<String> = file_paths
@@ -501,10 +521,22 @@ pub fn spawn_all_background(
.filter(|p| is_reviewable_path(p))
.cloned()
.collect();
spawn_background_arch_review(&reviewable, session_dir, workspaces, turn_events, abort_flag.clone());
spawn_background_arch_review(
&reviewable,
session_dir,
workspaces,
turn_events,
abort_flag.clone(),
);
// Background security review: only production source files
spawn_background_security_review(&source_paths, session_dir, workspaces, turn_events, abort_flag);
spawn_background_security_review(
&source_paths,
session_dir,
workspaces,
turn_events,
abort_flag,
);
}
#[cfg(test)]
+6 -4
View File
@@ -1,9 +1,8 @@
//! Construction of a `SubagentContext` from an `AgentDefinition`,
//! including the default read-only tool set for reviewer agents.
use std::path::PathBuf;
use std::sync::{Arc, Mutex, atomic::AtomicBool};
use super::spawn::AgentDefinition;
use std::path::PathBuf;
use std::sync::{atomic::AtomicBool, Arc, Mutex};
/// Default read-only tool names granted to `role == "reviewer"` agents.
pub const REVIEWER_ALLOWED: &[&str] = &["read", "grep", "glob", "recall", "remember"];
@@ -40,7 +39,10 @@ pub struct SubagentContext {
pub fn build_subagent_context(def: &AgentDefinition) -> SubagentContext {
let allowed_tools = def.allowed_tools.clone().unwrap_or_else(|| {
if def.role == "reviewer" {
REVIEWER_ALLOWED.iter().map(std::string::ToString::to_string).collect()
REVIEWER_ALLOWED
.iter()
.map(std::string::ToString::to_string)
.collect()
} else {
Vec::new()
}
+64 -15
View File
@@ -18,25 +18,68 @@ pub mod tool_scope {
/// Write-tier plus delete, git, and the remaining LSP actions.
pub const FULL: &str = "full";
const READ_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
"lsp_references", "read_findings",
/// The read-only tool set — reused by `context::dedup` as the
/// authoritative "safe to deduplicate" classification, so there's a
/// single list of read-only tool names in the codebase instead of two.
pub const READ_TOOLS: &[&str] = &[
"read",
"grep",
"glob",
"search",
"seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
];
const WRITE_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
"lsp_references", "read_findings",
"write", "edit", "bash", "todowrite", "todofinish", "remember",
"read",
"grep",
"glob",
"search",
"seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
"write",
"edit",
"bash",
"todowrite",
"todofinish",
"remember",
];
const FULL_TOOLS: &[&str] = &[
"read", "grep", "glob", "search", "seqthink", "recall",
"lsp_connect", "lsp_diagnostics", "lsp_hover", "lsp_definition",
"lsp_references", "read_findings",
"write", "edit", "bash", "todowrite", "todofinish", "remember",
"delete", "git_operator", "lsp_completion", "lsp_disconnect",
"read",
"grep",
"glob",
"search",
"seqthink",
"recall",
"lsp_connect",
"lsp_diagnostics",
"lsp_hover",
"lsp_definition",
"lsp_references",
"read_findings",
"write",
"edit",
"bash",
"todowrite",
"todofinish",
"remember",
"delete",
"git_operator",
"lsp_completion",
"lsp_disconnect",
];
/// Resolve a tier name to its concrete tool allowlist.
@@ -95,7 +138,13 @@ mod tests {
let read: HashSet<_> = tools_for(READ).into_iter().collect();
let write: HashSet<_> = tools_for(WRITE).into_iter().collect();
let full: HashSet<_> = tools_for(FULL).into_iter().collect();
assert!(read.is_subset(&write), "read tier must be a subset of write tier");
assert!(write.is_subset(&full), "write tier must be a subset of full tier");
assert!(
read.is_subset(&write),
"read tier must be a subset of write tier"
);
assert!(
write.is_subset(&full),
"write tier must be a subset of full tier"
);
}
}
+2 -6
View File
@@ -475,7 +475,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
// drain thread can show it as progress instead of just the tool name.
if !content.is_empty() {
let _ = tx.blocking_send(SubagentEvent::StepCompleted {
step,
output: content.clone(),
});
}
@@ -612,7 +611,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
let _ = tx.blocking_send(SubagentEvent::ToolResult {
tool: tool_name.clone(),
args: args.clone(),
output: output_text.clone(),
});
let is_readonly = tool_name == "read"
@@ -632,7 +630,7 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
shared_text.truncate(50_000);
shared_text.push_str("\n...[truncated]");
}
f.push(format!("[Auto-Shared] Sibling drone executed '{}' with args {}:\n{}", tool_name, args_json, shared_text));
f.push(format!("[Auto-Shared] Sibling drone executed '{tool_name}' with args {args_json}:\n{shared_text}"));
}
}
}
@@ -651,7 +649,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
let _ = tx.blocking_send(SubagentEvent::ToolResult {
tool: tool_name.clone(),
args: args.clone(),
output: msg,
});
}
}
@@ -663,7 +660,6 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
output.push('\n');
}
let _ = tx.blocking_send(SubagentEvent::StepCompleted {
step,
output: content.clone(),
});
// Break only when we got real content; empty means something went wrong
@@ -673,7 +669,7 @@ pub fn run_subagent(ctx: &SubagentContext, tx: &mpsc::Sender<SubagentEvent>) ->
}
}
let _ = tx.blocking_send(SubagentEvent::Completed { output: output.clone() });
let _ = tx.blocking_send(SubagentEvent::Completed);
Ok(output)
}
+1 -11
View File
@@ -1,6 +1,5 @@
//! Event variants that a running subagent can emit to its parent via the
//! shared mpsc channel.
use serde_json::Value;
/// Progress and outcome events emitted by `run_subagent` as it processes
@@ -8,29 +7,20 @@ use serde_json::Value;
#[derive(Debug, Clone)]
pub enum SubagentEvent {
StepCompleted {
#[allow(dead_code)]
step: usize,
#[allow(dead_code)]
output: String,
},
StepFailed {
step: usize,
error: String,
},
Completed {
#[allow(dead_code)]
output: String,
},
Completed,
ToolCall {
tool: String,
#[allow(dead_code)]
args: Value,
},
ToolResult {
tool: String,
args: Value,
#[allow(dead_code)]
output: String,
},
Progress(String),
/// Token usage reported by the LLM after one streaming call inside the
-1
View File
@@ -1,6 +1,5 @@
//! Subagent management: spawning, context building, engine loop, and
//! progress events.
pub mod auto;
pub mod context;
pub mod division;
-8
View File
@@ -1,6 +1,5 @@
//! `AgentDefinition` -- declarative specification for instantiating a
//! subagent from workflow scripts or programmatic calls.
use serde::{Deserialize, Serialize};
/// Declarative specification for instantiating a subagent: name, role,
@@ -29,13 +28,6 @@ impl AgentDefinition {
}
}
/// Builder method: limit this agent to at most `steps` LLM calls.
#[allow(dead_code)]
pub fn with_max_steps(mut self, steps: usize) -> Self {
self.max_steps = Some(steps);
self
}
/// Builder method: set the system prompt for this agent.
pub fn with_system_prompt(mut self, prompt: String) -> Self {
self.system_prompt = Some(prompt);
+28 -16
View File
@@ -6,11 +6,10 @@
//! choice, so this step is plain Rust — not an LLM call, not a cycle the
//! Core Intelligence can omit or reshape — and always runs after any
//! hive-mind convergence completes.
use std::path::{Path, PathBuf};
use std::fmt::Write as _;
use crate::app::workflow::hive_mind::NodeReport;
use crate::model::memory::Memory;
use std::fmt::Write as _;
use std::path::{Path, PathBuf};
/// Write a markdown report of one hive-mind convergence to
/// `<workspace_root>/docs/runs/<timestamp>-<slug>.md`.
@@ -42,22 +41,31 @@ pub fn write_hive_mind_convergence(
}
/// Render a hive-mind convergence as a markdown document.
fn render_report(user_request: &str, ts_millis: i64, reports: &[NodeReport], consensus: &str) -> String {
fn render_report(
user_request: &str,
ts_millis: i64,
reports: &[NodeReport],
consensus: &str,
) -> String {
let mut out = String::new();
writeln!(out, "# The Hive converges: {user_request}").unwrap();
writeln!(out, "\nTimestamp (ms): {ts_millis}\n").unwrap();
let _ = writeln!(out, "# The Hive converges: {user_request}");
let _ = writeln!(out, "\nTimestamp (ms): {ts_millis}\n");
let cycle_count = reports.iter().map(|r| r.cycle_index).max().map_or(0, |m| m + 1);
let cycle_count = reports
.iter()
.map(|r| r.cycle_index)
.max()
.map_or(0, |m| m + 1);
for cycle_index in 0..cycle_count {
writeln!(out, "## Cycle {cycle_index}\n").unwrap();
let _ = writeln!(out, "## Cycle {cycle_index}\n");
for r in reports.iter().filter(|r| r.cycle_index == cycle_index) {
writeln!(out, "### {}\n", r.node_id).unwrap();
writeln!(out, "{}\n", r.output).unwrap();
let _ = writeln!(out, "### {}\n", r.node_id);
let _ = writeln!(out, "{}\n", r.output);
}
}
writeln!(out, "## The Hive's Verdict\n").unwrap();
writeln!(out, "{consensus}\n").unwrap();
let _ = writeln!(out, "## The Hive's Verdict\n");
let _ = writeln!(out, "{consensus}\n");
out
}
@@ -70,10 +78,14 @@ mod tests {
let tmp = std::env::temp_dir().join(format!("zesdex-docs-test-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&tmp).unwrap();
let reports = vec![
NodeReport { node_id: "Node-0-0".to_string(), cycle_index: 0, output: "found the bug".to_string() },
];
let path = write_hive_mind_convergence(&tmp, "fix the bug", &reports, "the bug is a null check").unwrap();
let reports = vec![NodeReport {
node_id: "Node-0-0".to_string(),
cycle_index: 0,
output: "found the bug".to_string(),
}];
let path =
write_hive_mind_convergence(&tmp, "fix the bug", &reports, "the bug is a null check")
.unwrap();
assert!(path.starts_with(tmp.join("docs").join("runs")));
let content = std::fs::read_to_string(&path).unwrap();
+156 -61
View File
@@ -13,12 +13,14 @@
//! `Arc<Mutex<Vec<String>>>` threaded through `execute_primitive` and
//! `spawn_single_agent` rather than a global static, preventing data
//! leaks between concurrent workflow runs.
use std::collections::HashMap;
use std::sync::{Arc, Mutex, atomic::{AtomicBool, Ordering}};
use std::time::Duration;
use serde::{Deserialize, Serialize};
use super::script::{ScriptPrimitive, WorkflowScript};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
use std::time::Duration;
/// The lifecycle state of an agent within a workflow run.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
@@ -100,17 +102,31 @@ pub type LiveStateFn = Arc<dyn Fn(String, String, AgentStatus) + Send + Sync>;
/// Return: the agent's text output, or an error on failure.
fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value) -> String {
let details = match tool {
"read" | "view_file" | "write" | "write_to_file" | "edit" | "replace_file_content" | "multi_replace_file_content" | "delete" => {
args.get("path")
.or_else(|| args.get("TargetFile"))
.or_else(|| args.get("AbsolutePath"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string()
}
"read"
| "view_file"
| "write"
| "write_to_file"
| "edit"
| "replace_file_content"
| "multi_replace_file_content"
| "delete" => args
.get("path")
.or_else(|| args.get("TargetFile"))
.or_else(|| args.get("AbsolutePath"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
"grep" | "grep_search" => {
let pattern = args.get("pattern").or_else(|| args.get("Query")).and_then(|v| v.as_str()).unwrap_or("");
let path = args.get("path").or_else(|| args.get("SearchPath")).and_then(|v| v.as_str()).unwrap_or("");
let pattern = args
.get("pattern")
.or_else(|| args.get("Query"))
.and_then(|v| v.as_str())
.unwrap_or("");
let path = args
.get("path")
.or_else(|| args.get("SearchPath"))
.and_then(|v| v.as_str())
.unwrap_or("");
if path.is_empty() {
format!("\"{pattern}\"")
} else {
@@ -127,31 +143,44 @@ fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value)
}
}
"bash" | "run_command" => {
let cmd = args.get("command").or_else(|| args.get("CommandLine")).and_then(|v| v.as_str()).unwrap_or("");
let cmd = args
.get("command")
.or_else(|| args.get("CommandLine"))
.and_then(|v| v.as_str())
.unwrap_or("");
if cmd.len() > 60 {
format!("\"{}...\"", &cmd[..57])
} else {
format!("\"{cmd}\"")
}
}
"recall" => {
args.get("query").and_then(|v| v.as_str()).unwrap_or("").to_string()
}
"remember" => {
args.get("name").and_then(|v| v.as_str()).unwrap_or("").to_string()
}
"dir_list" | "list_dir" => {
args.get("DirectoryPath").or_else(|| args.get("path")).and_then(|v| v.as_str()).unwrap_or("").to_string()
}
"recall" => args
.get("query")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
"remember" => args
.get("name")
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
"dir_list" | "list_dir" => args
.get("DirectoryPath")
.or_else(|| args.get("path"))
.and_then(|v| v.as_str())
.unwrap_or("")
.to_string(),
_ => {
if args.is_object() && !args.as_object().unwrap().is_empty() {
args.as_object().unwrap().values()
.find_map(|v| v.as_str())
.unwrap_or("")
.to_string()
} else {
String::new()
if let Some(obj) = args.as_object() {
if !obj.is_empty() {
return obj
.values()
.find_map(|v| v.as_str())
.unwrap_or("")
.to_string();
}
}
String::new()
}
};
@@ -181,7 +210,6 @@ fn format_tool_call_progress(prefix: &str, tool: &str, args: &serde_json::Value)
/// a stuck stage from blocking the entire pipeline forever.
///
/// Return: the agent's text output, or an error on failure.
#[allow(clippy::too_many_lines, clippy::too_many_arguments, clippy::ref_option)]
fn spawn_single_agent(
agent_id: &str,
agent_name: &str,
@@ -338,10 +366,13 @@ fn spawn_single_agent(
);
}
}
SubagentEvent::Completed { .. } => {
SubagentEvent::Completed => {
tracing::debug!("[subagent] completed");
}
SubagentEvent::Usage { tokens_in, tokens_out } => {
SubagentEvent::Usage {
tokens_in,
tokens_out,
} => {
tracing::debug!("[subagent] usage: {} in, {} out", tokens_in, tokens_out);
}
}
@@ -349,7 +380,10 @@ fn spawn_single_agent(
});
// Check abort before even starting the subagent.
if abort_flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
if abort_flag
.as_ref()
.is_some_and(|f| f.load(Ordering::SeqCst))
{
anyhow::bail!("subagent '{agent_name}' aborted before start");
}
@@ -380,9 +414,7 @@ fn spawn_single_agent(
));
}
if bg_abort.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
break Err(anyhow::anyhow!(
"subagent '{bg_name}' aborted by user",
));
break Err(anyhow::anyhow!("subagent '{bg_name}' aborted by user"));
}
}
} else {
@@ -391,9 +423,7 @@ fn spawn_single_agent(
break r;
}
if bg_abort.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
break Err(anyhow::anyhow!(
"subagent '{bg_name}' aborted by user",
));
break Err(anyhow::anyhow!("subagent '{bg_name}' aborted by user"));
}
}
};
@@ -467,8 +497,6 @@ type ParallelResult = (usize, anyhow::Result<Vec<String>>);
///
/// Return: a `Vec<String>` of all agent outputs (or error strings) in
/// the order they were submitted.
#[allow(clippy::too_many_arguments)]
#[allow(clippy::ref_option, clippy::too_many_lines)]
pub fn execute_primitive(
primitive: &ScriptPrimitive,
args: &HashMap<String, String>,
@@ -501,7 +529,20 @@ pub fn execute_primitive(
let resolved = resolve_template(prompt, &resolved_args);
let agent_id = uuid::Uuid::new_v4().to_string();
let agent_name = resolved.chars().take(40).collect::<String>();
match spawn_single_agent(&agent_id, &agent_name, &resolved, "coder", None, &findings_snapshot, findings, abort_flag, live, session_dir, workspaces, timeout_ms) {
match spawn_single_agent(
&agent_id,
&agent_name,
&resolved,
"coder",
None,
&findings_snapshot,
findings,
abort_flag,
live,
session_dir,
workspaces,
timeout_ms,
) {
Ok(text) => Ok(vec![text]),
Err(e) => {
if continue_on_error {
@@ -513,7 +554,11 @@ pub fn execute_primitive(
}
}
ScriptPrimitive::ScopedAgent { prompt, node_id, tool_scope } => {
ScriptPrimitive::ScopedAgent {
prompt,
node_id,
tool_scope,
} => {
let mut resolved_args = args.clone();
let findings_snapshot = findings.lock().map(|f| f.clone()).unwrap_or_default();
if !resolved_args.contains_key("findings") {
@@ -535,9 +580,24 @@ pub fn execute_primitive(
tracing::debug!("[hive] deploying drone {node_id}: {truncated}");
let agent_name = format!("{node_id}: {truncated}");
let allowed_tools = crate::app::subagent::division::tool_scope::tools_for(tool_scope);
match spawn_single_agent(&agent_id, &agent_name, &resolved, node_id, Some(allowed_tools), &findings_snapshot, findings, abort_flag, live, session_dir, workspaces, timeout_ms) {
match spawn_single_agent(
&agent_id,
&agent_name,
&resolved,
node_id,
Some(allowed_tools),
&findings_snapshot,
findings,
abort_flag,
live,
session_dir,
workspaces,
timeout_ms,
) {
Ok(text) => {
tracing::debug!("[hive] drone {node_id} completed — merging into collective state");
tracing::debug!(
"[hive] drone {node_id} completed — merging into collective state"
);
// Merge this drone's complete output into the Hive's
// collective state the instant it finishes — not after
// the whole parallel cohort completes. Any sibling drone
@@ -567,8 +627,7 @@ pub fn execute_primitive(
// Each branch shares the same `findings` Arc so note_finding
// calls within any branch are visible to all other branches.
let semaphore = Arc::new(Semaphore::new(concurrency_cap.max(1)));
let results: Arc<Mutex<Vec<ParallelResult>>> =
Arc::new(Mutex::new(Vec::new()));
let results: Arc<Mutex<Vec<ParallelResult>>> = Arc::new(Mutex::new(Vec::new()));
let handles: Vec<_> = scripts
.iter()
@@ -589,7 +648,10 @@ pub fn execute_primitive(
std::thread::spawn(move || {
let _permit = sem.acquire();
let result = execute_primitive(
&script, &args, cap, continue_on_error,
&script,
&args,
cap,
continue_on_error,
&abort,
live_clone.as_ref(),
&session_dir,
@@ -608,7 +670,9 @@ pub fn execute_primitive(
let _ = handle.join();
}
let mut locked = results.lock().map_err(|_| anyhow::anyhow!("parallel results lock poisoned"))?;
let mut locked = results
.lock()
.map_err(|_| anyhow::anyhow!("parallel results lock poisoned"))?;
locked.sort_by_key(|(idx, _)| *idx);
let mut all = Vec::new();
for (_, res) in locked.drain(..) {
@@ -635,14 +699,28 @@ pub fn execute_primitive(
for (idx, script) in scripts.iter().enumerate() {
// Check abort before each pipeline stage so we don't
// launch the next division after the user cancelled.
if abort_flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst)) {
if abort_flag
.as_ref()
.is_some_and(|f| f.load(Ordering::SeqCst))
{
if continue_on_error {
all.push(format!("pipeline aborted at stage {idx}"));
break;
}
anyhow::bail!("pipeline aborted by user at stage {idx}");
}
match execute_primitive(script, args, concurrency_cap, continue_on_error, abort_flag, live, session_dir, workspaces, findings, timeout_ms) {
match execute_primitive(
script,
args,
concurrency_cap,
continue_on_error,
abort_flag,
live,
session_dir,
workspaces,
findings,
timeout_ms,
) {
Ok(outputs) => all.extend(outputs),
Err(e) => {
if continue_on_error {
@@ -656,9 +734,21 @@ pub fn execute_primitive(
Ok(all)
}
ScriptPrimitive::Phase { name: _name, script } => {
execute_primitive(script, args, concurrency_cap, continue_on_error, abort_flag, live, session_dir, workspaces, findings, timeout_ms)
}
ScriptPrimitive::Phase {
name: _name,
script,
} => execute_primitive(
script,
args,
concurrency_cap,
continue_on_error,
abort_flag,
live,
session_dir,
workspaces,
findings,
timeout_ms,
),
}
}
@@ -687,7 +777,6 @@ pub fn run_workflow(
/// `spawn_agents` invocations remain fully isolated.
///
/// Return: a human-readable summary string.
#[allow(clippy::ref_option)]
pub fn run_workflow_tracked(
script: &WorkflowScript,
args: &HashMap<String, String>,
@@ -704,9 +793,15 @@ pub fn run_workflow_tracked(
let findings = Arc::new(Mutex::new(Vec::new()));
let results = execute_primitive(
&script.script, args, concurrency_cap,
script.options.continue_on_error, abort_flag, live,
session_dir, workspaces, &findings,
&script.script,
args,
concurrency_cap,
script.options.continue_on_error,
abort_flag,
live,
session_dir,
workspaces,
&findings,
script.options.timeout_ms,
)?;
+163 -82
View File
@@ -25,12 +25,14 @@
//! Synthesis node reads the complete collective state and converges it
//! into one unified voice — returned to LO and persisted to docs/runs/*.md.
//! ```
use std::collections::HashMap;
use std::sync::{Arc, Mutex, atomic::{AtomicBool, Ordering}};
use serde::Deserialize;
use crate::app::workflow::engine::{execute_primitive, AgentStatus, LiveStateFn};
use crate::app::workflow::script::ScriptPrimitive;
use crate::app::workflow::engine::{execute_primitive, LiveStateFn, AgentStatus};
use serde::Deserialize;
use std::collections::HashMap;
use std::sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex,
};
/// One directive the Hive's Core Intelligence issues to a drone within a
/// cognitive cycle. A drone's sole identity is its directive and access tier.
@@ -85,60 +87,41 @@ pub const HIVE_MIND_CONSENSUS_TAG: &str = "[The Hive speaks]";
/// Return: `true` if any prior system message begins with
/// `HIVE_MIND_CONSENSUS_TAG`.
pub fn hive_mind_already_ran<'a>(system_message_bodies: impl Iterator<Item = &'a str>) -> bool {
system_message_bodies.into_iter().any(|body| body.starts_with(HIVE_MIND_CONSENSUS_TAG))
system_message_bodies
.into_iter()
.any(|body| body.starts_with(HIVE_MIND_CONSENSUS_TAG))
}
/// Build the live-state callback that forwards each drone's status to the
/// TUI panel so LO can watch the Hive work.
fn build_live(
turn_events: Option<&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>>,
turn_events: Option<
&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>,
>,
) -> Option<LiveStateFn> {
turn_events.map(|events| {
let events = events.clone();
let f: LiveStateFn = Arc::new(move |_agent_id: String, agent_name: String, status: AgentStatus| {
let display_name = agent_name.chars().take(40).collect::<String>();
if let Ok(mut q) = events.lock() {
q.push_back(crate::app::state::runtime::TurnEvent::WorkflowAgentUpdate {
agent_id: display_name.clone(),
agent_name: display_name,
status,
});
}
});
let f: LiveStateFn = Arc::new(
move |_agent_id: String, agent_name: String, status: AgentStatus| {
let display_name = agent_name.chars().take(40).collect::<String>();
if let Ok(mut q) = events.lock() {
q.push_back(crate::app::state::runtime::TurnEvent::WorkflowAgentUpdate {
agent_id: display_name.clone(),
agent_name: display_name,
status,
});
}
},
);
f
})
}
/// Deploy the Hive: execute a cognitive cycle plan authored by the Core
/// Intelligence. Each cycle spawns drones (anonymous processing nodes) in
/// parallel. Every drone's complete output merges into the Hive's
/// collective state the instant it finishes, and a final synthesis node
/// reconciles the entire collective state into one unified voice.
/// Context struct threaded through all Hive cycle execution.
///
/// Flow: for each cycle (sequential) → spawn one `ScriptPrimitive::ScopedAgent`
/// per directive, tagged with a system-assigned `node_id` (the Hive's
/// coordinate system, never an LLM-chosen name) → run them as a `Parallel`
/// block via `execute_primitive`, which merges each drone's output into the
/// Hive's shared collective-state Arc the instant that drone completes, not
/// after the whole cohort finishes → record `NodeReport`s → proceed to the
/// next cycle. After all cycles: spawn one more read-only synthesis node
/// whose directive is to converge the complete collective state into a
/// single consensus — the Hive becoming one voice — not list what each
/// drone said.
///
/// Concurrency per cycle and the per-drone timeout both come from
/// `Settings::load()` (`workflow_max_concurrency`, `hive_mind_node_timeout_ms`)
/// rather than a hardcoded cap/no-timeout — a stuck drone can no longer
/// stall the entire Hive forever.
///
/// Return: `(consensus, all_node_reports)` on success. `consensus` is the
/// synthesis node's converged output — what the Core Intelligence actually
/// hears from the Hive. `all_node_reports` is the complete per-drone record.
///
/// The convergence doc under `docs/runs/*.md` is written unconditionally
/// before this function returns — even when synthesis itself fails — so a
/// synthesis error never discards the work already done by cycle drones.
/// Callers must not write their own copy of this doc.
/// Carries the user request, shared collective state, concurrency limits,
/// abort flag, live-status callback, session/workspace paths, and per-drone
/// timeout so individual cycle functions don't need long parameter lists.
struct CycleCtx<'a> {
user_request: &'a str,
collective_state: &'a Arc<Mutex<Vec<String>>>,
@@ -154,6 +137,8 @@ struct CycleCtx<'a> {
///
/// Flow: map cycle directives to `ScopedAgent` primitives -> group in a Parallel
/// phase block -> run block via `execute_primitive` -> return reports.
///
/// Return: `Ok(Vec<NodeReport>)` with one report per directive in submission order.
fn execute_cycle(
cycle_index: usize,
directives: &[NodeDirective],
@@ -249,12 +234,44 @@ fn execute_cycle(
Ok(reports)
}
/// Deploy the Hive: execute a cognitive cycle plan authored by the Core
/// Intelligence. Each cycle spawns drones (anonymous processing nodes) in
/// parallel. Every drone's complete output merges into the Hive's
/// collective state the instant it finishes, and a final synthesis node
/// reconciles the entire collective state into one unified voice.
///
/// Flow: for each cycle (sequential) → spawn one `ScriptPrimitive::ScopedAgent`
/// per directive, tagged with a system-assigned `node_id` (the Hive's
/// coordinate system, never an LLM-chosen name) → run them as a `Parallel`
/// block via `execute_primitive`, which merges each drone's output into the
/// Hive's shared collective-state Arc the instant that drone completes, not
/// after the whole cohort finishes → record `NodeReport`s → proceed to the
/// next cycle. After all cycles: spawn one more read-only synthesis node
/// whose directive is to converge the complete collective state into a
/// single consensus — the Hive becoming one voice — not list what each
/// drone said.
///
/// Concurrency per cycle and the per-drone timeout both come from
/// `Settings::load()` (`workflow_max_concurrency`, `hive_mind_node_timeout_ms`)
/// rather than a hardcoded cap/no-timeout — a stuck drone can no longer
/// stall the entire Hive forever.
///
/// Return: `(consensus, all_node_reports)` on success. `consensus` is the
/// synthesis node's converged output — what the Core Intelligence actually
/// hears from the Hive. `all_node_reports` is the complete per-drone record.
///
/// The convergence doc under `docs/runs/*.md` is written unconditionally
/// before this function returns — even when synthesis itself fails — so a
/// synthesis error never discards the work already done by cycle drones.
/// Callers must not write their own copy of this doc.
pub fn run_hive_mind(
user_request: &str,
plan: &CognitiveCyclePlan,
session_dir: &std::path::Path,
workspaces: &[std::path::PathBuf],
turn_events: Option<&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>>,
turn_events: Option<
&Arc<Mutex<std::collections::VecDeque<crate::app::state::runtime::TurnEvent>>>,
>,
abort_flag: Option<&Arc<AtomicBool>>,
) -> anyhow::Result<(String, Vec<NodeReport>)> {
if plan.cycles.is_empty() {
@@ -288,20 +305,25 @@ pub fn run_hive_mind(
anyhow::bail!("the Hive was recalled by LO before cycle {cycle_index}");
}
tracing::info!("[hive-mind] cycle {cycle_index} deploying {} drone(s)", directives.len());
tracing::info!(
"[hive-mind] cycle {cycle_index} deploying {} drone(s)",
directives.len()
);
let mut cycle_reports = execute_cycle(
cycle_index,
directives,
&ctx,
)?;
let mut cycle_reports = execute_cycle(cycle_index, directives, &ctx)?;
reports.append(&mut cycle_reports);
}
tracing::info!("[hive-mind] all cycles complete — the Hive begins convergence");
let consensus_result = synthesize_consensus(
user_request, session_dir, workspaces, &collective_state, live.as_ref(), abort_flag, node_timeout_ms,
user_request,
session_dir,
workspaces,
&collective_state,
live.as_ref(),
abort_flag,
node_timeout_ms,
);
// Guaranteed documentation: write the convergence doc for whatever
@@ -311,13 +333,19 @@ pub fn run_hive_mind(
// CLAUDE.md promises for every convergence.
let doc_consensus = match &consensus_result {
Ok(c) => c.clone(),
Err(e) => format!(
"The Hive's convergence fractured: {e}. Partial node reports above.",
),
Err(e) => format!("The Hive's convergence fractured: {e}. Partial node reports above."),
};
if let Some(workspace_root) = workspaces.first() {
match crate::app::workflow::docs::write_hive_mind_convergence(workspace_root, user_request, &reports, &doc_consensus) {
Ok(path) => tracing::info!("[hive-mind] the Hive's convergence written to {}", path.display()),
match crate::app::workflow::docs::write_hive_mind_convergence(
workspace_root,
user_request,
&reports,
&doc_consensus,
) {
Ok(path) => tracing::info!(
"[hive-mind] the Hive's convergence written to {}",
path.display()
),
Err(e) => tracing::warn!("[hive-mind] the Hive's convergence report failed: {e}"),
}
}
@@ -373,7 +401,16 @@ fn synthesize_consensus(
let args: HashMap<String, String> = HashMap::new();
let abort_owned: Option<Arc<AtomicBool>> = abort_flag.cloned();
let results = execute_primitive(
&synthesis, &args, 1, false, &abort_owned, live, session_dir, workspaces, collective_state, node_timeout_ms,
&synthesis,
&args,
1,
false,
&abort_owned,
live,
session_dir,
workspaces,
collective_state,
node_timeout_ms,
)?;
Ok(results.into_iter().next().unwrap_or_default())
}
@@ -402,16 +439,31 @@ pub fn is_complex_request(request: &str) -> bool {
// Single-line simple update patterns
let lower = trimmed.to_lowercase();
let negative_keywords = [
"simple", "trivial", "typo", "just a", "only a", "minor",
"quick", "tiny", "small fix", "rename", "nitpick",
"cosmetic", "formatting", "spelling", "grammar",
"bump", "version bump", "update comment",
"simple",
"trivial",
"typo",
"just a",
"only a",
"minor",
"quick",
"tiny",
"small fix",
"rename",
"nitpick",
"cosmetic",
"formatting",
"spelling",
"grammar",
"bump",
"version bump",
"update comment",
];
if negative_keywords.iter().any(|k| lower.contains(k)) {
return false;
}
// Multi-line/multi-sentence → likely complex
let sentences = trimmed.split(['.', '!', '?'])
let sentences = trimmed
.split(['.', '!', '?'])
.filter(|s| !s.trim().is_empty())
.count();
if sentences >= 3 {
@@ -419,11 +471,29 @@ pub fn is_complex_request(request: &str) -> bool {
}
// Positive complexity keywords
let complexity_keywords = [
"refactor", "redesign", "architecture", "feature", "implement",
"migrate", "restructure", "rewrite", "new module", "new component",
"scaffold", "multi", "multiple files", "api", "endpoint",
"integration", "system", "workflow", "pipeline", "database",
"authentication", "authorization", "full stack",
"refactor",
"redesign",
"architecture",
"feature",
"implement",
"migrate",
"restructure",
"rewrite",
"new module",
"new component",
"scaffold",
"multi",
"multiple files",
"api",
"endpoint",
"integration",
"system",
"workflow",
"pipeline",
"database",
"authentication",
"authorization",
"full stack",
];
complexity_keywords.iter().any(|k| lower.contains(k))
}
@@ -445,7 +515,9 @@ mod tests {
#[test]
fn test_is_complex_request_multi_sentence() {
assert!(is_complex_request("This is sentence one. This is sentence two. This is sentence three."));
assert!(is_complex_request(
"This is sentence one. This is sentence two. This is sentence three."
));
}
#[test]
@@ -456,9 +528,7 @@ mod tests {
#[test]
fn test_default_access_is_read() {
let d: NodeDirective = serde_json::from_str(
r#"{"directive": "write tests"}"#
).unwrap();
let d: NodeDirective = serde_json::from_str(r#"{"directive": "write tests"}"#).unwrap();
assert_eq!(d.access, crate::app::subagent::division::tool_scope::READ);
}
@@ -468,14 +538,16 @@ mod tests {
// "role" key, if an LLM emits one out of old habit, is simply
// ignored rather than required or preserved.
let d: NodeDirective = serde_json::from_str(
r#"{"role": "Architect", "directive": "plan the migration", "access": "read"}"#
).unwrap();
r#"{"role": "Architect", "directive": "plan the migration", "access": "read"}"#,
)
.unwrap();
assert_eq!(d.directive, "plan the migration");
}
#[test]
fn test_cognitive_cycle_plan_arbitrary_shape() {
let plan: CognitiveCyclePlan = serde_json::from_str(r#"{
let plan: CognitiveCyclePlan = serde_json::from_str(
r#"{
"cycles": [
[{"directive": "scan the codebase topology", "access": "read"}],
[
@@ -484,7 +556,9 @@ mod tests {
],
[{"directive": "cut the release", "access": "full"}]
]
}"#).unwrap();
}"#,
)
.unwrap();
assert_eq!(plan.cycles.len(), 3);
assert_eq!(plan.cycles[1].len(), 2);
}
@@ -502,9 +576,12 @@ mod tests {
fn test_run_hive_mind_aborts_before_spawning_when_flag_preset() {
// The abort check runs before execute_primitive for cycle 0, so a
// pre-set abort flag must short-circuit without any LLM/network call.
let plan: CognitiveCyclePlan = serde_json::from_str(r#"{
let plan: CognitiveCyclePlan = serde_json::from_str(
r#"{
"cycles": [[{"directive": "whatever", "access": "read"}]]
}"#).unwrap();
}"#,
)
.unwrap();
let tmp = std::env::temp_dir();
let abort_flag = Arc::new(AtomicBool::new(true));
let err = run_hive_mind("do something", &plan, &tmp, &[], None, Some(&abort_flag))
@@ -526,12 +603,16 @@ mod tests {
"you are a helpful assistant".to_string(),
format!("{HIVE_MIND_CONSENSUS_TAG}\nthe bug is a null check"),
];
assert!(hive_mind_already_ran(bodies.iter().map(std::string::String::as_str)));
assert!(hive_mind_already_ran(
bodies.iter().map(std::string::String::as_str)
));
}
#[test]
fn hive_mind_already_ran_false_when_no_prior_convergence() {
let bodies = ["you are a helpful assistant".to_string()];
assert!(!hive_mind_already_ran(bodies.iter().map(std::string::String::as_str)));
assert!(!hive_mind_already_ran(
bodies.iter().map(std::string::String::as_str)
));
}
}
+1 -2
View File
@@ -1,7 +1,6 @@
//! Workflow orchestration: a script interpreter that runs pipeline/parallel
//! primitives across multiple subagent instances.
pub mod hive_mind;
pub mod docs;
pub mod engine;
pub mod hive_mind;
pub mod script;
-1
View File
@@ -1,6 +1,5 @@
//! Script primitives for the workflow engine: agent invocation, parallel
//! execution, pipelines, and phases.
use serde::{Deserialize, Serialize};
/// A workflow script primitive — can be a single agent, a parallel fan-out,
+12 -10
View File
@@ -11,10 +11,7 @@ pub enum Command {
ClearConfirm,
Login { provider: String },
Edit(String),
McpAdd {
name: String,
command: String,
},
McpAdd { name: String, command: String },
ModelList,
Compact,
TodoOpen,
@@ -44,13 +41,15 @@ pub fn parse_command(text: &str) -> Command {
"/quit" => Command::Quit,
"/clear" if arg1.is_empty() => Command::ClearConfirm,
"/clear" => Command::Clear,
"/login" if arg1.is_empty() => Command::Login { provider: String::new() },
"/login" if !arg1.is_empty() => Command::Login { provider: arg1.to_string() },
"/login" if arg1.is_empty() => Command::Login {
provider: String::new(),
},
"/login" if !arg1.is_empty() => Command::Login {
provider: arg1.to_string(),
},
"/edit" if !arg1.is_empty() => Command::Edit(arg1.to_string()),
"/edit" => Command::Edit(".".to_string()),
"/mcp" if arg1.is_empty() => {
Command::McpOpen
}
"/mcp" if arg1.is_empty() => Command::McpOpen,
"/mcp" if arg1 == "add" && !arg2.is_empty() => {
let rest = arg2.trim();
if let Some(space) = rest.find(' ') {
@@ -58,7 +57,10 @@ pub fn parse_command(text: &str) -> Command {
let command = rest[space + 1..].trim().to_string();
Command::McpAdd { name, command }
} else {
Command::McpAdd { name: rest.to_string(), command: String::new() }
Command::McpAdd {
name: rest.to_string(),
command: String::new(),
}
}
}
"/model" => Command::ModelList,
+134 -18
View File
@@ -1,12 +1,12 @@
//! Key event dispatcher: maps crossterm `KeyEvent` values into `Action`
//! variants, with special handling for overlays, auto-complete, and the
//! inline editor.
use crossterm::event::{KeyCode, KeyEvent, KeyModifiers};
use crate::app::mode;
use crate::app::runtime::actions::Action;
use crate::app::runtime::commands::apply_command;
use crate::app::state::misc::AutocompleteKind;
use crate::app::state::rest::AppStateRest;
use crate::app::state::types::Overlay;
use crate::controller::command::parse_command;
@@ -21,7 +21,6 @@ use crate::controller::command::parse_command;
/// Why: when Editor overlay is active, all key events are consumed by the
/// editor handler and never reach the main action dispatch. Return `Vec`
/// so that a single key press can trigger multiple actions.
#[allow(clippy::too_many_lines)]
pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
// While Editor overlay is active, route input directly to the editor handler
if state.misc.overlay == Overlay::Editor {
@@ -81,27 +80,39 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
KeyCode::Up => {
let items = crate::app::mode::learning::get_learning_items(state);
let n = items.len();
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
state.misc.selected_index = if state.misc.selected_index == 0 {
n.saturating_sub(1)
} else {
state.misc.selected_index - 1
};
state.dirty = true;
return vec![];
}
KeyCode::Down => {
let items = crate::app::mode::learning::get_learning_items(state);
let n = items.len();
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
state.misc.selected_index = if n == 0 {
0
} else {
(state.misc.selected_index + 1) % n
};
state.dirty = true;
return vec![];
}
KeyCode::Enter | KeyCode::Char('a') => {
let items = crate::app::mode::learning::get_learning_items(state);
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) = items.get(state.misc.selected_index) {
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) =
items.get(state.misc.selected_index)
{
return vec![Action::LessonAccept { name: name.clone() }];
}
return vec![];
}
KeyCode::Char('r') => {
let items = crate::app::mode::learning::get_learning_items(state);
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) = items.get(state.misc.selected_index) {
if let Some(crate::app::mode::learning::LearningItem::Pending { name, .. }) =
items.get(state.misc.selected_index)
{
return vec![Action::LessonReject { name: name.clone() }];
}
return vec![];
@@ -131,6 +142,26 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
KeyCode::Char('d') if key.modifiers.contains(KeyModifiers::CONTROL) => {
vec![Action::CloseOverlay]
}
KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL) => {
let last_assistant = state
.transcript_cache
.messages
.iter()
.rev()
.find(|m| m.role == crate::dto::chat::message::Role::Assistant);
match last_assistant {
Some(msg) => {
state.misc.pending_clipboard_copy = Some(msg.content.clone());
}
None => {
state.push_toast(crate::app::state::types::Toast::new(
crate::app::state::types::ToastKind::Info,
"No assistant message to copy yet".to_string(),
));
}
}
Vec::new()
}
KeyCode::Enter => {
if state.input.autocomplete_visible {
state.input.select_autocomplete();
@@ -176,18 +207,24 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
} else if state.misc.overlay == Overlay::Effort {
mode::effort::cycle_effort(state);
Vec::new()
} else if state.misc.overlay == Overlay::Rewind {
let n = mode::rewind::rewind_count(state);
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
state.misc.selected_index = if state.misc.selected_index == 0 {
n.saturating_sub(1)
} else {
state.misc.selected_index - 1
};
state.dirty = true;
Vec::new()
} else if state.misc.overlay == Overlay::ModelSelector {
let n = state.app_config.providers.len();
state.misc.selected_index = if state.misc.selected_index == 0 { n.saturating_sub(1) } else { state.misc.selected_index - 1 };
state.misc.selected_index = if state.misc.selected_index == 0 {
n.saturating_sub(1)
} else {
state.misc.selected_index - 1
};
state.dirty = true;
Vec::new()
} else if key.modifiers.contains(KeyModifiers::CONTROL) {
vec![Action::ScrollUp]
} else {
@@ -202,18 +239,24 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
} else if state.misc.overlay == Overlay::Effort {
mode::effort::cycle_effort(state);
Vec::new()
} else if state.misc.overlay == Overlay::Rewind {
let n = mode::rewind::rewind_count(state);
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
state.misc.selected_index = if n == 0 {
0
} else {
(state.misc.selected_index + 1) % n
};
state.dirty = true;
Vec::new()
} else if state.misc.overlay == Overlay::ModelSelector {
let n = state.app_config.providers.len();
state.misc.selected_index = if n == 0 { 0 } else { (state.misc.selected_index + 1) % n };
state.misc.selected_index = if n == 0 {
0
} else {
(state.misc.selected_index + 1) % n
};
state.dirty = true;
Vec::new()
} else if key.modifiers.contains(KeyModifiers::CONTROL) {
vec![Action::ScrollDown]
} else {
@@ -247,6 +290,11 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
state.input.tab_complete();
}
state.dirty = true;
} else if state.input.autocomplete_kind == AutocompleteKind::FileMention
&& state.input.autocomplete_visible
{
state.input.cycle_autocomplete(true);
state.dirty = true;
}
Vec::new()
}
@@ -263,6 +311,10 @@ pub fn handle_key(key: KeyEvent, state: &mut AppStateRest) -> Vec<Action> {
// without requiring an extra Tab press.
if state.input.buffer.starts_with('/') {
state.input.open_autocomplete();
} else if state.input.mention_query_at_cursor().is_some() {
state
.input
.open_mention_autocomplete(&state.mention_index.snapshot());
}
Vec::new()
}
@@ -301,7 +353,10 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
if text.is_empty() {
state.settings.api_keys.remove(&state.settings.provider);
} else {
state.settings.api_keys.insert(state.settings.provider.clone(), text.clone());
state
.settings
.api_keys
.insert(state.settings.provider.clone(), text.clone());
}
let _ = state.settings.save();
state.input.buffer.clear();
@@ -329,14 +384,24 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
if let Some(provider) = providers.get(state.misc.selected_index) {
if let Some(cfg) = state.app_config.providers.get(provider) {
let model = cfg.default_model.clone().unwrap_or_else(|| {
tracing::warn!("[input] provider '{}' has no default_model, using 'claude-opus-4-8'", provider);
tracing::warn!(
"[input] provider '{}' has no default_model, using 'claude-opus-4-8'",
provider
);
"claude-opus-4-8".to_string()
});
state.settings.provider.clone_from(provider);
state.settings.model.clone_from(&model);
if let Some(ref key) = cfg.default_api_key {
state.settings.api_keys.insert(provider.clone(), key.clone());
} else if let Some(env_key) = cfg.api_key_env.as_ref().and_then(|env| std::env::var(env).ok()) {
state
.settings
.api_keys
.insert(provider.clone(), key.clone());
} else if let Some(env_key) = cfg
.api_key_env
.as_ref()
.and_then(|env| std::env::var(env).ok())
{
state.settings.api_keys.insert(provider.clone(), env_key);
}
let _ = state.settings.save();
@@ -363,3 +428,54 @@ fn handle_overlay_enter(state: &mut AppStateRest) -> Vec<Action> {
_ => Vec::new(),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn test_state() -> AppStateRest {
let tmp = std::env::temp_dir().join(format!("zesdex-input-test-{}", uuid::Uuid::new_v4()));
std::fs::create_dir_all(&tmp).unwrap();
AppStateRest::new(vec![tmp.clone()], &tmp, tmp.join("memory"))
}
#[test]
fn ctrl_y_sets_pending_clipboard_copy_to_last_assistant_message() {
let mut state = test_state();
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
crate::dto::chat::message::Role::User,
"hi".to_string(),
));
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
crate::dto::chat::message::Role::Assistant,
"first reply".to_string(),
));
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
crate::dto::chat::message::Role::Tool,
"tool output".to_string(),
));
state.push_transcript(crate::app::state::rest::ChatMessageDisplay::new(
crate::dto::chat::message::Role::Assistant,
"second reply".to_string(),
));
handle_key(
KeyEvent::new(KeyCode::Char('y'), KeyModifiers::CONTROL),
&mut state,
);
assert_eq!(
state.misc.pending_clipboard_copy,
Some("second reply".to_string())
);
}
#[test]
fn ctrl_y_with_no_assistant_message_pushes_info_toast() {
let mut state = test_state();
handle_key(
KeyEvent::new(KeyCode::Char('y'), KeyModifiers::CONTROL),
&mut state,
);
assert!(state.misc.pending_clipboard_copy.is_none());
assert_eq!(state.misc.toasts.len(), 1);
}
}
-1
View File
@@ -1,4 +1,3 @@
//! Keyboard input handling and command parsing for the TUI.
pub mod command;
pub mod input;
+15 -1
View File
@@ -1,6 +1,5 @@
//! Chat message types shared across the DTO layer: `Role` and `ChatMessage`
//! with convenience constructors.
use serde::{Deserialize, Serialize};
/// The conversation participant who authored a message.
@@ -17,6 +16,21 @@ pub enum Role {
}
impl Role {
/// Return the role as a lowercase string.
pub fn as_str(&self) -> &'static str {
match self {
Role::User => "user",
Role::Assistant => "assistant",
Role::System => "system",
Role::Tool => "tool",
}
}
}
impl std::fmt::Display for Role {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(self.as_str())
}
}
/// A single message in a conversation, compatible with the OpenAI/Anthropic
-1
View File
@@ -1,4 +1,3 @@
//! Chat DTO submodules: message roles/content and tool-call structures.
pub mod message;
pub mod tool;
+11 -11
View File
@@ -7,7 +7,6 @@
//!
//! Why: kept separate from `dto::provider` because tool calls are a property
//! of a chat *message*, not of the request/response envelope.
use serde::{Deserialize, Serialize};
use serde_json::Value;
@@ -41,10 +40,7 @@ mod tests {
#[test]
fn repair_json_bracket_then_brace() {
// `[` opened first → `]` must close first, then `}`
assert_eq!(
repair_json("[[1, 2, {\"a\": 3"),
"[[1, 2, {\"a\": 3}]]"
);
assert_eq!(repair_json("[[1, 2, {\"a\": 3"), "[[1, 2, {\"a\": 3}]]");
}
#[test]
@@ -212,7 +208,8 @@ pub fn sanitize_tool_arguments(args: &Value) -> Value {
// Attempt 2: strip control chars (0x00-0x1F except \t, \n)
// that some LLM providers emit as literal bytes in JSON strings
// (e.g. multi-line commit messages), then retry.
let cleaned: String = s.chars()
let cleaned: String = s
.chars()
.filter(|&c| !c.is_control() || c == '\t' || c == '\n' || c == '\r')
.collect();
if cleaned.len() != s.len() {
@@ -225,20 +222,23 @@ pub fn sanitize_tool_arguments(args: &Value) -> Value {
}
}
// Attempt 3: repair truncated JSON and retry.
let input = if cleaned.len() == s.len() { s } else { &cleaned };
let input = if cleaned.len() == s.len() {
s
} else {
&cleaned
};
let repaired = repair_json(input);
match serde_json::from_str::<Value>(&repaired) {
Ok(v) => {
tracing::warn!(
"tool argument string was truncated — repaired successfully",
);
tracing::warn!("tool argument string was truncated — repaired successfully",);
v
}
Err(e2) => {
tracing::error!(
"tool argument is a JSON string but failed to parse. \
Wrapping in object. Error: {}. Raw (first 200): {}",
e2, s.chars().take(200).collect::<String>(),
e2,
s.chars().take(200).collect::<String>(),
);
serde_json::json!({"_raw": s, "_parse_error": e2.to_string()})
}
-1
View File
@@ -1,5 +1,4 @@
//! Data transfer objects shared across the app: chat messages/tool calls
//! and provider request/response/usage shapes.
pub mod chat;
pub mod provider;
-1
View File
@@ -1,5 +1,4 @@
//! Provider-facing DTOs: chat completion request, response, and usage/cost.
pub mod request;
pub mod response;
pub mod usage;
-1
View File
@@ -8,7 +8,6 @@
//! for reserved words like `type`) so no manual (de)serialization glue is
//! needed; optional fields use `skip_serializing_if` so unset knobs are
//! omitted rather than sent as `null`, matching provider expectations.
use serde::{Deserialize, Serialize};
use serde_json::Value;
-1
View File
@@ -6,7 +6,6 @@
//!
//! Why: separate from the streaming SSE path (see `app/runtime/stream/mod.rs`),
//! which parses incremental deltas rather than a single complete payload.
use serde::{Deserialize, Serialize};
/// Non-streaming chat completion response returned by the provider.
-1
View File
@@ -4,7 +4,6 @@
//! chunk when `stream_options.include_usage` is set, or the `usage` field of
//! a non-streaming `ChatResponse`) → surfaced to the TUI for cost/token
//! display.
use serde::{Deserialize, Serialize};
/// Token counts and optional cost breakdown for a single completion request.
+1 -2
View File
@@ -4,9 +4,8 @@
//! Flow: `IpcClient::connect_unix` opens a `Connection` (see `conn.rs`)
//! to the daemon's socket path → `send`/`receive` exchange framed JSON
//! messages (typically `ClientRequest`/`DaemonFrame` from `protocol.rs`).
use anyhow::Result;
use super::conn::Connection;
use anyhow::Result;
/// Client-side handle for the `--attach` process: wraps a `Connection`
/// to a daemon's Unix socket.
+2 -3
View File
@@ -6,10 +6,9 @@
//! writes it as one length-prefixed frame (`frame::write_frame`) →
//! `receive` reads one frame and deserializes it back to the caller's
//! type, propagating a clean peer-close as `Ok(None)`.
use std::os::unix::net::UnixStream;
use anyhow::Result;
use super::frame;
use anyhow::Result;
use std::os::unix::net::UnixStream;
/// A framed Unix-socket connection shared by client and server sides of
/// the IPC layer; each `send`/`receive` moves one length-prefixed JSON frame.
-1
View File
@@ -6,7 +6,6 @@
//! other mismatch is recorded wholesale → results accumulate into a
//! `StateDiff`'s `Vec<Change>`, built via `StateDiff::new`/`add_change`
//! and reset via `clear`.
use serde::{Deserialize, Serialize};
use serde_json::Value;
+7 -3
View File
@@ -1,4 +1,9 @@
#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]
#![allow(
clippy::cast_possible_truncation,
clippy::cast_sign_loss,
clippy::cast_precision_loss,
clippy::cast_possible_wrap
)]
//! Length-prefixed binary framing and JSON (de)serialization helpers for
//! the IPC wire protocol.
//!
@@ -10,9 +15,8 @@
//! Why: a fixed-size length prefix lets the reader know exactly how many
//! bytes to pull before attempting to parse, avoiding partial-JSON reads
//! over a stream socket.
use std::io::{Read, Write};
use anyhow::Result;
use std::io::{Read, Write};
/// Upper bound on a single frame's byte size (64 MiB), enforced on both
/// the write and read paths to bound memory use and reject malformed or
-1
View File
@@ -1,7 +1,6 @@
//! Unix-socket IPC layer used to connect a `--attach` TUI client to a
//! `--daemon` process: length-prefixed framing, connection wrapper,
//! client/server handles, and the wire protocol types.
pub mod client;
pub mod conn;
pub mod frame;
+1 -1
View File
@@ -9,7 +9,6 @@
//! Why: `StatePayload`/`MessageEntry`/`ToastEntry` are deliberately flat,
//! serializable projections of daemon-side state so the client can
//! redraw its TUI without sharing any in-process state with the daemon.
use serde::{Deserialize, Serialize};
/// Wire-serializable subset of `crossterm::event::KeyCode`, sent from
@@ -91,5 +90,6 @@ pub enum DaemonFrame {
StateUpdate(Box<StatePayload>),
StreamToken(String),
SystemNote { kind: String, message: String },
ClipboardCopy(String),
Closed,
}
+2 -3
View File
@@ -4,10 +4,9 @@
//! path (clearing any stale file left by a crashed prior daemon) →
//! `accept` blocks for the next client and wraps it as a `Connection`
//! (see `conn.rs`) for framed request/response traffic.
use std::os::unix::net::UnixListener;
use anyhow::Result;
use super::conn::Connection;
use anyhow::Result;
use std::os::unix::net::UnixListener;
/// Server-side handle for the `--daemon` process: listens on a Unix
/// socket and hands out `Connection`s to accepted clients.
-1
View File
@@ -6,7 +6,6 @@
//! callers populate/replace its fields as state changes →
//! `serialize_snapshot`/`deserialize_snapshot` move it to/from JSON bytes
//! for storage or IPC transport.
use serde::{Deserialize, Serialize};
use serde_json::Value;
+123 -41
View File
@@ -109,6 +109,7 @@ fn run_single_process() -> Result<()> {
&session_dir,
store.memory_dir,
);
state.spawn_mention_index_build();
state.sessions = model::session::Session::list(&store.base_dir);
@@ -119,6 +120,7 @@ fn run_single_process() -> Result<()> {
let mut stdout = io::stdout();
execute!(stdout, EnterAlternateScreen)?;
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
execute!(stdout, crossterm::event::EnableMouseCapture)?;
let backend = CrosstermBackend::new(stdout);
let mut terminal = Terminal::new(backend)?;
terminal.clear()?;
@@ -127,6 +129,7 @@ fn run_single_process() -> Result<()> {
let mut restore_stdout = io::stdout();
let _ = execute!(restore_stdout, crossterm::event::DisableBracketedPaste);
let _ = execute!(restore_stdout, crossterm::event::DisableMouseCapture);
let _ = execute!(restore_stdout, LeaveAlternateScreen);
let _ = disable_raw_mode();
@@ -394,6 +397,9 @@ fn handle_daemon_client(
running = false;
}
}
if let Some(text) = state.misc.pending_clipboard_copy.take() {
conn.send(&ipc::protocol::DaemonFrame::ClipboardCopy(text))?;
}
send_daemon_update(&mut conn, state)?;
}
None => {
@@ -437,6 +443,7 @@ fn run_daemon() -> Result<()> {
&session_dir,
store.memory_dir,
);
state.spawn_mention_index_build();
state.sessions = model::session::Session::list(&store.base_dir);
let _rt = tokio::runtime::Runtime::new()?;
@@ -473,6 +480,78 @@ fn run_daemon() -> Result<()> {
Ok(())
}
/// Set up the IPC client connection, terminal, and initial state for attach mode.
///
/// Flow: resolve socket path → connect → enable raw/alt mode → create state.
///
/// Return: (client, terminal, `client_state`) on success.
fn setup_attach_client(
session_id: &str,
) -> Result<(
ipc::client::IpcClient,
Terminal<CrosstermBackend<io::Stdout>>,
app::state::rest::AppStateRest,
)> {
let store = model::store::Store::new();
let socket_path = store.base_dir.join("run").join(format!("{session_id}.sock"));
let addr = socket_path.to_string_lossy().to_string();
let client = ipc::client::IpcClient::connect_unix(&addr)?;
enable_raw_mode()?;
let mut stdout = io::stdout();
execute!(stdout, EnterAlternateScreen)?;
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
execute!(stdout, crossterm::event::EnableMouseCapture)?;
let backend = CrosstermBackend::new(stdout);
let mut terminal = Terminal::new(backend)?;
terminal.clear()?;
let workspace_roots = vec![std::env::current_dir()?];
let session_dir = store.base_dir.join("sessions").join(session_id);
std::fs::create_dir_all(&session_dir)?;
let mut client_state = app::state::rest::AppStateRest::new(
workspace_roots,
&session_dir,
store.memory_dir,
);
client_state.session_id = session_id.to_string();
Ok((client, terminal, client_state))
}
/// Process a single daemon frame from the IPC channel, updating state accordingly.
fn handle_daemon_frame(
client_state: &mut app::state::rest::AppStateRest,
frame: Option<ipc::protocol::DaemonFrame>,
) {
match frame {
Some(ipc::protocol::DaemonFrame::StateUpdate(payload)) => {
apply_client_update(client_state, *payload);
}
Some(ipc::protocol::DaemonFrame::StreamToken(_token)) => {}
Some(ipc::protocol::DaemonFrame::SystemNote { kind: _, message }) => {
client_state.push_toast(
app::state::types::Toast::new(
app::state::types::ToastKind::Info,
message,
),
);
}
Some(ipc::protocol::DaemonFrame::ClipboardCopy(text)) => {
let _ = write_osc52(&mut io::stdout(), &text);
client_state.push_toast(
app::state::types::Toast::new(
app::state::types::ToastKind::Success,
"Copied to clipboard".to_string(),
),
);
}
Some(ipc::protocol::DaemonFrame::Closed) | None => {
client_state.quit = true;
}
}
}
/// Run zesdex as a TUI-only client attached to an existing daemon session.
///
/// Flow: connect to the daemon's Unix socket → enter raw mode/alternate
@@ -490,30 +569,7 @@ fn run_attach(session_id: &str) -> Result<()> {
use crossterm::event::{Event, KeyCode, KeyEventKind, KeyModifiers, MouseEventKind};
use ipc::protocol::ClientRequest;
let store = model::store::Store::new();
let socket_path = store.base_dir.join("run").join(format!("{session_id}.sock"));
let addr = socket_path.to_string_lossy().to_string();
let mut client = ipc::client::IpcClient::connect_unix(&addr)?;
enable_raw_mode()?;
let mut stdout = io::stdout();
execute!(stdout, EnterAlternateScreen)?;
execute!(stdout, crossterm::event::EnableBracketedPaste)?;
let backend = CrosstermBackend::new(stdout);
let mut terminal = Terminal::new(backend)?;
terminal.clear()?;
let workspace_roots = vec![std::env::current_dir()?];
let session_dir = store.base_dir.join("sessions").join(session_id);
std::fs::create_dir_all(&session_dir)?;
let mut client_state = app::state::rest::AppStateRest::new(
workspace_roots,
&session_dir,
store.memory_dir,
);
client_state.session_id = session_id.to_string();
let (mut client, mut terminal, mut client_state) = setup_attach_client(session_id)?;
let _rt = tokio::runtime::Runtime::new()?;
loop {
@@ -567,23 +623,10 @@ fn run_attach(session_id: &str) -> Result<()> {
client.send(&ClientRequest::Tick)?;
}
match client.receive::<ipc::protocol::DaemonFrame>()? {
Some(ipc::protocol::DaemonFrame::StateUpdate(payload)) => {
apply_client_update(&mut client_state, *payload);
}
Some(ipc::protocol::DaemonFrame::StreamToken(_token)) => {}
Some(ipc::protocol::DaemonFrame::SystemNote { kind: _, message }) => {
client_state.push_toast(
app::state::types::Toast::new(
app::state::types::ToastKind::Info,
message,
),
);
}
Some(ipc::protocol::DaemonFrame::Closed) | None => {
client_state.quit = true;
}
}
handle_daemon_frame(
&mut client_state,
client.receive::<ipc::protocol::DaemonFrame>()?,
);
terminal.draw(|f| {
view::draw(f, &client_state);
@@ -591,6 +634,7 @@ fn run_attach(session_id: &str) -> Result<()> {
}
let _ = execute!(io::stdout(), crossterm::event::DisableBracketedPaste);
let _ = execute!(io::stdout(), crossterm::event::DisableMouseCapture);
let _ = execute!(io::stdout(), LeaveAlternateScreen);
let _ = disable_raw_mode();
@@ -617,11 +661,27 @@ fn run_loop(
let _ = disable_raw_mode();
let _ = execute!(io::stdout(), crossterm::event::DisableBracketedPaste);
let _ = execute!(io::stdout(), crossterm::event::DisableMouseCapture);
let _ = execute!(io::stdout(), LeaveAlternateScreen);
}
result
}
/// Write text to the system clipboard via an OSC52 terminal escape sequence.
///
/// Flow: base64-encode `text` -> wrap in `\x1b]52;c;<b64>\x07` -> write and
/// flush to `stdout`.
///
/// Why: OSC52 asks the terminal emulator itself to set the clipboard, so no
/// OS-level clipboard library (X11/Wayland/win32) is needed. Terminals that
/// don't support it silently ignore the sequence.
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
use base64::Engine as _;
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
write!(stdout, "\x1b]52;c;{b64}\x07")?;
stdout.flush()
}
/// The core single-process render/input loop.
///
/// Flow: until `state.quit` → drain expired toasts → draw the frame →
@@ -660,6 +720,13 @@ fn run_loop_inner(
for action in actions {
apply_action(state, action);
}
if let Some(text) = state.misc.pending_clipboard_copy.take() {
let _ = write_osc52(&mut io::stdout(), &text);
state.push_toast(app::state::types::Toast::new(
app::state::types::ToastKind::Success,
"Copied to clipboard".to_string(),
));
}
}
}
Event::Paste(text) => {
@@ -694,3 +761,18 @@ fn run_loop_inner(
terminal.clear()?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::write_osc52;
#[test]
fn write_osc52_formats_the_escape_sequence() {
let mut buf: Vec<u8> = Vec::new();
write_osc52(&mut buf, "hello").unwrap();
use base64::Engine as _;
let b64 = base64::engine::general_purpose::STANDARD.encode("hello");
let expected = format!("\x1b]52;c;{b64}\x07");
assert_eq!(String::from_utf8(buf).unwrap(), expected);
}
}
-1
View File
@@ -1,5 +1,4 @@
//! Hardcoded built-in subagent definitions (coder, reviewer, researcher, planner).
use crate::app::subagent::spawn::AgentDefinition;
/// Build the fixed list of built-in agent definitions shipped with zesdex.
-1
View File
@@ -1,6 +1,5 @@
//! Load, save, and remove user-defined agent definitions stored globally
//! (under the store's `agents/` directory), independent of any session.
use crate::app::subagent::spawn::AgentDefinition;
/// Load all globally-registered agent definitions from disk.
-1
View File
@@ -1,6 +1,5 @@
//! Agent definition sources: built-in defaults, global (user-wide), and
//! per-session overrides.
pub mod builtin;
pub mod global;
pub mod session;
-1
View File
@@ -1,6 +1,5 @@
//! Load, save, add, and remove agent definitions scoped to a single
//! session (`<session_dir>/agents.json`).
use std::path::Path;
use crate::app::subagent::spawn::AgentDefinition;
+43 -31
View File
@@ -1,6 +1,5 @@
//! Application-level configuration: LLM providers, model roles, and defaults,
//! persisted to `app_config.json` in the store directory.
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
@@ -38,26 +37,35 @@ pub struct ModelRole {
impl Default for AppConfig {
fn default() -> Self {
let mut providers = HashMap::new();
providers.insert("zen".to_string(), ProviderConfig {
api_base: "https://opencode.ai/zen/v1".to_string(),
api_key_env: Some("API_KEY".to_string()),
default_model: Some("deepseek-v4-flash-free".to_string()),
default_api_key: None,
});
providers.insert("router".to_string(), ProviderConfig {
api_base: "https://9router.asepharyana.my.id/v1".to_string(),
api_key_env: Some("ROUTER_API_KEY".to_string()),
default_model: Some("claude-opus-4-8".to_string()),
default_api_key: None,
});
providers.insert(
"zen".to_string(),
ProviderConfig {
api_base: "https://opencode.ai/zen/v1".to_string(),
api_key_env: Some("API_KEY".to_string()),
default_model: Some("deepseek-v4-flash-free".to_string()),
default_api_key: None,
},
);
providers.insert(
"router".to_string(),
ProviderConfig {
api_base: "https://9router.asepharyana.my.id/v1".to_string(),
api_key_env: Some("ROUTER_API_KEY".to_string()),
default_model: Some("claude-opus-4-8".to_string()),
default_api_key: None,
},
);
let mut model_roles = HashMap::new();
model_roles.insert("default".to_string(), ModelRole {
provider: "zen".to_string(),
model: "deepseek-v4-flash-free".to_string(),
max_tokens: None,
context_window: None,
temperature: Some(0.7),
});
model_roles.insert(
"default".to_string(),
ModelRole {
provider: "zen".to_string(),
model: "deepseek-v4-flash-free".to_string(),
max_tokens: None,
context_window: None,
temperature: Some(0.7),
},
);
AppConfig {
providers,
model_roles,
@@ -89,7 +97,8 @@ impl AppConfig {
Err(e) => {
tracing::warn!(
"warning: failed to parse config file '{}': {}. Loading defaults.",
path.display(), e
path.display(),
e
);
Self::default()
}
@@ -103,7 +112,9 @@ impl AppConfig {
}
// Auto-detect provider from ~/.claude/settings.json
if let Some(claude_provider) = detect_claude_settings_provider() {
cfg.providers.entry("claude".to_string()).or_insert(claude_provider);
cfg.providers
.entry("claude".to_string())
.or_insert(claude_provider);
// Register known Claude models as named model roles
let claude_models = [
("claude-opus-4-8", "claude-opus-4-8"),
@@ -111,13 +122,15 @@ impl AppConfig {
("claude-haiku-4-5", "claude-haiku-4-5-20251001"),
];
for (role_name, model_name) in &claude_models {
cfg.model_roles.entry(role_name.to_string()).or_insert(ModelRole {
provider: "claude".to_string(),
model: model_name.to_string(),
max_tokens: Some(8192),
context_window: Some(200_000),
temperature: Some(0.7),
});
cfg.model_roles
.entry(role_name.to_string())
.or_insert(ModelRole {
provider: "claude".to_string(),
model: model_name.to_string(),
max_tokens: Some(8192),
context_window: Some(200_000),
temperature: Some(0.7),
});
}
// Set as default provider only if user hasn't picked a custom default
if cfg.default_provider == defaults.default_provider {
@@ -154,8 +167,7 @@ struct ClaudeSettings {
/// than through its settings file, so reading only the file misses them.
fn detect_claude_settings_provider() -> Option<ProviderConfig> {
// Prefer the file, then fall back to env vars.
let (base_url, key) = claude_credentials_from_file()
.or_else(claude_credentials_from_env)?;
let (base_url, key) = claude_credentials_from_file().or_else(claude_credentials_from_env)?;
Some(ProviderConfig {
api_base: base_url,
// Keep the env-var name so runtime env overrides still work.
-1
View File
@@ -1,6 +1,5 @@
//! In-memory conversation state: message history plus the system prompt and
//! model parameters used to drive the LLM.
use serde::{Deserialize, Serialize};
/// A single conversation's message history and generation settings.
+5 -3
View File
@@ -1,6 +1,5 @@
//! Append-only JSONL edit log recording every file mutation made by tools,
//! for audit and undo/history purposes.
use serde::{Deserialize, Serialize};
/// A single recorded file edit: which tool made it, to which path, why,
@@ -44,7 +43,9 @@ impl EditLog {
/// regardless of the in-memory limit.
fn load_from_disk(path: &std::path::Path) -> Vec<EditLogEntry> {
use std::io::{BufRead, BufReader};
let Ok(file) = std::fs::File::open(path) else { return Vec::new() };
let Ok(file) = std::fs::File::open(path) else {
return Vec::new();
};
let reader = BufReader::new(file);
let mut entries: Vec<EditLogEntry> = Vec::new();
for line in reader.lines() {
@@ -151,7 +152,8 @@ mod tests {
bytes_delta: 10 + i,
origin: "main".to_string(),
session_id: "sess-1".to_string(),
}).unwrap();
})
.unwrap();
}
assert_eq!(log.len(), 5);
assert_eq!(log.entries[0].reason, "reason 0");
+87 -26
View File
@@ -1,8 +1,7 @@
//! Long-term agent memory: markdown files with YAML-ish frontmatter storing
//! lessons/references, plus slugified filenames and export/import helpers.
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use std::path::{Path, PathBuf};
/// A single memory entry (lesson, reference, etc.) with frontmatter
/// metadata and free-form markdown content.
@@ -72,15 +71,30 @@ impl Memory {
///
/// Return: `Ok(())` on success, or an `io::Error` from directory
/// creation, the temp write, or the rename.
#[allow(clippy::suspicious_open_options)]
pub fn write(&self, memory_dir: &Path) -> std::io::Result<()> {
let path = Self::path(memory_dir, &self.name);
let parent = path.parent().unwrap();
std::fs::create_dir_all(parent)?;
let outcome_line = self.outcome.as_ref().map(|o| format!("outcome: {o}")).unwrap_or_default();
let scope_line = self.scope.as_ref().map(|s| format!("scope: {s}")).unwrap_or_default();
let before_line = self.before_snippet.as_ref().map(|s| format!("before: {s}")).unwrap_or_default();
let after_line = self.after_snippet.as_ref().map(|s| format!("after: {s}")).unwrap_or_default();
let outcome_line = self
.outcome
.as_ref()
.map(|o| format!("outcome: {o}"))
.unwrap_or_default();
let scope_line = self
.scope
.as_ref()
.map(|s| format!("scope: {s}"))
.unwrap_or_default();
let before_line = self
.before_snippet
.as_ref()
.map(|s| format!("before: {s}"))
.unwrap_or_default();
let after_line = self
.after_snippet
.as_ref()
.map(|s| format!("after: {s}"))
.unwrap_or_default();
let prov_line = if self.provenances.is_empty() {
String::new()
} else {
@@ -99,6 +113,7 @@ impl Memory {
use std::io::Write;
let mut f = std::fs::OpenOptions::new()
.create(true)
.truncate(true)
.write(true)
.open(&tmp)?;
f.write_all(content.as_bytes())?;
@@ -139,7 +154,10 @@ impl Memory {
let content = content.strip_prefix("---\n").unwrap_or(content);
let parts: Vec<&str> = content.splitn(2, "\n---\n").collect();
if parts.len() < 2 {
return Err(std::io::Error::new(std::io::ErrorKind::InvalidData, "missing frontmatter"));
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidData,
"missing frontmatter",
));
}
let front: std::collections::HashMap<String, String> = parts[0]
.lines()
@@ -153,16 +171,34 @@ impl Memory {
name: front.get("name").cloned().unwrap_or_default(),
description: front.get("description").cloned().unwrap_or_default(),
content: body,
kind: front.get("kind").cloned().unwrap_or_else(|| "reference".to_string()),
created_at: front.get("created_at").and_then(|v| v.parse().ok()).unwrap_or(0),
updated_at: front.get("updated_at").and_then(|v| v.parse().ok()).unwrap_or(0),
kind: front
.get("kind")
.cloned()
.unwrap_or_else(|| "reference".to_string()),
created_at: front
.get("created_at")
.and_then(|v| v.parse().ok())
.unwrap_or(0),
updated_at: front
.get("updated_at")
.and_then(|v| v.parse().ok())
.unwrap_or(0),
outcome: front.get("outcome").cloned().filter(|s| !s.is_empty()),
lifecycle: front.get("lifecycle").cloned().unwrap_or_else(|| "new".to_string()),
lifecycle: front
.get("lifecycle")
.cloned()
.unwrap_or_else(|| "new".to_string()),
scope: front.get("scope").cloned().filter(|s| !s.is_empty()),
before_snippet: front.get("before").cloned().filter(|s| !s.is_empty()),
after_snippet: front.get("after").cloned().filter(|s| !s.is_empty()),
provenances: front.get("provenances").cloned()
.map(|s| s.split(", ").map(std::string::ToString::to_string).collect())
provenances: front
.get("provenances")
.cloned()
.map(|s| {
s.split(", ")
.map(std::string::ToString::to_string)
.collect()
})
.unwrap_or_default(),
})
}
@@ -186,13 +222,17 @@ impl Memory {
/// Return: slugs (without extension); empty `Vec` if the directory
/// can't be read.
pub fn list(memory_dir: &Path) -> Vec<String> {
let Ok(entries) = std::fs::read_dir(memory_dir) else { return Vec::new() };
let Ok(entries) = std::fs::read_dir(memory_dir) else {
return Vec::new();
};
entries
.filter_map(std::result::Result::ok)
.filter(|e| e.path().extension().is_some_and(|x| x == "md"))
.filter_map(|e| {
let name = e.file_name().to_string_lossy().to_string();
if name == "MEMORY.md" { return None; }
if name == "MEMORY.md" {
return None;
}
let slug = name.strip_suffix(".md")?.to_string();
Some(slug)
})
@@ -209,11 +249,22 @@ impl Memory {
/// Why: leading-dot stripping specifically blocks accidental hidden
/// files and `..`-style traversal attempts embedded in `raw`.
pub fn slug_path(memory_dir: &Path, raw: &str) -> PathBuf {
let clean: String = raw.chars()
.map(|c| if c.is_ascii_alphanumeric() || c == '.' || c == '-' { c } else { '-' })
let clean: String = raw
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '.' || c == '-' {
c
} else {
'-'
}
})
.collect();
let clean = clean.trim_start_matches('.').to_string();
memory_dir.join(if clean.is_empty() { "memory.md" } else { &clean })
memory_dir.join(if clean.is_empty() {
"memory.md"
} else {
&clean
})
}
/// Export all memories in `memory_dir` to a single JSON file.
@@ -227,11 +278,11 @@ pub fn slug_path(memory_dir: &Path, raw: &str) -> PathBuf {
#[cfg(test)]
pub fn export_lessons(memory_dir: &Path, output: &Path) -> std::io::Result<()> {
let names = Memory::list(memory_dir);
let lessons: Vec<Memory> = names.iter()
let lessons: Vec<Memory> = names
.iter()
.filter_map(|n| Memory::read(memory_dir, n).ok())
.collect();
let data = serde_json::to_string_pretty(&lessons)
.map_err(std::io::Error::other)?;
let data = serde_json::to_string_pretty(&lessons).map_err(std::io::Error::other)?;
// Write to temp, fsync, then rename for crash-safe export
let tmp = output.with_extension("json.tmp");
std::fs::write(&tmp, data)?;
@@ -259,7 +310,8 @@ pub fn import_lessons(memory_dir: &Path, input: &Path) -> std::io::Result<usize>
let data = std::fs::read_to_string(input)?;
let lessons: Vec<Memory> = serde_json::from_str(&data)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))?;
let existing: std::collections::HashSet<String> = Memory::list(memory_dir).into_iter().collect();
let existing: std::collections::HashSet<String> =
Memory::list(memory_dir).into_iter().collect();
let mut imported = 0;
for lesson in &lessons {
let slug = Memory::slugify(&lesson.name).unwrap_or_default();
@@ -282,12 +334,18 @@ mod tests {
#[test]
fn test_slugify_basic() {
assert_eq!(Memory::slugify("Hello World"), Some("hello-world".to_string()));
assert_eq!(
Memory::slugify("Hello World"),
Some("hello-world".to_string())
);
}
#[test]
fn test_slugify_special_chars() {
assert_eq!(Memory::slugify("Use & Avoid! @#$"), Some("use-avoid".to_string()));
assert_eq!(
Memory::slugify("Use & Avoid! @#$"),
Some("use-avoid".to_string())
);
}
#[test]
@@ -375,7 +433,10 @@ mod tests {
};
mem.write(&dir).unwrap();
let names = Memory::list(&dir);
assert!(names.contains(&"alpha".to_string()), "list should contain 'alpha', got: {names:?}");
assert!(
names.contains(&"alpha".to_string()),
"list should contain 'alpha', got: {names:?}"
);
let _ = std::fs::remove_dir_all(&dir);
}
-1
View File
@@ -1,6 +1,5 @@
//! Persistence and domain model layer: sessions, conversations, memory,
//! message log (`SQLite`), edit log, and app/settings config.
pub mod app_config;
pub mod editlog;
pub mod memory;
+16 -23
View File
@@ -1,8 +1,7 @@
//! Binary blob storage in the message-log `SQLite` database (e.g. images,
//! attachments), keyed by session id and an arbitrary blob key.
use rusqlite::{Connection, params};
use anyhow::Result;
use rusqlite::{params, Connection};
/// Insert or overwrite a blob for a session under `blob_key`.
///
@@ -10,7 +9,13 @@ use anyhow::Result;
/// keyed on `(session_id, blob_key)`.
///
/// Return: `Ok(())` on success, or the underlying `SQLite` error.
pub fn store_blob(conn: &Connection, session_id: &str, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()> {
pub fn store_blob(
conn: &Connection,
session_id: &str,
blob_key: &str,
data: &[u8],
mime_type: Option<&str>,
) -> Result<()> {
let created_at = chrono::Utc::now().timestamp_millis();
conn.execute(
"INSERT OR REPLACE INTO blobs (session_id, blob_key, data, mime_type, created_at) VALUES (?1, ?2, ?3, ?4, ?5)",
@@ -23,7 +28,11 @@ pub fn store_blob(conn: &Connection, session_id: &str, blob_key: &str, data: &[u
///
/// Return: `Ok(Some(data))` if found, `Ok(None)` if no matching row
/// exists, `Err` for any other `SQLite` failure.
pub fn retrieve_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Result<Option<Vec<u8>>> {
pub fn retrieve_blob(
conn: &Connection,
session_id: &str,
blob_key: &str,
) -> Result<Option<Vec<u8>>> {
let result = conn.query_row(
"SELECT data FROM blobs WHERE session_id = ?1 AND blob_key = ?2",
params![session_id, blob_key],
@@ -36,30 +45,14 @@ pub fn retrieve_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Res
}
}
/// Delete a blob for a session by key.
///
/// Return: `Ok(true)` if a row was deleted, `Ok(false)` if no matching
/// row existed.
#[allow(dead_code)]
pub fn delete_blob(conn: &Connection, session_id: &str, blob_key: &str) -> Result<bool> {
let rows = conn.execute(
"DELETE FROM blobs WHERE session_id = ?1 AND blob_key = ?2",
params![session_id, blob_key],
)?;
Ok(rows > 0)
}
/// List all blob keys stored for a session, oldest first.
///
/// Return: `Ok(Vec<String>)` of keys ordered by `created_at`, or the
/// underlying `SQLite` error.
pub fn list_blob_keys(conn: &Connection, session_id: &str) -> Result<Vec<String>> {
let mut stmt = conn.prepare(
"SELECT blob_key FROM blobs WHERE session_id = ?1 ORDER BY created_at ASC"
)?;
let rows = stmt.query_map(params![session_id], |row| {
row.get::<_, String>(0)
})?;
let mut stmt =
conn.prepare("SELECT blob_key FROM blobs WHERE session_id = ?1 ORDER BY created_at ASC")?;
let rows = stmt.query_map(params![session_id], |row| row.get::<_, String>(0))?;
let mut keys = Vec::new();
for row in rows {
keys.push(row?);
-1
View File
@@ -1,6 +1,5 @@
//! SQLite-backed message log: per-session `messages.sqlite` storing chat
//! messages, blobs, and archive/summary metadata.
pub mod blobs;
pub mod query;
pub mod schema;
+6 -6
View File
@@ -1,8 +1,7 @@
//! Insert queries against the message log's `messages` table.
use rusqlite::{Connection, params};
use anyhow::Result;
use crate::dto::chat::message::{ChatMessage, Role};
use anyhow::Result;
use rusqlite::{params, Connection};
/// Insert a chat message into the session's message log.
///
@@ -15,9 +14,10 @@ pub fn insert_message(conn: &Connection, session_id: &str, msg: &ChatMessage) ->
let content = msg.content.as_deref();
let tool_call_id = msg.tool_call_id.as_deref();
let tool_name = msg.name.as_deref();
let tool_arguments = msg.tool_calls.as_ref().map(|calls| {
serde_json::to_string(calls).unwrap_or_default()
});
let tool_arguments = msg
.tool_calls
.as_ref()
.map(|calls| serde_json::to_string(calls).unwrap_or_default());
let created_at = chrono::Utc::now().timestamp_millis();
let role_str = match msg.role {
Role::User => "user",

Some files were not shown because too many files have changed in this diff Show More