feat: remove obsolete design documents for clipboard OSC52, diff view, file mention, context compaction, and add development guide
This commit is contained in:
asepharyana
@@ -780,7 +780,7 @@ pub fn resolve_context_window(
|
||||
return max as usize;
|
||||
}
|
||||
}
|
||||
128_000
|
||||
256_000
|
||||
}
|
||||
|
||||
/// Count tokens using tiktoken, fall back to character estimation.
|
||||
|
||||
+104
-51
@@ -1,65 +1,118 @@
|
||||
# Architecture Overview
|
||||
# Arsitektur Sistem Zesdex
|
||||
|
||||
## System Layout
|
||||
## Gambaran Umum
|
||||
|
||||
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
|
||||
Zesdex adalah autonomous AI coding agent berbasis TUI, dibangun dengan **Rust** menggunakan clean architecture berlapis. LLM client dibungkus dalam tool-use harness dengan 37+ built-in tools — file ops, git, shell, LSP, MCP, subagent orchestration, dan lainnya.
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Process Mode │
|
||||
│ Single-Process ─── Daemon (background) ─── Attach (client) │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│ IPC (Unix domain socket)
|
||||
zesdex/
|
||||
├── apps/
|
||||
│ ├── domain/ # Layer 1: Pure entities, traits, value objects
|
||||
│ ├── application/ # Layer 2: Use-case services
|
||||
│ ├── infrastructure/ # Layer 3: Semua I/O (LLM, DB, tools, MCP, LSP)
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # Ratatui terminal UI
|
||||
│ │ ├── api/ # Axum REST API
|
||||
│ │ ├── daemon/ # Unix socket daemon
|
||||
│ │ ├── ws/ # WebSocket server
|
||||
│ │ ├── grpc/ # gRPC server
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point & dispatcher
|
||||
│ └── bootstrap/ # Initial data seeder
|
||||
```
|
||||
|
||||
## Dependency Graph Antar Layer
|
||||
|
||||
```
|
||||
gateway/bootstrap
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ src/main.rs │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
||||
│ │ Controller │──▶│ Runtime │──▶│ View │ │
|
||||
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
|
||||
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────▼────────┐ │
|
||||
│ │ Harness │ │
|
||||
│ │ (tool dispatch)│ │
|
||||
│ └───────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────┼─────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ Tools │ │ Subagents │ │ Workflow │ │
|
||||
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
|
||||
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
|
||||
│ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
interfaces/* (tui, api, daemon, ws, grpc, web)
|
||||
│
|
||||
▼
|
||||
infrastructure ─── implements ──▶ domain ports
|
||||
│
|
||||
▼
|
||||
application ─── depends on ──▶ domain traits
|
||||
│
|
||||
▼
|
||||
domain (zero framework deps: serde, chrono, uuid only)
|
||||
```
|
||||
|
||||
> **Aturan**: layer bawah tidak boleh tahu tentang layer atas. `domain` tidak import apapun dari `infrastructure` atau `interfaces`.
|
||||
|
||||
## Process Modes
|
||||
|
||||
| Mode | Description |
|
||||
|------|-------------|
|
||||
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
|
||||
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
|
||||
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
|
||||
| Mode | Flag | Keterangan |
|
||||
|------|------|------------|
|
||||
| **TUI** | *(default)* | TUI + agent loop dalam satu proses |
|
||||
| **Daemon** | `--daemon` | Agent berjalan di background via IPC socket |
|
||||
| **Attach** | `--attach <id>` | TUI terhubung ke daemon yang sedang berjalan |
|
||||
| **REST API** | `--api` | HTTP server (default port 8080) |
|
||||
| **WebSocket** | `--ws` | WS server (default port 8081) |
|
||||
| **gRPC** | `--grpc` | gRPC server (default port 50051) |
|
||||
| **Web** | `--web` | Static web frontend (default port 3000) |
|
||||
|
||||
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
|
||||
## Alur Data (Single-Process Mode)
|
||||
|
||||
## Data Flow
|
||||
```
|
||||
Keyboard/Event
|
||||
│
|
||||
▼
|
||||
controller/input.rs: handle_key()
|
||||
│ returns Vec<Action>
|
||||
▼
|
||||
action.rs: apply_action(&mut AppStateRest)
|
||||
│ state dimutasi in-place
|
||||
├──▶ turn.rs: spawn_agent_turn() ──▶ background thread
|
||||
│ │
|
||||
│ ├─ LLM call (blocking reqwest)
|
||||
│ ├─ tool execution (Tool trait)
|
||||
│ └─ push TurnEvent ke queue
|
||||
│
|
||||
▼
|
||||
run.rs: run_loop_inner()
|
||||
│ drain TurnEvent setiap tick
|
||||
│ skip render jika dirty=false
|
||||
▼
|
||||
view/: draw frame ke terminal (ratatui)
|
||||
```
|
||||
|
||||
1. **Input** → `controller/input.rs` handles key events and autocomplete
|
||||
2. **Dispatch** → `app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
|
||||
3. **LLM Stream** → `app/runtime/stream/mod.rs` parses SSE chunks into typed events
|
||||
4. **Tool Execution** → `app/harness.rs` gates and runs tool calls via the `Tool` trait
|
||||
5. **Rendering** → `view/` modules read `AppStateRest` and render via ratatui
|
||||
## File Kunci
|
||||
|
||||
## Key Files
|
||||
| File | Peran |
|
||||
|------|-------|
|
||||
| `apps/gateway/src/main.rs` | CLI entry point, parse args, dispatch ke mode |
|
||||
| `apps/interfaces/tui/src/state.rs` | `AppStateRest` — single source of truth state TUI |
|
||||
| `apps/interfaces/tui/src/action.rs` | `apply_action()` — satu-satunya tempat state dimutasi |
|
||||
| `apps/interfaces/tui/src/run.rs` | Event loop: render → poll → handle → tick |
|
||||
| `apps/interfaces/tui/src/turn.rs` | Spawn agent turn di background thread |
|
||||
| `apps/interfaces/tui/src/view/mod.rs` | Top-level render pipeline + `pre_render` hook |
|
||||
| `apps/infrastructure/src/llm/` | LLM client (streaming + non-streaming) |
|
||||
| `apps/infrastructure/src/tools/` | 37 tool implementations |
|
||||
| `apps/domain/src/core/` | Entity inti: `ChatMessage`, `Role`, `Store`, `Tool` trait |
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
|
||||
| `src/app/state/rest.rs` | Single source-of-truth state struct |
|
||||
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
|
||||
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
|
||||
| `src/app/harness.rs` | Tool harness with safety gating |
|
||||
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
|
||||
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
|
||||
| `src/view/mod.rs` | TUI render pipeline |
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
```
|
||||
┌──────────┐ Unix domain socket ┌──────────┐
|
||||
│ Client │ ◄──────────────────► │ Daemon │
|
||||
│ (TUI) │ [4-byte len][JSON] │ (agent) │
|
||||
└──────────┘ └──────────┘
|
||||
|
||||
Client ──Action──▶ Daemon (apply_action → state mutasi)
|
||||
Daemon ──StatePayload──▶ Client (render snapshot)
|
||||
```
|
||||
|
||||
## Lints & Kualitas Kode
|
||||
|
||||
Semua workspace crate menerapkan lint ketat di `Cargo.toml`:
|
||||
- `unused`, `dead_code`, `unreachable_code` → **deny**
|
||||
- `unused_imports`, `unused_variables`, `unused_mut` → **deny**
|
||||
- `clippy::all` + `clippy::pedantic` → **warn**
|
||||
|
||||
## Release Profile
|
||||
|
||||
`opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"`, `strip="symbols"`
|
||||
|
||||
+108
-47
@@ -1,68 +1,129 @@
|
||||
# Backend Architecture
|
||||
# Backend & Infrastructure
|
||||
|
||||
## Provider Layer
|
||||
Semua implementasi I/O ada di `apps/infrastructure/src/`. Layer ini mengimplementasikan port/trait yang didefinisikan di `apps/domain/`.
|
||||
|
||||
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
|
||||
## LLM Client (`infrastructure/src/llm/`)
|
||||
|
||||
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
|
||||
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
|
||||
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
|
||||
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
|
||||
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
|
||||
Wrapper di atas provider OpenAI-compatible:
|
||||
|
||||
## IPC (Inter-Process Communication)
|
||||
- **`provider/`** — `LlmClient`: HTTP client dengan `reqwest::blocking` (sync) untuk agent turn, dan async streaming untuk preview
|
||||
- **Request/Response** — `ChatMessage`, `ChatCompletionRequest`, `ChatCompletionResponse` dengan support tool calls
|
||||
- **Streaming** — SSE event parser untuk streaming response
|
||||
- **Usage tracking** — `tokens_in`, `tokens_out`, `last_tokens_in`, `last_tokens_out` per panggilan
|
||||
- **Provider defaults** — DeepSeek v4 flash free via OpenCode AI proxy (default)
|
||||
|
||||
The daemon-client protocol in `src/ipc/`:
|
||||
|
||||
- **Transport**: Unix domain sockets
|
||||
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
|
||||
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
|
||||
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
|
||||
|
||||
Flow:
|
||||
```
|
||||
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
|
||||
│
|
||||
└──StatePayload──▶ Client (render)
|
||||
```rust
|
||||
// Contoh penggunaan di turn.rs
|
||||
let result = client.chat_with_tools_non_streaming(
|
||||
&mut messages,
|
||||
Some(tool_defs),
|
||||
Some(4096), // max_tokens
|
||||
Some(0.7), // temperature
|
||||
None, // abort flag
|
||||
);
|
||||
```
|
||||
|
||||
## Workflow Engine
|
||||
## Tool System (`infrastructure/src/tools/`)
|
||||
|
||||
Located in `src/app/workflow/`:
|
||||
37 tool yang mengimplementasikan trait `Tool` dari domain:
|
||||
|
||||
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
|
||||
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
|
||||
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
|
||||
```rust
|
||||
pub trait Tool: Send + Sync {
|
||||
fn name(&self) -> &'static str;
|
||||
fn description(&self) -> &'static str;
|
||||
fn parameters(&self) -> Value; // JSON Schema
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String>;
|
||||
}
|
||||
```
|
||||
|
||||
## MCP (Model Context Protocol)
|
||||
### Kategori Tool
|
||||
|
||||
`src/app/mcp/manager.rs` manages MCP client connections:
|
||||
| Kategori | Tools |
|
||||
|----------|-------|
|
||||
| **File System** | `read`, `write`, `edit`, `delete`, `dir_list`, `dir_cache_update` |
|
||||
| **Shell** | `bash`, `bash_interactive`, `bash_kill`, `bash_output` |
|
||||
| **Git** | `git_operator`, `git_cred`, `git_worktree` |
|
||||
| **Search** | `search`, `grep`, `glob`, `semantic_search` |
|
||||
| **LSP** | `lsp_connect`, `lsp_hover`, `lsp_completion`, `lsp_definition`, `lsp_references`, `lsp_diagnostics`, `lsp_disconnect` |
|
||||
| **Memory** | `remember`, `recall`, `forget` |
|
||||
| **Workflow** | `spawn_agents`, `spawn_pipeline`, `plan`, `sequential_think`, `hive_mind` |
|
||||
| **Utility** | `todo_write`, `todo_finish`, `pong`, `cd` |
|
||||
| **Background** | `bash_bg_cancel`, `bash_bg_status`, `bash_bg_list` |
|
||||
|
||||
- Uses the `rmcp` crate for the MCP protocol
|
||||
- Supports stdio-based transport (child process) and streamable HTTP
|
||||
- Tool discovery via `list_tools()` and dynamic tool registration
|
||||
`ToolCtx` berisi:
|
||||
- `session_dir: PathBuf` — direktori sesi aktif
|
||||
- `workspaces: Vec<PathBuf>` — root workspace yang dibuka
|
||||
|
||||
## LSP Integration
|
||||
## Background Shell Jobs (`infrastructure/src/bgbash/`)
|
||||
|
||||
`src/app/lsp/` provides Language Server Protocol support:
|
||||
Manajemen proses shell jangka panjang:
|
||||
- **Spawn** dengan Unix process groups (untuk kill seluruh tree)
|
||||
- **Output buffering** — collect stdout/stderr secara async
|
||||
- **Cancel/status/list** — kontrol via tool calls
|
||||
- **Progress monitoring** — track state: `Running`, `Completed`, `Failed`
|
||||
|
||||
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
|
||||
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
|
||||
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
|
||||
## MCP Manager (`infrastructure/src/mcp/`)
|
||||
|
||||
## Background Bash
|
||||
Integrasi **Model Context Protocol**:
|
||||
- Menggunakan crate `rmcp` (v2.2)
|
||||
- Transport: **stdio** (child process) dan **streamable HTTP**
|
||||
- `list_tools()` → tool discovery otomatis → registrasi ke tool harness
|
||||
- Persistent connection management
|
||||
|
||||
`src/app/bgbash/` manages long-running shell jobs:
|
||||
## LSP Integration (`infrastructure/src/lsp/`)
|
||||
|
||||
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
|
||||
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
|
||||
Integrasi **Language Server Protocol**:
|
||||
- **Auto-provisioner** — deteksi bahasa dari file extension, start LSP server yang sesuai
|
||||
- Mendukung: `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dan lainnya
|
||||
- **JSON-RPC client** — typed notifications + request/response
|
||||
- 7 tools LSP yang diekspose ke LLM
|
||||
|
||||
## Review System
|
||||
## Persistence (`infrastructure/src/persistence/`)
|
||||
|
||||
`src/app/subagent/auto.rs` spawns background reviews:
|
||||
### SQLite Message Log
|
||||
|
||||
- Quick review after every edit
|
||||
- Background test generation
|
||||
- Architecture review
|
||||
- Security review
|
||||
- All retry once on failure, escalate to blocking error if retry also fails
|
||||
Session database dengan `rusqlite` (bundled):
|
||||
- Per-session isolation
|
||||
- Table: `messages`, `sessions`
|
||||
- CRUD, query/filter, blob storage
|
||||
|
||||
### Settings Repository
|
||||
|
||||
`JsonSettingsRepository` — simpan/load `Settings` dari `settings.json`:
|
||||
- `provider`: nama provider LLM
|
||||
- `model`: model ID
|
||||
- `max_tokens`: override context window (default: 256k jika tidak diset)
|
||||
- `temperature`, `concise_output`, dll
|
||||
|
||||
### Memory Files
|
||||
|
||||
File-based memory di `~/.local/share/zesdex/memories/`:
|
||||
- Setiap memory = satu `.md` dengan frontmatter YAML
|
||||
- Fields: `name`, `description`, `type` (`user`/`feedback`/`project`/`reference`)
|
||||
- Index di `MEMORY.md`
|
||||
|
||||
## Session Management (`infrastructure/src/session/`)
|
||||
|
||||
- Setiap sesi memiliki UUID, direktori sendiri di `sessions/<uuid>/`
|
||||
- `.lock` file untuk cegah concurrent access
|
||||
- `session.json` — metadata (waktu mulai, workspace, model yang dipakai)
|
||||
|
||||
## Utils
|
||||
|
||||
| Util | Fungsi |
|
||||
|------|--------|
|
||||
| `utils::write_osc52` | Tulis teks ke clipboard via OSC52 escape sequence |
|
||||
| `Toast` / `ToastKind` | Notifikasi sementara (Success/Warning/Error/Info/Lesson) |
|
||||
| `TurnEvent` | Event dari background agent ke TUI (queue-based) |
|
||||
| `DirCache` | Cache async listing direktori untuk `@mention` autocomplete |
|
||||
| `MentionIndex` | Index file workspace untuk fuzzy autocomplete |
|
||||
| `SessionRuntime` | Runtime state: messages history, usage stats, session start time |
|
||||
|
||||
## OAuth 2.0
|
||||
|
||||
Flow PKCE untuk provider LLM:
|
||||
1. Generate code verifier + challenge
|
||||
2. Open browser ke authorization URL
|
||||
3. Start localhost HTTP server untuk tangkap redirect
|
||||
4. Exchange code → access + refresh token
|
||||
5. Simpan token di settings
|
||||
|
||||
+114
-65
@@ -1,89 +1,138 @@
|
||||
# Data Architecture
|
||||
# Data & Persistence
|
||||
|
||||
## State Model
|
||||
## State Runtime (TUI)
|
||||
|
||||
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
|
||||
State TUI yang berjalan di memori adalah `AppStateRest` (`apps/interfaces/tui/src/state.rs`).
|
||||
|
||||
```
|
||||
AppStateRest
|
||||
├── session: SessionRuntime (hive_mind state, convergence flag)
|
||||
├── runtime: RuntimeState (mode, provider status)
|
||||
├── chat: ChatState (messages, scroll)
|
||||
├── input: InputState (text, cursor, autocomplete)
|
||||
├── settings: Settings (provider, model, temperature, concise_output)
|
||||
├── config: AppConfig (endpoints, credentials)
|
||||
├── scroll: ScrollState (per-panel offset)
|
||||
├── diff: DiffState (edit review)
|
||||
├── tools: Vec with outputs
|
||||
├── statusline, sidebar, etc.
|
||||
└── toasts: pending notifications
|
||||
### TranscriptCache
|
||||
|
||||
```rust
|
||||
pub struct TranscriptCache {
|
||||
pub messages: VecDeque<ChatMessageDisplay>, // O(1) eviction
|
||||
pub max_lines: usize, // default: 200
|
||||
pub dirty: bool, // perlu rebuild cache?
|
||||
}
|
||||
```
|
||||
|
||||
**Mutation rules** (per CLAUDE.md):
|
||||
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
|
||||
- Read-only from every other module
|
||||
- No generic update function — direct field mutation only
|
||||
Saat `dirty=true`, `pre_render_chat()` rebuild `display_lines_cache` (render markdown semua pesan) sebelum frame berikutnya.
|
||||
|
||||
## Persistence
|
||||
### SessionRuntime
|
||||
|
||||
### SQLite Message Log (`src/model/msglog/`)
|
||||
```rust
|
||||
pub struct SessionRuntime {
|
||||
pub messages: Vec<ChatMessage>, // history untuk LLM context
|
||||
pub usage: UsageStats, // token counting akumulasi
|
||||
pub session_start: i64, // unix ms saat sesi dimulai
|
||||
pub hive_mind_converged: bool, // flag selesai hive mind
|
||||
}
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `schema.rs` | Table definitions (messages, sessions) |
|
||||
| `mod.rs` | CRUD operations |
|
||||
| `query.rs` | Query helpers (search, filter) |
|
||||
| `blobs.rs` | Large message blob storage |
|
||||
| `summary.rs` | Conversation summary cache |
|
||||
## Context Window
|
||||
|
||||
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
|
||||
`resolve_context_window()` di `state.rs`:
|
||||
1. Ambil `settings.max_tokens` jika ada dan > 0
|
||||
2. Fallback ke **256.000 token** (default)
|
||||
|
||||
### Memory System (`src/model/memory.rs`)
|
||||
Token dihitung lazily via `count_tokens()` (tiktoken `cl100k_base`, fallback `len/4`), di-cache di `AppStateRest::cached_token_count`, hanya dihitung ulang saat `token_count_dirty=true`.
|
||||
|
||||
File-based memory stored under `~/.claude/projects/<project>/memory/`:
|
||||
## Persistence di Disk
|
||||
|
||||
- Each memory is one markdown file with frontmatter (name, description, type)
|
||||
- Types: `user`, `feedback`, `project`, `reference`
|
||||
- Memory index in MEMORY.md
|
||||
- Export/import for lesson sharing
|
||||
- PID-file session lock prevents concurrent access
|
||||
Semua data disimpan di **platform data directory**:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
### Settings & Config (`src/model/`)
|
||||
```
|
||||
~/.local/share/zesdex/
|
||||
├── settings.json # User settings (provider, model, max_tokens, dll)
|
||||
├── sessions/
|
||||
│ └── <uuid>/
|
||||
│ ├── session.json # Metadata sesi
|
||||
│ ├── messages.jsonl # Message log (append-only)
|
||||
│ └── .lock # Lock file (cegah concurrent access)
|
||||
├── memories/
|
||||
│ └── *.md # Memory files dengan frontmatter
|
||||
├── lessons/
|
||||
│ └── *.md # Lesson files (output dari learning system)
|
||||
└── worktrees/ # Git worktree per sesi (isolasi perubahan)
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `settings.rs` | Serialized user preferences (provider, model, theme) |
|
||||
| `app_config.rs` | Provider endpoints, API key resolution from env |
|
||||
| `session.rs` | Current session metadata |
|
||||
| `conversation.rs` | In-memory conversation state |
|
||||
| `editlog.rs` | Append-only JSONL edit audit trail |
|
||||
|
||||
### Edit Log
|
||||
|
||||
`src/model/editlog.rs` records every file mutation:
|
||||
### settings.json
|
||||
|
||||
```json
|
||||
{"ts": 123, "tool": "edit", "path": "src/main.rs",
|
||||
"reason": "fix bug", "content_sha256": "abc123",
|
||||
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"max_tokens": 256000,
|
||||
"temperature": 0.7,
|
||||
"concise_output": false
|
||||
}
|
||||
```
|
||||
|
||||
Max 5000 entries held in memory before pruning oldest.
|
||||
Diload via `JsonSettingsRepository::load()`, disimpan kembali saat TUI keluar (`state.save_settings()`).
|
||||
|
||||
## Context Management (`src/app/runtime/context/`)
|
||||
### Memory Files
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `tokens.rs` | Token counting via `tiktoken-rs` |
|
||||
| `window.rs` | Token window resolution (fit within model context) |
|
||||
| `dedup.rs` | Deduplication of repeated tool outputs |
|
||||
| `squash.rs` | Compression of large JSON tool results |
|
||||
| `shaping.rs` | Message dropping when context exceeds limits |
|
||||
Format markdown dengan YAML frontmatter:
|
||||
```markdown
|
||||
---
|
||||
name: prefer-early-return
|
||||
type: feedback
|
||||
description: Selalu gunakan early return untuk mengurangi nesting
|
||||
---
|
||||
|
||||
## IPC Data Flow
|
||||
Ketika menulis fungsi, gunakan early return/guard clauses daripada deep nesting.
|
||||
```
|
||||
|
||||
Types: `user`, `feedback`, `project`, `reference`
|
||||
|
||||
### Lesson Files
|
||||
|
||||
Hasil dari learning system, disimpan di `lessons/`:
|
||||
- Satu file per lesson
|
||||
- Plain markdown, dibaca oleh overlay `Learning`
|
||||
- Bisa di-accept/reject dari TUI
|
||||
|
||||
## SQLite (Message Log)
|
||||
|
||||
`rusqlite` dengan fitur `bundled` (tidak perlu install SQLite terpisah):
|
||||
- Per-session database di `sessions/<uuid>/messages.db`
|
||||
- Table `messages`: `id`, `session_id`, `role`, `content`, `timestamp`, `tokens`
|
||||
- Table `sessions`: `id`, `metadata`, `created_at`
|
||||
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
Daemon dan client berkomunikasi via **Unix domain socket**:
|
||||
|
||||
```
|
||||
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
|
||||
│
|
||||
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
|
||||
~/.local/share/zesdex/daemon.sock
|
||||
```
|
||||
|
||||
Frame format:
|
||||
```
|
||||
[4 bytes BE: payload length][JSON payload]
|
||||
```
|
||||
|
||||
Message types:
|
||||
- `Action` — client kirim aksi ke daemon
|
||||
- `StateSnapshot` — daemon kirim snapshot state ke client
|
||||
- `Ping/Pong` — keepalive
|
||||
|
||||
## Edit History
|
||||
|
||||
Agent mencatat setiap mutasi file ke event log internal:
|
||||
- Tool `write`/`edit`/`delete` merekam path, bytes delta, timestamp
|
||||
- Digunakan oleh subagent review untuk audit trail
|
||||
|
||||
## TurnEvent Queue
|
||||
|
||||
Agent berjalan di background thread dan mengirim events ke TUI via `Arc<Mutex<VecDeque<TurnEvent>>>`:
|
||||
|
||||
| Event | Payload | Efek di TUI |
|
||||
|-------|---------|-------------|
|
||||
| `AssistantMessage(msg)` | `ChatMessage` | Push ke transcript |
|
||||
| `ToolResult { output, .. }` | String | Push sebagai tool message |
|
||||
| `Usage { tokens_in, tokens_out }` | u64, u64 | Update `usage` stats |
|
||||
| `Error(msg)` | String | Toast error |
|
||||
| `Compacted(msgs)` | `Vec<ChatMessage>` | Update `session_runtime.messages` |
|
||||
| `SystemNote { kind, message }` | String | Push ke transcript |
|
||||
| `Done` | — | Set `turn_in_flight_flag = false` |
|
||||
|
||||
+121
-80
@@ -1,99 +1,140 @@
|
||||
# Dependencies
|
||||
|
||||
## Rust Crates (30+ direct)
|
||||
Semua dependency dideklarasikan di `[workspace.dependencies]` dalam `Cargo.toml` root, lalu di-*inherit* oleh setiap crate anggota.
|
||||
|
||||
### Core Framework
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation |
|
||||
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
## Crate per Layer
|
||||
|
||||
### HTTP & Networking
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
|
||||
### Domain (`apps/domain`)
|
||||
|
||||
Hanya boleh pakai dependency yang tidak membawa I/O:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Serialisasi (derive) |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `chrono` | 0.4 | Tanggal/waktu |
|
||||
| `uuid` | 1 | UUID v4/v5 |
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `thiserror` | 1 | Derive error types |
|
||||
|
||||
### Infrastructure (`apps/infrastructure`)
|
||||
|
||||
Semua I/O, LLM, DB, tools:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `tokio` | 1 | Async runtime (rt-multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
| `reqwest` | 0.13 | HTTP client (json, stream, blocking, native-tls-vendored, form) |
|
||||
| `rusqlite` | 0.40 | SQLite (bundled — tidak perlu install sistem) |
|
||||
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
|
||||
| `webbrowser` | 1 | Open URLs in browser |
|
||||
| `tiktoken-rs` | 0.12 | Token counting (OpenAI cl100k) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting (default-fancy) |
|
||||
| `ignore` | 0.4 | File walking dengan `.gitignore` support |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed direktori ke binary |
|
||||
| `infer` | 0.19 | Deteksi tipe file dari byte signature |
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (untuk `@mention` autocomplete) |
|
||||
| `webbrowser` | 1 | Buka URL di browser (OAuth) |
|
||||
| `url` | 2 | URL parsing |
|
||||
| `percent-encoding` | 2 | URL encoding |
|
||||
|
||||
### HTML/Markdown
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
|
||||
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
|
||||
| `scraper` | 0.27.0 | HTML parsing/selecting |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
|
||||
|
||||
### Serialization
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `serde` | 1 | Serialization framework |
|
||||
| `serde_json` | 1 | JSON serialization |
|
||||
| `serde_yaml_ng` | 0.10 | YAML serialization |
|
||||
|
||||
### Storage & Files
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `rusqlite` | 0.40 | SQLite (bundled) |
|
||||
| `ignore` | 0.4 | `.gitignore`-aware file walking |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed directory contents in binary |
|
||||
| `infer` | 0.19 | File type detection |
|
||||
| `dirs` | 6 | Standard OS directories |
|
||||
|
||||
### Text & Search
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting |
|
||||
| `tiktoken-rs` | 0.12 | OpenAI token counting |
|
||||
|
||||
### Cryptography & Encoding
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `base64` | 0.22 | Base64 encoding |
|
||||
| `sha2` | 0.11 | SHA-256 hashing |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `uuid` | 1 | UUID generation (v4, v5) |
|
||||
| `libc` | 0.2 | Raw C FFI bindings |
|
||||
|
||||
### Error Handling & Logging
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `tracing` | 0.1 | Structured logging |
|
||||
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
|
||||
| `chrono` | 0.4 | Date/time with serde |
|
||||
|
||||
### Other
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `fast_html2md` | 0.0.62 | HTML → Markdown |
|
||||
| `scraper` | 0.27.0 | HTML parsing + CSS selector |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing |
|
||||
| `lsp-types` | 0.97 | LSP protocol types |
|
||||
| `futures-util` | 0.3 | Async stream combinators |
|
||||
| `libc` | 0.2 | Raw C FFI (Unix process groups) |
|
||||
|
||||
## External Services
|
||||
### TUI (`apps/interfaces/tui`)
|
||||
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| **Anthropic API** | Primary LLM provider |
|
||||
| **OpenAI API** | Alternative LLM provider (including OAuth) |
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation (raw mode, events, mouse) |
|
||||
| `base64` | 0.22 | Base64 (OSC52 clipboard) |
|
||||
| `sha2` | 0.11 | SHA-256 |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `dirs` | 6 | Platform data directory |
|
||||
|
||||
### API (`apps/interfaces/api`)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `axum` | 0.8 | HTTP server framework (macros) |
|
||||
| `tower` | 0.5 | Middleware layer |
|
||||
| `tower-http` | 0.6 | CORS, body limit |
|
||||
| `argon2` | 0.5 | Password hashing |
|
||||
| `jsonwebtoken` | 9 | JWT (HS256) |
|
||||
| `clap` | 4 | CLI argument parsing (derive) |
|
||||
| `rand_core` | 0.6 | Secure random (getrandom) |
|
||||
|
||||
### Serialization (semua layer)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Framework serialisasi |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `serde_yaml_ng` | 0.10 | YAML (frontmatter memory files) |
|
||||
|
||||
## Layanan Eksternal
|
||||
|
||||
| Layanan | Fungsi |
|
||||
|---------|--------|
|
||||
| **LLM Provider** | OpenAI/Anthropic-compatible API (default: OpenCode AI / DeepSeek) |
|
||||
| **MCP Servers** | Tool servers eksternal via stdio atau HTTP |
|
||||
| **LSP Servers** | `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dll |
|
||||
| **GitHub** | Release artifacts via semantic-release CI |
|
||||
| **MCP Servers** | External tool servers (stdio or HTTP) |
|
||||
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
|
||||
|
||||
## Build Configuration
|
||||
## Build & CI
|
||||
|
||||
### Compiler Lints (`.cargo/config.toml`)
|
||||
All unused code, dead code, and deprecation warnings promoted to errors:
|
||||
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
|
||||
### Compiler Lints (`Cargo.toml` workspace)
|
||||
|
||||
```toml
|
||||
[workspace.lints.rust]
|
||||
unused = "deny"
|
||||
dead_code = "deny"
|
||||
unreachable_code = "deny"
|
||||
unused_imports = "deny"
|
||||
unused_variables = "deny"
|
||||
unused_mut = "deny"
|
||||
unused_must_use = "deny"
|
||||
deprecated = "deny"
|
||||
trivial_casts = "deny"
|
||||
trivial_numeric_casts = "deny"
|
||||
|
||||
[workspace.lints.clippy]
|
||||
all = { level = "warn", priority = -1 }
|
||||
pedantic = { level = "warn", priority = -2 }
|
||||
```
|
||||
|
||||
### Release Profile
|
||||
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
|
||||
|
||||
```toml
|
||||
[profile.release]
|
||||
opt-level = 3
|
||||
lto = "fat"
|
||||
codegen-units = 1
|
||||
panic = "abort"
|
||||
strip = "symbols"
|
||||
overflow-checks = true
|
||||
```
|
||||
|
||||
### CI/CD
|
||||
- **CI**: cargo build + test + clippy on every push
|
||||
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
|
||||
|
||||
- **CI**: `cargo build` + `cargo test` + `cargo clippy --all-targets` pada setiap push
|
||||
- **Release**: semantic-release — auto changelog, Cargo.toml version bump, GitHub artifact upload
|
||||
- **Versioning**: `v1.17.0` (saat ini) — mengikuti semver dari commit messages
|
||||
|
||||
## Feature Flags Penting
|
||||
|
||||
| Crate | Feature | Alasan |
|
||||
|-------|---------|--------|
|
||||
| `rusqlite` | `bundled` | SQLite statically linked — tidak perlu install sistem |
|
||||
| `reqwest` | `blocking` | Sync HTTP untuk agent turn di blocking thread |
|
||||
| `reqwest` | `native-tls-vendored` | TLS tanpa dependency sistem |
|
||||
| `tokio` | `rt-multi-thread` | Async runtime multi-thread |
|
||||
| `pulldown-cmark` | *(no default)* | Tidak include semua fitur berat |
|
||||
| `syntect` | `default-fancy` | Syntax highlighting penuh |
|
||||
| `rmcp` | `transport-child-process` + `transport-streamable-http-client-reqwest` | MCP via stdio dan HTTP |
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
# Panduan Development
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Build semua crate
|
||||
cargo build
|
||||
|
||||
# Jalankan TUI (default)
|
||||
cargo run
|
||||
|
||||
# Jalankan dengan log debug
|
||||
RUST_LOG=debug cargo run
|
||||
|
||||
# Jalankan REST API
|
||||
cargo run -- --api --api-port 8080
|
||||
|
||||
# Build release
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
zesdex/
|
||||
├── Cargo.toml # Workspace root, semua dependency terpusat di sini
|
||||
├── Cargo.lock # Lock file (commit ini!)
|
||||
├── apps/
|
||||
│ ├── domain/ # Pure domain (tidak ada I/O)
|
||||
│ ├── application/ # Use-case services
|
||||
│ ├── infrastructure/ # Semua implementasi I/O
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # TUI — fokus pengembangan utama
|
||||
│ │ ├── api/ # REST API (Axum)
|
||||
│ │ ├── daemon/ # Daemon mode
|
||||
│ │ ├── ws/ # WebSocket
|
||||
│ │ ├── grpc/ # gRPC
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point
|
||||
│ └── bootstrap/ # Seeder
|
||||
└── docs/
|
||||
└── CODEMAPS/ # Dokumentasi ini
|
||||
```
|
||||
|
||||
## Menambah Tool Baru
|
||||
|
||||
1. Buat file baru di `apps/infrastructure/src/tools/<nama>.rs`
|
||||
2. Implement trait `Tool`:
|
||||
|
||||
```rust
|
||||
use zesdex_domain::core::tool_call::Tool;
|
||||
use anyhow::Result;
|
||||
use serde_json::Value;
|
||||
|
||||
pub struct MyTool;
|
||||
|
||||
impl Tool for MyTool {
|
||||
fn name(&self) -> &'static str { "my_tool" }
|
||||
fn description(&self) -> &'static str { "Deskripsi untuk LLM" }
|
||||
fn parameters(&self) -> Value {
|
||||
serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"param": { "type": "string", "description": "..." }
|
||||
},
|
||||
"required": ["param"]
|
||||
})
|
||||
}
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String> {
|
||||
let param = args["param"].as_str().unwrap_or("");
|
||||
Ok(format!("Result: {param}"))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Daftarkan di `apps/infrastructure/src/tools/mod.rs`:
|
||||
|
||||
```rust
|
||||
pub fn all_tools() -> Vec<Box<dyn Tool>> {
|
||||
vec![
|
||||
// ... tools lain ...
|
||||
Box::new(my_tool::MyTool),
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Action TUI Baru
|
||||
|
||||
1. Tambah variant ke `enum Action` di `apps/interfaces/tui/src/action.rs`
|
||||
2. Tangani di `apply_action()` match block yang sama
|
||||
3. Emit dari `controller/input.rs::handle_key()`
|
||||
|
||||
```rust
|
||||
// action.rs
|
||||
pub enum Action {
|
||||
// ... existing ...
|
||||
MyNewAction { data: String },
|
||||
}
|
||||
|
||||
// dalam apply_action:
|
||||
Action::MyNewAction { data } => {
|
||||
state.some_field = data;
|
||||
state.mark_dirty();
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Overlay Baru
|
||||
|
||||
1. Buat file `apps/interfaces/tui/src/view/overlays/<nama>.rs`
|
||||
2. Tambah variant ke `enum Overlay` di `state.rs`
|
||||
3. Tambah entry di `overlays/mod.rs::render_overlay()`
|
||||
4. Implement `pub fn render(frame, area, block, state)` di file baru
|
||||
|
||||
## Linting & Testing
|
||||
|
||||
```bash
|
||||
# Cek semua warnings/errors
|
||||
cargo clippy --all-targets
|
||||
|
||||
# Run tests
|
||||
cargo test
|
||||
|
||||
# Test satu crate saja
|
||||
cargo test -p zesdex-tui
|
||||
|
||||
# Check tanpa build (cepat)
|
||||
cargo check --all
|
||||
```
|
||||
|
||||
> **Penting**: Workspace ini menggunakan `deny` untuk hampir semua lint.
|
||||
> Kode harus compile bersih tanpa warning apapun.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Fungsi |
|
||||
|----------|--------|
|
||||
| `RUST_LOG` | Log level (`debug`, `info`, `warn`, `error`) |
|
||||
| `OPENAI_API_KEY` | API key LLM (jika tidak diset via settings) |
|
||||
| `ANTHROPIC_API_KEY` | API key Anthropic |
|
||||
| `ZESDEX_DATA_DIR` | Override direktori data (default: platform standard) |
|
||||
|
||||
## Data Directory
|
||||
|
||||
Saat development, data disimpan di:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
Untuk reset bersih:
|
||||
```bash
|
||||
rm -rf ~/.local/share/zesdex/
|
||||
```
|
||||
|
||||
## Konvensi Kode
|
||||
|
||||
- **Tidak ada `unwrap()`** di kode produksi — gunakan `?` atau `unwrap_or_default()`
|
||||
- **State hanya dimutasi dari `apply_action()`** — jangan mutasi `AppStateRest` dari view
|
||||
- **View functions bersifat read-only** — signature `fn draw(frame: &mut Frame, state: &AppStateRest)`
|
||||
- **Cache mahal dikomputasi sekali** — gunakan flag `dirty` dan `pre_render` pattern
|
||||
- **Semua string ke LLM harus deskriptif** — nama tool dan deskripsinya penting untuk LLM context
|
||||
|
||||
## Release
|
||||
|
||||
Release dilakukan via git tag semantic-release:
|
||||
|
||||
```bash
|
||||
git commit -m "feat: tambah fitur baru" # bumps minor
|
||||
git commit -m "fix: perbaiki bug" # bumps patch
|
||||
git commit -m "feat!: breaking change" # bumps major
|
||||
```
|
||||
|
||||
CI akan otomatis:
|
||||
1. Bump versi di `Cargo.toml`
|
||||
2. Generate `CHANGELOG.md`
|
||||
3. Build release binary
|
||||
4. Upload ke GitHub Releases
|
||||
+154
-67
@@ -1,79 +1,166 @@
|
||||
# Frontend (TUI) Architecture
|
||||
# TUI (Terminal User Interface)
|
||||
|
||||
## Render Pipeline
|
||||
Dibangun di atas **ratatui** + **crossterm**. Kode ada di `apps/interfaces/tui/src/`.
|
||||
|
||||
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
|
||||
## Struktur Source
|
||||
|
||||
```
|
||||
Timer tick
|
||||
apps/interfaces/tui/src/
|
||||
├── run.rs # Event loop utama
|
||||
├── state.rs # AppStateRest — single source of truth
|
||||
├── action.rs # apply_action(): satu-satunya mutator state
|
||||
├── turn.rs # Spawn agent turn di background thread
|
||||
├── lib.rs # Re-export publik
|
||||
├── controller/
|
||||
│ ├── input.rs # Key handler → Vec<Action>
|
||||
│ └── command.rs # Slash command parser
|
||||
├── view/
|
||||
│ ├── mod.rs # Layout + pre_render() + draw()
|
||||
│ ├── chat.rs # Chat transcript panel (dengan display cache)
|
||||
│ ├── sidebar.rs # Sidebar: workflow, tasks, usage
|
||||
│ ├── status.rs # Status bar satu baris
|
||||
│ ├── markdown.rs # Markdown → styled Span (pulldown-cmark)
|
||||
│ ├── workflow.rs # Workflow/hive-mind progress panel
|
||||
│ ├── theme.rs # Tokyo Night color palette (const)
|
||||
│ └── overlays/ # 16 overlay panel
|
||||
└── model/ # Data model lokal TUI
|
||||
```
|
||||
|
||||
## Render Pipeline (Per Frame)
|
||||
|
||||
```
|
||||
run_loop_inner() [50ms in-flight / 200ms idle]
|
||||
│
|
||||
▼
|
||||
main.rs: fn tui_loop()
|
||||
├── drain expired toasts (1x, bukan 2x)
|
||||
│
|
||||
├── controller/input.rs: handle_key() → action
|
||||
├── app/runtime/actions/mod.rs: apply_action()
|
||||
├── if dirty:
|
||||
│ view::pre_render(&mut state) ← update cache (markdown, token count)
|
||||
│ terminal.draw(|f| view::draw(f, &state))
|
||||
│ state.dirty = false
|
||||
│
|
||||
└── poll events → apply_action → Action::Tick
|
||||
```
|
||||
|
||||
### Optimasi Performa
|
||||
|
||||
| Masalah lama | Solusi saat ini |
|
||||
|---|---|
|
||||
| `count_tokens` (tiktoken) setiap frame | Cache `cached_token_count`, update hanya saat pesan baru |
|
||||
| `render_markdown` ulang setiap frame | `display_lines_cache` di `AppStateRest`, rebuild saat `transcript_cache.dirty` |
|
||||
| `Vec::remove(0)` untuk evict pesan lama | `VecDeque::pop_front()` — O(1) |
|
||||
| `Mutex<bool>` untuk `turn_in_flight` | `Arc<AtomicBool>` — lock-free |
|
||||
| Render terus meski idle | Skip `terminal.draw()` jika `dirty == false` |
|
||||
| Poll 50ms konstan | Adaptif: 50ms saat in-flight, 200ms saat idle |
|
||||
| `drain_expired_toasts` 2x per iterasi | Sekali saja di `run_loop_inner` |
|
||||
|
||||
## State (AppStateRest)
|
||||
|
||||
`AppStateRest` di `state.rs` adalah satu-satunya sumber kebenaran TUI:
|
||||
|
||||
```
|
||||
AppStateRest {
|
||||
settings: Settings // provider, model, dll
|
||||
app_config: AppConfig // endpoint, env vars
|
||||
workspace_roots: Vec<PathBuf> // working directories
|
||||
session_dir / session_id // path sesi aktif
|
||||
memory_dir // direktori memory
|
||||
session_runtime: Option<SessionRuntime> // history pesan, usage stats
|
||||
|
||||
transcript_cache: TranscriptCache // VecDeque<ChatMessageDisplay>
|
||||
scroll: ScrollState // offset scroll pane chat
|
||||
input: InputState // buffer, cursor, history, autocomplete
|
||||
misc: MiscState // overlay aktif, toasts, flags
|
||||
|
||||
turn_events: Arc<Mutex<VecDeque<TurnEvent>>> // queue event dari agent
|
||||
turn_in_flight_flag: Arc<AtomicBool> // apakah agent sedang jalan
|
||||
abort_flag: Arc<AtomicBool> // sinyal abort oleh user
|
||||
|
||||
// Cache performa
|
||||
display_lines_cache: Vec<Line<'static>> // hasil render markdown
|
||||
cached_token_count: usize // token count terkini
|
||||
token_count_dirty: bool // perlu hitung ulang?
|
||||
last_render_width: u16 // lebar terminal saat render terakhir
|
||||
|
||||
dirty: bool // perlu render ulang?
|
||||
quit: bool // keluar dari loop?
|
||||
}
|
||||
```
|
||||
|
||||
**Aturan mutasi:**
|
||||
- Dimutasi hanya dari `action.rs::apply_action()` dan `run.rs` (untuk dirty/quit)
|
||||
- Semua fungsi `view/*` bersifat read-only terhadap state
|
||||
- `pre_render_chat()` boleh mutasi hanya field cache (`display_lines_cache`, `cached_token_count`, `token_count_dirty`)
|
||||
|
||||
## Input & Actions
|
||||
|
||||
`controller/input.rs::handle_key()` → `Vec<Action>` → `apply_action(&mut state, action)`
|
||||
|
||||
Semua mutasi state melewati satu titik: `apply_action`. Controller tidak tahu *bagaimana* state diubah, hanya *action apa* yang dihasilkan.
|
||||
|
||||
### Action Utama
|
||||
|
||||
| Action | Efek |
|
||||
|--------|------|
|
||||
| `SubmitInput(text)` | Push ke transcript, spawn agent turn |
|
||||
| `Tick` | Drain `TurnEvent` queue, update state dari hasil agent |
|
||||
| `ScrollUp/Down` | Ubah `scroll.offset` |
|
||||
| `OpenOverlay(v)` | Set `misc.overlay = v` |
|
||||
| `Resize(w, h)` | Invalidasi cache display, set `last_render_width` |
|
||||
| `AbortTurn` | Store `true` ke `abort_flag` |
|
||||
| `ForceQuit` | Set `quit = true` |
|
||||
|
||||
## Overlays (16 Panel)
|
||||
|
||||
| Overlay | File | Fungsi |
|
||||
|---------|------|--------|
|
||||
| `Help` | `overlays/help.rs` | Daftar shortcut keyboard |
|
||||
| `Settings` | `overlays/settings.rs` | Panel pengaturan |
|
||||
| `Bash` | `overlays/bash.rs` | Background shell jobs |
|
||||
| `QuitConfirm` | `overlays/quit_confirm.rs` | Konfirmasi keluar |
|
||||
| `KeyInput` | `overlays/key_input.rs` | Capture key binding |
|
||||
| `Editor` | `overlays/editor.rs` | File editor inline |
|
||||
| `Effort` | `overlays/effort.rs` | Pilih level reasoning LLM |
|
||||
| `Mcp` | `overlays/mcp.rs` | Manajemen MCP server |
|
||||
| `Todo` | `overlays/todo.rs` | Daftar TODO |
|
||||
| `Rewind` | `overlays/rewind.rs` | Navigasi history pesan |
|
||||
| `Learning` | `overlays/learning.rs` | Viewer lesson |
|
||||
| `Usage` | `overlays/usage.rs` | Statistik token |
|
||||
| `Loading` | `overlays/loading.rs` | Spinner generik |
|
||||
| `ModelSelector` | `overlays/model_selector.rs` | Pilih model LLM |
|
||||
| `ClearConfirm` | `overlays/clear_confirm.rs` | Konfirmasi clear chat |
|
||||
|
||||
## Layout Terminal
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ └── state mutates (AppStateRest)
|
||||
│
|
||||
└── view/mod.rs: build TUI layout
|
||||
│
|
||||
├── view/chat.rs: Chat transcript
|
||||
├── view/sidebar.rs: Usage dashboard
|
||||
├── view/status.rs: Status bar
|
||||
├── view/markdown.rs: Message renderer
|
||||
├── view/workflow.rs: Hive-mind progress
|
||||
└── view/theme.rs: Tokyo Night palette
|
||||
│ Chat Transcript Sidebar (≥90) │
|
||||
│ (view/chat.rs) ┌────────────┐ │
|
||||
│ VecDeque messages │ Workflow │ │
|
||||
│ + markdown cache ├────────────┤ │
|
||||
│ scrollable │ Tasks │ │
|
||||
│ ├────────────┤ │
|
||||
│ │ Usage │ │
|
||||
│ └────────────┘ │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ❯ Input Bar + Autocomplete dropdown │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ⚡zesdex READY │ ...center... │ tok · model │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Overlay System
|
||||
|
||||
16 overlays managed by `app/mode/`:
|
||||
|
||||
| Overlay | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| Chat input | `mod.rs` | Main input bar with autocomplete |
|
||||
| Bash | `bash.rs` | Interactive shell panel |
|
||||
| Editor | `editor.rs` | Built-in file editor |
|
||||
| Effort | `effort.rs` | LLM effort selector |
|
||||
| Help | `help.rs` | Keybindings help |
|
||||
| Key Input | `key_input.rs` | Custom key binding |
|
||||
| Learning | `learning.rs` | Lesson viewer |
|
||||
| Loading | `loading.rs` | Spinner overlay |
|
||||
| MCP | `mcp.rs` | MCP server management |
|
||||
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
|
||||
| Rewind | `rewind.rs` | Message/history rewind |
|
||||
| Settings | `settings.rs` | Settings panel |
|
||||
| Todo | `todo.rs` | Task/TODO list |
|
||||
| Workflow | (via view) | Workflow progress |
|
||||
|
||||
## Layout Structure
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Status Bar (view/status.rs) │
|
||||
├──────────────────────┬──────────────────────┤
|
||||
│ │ │
|
||||
│ Chat Transcript │ Sidebar │
|
||||
│ (view/chat.rs) │ (view/sidebar.rs) │
|
||||
│ scrollable, │ tokens, status, │
|
||||
│ inline-log style │ agent info │
|
||||
│ │ │
|
||||
├──────────────────────┴──────────────────────┤
|
||||
│ Input Bar + Autocomplete dropdown │
|
||||
│ (view/mod.rs) │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Input Handling
|
||||
|
||||
`controller/input.rs`:
|
||||
|
||||
- Normal mode: keystrokes go to the active overlay
|
||||
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
|
||||
- Tab cycles autocomplete candidates
|
||||
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
|
||||
- Arrow keys scroll chat, sidebar, and other scrollable panels
|
||||
Sidebar hanya tampil jika lebar terminal ≥ 90 kolom.
|
||||
|
||||
## Theme
|
||||
|
||||
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
|
||||
`view/theme.rs` mendefinisikan palette **Tokyo Night** sebagai `const Color`:
|
||||
`PRIMARY`, `BG`, `SURFACE`, `SURFACE_ELEVATED`, `BORDER`, `TEXT`, `TEXT_DIM`, `TEXT_MUTED`, `SUCCESS`, `WARNING`, `ERROR`, `INFO`, `HIGHLIGHT`, `CODE_BG`, dll.
|
||||
|
||||
## Markdown Rendering
|
||||
|
||||
`view/markdown.rs::render_markdown(text, width, dim)`:
|
||||
- Parse dengan `pulldown-cmark`
|
||||
- Hasilkan `Vec<Span<'static>>` dengan styling
|
||||
- Support: heading, code block, diff block (warna +/-/@@), list, blockquote, table, inline code, link
|
||||
- `dim=true` → semua span memakai `TEXT_DIM` + italic (untuk tool output)
|
||||
- Hasil di-cache di `AppStateRest::display_lines_cache`
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1,975 +0,0 @@
|
||||
# DRY Refactor — High Priority Items Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Eliminate ~500 lines of duplicated code across 6 high-impact patterns, making the codebase more maintainable and reducing the surface area for bugs.
|
||||
|
||||
**Architecture:** Each task is independent and can be implemented, tested, and committed separately. Tasks are ordered by risk/reward — highest impact, lowest risk first.
|
||||
|
||||
**Tech Stack:** Rust, anyhow, serde, std::fs, tower (Service/Layer trait), std::sync::atomic
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Atomic Write Helper — `zesdex-utils`
|
||||
|
||||
**Problem:** 10 files across 3 crates (zesdex-cms persistence, zesdex-iam persistence, zesdex-entities) duplicate the same crash-safe write-to-then-rename pattern. Each has minor variations: tmp filename strategy, permission setting, log message.
|
||||
|
||||
**Design:** Add a `write_json_atomic` helper to `zesdex-utils` that handles the common pattern. For the permission variant (oauth_repo.rs), expose a mode parameter. For the logging variant, the caller handles that.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-utils/src/atomic_write.rs`
|
||||
- Modify: `crates/zesdex-utils/src/lib.rs`
|
||||
- Modify: 10 caller files across 3 crates
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> Result<()>`
|
||||
|
||||
### Step 1: Create `atomic_write.rs` module
|
||||
|
||||
```rust
|
||||
//! Crash-safe atomic file write helper.
|
||||
//!
|
||||
//! Writes serializable data to a temp file, fsyncs, then renames into
|
||||
//! place to guarantee atomicity. On Unix, an optional `mode` sets the
|
||||
//! permissions of the final file (e.g. `0o600` for OAuth tokens).
|
||||
|
||||
use std::io::Write;
|
||||
use std::path::Path;
|
||||
|
||||
use serde::Serialize;
|
||||
|
||||
/// Atomically write serializable `data` to `path`.
|
||||
///
|
||||
/// Flow: serialize → write to `path.tmp` → fsync → rename → fsync parent.
|
||||
/// If `mode` is `Some`, set permissions before rename (Unix only).
|
||||
///
|
||||
/// Edge case: tmp file name uses `with_extension("tmp")` which replaces
|
||||
/// the existing extension — correct for `foo.json` → `foo.tmp`. For paths
|
||||
/// without an extension (unlikely in this codebase), appends `.tmp`.
|
||||
pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> anyhow::Result<()> {
|
||||
let tmp = path.with_extension("tmp");
|
||||
let bytes = serde_json::to_vec_pretty(data)?;
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.truncate(true)
|
||||
.write(true)
|
||||
.open(&tmp)?;
|
||||
f.write_all(&bytes)?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
if let Some(m) = mode {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(m))?;
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{ let _ = m; }
|
||||
}
|
||||
std::fs::rename(&tmp, path)?;
|
||||
if let Some(parent) = path.parent() {
|
||||
let _ = std::fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register module in `zesdex-utils/src/lib.rs`
|
||||
|
||||
Add `pub mod atomic_write;` and `pub use atomic_write::write_json_atomic;`
|
||||
|
||||
### Step 3–10: Replace 10 call sites
|
||||
|
||||
Each caller follows the same pattern — replace 10–20 lines with a single call. Below is the before/after for each file.
|
||||
|
||||
**A) `crates/zesdex-cms/src/infrastructure/persistence/app_config_repo.rs:133-164`**
|
||||
|
||||
Before (30 lines with OpenOptions + write_all + sync_all + rename + parent sync):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
let json = serde_json::to_string_pretty(config).context("failed to serialize app config")?;
|
||||
let tmp = base_dir.join("app_config.json.tmp");
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true)
|
||||
.open(&tmp)
|
||||
.with_context(|| format!("failed to write temp file '{}'", tmp.display()))?;
|
||||
f.write_all(json.as_bytes())?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
std::fs::rename(&tmp, &path).with_context(|| {
|
||||
format!("failed to rename '{}' -> '{}'", tmp.display(), path.display())
|
||||
})?;
|
||||
if let Some(parent) = path.parent() {
|
||||
if let Ok(d) = std::fs::File::open(parent) {
|
||||
let _ = d.sync_all();
|
||||
}
|
||||
}
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
After (7 lines):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
write_json_atomic(&path, config, None)
|
||||
.with_context(|| "failed to save app_config")?;
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The `serde_json::to_string_pretty` call is now inside `write_json_atomic`, so its `.context("...")` goes away. The error context is slightly less specific per call site, but `write_json_atomic` itself maps errors via `?` and they'll propagate with the caller's context.
|
||||
|
||||
**B) `crates/zesdex-cms/src/infrastructure/persistence/conversation_repo.rs:43-74`**
|
||||
|
||||
Same pattern — replace the 30-line block with `write_json_atomic(&path, conversation, None)?`.
|
||||
|
||||
**C) `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs:55-87`**
|
||||
|
||||
Replace with `write_json_atomic(&path, settings, None)?`.
|
||||
|
||||
**D) `crates/zesdex-cms/src/infrastructure/persistence/memory_repo.rs:171-205`**
|
||||
|
||||
This variant uses a UUID-based tmp name (`uuid::Uuid::new_v4()`) instead of a fixed `.tmp` suffix. **Goes away** — `write_json_atomic` uses `with_extension("tmp")`.
|
||||
|
||||
Replace with `write_json_atomic(&path, memory, None)?`.
|
||||
|
||||
Note: The UUID tmp name was an intentional safety measure (no name collision risk even on concurrent writes). `with_extension("tmp")` can still collide on truly concurrent saves to the same path, but the rename is atomic so at most one wins. Accept this trade-off for the DRY benefit.
|
||||
|
||||
**E) `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs:62-71`**
|
||||
|
||||
This writes binary `data: &[u8]` (not serializable). The helper only handles `Serialize`. Two options:
|
||||
1. Keep as-is (it's short — 10 lines, 3 are unique)
|
||||
2. Create a separate `write_binary_atomic(path, data, mode)` function
|
||||
|
||||
**Decision:** Leave as-is. Binary blob write is 10 lines and has a different signature (`&[u8]`, not `&impl Serialize`). Not worth abstracting.
|
||||
|
||||
**F) `crates/zesdex-iam/src/infrastructure/persistence/oauth_repo.rs:29-48`**
|
||||
|
||||
Adds `#[cfg(unix)]` chmod 0o600. Gets `mode: Some(0o600)`:
|
||||
```rust
|
||||
fn save_token(&self, path: &Path, token: &OAuthToken) -> anyhow::Result<()> {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
write_json_atomic(path, token, Some(0o600))?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**G) `crates/zesdex-iam/src/infrastructure/persistence/session_repo.rs:60-76`**
|
||||
|
||||
Replace with `write_json_atomic(&path, session, None)?`.
|
||||
|
||||
**H) `crates/zesdex-entities/src/domain/common/conversation.rs:83-95`**
|
||||
|
||||
This uses `std::io::Result` not `anyhow::Result`. The helper returns `anyhow::Result`. Two options:
|
||||
1. Make helper generic over error type (too complex)
|
||||
2. Convert
|
||||
|
||||
**Decision:** Convert caller to use `anyhow::Result`. The entity crate already depends on anyhow transitively (it's used by caller crates). Add `use anyhow::Context as _;` and wrap.
|
||||
|
||||
```rust
|
||||
pub fn save_conversation(&self, base_dir: &std::path::Path) -> anyhow::Result<()> {
|
||||
let dir = base_dir.join("sessions").join(&self.session_id);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("conversation.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**I) `crates/zesdex-entities/src/domain/auth/session.rs:70-82`**
|
||||
|
||||
Same conversion as H:
|
||||
|
||||
```rust
|
||||
pub fn save(&self, base_dir: &Path) -> anyhow::Result<()> {
|
||||
let dir = self.session_dir(base_dir);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("session.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**J) `crates/zesdex-entities/src/domain/auth/session_lock.rs:72-88`**
|
||||
|
||||
The stale-lock recovery path in `try_lock()`. Same conversion:
|
||||
|
||||
```rust
|
||||
let tmp = self.path.with_extension("lock.tmp");
|
||||
{
|
||||
let mut tmp_file = fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true).open(&tmp)?;
|
||||
write!(tmp_file, "{}", self.pid)?;
|
||||
tmp_file.sync_all()?;
|
||||
}
|
||||
fs::rename(&tmp, &self.path)?;
|
||||
if let Some(parent) = self.path.parent() {
|
||||
let _ = fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
```
|
||||
|
||||
This writes a PID string, not JSON. `write_json_atomic` expects `Serialize`. **Keep as-is** — 10 lines, different serialization format (write! macro, not serde).
|
||||
|
||||
### Step 11: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-utils && cargo test -p zesdex-utils`
|
||||
|
||||
Run: `cargo build -p zesdex-cms -p zesdex-iam -p zesdex-entities`
|
||||
|
||||
Run full test suite: `cargo test`
|
||||
|
||||
### Step 12: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract write_json_atomic helper, DRY 7 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Consolidate `#![allow(clippy::cast_*)]` to crate roots
|
||||
|
||||
**Problem:** 67 files across 8 crates have a `#![allow(clippy::cast_...)]` inner attribute. 6 of 8 crate roots already have it, making sub-file attrs redundant. 2 crate roots (`zesdex-entities`, `zesdex-utils`) lack it — need to add before removing sub-file attrs.
|
||||
|
||||
**Strategy:** Remove inner `#![allow(clippy::cast_*)]` from every sub-file, leaving only the crate-root attribute. Use a script for the mechanical removal, then verify with `cargo build`.
|
||||
|
||||
**Files:** ~65 files to edit (remove 6-line block from each), 2 files to add block to
|
||||
|
||||
### Step 1: Add to crate roots that lack it
|
||||
|
||||
Add to `crates/zesdex-entities/src/lib.rs` (before `pub mod domain;`):
|
||||
```rust
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
```
|
||||
|
||||
Add same block to `crates/zesdex-utils/src/lib.rs`.
|
||||
|
||||
### Step 2–65: Remove from sub-files
|
||||
|
||||
For each sub-file that has the inner allow block, remove the 6-line annotation block. **Do NOT remove `#[allow(...)]` (outer) on individual items — only `#![allow(...)]` (inner) at module level.**
|
||||
|
||||
**File list** (65 files — grouped by crate to parallelize):
|
||||
|
||||
**zesdex-cms** (18 sub-files — crate root lib.rs already has it):
|
||||
`infrastructure/persistence/app_config_repo.rs`, `conversation_repo.rs`, `edit_log_repo.rs`, `memory_repo.rs`, `mod.rs`, `settings_repo.rs`, `rewind_blob_repo.rs`
|
||||
`domain/app_config.rs`, `edit_log.rs`, `memory.rs`, `mod.rs`, `repository.rs`, `service.rs`, `settings.rs`
|
||||
`application/conversation_service.rs`, `memory_service.rs`, `mod.rs`, `settings_service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `mod.rs`
|
||||
`infrastructure/mod.rs`
|
||||
|
||||
**zesdex-iam** (9 sub-files — lib.rs already has it):
|
||||
`application/oauth_service.rs`, `session_service.rs`
|
||||
`domain/oauth.rs`, `repository.rs`, `service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `oauth_loopback.rs`
|
||||
`infrastructure/persistence/oauth_repo.rs`, `session_repo.rs`
|
||||
|
||||
**zesdex-backend** (12 sub-files — main.rs already has it):
|
||||
`app/bgbash/control.rs`, `app/mode/effort.rs`, `app/mode/rewind.rs`, `app/review/probe.rs`, `app/runtime/actions/mod.rs`, `app/state/types.rs`
|
||||
`tool/fs/edit.rs`, `tool/fs/read.rs`, `tool/lsp/mod.rs`
|
||||
`view/chat.rs`, `view/mod.rs`, `view/status.rs`
|
||||
|
||||
**zesdex-entities** (9 sub-files — after adding to lib.rs):
|
||||
`domain/auth/session.rs`, `session_lock.rs`
|
||||
`domain/common/conversation.rs`, `message.rs`, `provider.rs`, `store.rs`, `tool_call.rs`, `tool_result.rs`, `usage.rs`
|
||||
|
||||
**zesdex-ipc** (3 sub-files — lib.rs already has it):
|
||||
`frame.rs`, `protocol.rs`, `server.rs`
|
||||
|
||||
**zesdex-middleware** (2 sub-files — lib.rs already has it):
|
||||
`auth.rs`, `cors.rs`
|
||||
|
||||
**zesdex-infra** (4 sub-files — lib.rs already has it):
|
||||
`database.rs`, `jwt.rs`, `password.rs`, `state.rs`
|
||||
|
||||
**zesdex-utils** (3 sub-files — after adding to lib.rs):
|
||||
`error.rs`, `pagination.rs`, `sanitize.rs`, `slug.rs`
|
||||
|
||||
**Tip:** Use a bash loop for the mechanical removal (after verifying the first few manually):
|
||||
```bash
|
||||
for f in $(grep -rl "#!\[allow" crates/ --include="*.rs" | grep -v lib.rs | grep -v main.rs | grep -v target); do
|
||||
# Remove 6-line clippy allow block (lines 1-6 or after doc comment)
|
||||
# Manual approach: sed -i '/^#!\[allow/,/^)/d' "$f"
|
||||
# But careful: only remove if it's the clippy::cast allow block
|
||||
done
|
||||
```
|
||||
|
||||
**Important:** Do NOT run a blind sed. Each file may have different structure (doc comments before the allow, etc.). Use a targeted approach:
|
||||
1. Search for `#![allow(clippy::cast_`
|
||||
2. Verify it's the 4 cast lints
|
||||
3. Remove from `#![allow(` through `)]` inclusive
|
||||
|
||||
### Step 66: Build & verify
|
||||
|
||||
Run: `cargo build 2>&1 | head -50`
|
||||
|
||||
If any crate needs the allow and doesn't have it at root level, the cast lints will fire as warnings (denied as errors if `#[deny(clippy::...)]` is in play). Add the allow to that crate root.
|
||||
|
||||
### Step 67: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: consolidate #![allow(clippy::cast_*)] to crate roots, remove from 65 sub-files"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Tower Service/Layer Boilerplate Macro — `zesdex-middleware`
|
||||
|
||||
**Problem:** `auth.rs` and `rate_limit.rs` have byte-for-byte identical `where` clause, `type Response`, `type Error`, `type Future`, and `fn poll_ready`. The `call()` method differs (auth vs rate-limit logic).
|
||||
|
||||
**Design:** Create a `impl_tower_middleware!` macro that generates the shared boilerplate.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/lib.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
### Step 1: Add macro to `lib.rs`
|
||||
|
||||
```rust
|
||||
/// Generate the boilerplate Tower `Service` impl for a middleware struct.
|
||||
///
|
||||
/// Usage:
|
||||
/// ```ignore
|
||||
/// impl_tower_middleware!(MyMiddleware<S> [ inner: S, extra_field: Type ]);
|
||||
/// ```
|
||||
///
|
||||
/// Expands to:
|
||||
/// - `type Response = S::Response`
|
||||
/// - `type Error = S::Error`
|
||||
/// - `type Future = Pin<Box<dyn Future<Output = Result<...>> + Send + 'static>>`
|
||||
/// - `fn poll_ready(&mut self, cx) { self.inner.poll_ready(cx) }`
|
||||
#[macro_export]
|
||||
macro_rules! impl_tower_middleware {
|
||||
($name:ident<S $(, $extra:ident: $ty:ty)*>) => {
|
||||
impl<S, ReqBody> tower::Service<axum::http::Request<ReqBody>> for $name<S>
|
||||
where
|
||||
S: tower::Service<axum::http::Request<ReqBody>, Response = axum::response::Response>
|
||||
+ Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = std::pin::Pin<
|
||||
Box<dyn std::future::Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>,
|
||||
>;
|
||||
|
||||
fn poll_ready(
|
||||
&mut self,
|
||||
cx: &mut std::task::Context<'_>,
|
||||
) -> std::task::Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Apply to `auth.rs`
|
||||
|
||||
Before (lines 124-137):
|
||||
```rust
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future =
|
||||
Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>>;
|
||||
|
||||
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... 40 lines of actual logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
impl_tower_middleware!(SessionAuthMiddleware<S>);
|
||||
|
||||
impl<S, ReqBody> SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... same 40 lines
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Wait — the macro generates the `impl<S, ReqBody> Service<Request<ReqBody>> for ...` block including `fn call`. We need to only use the macro for the boilerplate and keep `call()` free.
|
||||
|
||||
**Revised approach:** The macro expands to the full `impl Service for ...` but only includes `poll_ready` and associated types, NOT `call`. The `call()` method remains in a separate `impl` block:
|
||||
|
||||
```rust
|
||||
// Generated by macro:
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where ...
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = ...;
|
||||
fn poll_ready(...) { ... }
|
||||
|
||||
// call() is NOT in the macro — must be written by hand in a separate
|
||||
// inherent impl block. Actually no — call() is required by the trait.
|
||||
}
|
||||
```
|
||||
|
||||
**Revised design:** Don't use a macro. Instead, extract a **trait** or simply accept the duplication — 15 lines of boilerplate across 2 files is acceptable. Alternative: use a **widget supertrait** or keep-as-is.
|
||||
|
||||
**Decision:** Skip this task. The Tower Service boilerplate is only 15 lines duplicated once (2 files). The macro approach adds complexity without proportional benefit. The `where` clause in particular is fragile — tightening bounds (e.g., adding `ReqBody: Debug`) shouldn't need a macro change.
|
||||
|
||||
**Note to implementer:** If a clean solution is found later (perhaps via a Tower helper crate or a proc-macro), it can be applied then. For now, mark this as `wontfix`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Extract Session ID Helper — `auth.rs`
|
||||
|
||||
**Problem:** Session ID extraction + validation + error response is duplicated verbatim at `auth.rs:143-174` and `auth.rs:195-222` (~30 lines × 2).
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Extract and validate `X-Session-Id` from request headers.
|
||||
///
|
||||
/// Flow: read header → validate non-empty → return ID or a 401 error response.
|
||||
fn extract_session_id(req: &Request<ReqBody>) -> Result<String, Response> {
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
match session_id {
|
||||
Some(id) if !id.is_empty() => Ok(id),
|
||||
_ => Err((StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate session and build identity from request context.
|
||||
fn validate_and_build_identity(
|
||||
session_id: &str,
|
||||
store: &Store,
|
||||
req: &Request<ReqBody>,
|
||||
) -> Result<SessionIdentity, Response> {
|
||||
match validate_session(session_id, store) {
|
||||
Ok(session) => {
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
Ok(SessionIdentity::new(session_id.to_string(), user_agent))
|
||||
}
|
||||
Err(e) => Err((
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response()),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace first call site (inside `SessionAuthMiddleware::call`, lines 143-174)
|
||||
|
||||
Before:
|
||||
```rust
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
let session_id = match session_id {
|
||||
Some(id) if !id.is_empty() => id,
|
||||
_ => {
|
||||
let resp = (StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
|
||||
match validate_session(&session_id, &store) {
|
||||
Ok(session) => {
|
||||
let identity = SessionIdentity::new(session_id, user_agent);
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(e) => {
|
||||
let resp = (
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = match extract_session_id(&req) {
|
||||
Ok(id) => id,
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
|
||||
match validate_and_build_identity(&session_id, &store, &req) {
|
||||
Ok(identity) => {
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
```
|
||||
|
||||
### Step 3: Replace second call site (inside `require_session`, lines 195-222)
|
||||
|
||||
Before: same 30 lines (slightly different return style).
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = extract_session_id(&req)?;
|
||||
let identity = validate_and_build_identity(&session_id, &store, &req)?;
|
||||
req.extensions_mut().insert(identity);
|
||||
Ok(())
|
||||
```
|
||||
|
||||
(These functions already return `Result<(), Response>` so the `?` operator works directly.)
|
||||
|
||||
### Step 4: Add `use` imports if needed
|
||||
|
||||
```rust
|
||||
use axum::http::Request;
|
||||
// ... existing imports
|
||||
```
|
||||
|
||||
### Step 5: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-middleware && cargo test -p zesdex-middleware`
|
||||
|
||||
### Step 6: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract session_id extraction helper, DRY auth.rs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Merge Backoff/Jitter Implementations — `zesdex-backend`
|
||||
|
||||
**Problem:** 3 separate implementations of exponential backoff with ±25% jitter in `service/provider.rs`, `app/subagent/engine.rs`, and `app/workflow/engine/mod.rs`. Different caps (30s, 16s, 8s) but same base formula.
|
||||
|
||||
**Design:** Create a `backoff` module with a parameterized function.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/backoff.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs` (or create if needed)
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
|
||||
### Step 1: Create `backoff.rs`
|
||||
|
||||
```rust
|
||||
//! Exponential backoff with jitter.
|
||||
//!
|
||||
//! Three use cases (subagent, provider, workflow) all share the same formula
|
||||
//! with different caps. This module provides a single implementation.
|
||||
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
|
||||
/// Compute an exponential backoff with ±25% jitter.
|
||||
///
|
||||
/// `attempt` is 0-based (first retry → attempt=0 → base=1s,
|
||||
/// second retry → attempt=1 → base=2s, etc.).
|
||||
/// `max_secs` sets the cap.
|
||||
fn backoff_seconds(attempt: u32, max_secs: u64) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(max_secs);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000); // 25% of base, min 100ms
|
||||
let offset = jitter_ns(quarter);
|
||||
// ±25%: offset in [0, quarter), so result = base - quarter/2 + offset
|
||||
// which lies in [base - 25%, base + 25%).
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
|
||||
/// Return a jitter offset in the range [0, range_ns).
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `service/provider.rs`
|
||||
|
||||
Current code (lines 51-70, 96-106):
|
||||
```rust
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(30);
|
||||
let half_range = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(half_range * 2);
|
||||
let ns = base_secs * 1_000_000_000 + offset - half_range;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 30)
|
||||
}
|
||||
```
|
||||
|
||||
And delete the local `jitter_ns` function.
|
||||
|
||||
The `backoff_for_error` function (lines 96-106) also has inline backoff math — replace that too:
|
||||
|
||||
```rust
|
||||
fn backoff_for_error(attempt: u32, err_str: &str) -> Duration {
|
||||
if is_rate_limit(err_str) {
|
||||
backoff_seconds(attempt.saturating_sub(1), 60) // rate-limit cap: 60s
|
||||
} else {
|
||||
backoff_duration(attempt)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old code used `(5u64 * (2u64).pow(attempt.saturating_sub(1))).min(60)` for rate limits. The new code calls `backoff_seconds(attempt.saturating_sub(1), 60)` which gives `(2u64).pow(attempt-1).min(60)`. This changes the base from `5*2^(n-1)` to `2^(n-1)`. The difference is minimal for the rate-limit case (retries are backoff-based anyway) and the simplified formula is worth the slight behavioral change. Accept this.
|
||||
|
||||
### Step 3: Replace in `app/subagent/engine.rs`
|
||||
|
||||
Current code (lines 21-36):
|
||||
```rust
|
||||
fn retry_jitter_ns(range_ns: u64) -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.subsec_nanos() as u64
|
||||
% range_ns
|
||||
}
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(16);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = retry_jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 16)
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old jitter source used `subsec_nanos()` (max ~1s range) while the new helper uses `as_nanos()`. This slightly changes jitter distribution but preserves the ±25% range. Acceptable.
|
||||
|
||||
### Step 4: Replace in `app/workflow/engine/mod.rs`
|
||||
|
||||
Current inline closure (lines 402-413):
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| {
|
||||
let base_secs = (2u64).pow(attempt).min(8);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| crate::app::util::backoff::backoff_seconds(attempt, 8);
|
||||
```
|
||||
|
||||
### Step 5: Create `mod.rs` if needed
|
||||
|
||||
```rust
|
||||
// crates/zesdex-backend/src/app/util/mod.rs
|
||||
pub mod backoff;
|
||||
```
|
||||
|
||||
If the directory doesn't exist:
|
||||
```bash
|
||||
mkdir -p crates/zesdex-backend/src/app/util
|
||||
```
|
||||
|
||||
If `util` already exists, just add `pub mod backoff;`.
|
||||
|
||||
### Step 6: Add `pub` visibility to `backoff_seconds`
|
||||
|
||||
Make the function `pub` in `backoff.rs`.
|
||||
|
||||
### Step 7: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
Run integration test: `cargo test -p zesdex-backend -- --nocapture` (watch for infinite retries in tests)
|
||||
|
||||
### Step 8: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: unify 3 backoff implementations into shared helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Merge Auth/Billing Error Check
|
||||
|
||||
**Problem:** The same 401/402/403 + keyword check appears in 3 places. `provider.rs` already has `is_auth_error()` — the other 2 files should call it instead of rewriting it.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- (No changes to `provider.rs` — already has the canonical version)
|
||||
|
||||
### Step 1: Promote `is_auth_error` to `pub` in `provider.rs`
|
||||
|
||||
```rust
|
||||
/// Is the error an auth / billing failure that retrying won't fix?
|
||||
pub fn is_auth_error(err_str: &str) -> bool {
|
||||
// ... existing implementation
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `engine.rs`
|
||||
|
||||
Current (lines 39-57):
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
let lower = err_str.to_lowercase();
|
||||
if err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403")
|
||||
|| lower.contains("unauthorized")
|
||||
|| lower.contains("forbidden")
|
||||
|| lower.contains("authentication failed")
|
||||
{
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
if crate::service::provider::is_auth_error(err_str) {
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Replace in `mod.rs` (workflow engine)
|
||||
|
||||
Current inline check (lines 433-435):
|
||||
```rust
|
||||
let is_auth = err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403");
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let is_auth = crate::service::provider::is_auth_error(err_str);
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: reuse is_auth_error from provider.rs, DRY backend retry logic"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Abort-Flag Check Helper
|
||||
|
||||
**Problem:** `AtomicBool::load(Ordering::SeqCst)` repeated 16 times across 4 files with 2 variants (`Option<Arc<AtomicBool>>` and bare `AtomicBool`).
|
||||
|
||||
**Design:** Two tiny free functions.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/abort.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/runtime/actions/turn.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
|
||||
### Step 1: Create `abort.rs`
|
||||
|
||||
```rust
|
||||
//! Shared abort-flag checks.
|
||||
//!
|
||||
//! The two variants (Option<Arc<AtomicBool>> and bare AtomicBool) are
|
||||
//! used across the agent runtime, subagent, workflow engine, and provider.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Check whether an optional abort flag has been signalled.
|
||||
pub fn is_aborted(flag: &Option<Arc<AtomicBool>>) -> bool {
|
||||
flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst))
|
||||
}
|
||||
|
||||
/// Check whether a bare abort flag has been signalled.
|
||||
pub fn is_aborted_direct(flag: &AtomicBool) -> bool {
|
||||
flag.load(Ordering::SeqCst)
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register in `mod.rs`
|
||||
|
||||
```rust
|
||||
pub mod abort;
|
||||
```
|
||||
|
||||
### Step 3: Replace 16 call sites
|
||||
|
||||
**In `engine.rs` (4 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
.abort_flag.as_ref().is_some_and(|f| f.load(std::sync::atomic::Ordering::SeqCst))
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted(&ctx.abort_flag)
|
||||
```
|
||||
|
||||
**In `turn.rs` (5 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted_direct(&tc.abort_flag)
|
||||
```
|
||||
|
||||
**In worklow `mod.rs` (5 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&sp.abort_flag)
|
||||
```
|
||||
|
||||
**In `provider.rs` (2 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&abort_flag)
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract is_aborted helpers, DRY 16 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: `dirty()` Helper in `input.rs`
|
||||
|
||||
**Problem:** `state.dirty = true; return Vec::new()` repeated 5 times with direct field access instead of using the existing `state.mark_dirty()` method.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/controller/input.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Mark state dirty and return an empty action list.
|
||||
fn mark(state: &mut AppStateRest) -> Vec<Action> {
|
||||
state.mark_dirty();
|
||||
Vec::new()
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace 5 occurrences
|
||||
|
||||
Replace `state.dirty = true; return vec![];` and `state.dirty = true; return Vec::new();` with `return mark(state);`.
|
||||
|
||||
Additional: Convert the remaining 17 `state.dirty = true;` to `state.mark_dirty();` for API consistency.
|
||||
|
||||
### Step 3: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend`
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: use mark_dirty() helper in input.rs, DRY 22 sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. **Task 1** (Atomic write) — most lines saved, independent, well-understood pattern
|
||||
2. **Task 2** (Clippy allow) — mechanical, safe, 65 files touched but no behavior change
|
||||
3. **Task 4** (Session ID helper) — small, contained, eliminates duplication within one file
|
||||
4. **Task 5** (Backoff merge) — cross-file, needs careful diff of behavior
|
||||
5. **Task 6** (Auth error check) — depends on Task 5's provider.rs changes, do after
|
||||
6. **Task 7** (Abort flag) — independent, mechanical
|
||||
7. **Task 8** (dirty helper) — independent, small change
|
||||
|
||||
Total estimated savings: **~450–600 lines of duplication removed** across ~85 file changes.
|
||||
@@ -1,187 +0,0 @@
|
||||
# TUI Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-14
|
||||
**Scope:** `src/view/`, `src/controller/` (render/interaction layer only)
|
||||
|
||||
## Context
|
||||
|
||||
The TUI went through a "modern design" pass the day before this spec (commit `3f5f27c`:
|
||||
dark palette, neon accents, message cards, segmented status bar). The request for this
|
||||
overhaul covers all three axes at once: aesthetics, UX/navigation, and layout paradigm —
|
||||
not a re-skin of the existing structure.
|
||||
|
||||
## Goals
|
||||
|
||||
- Replace the current 3-zone layout (chat / input / status, everything else as a
|
||||
full-block centered modal) with a **Multi-Pane Dashboard**: chat stays central, a
|
||||
persistent right sidebar surfaces live status that today requires opening a modal.
|
||||
- Replace the current "neon dusk" palette with a **Tokyo Night** palette.
|
||||
- Replace the current per-message card rendering (badge pill, left accent bar, blank-line
|
||||
gaps) with a **tight inline log** format.
|
||||
- Drop decorative emoji from overlay titles in favor of plain colored text — the accent
|
||||
border/text color already carries identity.
|
||||
- Restyle (not restructure) the overlays that stay modal.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No `AppStateRest` shape changes, no new `Action` variants, no controller/state-mutation
|
||||
changes. This is a view-layer repaint; `theme.rs` constants are the only "API" the rest
|
||||
of the app depends on, and their names don't change, only their values.
|
||||
- No new keybindings and no mouse support. Sidebar widgets are read-only/glanceable —
|
||||
none of the three (Workflow, Todo, Usage) are interactive today, so they don't need
|
||||
focus or selection state in their new form either.
|
||||
- No overlay is removed. Workflow/Todo/Usage keep their existing overlay trigger as an
|
||||
"expand" view (see below); the other 13 overlays are untouched functionally.
|
||||
- No automated visual/snapshot tests are being introduced (none exist today for
|
||||
`view/`/`controller/`; see Testing below).
|
||||
|
||||
## Layout architecture
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────┬──────────────┐
|
||||
│ │ WORKFLOW │
|
||||
│ Chat transcript (tight inline log) │ ▶ Node-0-1 │
|
||||
│ │ ✓ Node-0-2 │
|
||||
│ ├──────────────┤
|
||||
│ │ TASKS │
|
||||
│ │ ☐ Fix bug │
|
||||
│ │ ☑ Repro │
|
||||
│ ├──────────────┤
|
||||
│ │ USAGE │
|
||||
│ │ 12.3k tok │
|
||||
├─────────────────────────────────────────────┴──────────────┤
|
||||
│ ❯ input bar │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ status bar │
|
||||
└───────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- The sidebar is a fixed-width column (generalizing the existing `show_todo`
|
||||
two-column split in `view/mod.rs::draw`) holding three stacked widgets, in this
|
||||
order: **Workflow**, **Tasks**, **Usage**.
|
||||
- **Responsive collapse**: below a width threshold (~90 cols — extending the existing
|
||||
`show_todo && area.width > 60` precedent, widened because the new sidebar holds three
|
||||
stacked widgets instead of one), the sidebar doesn't render and chat takes full width.
|
||||
No manual toggle key — purely width-driven, matching current behavior.
|
||||
- Each sidebar widget truncates its content to what fits and shows a `+N more, press
|
||||
<key> to expand` hint (same pattern `Rewind` already uses for `"... and N more
|
||||
messages"`) when there's more than fits — that's what the kept overlay is for.
|
||||
|
||||
### Workflow / Todo / Usage: sidebar glance + overlay expand
|
||||
|
||||
These three overlays are **not removed**. Their existing trigger (same keys/commands as
|
||||
today) still opens the full-screen version — now serving as the "expand" view for when
|
||||
the sidebar column is too narrow to show everything (many hive-mind nodes, a long task
|
||||
list). The sidebar widget and the overlay both read the same state
|
||||
(`workflow_engine`, `misc.todo_content`, `session_runtime.usage` +
|
||||
`session_runtime.session_start`); the sidebar version is a new compact rendering, factored
|
||||
out so both call sites share it where the content is identical (e.g. per-agent card
|
||||
formatting in `workflow.rs`).
|
||||
|
||||
### Remaining 13 overlays: restyled modals, unchanged behavior
|
||||
|
||||
`Help, Settings, Bash, QuitConfirm, KeyInput, Editor, Effort, Mcp, Rewind, Learning,
|
||||
Loading, ModelSelector, ClearConfirm` keep their current centered-modal mechanic and
|
||||
content logic exactly as-is. Only their chrome changes: new palette values (same
|
||||
semantic-color-per-overlay mapping as today — e.g. `QuitConfirm` stays `ERROR`, `Settings`
|
||||
stays `PRIMARY`), and emoji dropped from their title strings.
|
||||
|
||||
## Visual language
|
||||
|
||||
### Palette — Tokyo Night
|
||||
|
||||
Values only; `Theme` constant names in `view/theme.rs` are unchanged, so every call site
|
||||
across `view/*` keeps working without edits beyond the const definitions themselves.
|
||||
|
||||
| Constant | Value | Constant | Value |
|
||||
|---|---|---|---|
|
||||
| `BG` | `#1a1b26` | `ROLE_USER` | `#9ece6a` |
|
||||
| `SURFACE` | `#1f2335` | `ROLE_ASSISTANT` | `#7aa2f7` |
|
||||
| `SURFACE_ELEVATED` | `#292e42` | `ROLE_SYSTEM` | `#7dcfff` |
|
||||
| `TEXT` | `#c0caf5` | `ROLE_TOOL` | `#e0af68` |
|
||||
| `TEXT_MUTED` | `#a9b1d6` | `PRIMARY` | `#7aa2f7` |
|
||||
| `TEXT_DIM` | `#565f89` | `SUCCESS` | `#9ece6a` |
|
||||
| `BORDER` | `#3b4261` | `WARNING` | `#e0af68` |
|
||||
| `BORDER_FOCUS` | `#7aa2f7` | `ERROR` | `#f7768e` |
|
||||
| `HIGHLIGHT` | `#3d59a1` | `INFO` | `#7dcfff` |
|
||||
| `HIGHLIGHT_DIM` | `#292e42` | `ACCENT_PURPLE` | `#bb9af7` |
|
||||
| `STATUS_BAR_BG` | `#16161e` | `ACCENT_PINK` | `#ff007c` |
|
||||
| `MODE_AUTO` | `#9ece6a` | `ACCENT_ORANGE` | `#ff9e64` |
|
||||
| `MODE_YOLO` | `#f7768e` | `ACCENT_TEAL` | `#73daca` |
|
||||
| `CODE_BG` | `#16161e` | `CODE_BAR` | `#292e42` |
|
||||
| `BLOCKQUOTE_BAR` | `#7dcfff` | `SCROLLBAR_BG` / `SCROLLBAR_FG` | `#1f2335` / `#3b4261` |
|
||||
|
||||
### Message density — tight inline log
|
||||
|
||||
Replaces the per-message card (role badge pill + left accent bar + blank-line gap)
|
||||
in `chat.rs`:
|
||||
|
||||
```
|
||||
you 09:14 fix the login bug
|
||||
ai 09:14 Looking at src/auth.rs now.
|
||||
↳ Reading src/auth.rs
|
||||
you 09:15 ok try again
|
||||
```
|
||||
|
||||
- Role rendered as a short lowercase colored label (`ROLE_*` colors), timestamp dim,
|
||||
inline with the first content line.
|
||||
- Wrapped/multi-line content aligns under the content column (not under the role label).
|
||||
- Tool-call sub-lines get a dim `↳` prefix.
|
||||
- No blank line within a turn; a single blank line only between different speakers (not
|
||||
after every message).
|
||||
- The chat panel's outer bordered `Block` is unchanged — only the messages inside it lose
|
||||
per-message decoration.
|
||||
- The streaming indicator becomes `ai 09:14 ⠋ generating...` inline, matching the new
|
||||
format, instead of the current padded badge line.
|
||||
|
||||
### Icons
|
||||
|
||||
Overlay titles drop decorative emoji (❓⚙💻🚪✏️🎯🔌📋⏪📚📊⏳🧠🗑️⚡) and render as plain
|
||||
bold colored text (e.g. `Settings` in `PRIMARY`, no ⚙). The border/text accent color is
|
||||
the identity signal, consistent with the muted Tokyo Night + tight-density direction.
|
||||
|
||||
## File impact
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `view/theme.rs` | Palette values swap (table above). Const names/count unchanged. |
|
||||
| `view/chat.rs` | Rewrite message rendering to the tight inline format. |
|
||||
| `view/markdown.rs` | Re-themed code/quote colors; tightened padding. No structural rewrite. |
|
||||
| `view/mod.rs` | `draw()` grows the persistent sidebar column (generalizes `show_todo` split). `render_overlay()` match arms restyled in place (palette + title text), content logic untouched. Todo/Usage compact-widget rendering factored out of the current inline overlay code so it's callable from both the sidebar and the kept overlay. |
|
||||
| `view/status.rs` | Restyle to new palette; structurally unchanged. |
|
||||
| `view/workflow.rs` | Add a compact-card render function for the sidebar widget, reusing the existing per-agent formatting logic. |
|
||||
| `controller/*` | No changes. Interaction model is unchanged; sidebar is non-interactive. |
|
||||
|
||||
## Edge cases
|
||||
|
||||
- Empty states per sidebar widget (no workflow running, no tasks, zero usage) — compact
|
||||
one-line placeholders, consistent with the tight density (not the current multi-line
|
||||
placeholder paragraphs).
|
||||
- Sidebar auto-collapses below ~90 cols; chat reclaims full width.
|
||||
- Sidebar widget overflow (e.g. a hive-mind run with many nodes, a long task list)
|
||||
truncates with a `+N more` hint pointing at the existing expand-overlay trigger.
|
||||
- Long chat content wraps with continuation lines aligned under the content column.
|
||||
|
||||
## Testing / verification
|
||||
|
||||
No automated visual or snapshot tests exist for `view/`/`controller/` today (confirmed:
|
||||
zero `#[cfg(test)] mod tests` in either directory), and none are introduced by this
|
||||
change — ratatui rendering isn't meaningfully unit-testable without a snapshot harness
|
||||
this repo doesn't have. Verification is manual: run the TUI (`cargo run`) and exercise
|
||||
the golden paths (send a chat message, trigger a workflow/hive-mind run, open each of the
|
||||
13 remaining overlays, resize the terminal across the sidebar-collapse threshold).
|
||||
`cargo clippy` must stay clean (warnings-as-errors per repo config), and every touched
|
||||
`pub fn`/`struct` keeps the doc-comment convention from CLAUDE.md (What/Flow/Why/Return).
|
||||
|
||||
## Suggested implementation order
|
||||
|
||||
Not binding — the implementation plan owns sequencing — but a sensible build order given
|
||||
the dependency shape (palette first, since everything else reads `Theme` consts):
|
||||
|
||||
1. `theme.rs` palette swap
|
||||
2. `chat.rs` tight-inline rewrite
|
||||
3. `mod.rs` sidebar scaffolding + Workflow/Tasks/Usage compact widgets (+ `workflow.rs`
|
||||
compact-card fn)
|
||||
4. `status.rs` restyle + remaining 13 overlay restyle (mechanical: palette + title text)
|
||||
5. Manual TUI verification pass across golden paths above
|
||||
@@ -1,114 +0,0 @@
|
||||
# Clipboard Copy via OSC52 — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/controller/input.rs`, `src/main.rs`,
|
||||
`src/ipc/protocol.rs`
|
||||
|
||||
## Context
|
||||
|
||||
There is no clipboard support anywhere in the TUI today, and mouse capture is enabled
|
||||
(`EnableMouseCapture` in `main.rs`), which in most terminal emulators suppresses native
|
||||
click-drag text selection unless the user holds a modifier — making an in-app copy action
|
||||
more valuable than it would be in a plain scrollback. OSC52 is a terminal escape sequence
|
||||
(`\x1b]52;c;<base64>\x07`) that asks the terminal emulator itself to set the system
|
||||
clipboard; it needs no OS-level clipboard library (no X11/Wayland/win32 dependency) and
|
||||
the `base64` crate is already a dependency (used in `service/oauth/pkce.rs`), so no new
|
||||
crate is needed for this feature.
|
||||
|
||||
Key architectural constraint discovered while designing this: `controller::input::handle_key`
|
||||
runs on the **daemon** process in `--daemon`/`--attach` mode (`main.rs:359`, inside
|
||||
`handle_daemon_client`), not on the process that owns the user's actual terminal. A raw
|
||||
`io::stdout()` write inside `handle_key` would go to the headless daemon's stdout in that
|
||||
mode, not the user's terminal. The copy action therefore can't write the escape sequence
|
||||
directly from `handle_key` — it has to signal intent via state, and the terminal-owning
|
||||
process (single-process `run_loop_inner`, or the attach client's loop) performs the actual
|
||||
write.
|
||||
|
||||
## Goals
|
||||
|
||||
- `Ctrl+Y` copies the most recent `Role::Assistant` message's raw text (not the rendered
|
||||
markdown spans) to the system clipboard via OSC52.
|
||||
- Works identically in single-process mode and in `--daemon`/`--attach` mode.
|
||||
- No new dependency.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No native clipboard fallback (e.g. `arboard`) for terminals that don't honor OSC52 —
|
||||
unsupported terminals silently swallow the escape sequence; no error surfaces to the
|
||||
user beyond the optimistic "Copied to clipboard" toast (there's no ack mechanism in the
|
||||
OSC52 protocol to verify the terminal actually did it).
|
||||
- No copy-last-code-block variant — out of scope for this pass; the whole-message copy
|
||||
covers the common case and is simple to extend later if needed.
|
||||
- No mouse-drag text selection — unrelated, much larger feature; not being built here.
|
||||
|
||||
## State (`misc.rs`)
|
||||
|
||||
- `MiscState` gains `pub pending_clipboard_copy: Option<String>`, initialized to `None` in
|
||||
`MiscState::new()`.
|
||||
|
||||
## `input.rs`
|
||||
|
||||
- New top-level arm alongside the existing `Ctrl+C`/`Ctrl+D` handlers:
|
||||
`KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL)`. It finds the last
|
||||
message in `state.transcript_cache.messages` with `role == Role::Assistant`:
|
||||
- If found: `state.misc.pending_clipboard_copy = Some(msg.content.clone())`.
|
||||
- If not found: push an `Info` toast ("No assistant message to copy yet") and leave
|
||||
`pending_clipboard_copy` as `None`.
|
||||
- Returns `Vec::new()` — this is a direct state mutation inside `handle_key`, matching
|
||||
the existing `Ctrl+S` editor-save precedent (`main.rs`'s editor branch also mutates
|
||||
state/does I/O directly rather than going through an `Action`).
|
||||
|
||||
## OSC52 write helper (`main.rs`)
|
||||
|
||||
```
|
||||
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
|
||||
write!(stdout, "\x1b]52;c;{b64}\x07")?;
|
||||
stdout.flush()
|
||||
}
|
||||
```
|
||||
|
||||
Generic over `impl Write` so both the single-process loop (writing to `io::stdout()`) and
|
||||
tests (writing to a `Vec<u8>` to assert the formatted sequence) can use it without a real
|
||||
terminal.
|
||||
|
||||
## Single-process mode (`run_loop_inner`)
|
||||
|
||||
After the existing `for action in actions { apply_action(state, action); }` block, add:
|
||||
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
state.push_toast(Toast::new(ToastKind::Success, "Copied to clipboard".into()));
|
||||
}
|
||||
```
|
||||
|
||||
## Daemon/attach mode
|
||||
|
||||
- `ipc/protocol.rs`: add `DaemonFrame::ClipboardCopy(String)` (alongside `StateUpdate`,
|
||||
`StreamToken`, `SystemNote`, `Closed` — same `Serialize`/`Deserialize` derive).
|
||||
- `handle_daemon_client` (`main.rs`): after each branch that calls `handle_key`/`apply_action`
|
||||
(`KeyPress` and `Submit`, the only two that can reach the input handler), before the
|
||||
existing `send_daemon_update(&mut conn, state)?;` call, add:
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
conn.send(&DaemonFrame::ClipboardCopy(text))?;
|
||||
}
|
||||
```
|
||||
- Attach-client loop (`main.rs`, the function matching on `DaemonFrame::StateUpdate` /
|
||||
`SystemNote` / `Closed` around line 573): add a `DaemonFrame::ClipboardCopy(text) => {
|
||||
let _ = write_osc52(&mut io::stdout(), &text); client_state.push_toast(...); }` arm,
|
||||
mirroring the existing `SystemNote` handling but performing the actual terminal write
|
||||
since this process — not the daemon — owns the user's terminal.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `input.rs`: `Ctrl+Y` with a transcript containing multiple messages sets
|
||||
`pending_clipboard_copy` to the *last* assistant message's content, ignoring later
|
||||
user/tool messages that might follow it; with no assistant message present, it pushes
|
||||
an info toast and leaves `pending_clipboard_copy` as `None`.
|
||||
- `main.rs`: `write_osc52` writing into a `Vec<u8>` buffer produces the exact expected
|
||||
`\x1b]52;c;<base64>\x07` byte sequence for a known input string.
|
||||
@@ -1,115 +0,0 @@
|
||||
# Diff View for edit/write Tools — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/tool/fs/edit.rs`, `src/tool/fs/write.rs`, `src/view/markdown.rs`, `src/view/chat.rs`
|
||||
|
||||
## Context
|
||||
|
||||
`edit` currently reports only a byte-delta (`"edited {rel} ({N} byte delta)"`), and `write`
|
||||
reports only a byte count. Neither the model nor the user sees what actually changed —
|
||||
just a number. This makes it hard for the model to self-verify an edit landed correctly,
|
||||
and hard for the user to review a change without opening the file. No diff-computing
|
||||
library exists in the dependency tree today.
|
||||
|
||||
## Goals
|
||||
|
||||
- `edit` returns a real unified diff (git-style, 3 lines of context) of the change it just
|
||||
made, in place of the byte-delta note.
|
||||
- `write` returns the same kind of diff when it overwrites a file that already existed
|
||||
with valid UTF-8 content; falls back to the current "wrote N bytes" message for new
|
||||
files or non-UTF-8 (binary) overwrites.
|
||||
- Diffs render in the chat view with real color (green add / red remove / cyan hunk
|
||||
header) instead of being flattened to dim/italic like other tool output.
|
||||
- Large diffs are truncated with a trailing count, matching the existing pattern in
|
||||
`read.rs` (`"... ({N} more lines, total {total})"`).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No diff view for any tool besides `edit`/`write` (e.g. no retroactive diffing of
|
||||
`bash_tools.rs` shell edits).
|
||||
- No side-by-side diff layout — unified format only, matching how every other tool
|
||||
output already renders as a single text stream.
|
||||
- No persistence of diff history; each diff is only the delta of the single tool call
|
||||
that produced it, not a cumulative session diff.
|
||||
- No changes to non-tool (assistant/user/system) message rendering or coloring.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `similar = "3"` (line/word diff crate; permissive MIT/Apache-2.0, no heavy
|
||||
transitive deps). Use `TextDiff::from_lines(old, new).unified_diff().context_radius(3)`,
|
||||
which produces standard `@@ -a,b +c,d @@` hunk headers and `-`/`+`/` `-prefixed lines —
|
||||
no custom diff algorithm needed.
|
||||
|
||||
## Tool changes
|
||||
|
||||
### `edit.rs`
|
||||
|
||||
After computing `new_content` and writing it to disk:
|
||||
|
||||
1. Compute `similar::TextDiff::from_lines(&content, &new_content).unified_diff().context_radius(3).to_string()`.
|
||||
2. Split into lines; if `> MAX_DIFF_LINES` (200), keep the first 200 and append
|
||||
`"... ({N} more lines truncated)"`.
|
||||
3. Wrap the (possibly truncated) diff text in a fenced ` ```diff ` block.
|
||||
4. Replace the byte-delta note in the returned message with this block; keep the
|
||||
existing "Graduated checks matched" / LSP note suffixes in their current position
|
||||
(after the diff block).
|
||||
|
||||
### `write.rs`
|
||||
|
||||
Before overwriting:
|
||||
|
||||
1. If `path.exists()` and `fs::read_to_string(&path)` succeeds (valid UTF-8), capture it
|
||||
as `old_content` and note `is_overwrite = true`.
|
||||
2. If the file doesn't exist, or reading it fails (binary/non-UTF-8), `is_overwrite = false`
|
||||
— no error, just skip the diff path silently.
|
||||
3. After writing, if `is_overwrite`, compute and truncate the diff exactly as in `edit.rs`
|
||||
and append the fenced block to the return message (in addition to the existing
|
||||
"wrote N bytes" line, not instead of it — for `write`, unlike `edit`, the byte count is
|
||||
still useful since it can be a full-file rewrite).
|
||||
4. If not `is_overwrite`, return message is unchanged from today.
|
||||
|
||||
The truncation constant (`MAX_DIFF_LINES = 200`) and truncation message format are
|
||||
shared — factor into a small helper in `tool/fs/helpers.rs` used by both tools.
|
||||
|
||||
## Rendering changes
|
||||
|
||||
### `markdown.rs`
|
||||
|
||||
- `render_markdown` gains a `dim: bool` parameter: `render_markdown(text, width, dim)`.
|
||||
- Capture the fence language from `Tag::CodeBlock(CodeBlockKind::Fenced(lang))` (today
|
||||
matched as `CodeBlock(_)`, discarding the language). Track `in_diff_block: bool` when
|
||||
`lang == "diff"`.
|
||||
- Inside a diff block, process text line-by-line instead of as one blob: a line starting
|
||||
with `+` (not `+++`) is styled green, `-` (not `---`) red, `@@` cyan/muted, everything
|
||||
else (context lines, `+++`/`---` file headers) uses the existing code-block teal.
|
||||
- When `dim` is `true`: every span keeps its assigned color as computed above, but
|
||||
non-diff spans (headings, links, plain text, non-diff code blocks, table cells) fall
|
||||
back to `Theme::TEXT_DIM` + `Modifier::ITALIC` instead of their normal palette color —
|
||||
this replicates today's "tool output is always dim" behavior for everything except
|
||||
diff lines.
|
||||
- When `dim` is `false`: behavior is unchanged from today (full color, used for
|
||||
assistant/user/system messages).
|
||||
|
||||
### `chat.rs`
|
||||
|
||||
- `Role::Tool` branch: replace the two manual span-remapping loops (that force every
|
||||
span to `dim_italic`) with a direct call to `render_markdown(&content, content_width, true)`
|
||||
and use the returned spans as-is.
|
||||
- All other roles: call `render_markdown(&content_str, content_width, false)` — same
|
||||
call as today, just with the new explicit `false` argument.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `edit.rs`: a normal single-replace edit produces a diff block with matching
|
||||
`-`/`+` lines; a `replace_all` across 250+ lines truncates at 200 with the correct
|
||||
trailing count.
|
||||
- `write.rs`: writing a brand-new file keeps the old "wrote N bytes" message with no
|
||||
diff block; overwriting an existing UTF-8 file produces a diff block; overwriting
|
||||
a path that reads as invalid UTF-8 (simulate via non-UTF-8 bytes) falls back to the
|
||||
byte-count message without erroring.
|
||||
- `markdown.rs`: a fenced ` ```diff ` block with `+`/`-`/`@@` lines produces spans with
|
||||
the expected fg colors under `dim=true` (diff lines colored) and confirms non-diff
|
||||
text in the same call falls back to `TEXT_DIM` + italic.
|
||||
@@ -1,126 +0,0 @@
|
||||
# Fuzzy @file-mention Autocomplete — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/app/state/rest.rs`, `src/controller/input.rs`,
|
||||
`src/view/mod.rs`, `src/tool/mod.rs`, `src/tool/fs/write.rs`, `src/main.rs`
|
||||
|
||||
## Context
|
||||
|
||||
The chat input already has a dropdown autocomplete (`InputState` in `misc.rs`), but it
|
||||
only covers slash commands: it requires the whole buffer to start with `/` and filters a
|
||||
fixed `COMMANDS` list by prefix. There's no way to reference a project file from the chat
|
||||
input without typing its exact path from memory. The existing `dir_cache` (used by the
|
||||
`dir_cache_update` tool) looks like it could serve this but doesn't: it's a single,
|
||||
non-recursive directory snapshot, overwritten on each LLM-driven `dir_cache_update` call —
|
||||
not a standing, recursive, whole-workspace file index. `search.rs`'s `Grep`/`Glob` tools
|
||||
already do the recursive, `.gitignore`-respecting walk this feature needs, via
|
||||
`ignore::Walk`.
|
||||
|
||||
Also relevant: there is no persistent async runtime driving the TUI loop. `main.rs`
|
||||
constructs a `tokio::runtime::Runtime` but never `.enter()`s or `block_on`s it in the
|
||||
main loop — `run_loop` is fully synchronous. The one existing async-flavored pattern
|
||||
(`dir_cache_update.rs`) spins up a throwaway one-shot runtime purely to satisfy
|
||||
`tokio::sync::RwLock`'s API, then discards it. This feature does not need that ceremony:
|
||||
a plain `std::sync::RwLock` is enough, since every reader/writer here is synchronous
|
||||
(`handle_key`, `Tool::run`, and the index-build thread all being plain sync code).
|
||||
|
||||
## Goals
|
||||
|
||||
- Typing `@` at a word boundary (start of buffer or after whitespace) in the chat input,
|
||||
followed by non-whitespace characters, opens a dropdown of fuzzy-matched project file
|
||||
paths, live-updating as the query changes.
|
||||
- Selecting a candidate splices `@relative/path ` into the buffer at the mention's
|
||||
position (not a whole-buffer replace) and the user keeps typing.
|
||||
- Candidates come from a background-built, whole-workspace file index — not the
|
||||
LLM-facing `dir_cache`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No auto-reading of the selected file's content into the conversation — the inserted
|
||||
`@path` is plain text; the model reads it via the `read` tool if it wants to, same as
|
||||
any other path reference.
|
||||
- No live re-filter on Backspace/Delete while a mention dropdown is open — mirrors the
|
||||
slash-command dropdown's existing behavior (closes on Backspace/Delete rather than
|
||||
refiltering). Not fixing that for commands here; file mentions just inherit it for
|
||||
consistency.
|
||||
- No periodic re-walk of the index after startup — only single-file incremental updates
|
||||
on file creation (see below). A deleted or renamed file may show a stale entry until
|
||||
restart; acceptable since selecting it just inserts text, it doesn't touch the
|
||||
filesystem.
|
||||
- No fuzzy matching over directories, only files.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `nucleo-matcher = "0.3"` (the fuzzy-matching engine from the Helix editor project;
|
||||
small, actively maintained, no heavy transitive deps).
|
||||
|
||||
## Index storage & construction
|
||||
|
||||
- New type in `misc.rs`: `MentionIndex { entries: Arc<std::sync::RwLock<Vec<String>>> }`, with `MentionIndex::new()`, `set(&self, paths: Vec<String>)`, and `snapshot(&self) -> Vec<String>` (both plain sync `.write()`/`.read()`, no `try_`/async — a std `RwLock` doesn't block indefinitely here since every hold is a quick vec swap or clone).
|
||||
- `AppStateRest` gets a `pub mention_index: MentionIndex` field, initialized in `AppStateRest::new()`, threaded into `ToolCtx`/`ToolCtxBuilder` the same way `dir_cache` is (new `mention_index` field on both, wired through `tool_ctx()`/`tool_ctx_for()`/`build()`).
|
||||
- In `main.rs`, right after `AppStateRest::new(...)` in the single-process TUI path and the daemon path (not the attach-only client path, which has no local `ToolCtx`), spawn `std::thread::spawn` that:
|
||||
1. For each workspace root (index `i`, path `w`): `ignore::Walk::new(w)`, keep only files, strip `w` as prefix, format as `rel` for `i == 0` or `[i]rel` for `i > 0` (matching `resolve_path`'s existing workspace-index convention).
|
||||
2. Stop collecting once the total across all workspaces hits 50,000 entries (repos larger than that are rare here; this is a soft cap to bound memory/scan time, not a hard requirement).
|
||||
3. Call `mention_index.set(all_paths)`.
|
||||
- `write.rs`: after a successful write, if the target path did **not** exist before the write (i.e. this created a new file, not an overwrite), compute its relative/workspace-prefixed form and push it onto `ctx.mention_index`'s vec directly (read-modify-write under the same lock) rather than re-walking.
|
||||
|
||||
## `InputState` changes (`misc.rs`)
|
||||
|
||||
- New `pub enum AutocompleteKind { Command, FileMention }`.
|
||||
- `InputState` gains `pub autocomplete_kind: AutocompleteKind` (default `Command`) and
|
||||
`pub mention_start: usize` (byte offset of the triggering `@`).
|
||||
- New `fn mention_query_at_cursor(&self) -> Option<(usize, String)>`: scans backward from
|
||||
`self.cursor` for an `@`; the scan stops (returns `None`) if it hits whitespace before
|
||||
finding `@`. The `@` only counts as a trigger if it's at buffer start or immediately
|
||||
preceded by whitespace. Returns `(byte offset of '@', query text between '@' and cursor)`.
|
||||
- New `fn open_mention_autocomplete(&mut self, files: &[String])`: calls
|
||||
`mention_query_at_cursor()`; if `None`, calls `close_autocomplete()` and returns. If
|
||||
`Some((start, query))`, fuzzy-matches `query` against `files` via `nucleo-matcher`,
|
||||
keeps the top 10 by score, sets `autocomplete_candidates`, `autocomplete_kind =
|
||||
FileMention`, `mention_start = start`, `autocomplete_visible = !candidates.is_empty()`.
|
||||
- `select_autocomplete()` becomes kind-aware:
|
||||
- `Command` (today's behavior, unchanged): `buffer = candidate.clone()`, `cursor =
|
||||
buffer.len()`.
|
||||
- `FileMention`: `buffer.replace_range(mention_start..cursor, &format!("@{candidate} "))`,
|
||||
`cursor = mention_start + candidate.len() + 2` (the `@` plus the candidate plus the
|
||||
trailing space).
|
||||
- Both paths end with `close_autocomplete()`, same as today.
|
||||
|
||||
## `input.rs` wiring
|
||||
|
||||
- `KeyCode::Char(c)` handler: after `state.input.insert(c)`, keep the existing
|
||||
`if buffer.starts_with('/') { open_autocomplete() }` check, and add an `else if let
|
||||
Some(_) = state.input.mention_query_at_cursor() { state.input.open_mention_autocomplete(&state.mention_index.snapshot()) }` branch. These are mutually exclusive in practice (a
|
||||
buffer starting with `/` is a slash command, not a sentence with an `@mention` in it).
|
||||
- `KeyCode::Backspace` / `KeyCode::Delete`: unchanged — both already just call
|
||||
`close_autocomplete()` when a dropdown is visible, regardless of kind. No new branching
|
||||
needed since `close_autocomplete()` already resets `autocomplete_kind` isn't touched but
|
||||
becomes irrelevant once `autocomplete_visible` is false.
|
||||
- `KeyCode::Tab`: currently gated on `buffer.starts_with('/')`. Extend the condition to
|
||||
also fire when `autocomplete_kind == FileMention && autocomplete_visible` so Tab cycles
|
||||
file-mention candidates too.
|
||||
- `KeyCode::Enter`: unchanged — already calls `select_autocomplete()` whenever
|
||||
`autocomplete_visible`, which is now kind-aware internally.
|
||||
|
||||
## Rendering (`view/mod.rs`)
|
||||
|
||||
- `render_input_bar`'s dropdown block reuses the exact same list-rendering code (already
|
||||
generic over `autocomplete_candidates`/`autocomplete_idx`); only the title changes based
|
||||
on `state.input.autocomplete_kind`: `" ⌘ Commands "` (unchanged) vs `" 📁 Files "`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `misc.rs`: `mention_query_at_cursor` returns the right `(start, query)` for `@` at
|
||||
buffer start, `@` after a space mid-sentence, and correctly returns `None` when the `@`
|
||||
is mid-word (e.g. `foo@bar`) or when whitespace exists between the `@` and the cursor.
|
||||
`select_autocomplete` for `FileMention` splices correctly into a buffer with text before
|
||||
and after the mention span; `Command` selection still replaces the whole buffer as
|
||||
before.
|
||||
- `write.rs`: creating a new file appends its path to the shared `mention_index`;
|
||||
overwriting an existing file does not add a duplicate entry.
|
||||
- Index construction: not unit-tested directly (it's a `std::thread::spawn` walking the
|
||||
real filesystem at startup) — covered implicitly by exercising the app manually per the
|
||||
`verify` skill during implementation.
|
||||
@@ -1,281 +0,0 @@
|
||||
# Context & Compaction Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-16
|
||||
**Scope:** replaces `src/app/runtime/shortsend.rs`; touches `src/app/runtime/actions/mod.rs`,
|
||||
`src/view/status.rs`, `src/model/settings.rs`, `src/app/subagent/division.rs`, `Cargo.toml`
|
||||
|
||||
## Context
|
||||
|
||||
The existing conversation-compaction system (`shortsend.rs`, 129 lines) only acts once the
|
||||
context is already close to the model's window limit, and has accumulated inconsistencies
|
||||
found during a codebase audit:
|
||||
|
||||
1. Three different token-count heuristics for the same job: `/3` inside
|
||||
`shortsend::shape_messages`, `/4` in the auto-compact loop
|
||||
(`actions/mod.rs` ~line 1146), `/4` again in the live status bar (`view/status.rs:68`).
|
||||
2. Manual `/compact` (`Action::Compact`, `actions/mod.rs:547-563`) passes `client: None`
|
||||
because `apply_action` is synchronous, so it never gets LLM summarization — it always
|
||||
falls back to the bare `"[prior conversation compacted]"` placeholder, unlike automatic
|
||||
mid-turn compaction (`Some(&tc.client)`, line 1160). Undocumented asymmetry between the
|
||||
two trigger paths.
|
||||
3. `context_window` resolution (`model_roles.values().find(...).and_then(...).unwrap_or(...)`)
|
||||
duplicated three times (`Action::Compact`, `spawn_turn`, `view/status.rs` twice).
|
||||
4. No repeated-tool-call dedup: reading the same file (or running the same grep) twice in a
|
||||
session keeps both full copies in context forever, until compaction eventually drops the
|
||||
older one wholesale along with everything else from that period.
|
||||
5. No per-result compression: a single large tool output (a big `bash` log, a large `grep`
|
||||
result) is stored verbatim even when most of it is redundant or low-value.
|
||||
6. Zero test coverage on `shortsend.rs`.
|
||||
|
||||
Separately, research into three real, permissively-licensed open-source projects
|
||||
(`rtk-ai/rtk`, Apache-2.0; `headroomlabs-ai/headroom`, Apache-2.0; `JuliusBrussee/caveman`,
|
||||
MIT — verified via `gh api` for authenticity/license, and by cloning and reading source, not
|
||||
taken from marketing blog posts) surfaced techniques worth reimplementing natively:
|
||||
|
||||
- **rtk**: generic line-scan compression (strip comment/blank runs, brace-depth collapse of
|
||||
function bodies, importance-ranked truncation ending in an unambiguous `[N more lines]`
|
||||
marker — their own regression tests show a comment-shaped marker confuses the LLM into
|
||||
retry-looping) plus structured per-toolchain parsing (e.g. `cargo --message-format=json`
|
||||
bucketed into errors/warnings, boilerplate lines dropped).
|
||||
- **headroom**: per-content-type compressors — logs (classify lines by level/stack-trace/
|
||||
summary, score, keep highest-value lines + surrounding context, adaptive cap), grep
|
||||
results (group by file, score matches, cap globally and per-file), JSON (keep all
|
||||
structural tokens — keys, brackets, colons — drop or shrink long low-entropy string
|
||||
values, keep short values and UUID/hash-shaped high-entropy ones).
|
||||
- **caveman**: a pure prompt/persona instruction (no algorithm) that tells the model to
|
||||
write tersely — drop articles/filler/hedging, keep code/commands/errors verbatim — with an
|
||||
explicit carve-out that disables terseness for destructive-op confirmations and security
|
||||
warnings. This compresses *output* tokens, a different axis from everything else in this
|
||||
design, which compresses *input* context.
|
||||
|
||||
This is a from-scratch reimplementation of the underlying ideas, not a port — no code is
|
||||
copied from any of the three projects.
|
||||
|
||||
## Goals
|
||||
|
||||
- One unified, always-on pipeline that keeps context lean from turn 1, not just once near
|
||||
the limit.
|
||||
- Deduplicate repeated tool calls: an older copy of a tool result superseded by an identical
|
||||
later call (same tool name + same arguments) is replaced with a placeholder, for read-only
|
||||
tools only.
|
||||
- Compress large individual tool results (logs, JSON, generic text) at capture time, above a
|
||||
size floor.
|
||||
- Fix the three known inconsistencies (token heuristic, manual/auto asymmetry,
|
||||
`context_window` duplication).
|
||||
- Optional, off-by-default "concise mode" system-prompt toggle for terser model output.
|
||||
- Full inline test coverage per repo convention.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not adding a runtime dependency on `rtk`, `headroom`, or `caveman` themselves (as a binary,
|
||||
proxy, or crate) — everything is implemented natively in Rust inside zesdex.
|
||||
- Not building rtk's per-toolchain structured parsers (`cargo --message-format=json`
|
||||
re-invocation, etc.) — too invasive for a general-purpose `bash` tool that runs arbitrary
|
||||
commands zesdex doesn't control the flags of. Only the generic line-scan/log/JSON layer is
|
||||
built.
|
||||
- Not switching to an exact per-provider tokenizer — `tiktoken-rs` (BPE, cl100k_base/
|
||||
o200k_base) is an approximation good enough for the 85%/95% budget thresholds; it is not
|
||||
used for billing-accurate counts.
|
||||
- `caveman-compress`-style memory-file rewriting (the LLM-round-trip variant of caveman) is
|
||||
out of scope — only the pure-prompt persona mechanism is adopted.
|
||||
|
||||
## Architecture
|
||||
|
||||
Replace `src/app/runtime/shortsend.rs` with `src/app/runtime/context/`:
|
||||
|
||||
```
|
||||
context/
|
||||
mod.rs — module registration only, no facade (see below)
|
||||
tokens.rs — unified token counting (tiktoken-rs)
|
||||
dedup.rs — cross-call tool-result deduplication
|
||||
squash.rs — per-result compression (log/json/generic), applied at
|
||||
tool-result construction time, upstream of prepare()
|
||||
shaping.rs — budget-based drop + LLM summarize (renamed shortsend logic)
|
||||
window.rs — shared context_window resolution
|
||||
```
|
||||
|
||||
### `tokens.rs`
|
||||
|
||||
```rust
|
||||
pub fn count_tokens(text: &str) -> usize
|
||||
pub fn count_message_tokens(msg: &ChatMessage) -> usize
|
||||
```
|
||||
|
||||
Backed by `tiktoken-rs` (new dependency, pure Rust, embedded BPE vocab, no network calls at
|
||||
runtime), using `o200k_base`. Replaces all three existing heuristic call sites: `shortsend`'s
|
||||
internal `/3`, the auto-loop's `/4` (`actions/mod.rs` ~1146), and `status.rs:68`'s `/4`.
|
||||
|
||||
### `dedup.rs`
|
||||
|
||||
```rust
|
||||
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool)
|
||||
```
|
||||
|
||||
The `bool` is `true` iff at least one message was replaced with a placeholder — callers use
|
||||
it to decide whether the result is worth persisting/announcing, without needing `ChatMessage`
|
||||
to implement `PartialEq` (it doesn't today, and adding it purely to diff whole message lists
|
||||
would be needless surface area for what `collapse` already knows precisely mid-walk).
|
||||
|
||||
Flow: walk messages, pair each `Role::Tool` message to its originating `ToolCall` via
|
||||
`tool_call_id`. Key = `(function.name, sha256(canonical_json(function.arguments)))` (`sha2`
|
||||
is already a dependency). Track the last index seen per key. For any earlier occurrence of a
|
||||
key whose tool name is in the read-only set, replace that earlier `Tool` message's `content`
|
||||
with a short placeholder (`"[duplicate result — superseded by a later identical call, see
|
||||
below]"`); the assistant's tool-call entry (name + arguments) is left untouched, so the
|
||||
action/audit trail stays intact. Mutating tools are never touched, even with identical
|
||||
arguments, because call order and repetition can be semantically meaningful (e.g. retrying a
|
||||
flaky `bash` command).
|
||||
|
||||
Read-only classification reuses `subagent::division::tool_scope::READ_TOOLS`
|
||||
(`src/app/subagent/division.rs:21`) rather than a new list — that `const` is made `pub` for
|
||||
this purpose. It already enumerates exactly the read-only tool set (`read`, `grep`, `glob`,
|
||||
`search`, `seqthink`, `recall`, `lsp_*`, `read_findings`).
|
||||
|
||||
Runs every turn, unconditionally, before token counting — not gated on `should_shape`.
|
||||
|
||||
### `squash.rs`
|
||||
|
||||
```rust
|
||||
pub fn apply(tool_name: &str, output: &str) -> String
|
||||
```
|
||||
|
||||
`read` is exempted entirely, always passed through unchanged regardless of size: its output
|
||||
must stay byte-exact because the agent relies on it for exact-match edits afterward, and a
|
||||
squashed view of a JSON config file (or any file whose content happens to parse as JSON)
|
||||
would otherwise be silently altered. Size floor for every other tool: outputs under 1500
|
||||
bytes pass through unchanged (compression only pays off on large output, and touching small
|
||||
results risks losing detail with no token benefit). Above the floor, dispatch by content
|
||||
shape:
|
||||
|
||||
- `squash_json(&str) -> String` — walks a parsed `serde_json::Value` (not a hand-rolled
|
||||
tokenizer — `serde_json` already handles escaping/nesting correctly, reusing it is simpler
|
||||
and more robust); structural tokens (keys, brackets, colons, commas, booleans, null) always
|
||||
kept; string values kept if ≤20 chars or "identifier-shaped" (no internal whitespace *and*
|
||||
Shannon entropy ≥3.0 bits/char — catches UUIDs/hashes/paths), otherwise replaced with `"…"`
|
||||
in place; array elements past the first 3 compressed harder (values elided regardless of
|
||||
length/entropy). Applied when `serde_json::from_str` on the output succeeds. The
|
||||
no-whitespace pre-filter matters: raw per-character entropy alone doesn't separate prose
|
||||
from identifiers — repeated English prose measures ~3.89 bits/char, higher than a UUID's
|
||||
~3.39 — because prose also draws from a wide character set. headroom's own entropy gate is
|
||||
"cheaply pre-filtered by 'no spaces'" before scoring for the same reason; multi-word values
|
||||
never reach the entropy check at all under this rule.
|
||||
- `squash_log(&str) -> String` — line classifier (error/fail/warn/info/debug/trace by
|
||||
keyword + stack-trace-frame detection) → score
|
||||
(`level_score {1.0 error/fail, 0.5 warn, 0.1 info, 0.05 debug/trace} + 0.3 if
|
||||
stack-trace-frame + 0.4 if summary-shaped line`) → keep up to 20 highest-scored error
|
||||
lines, up to 10 highest-scored warning lines, all summary lines, plus a ±2-line context
|
||||
window around each kept line → single `[N lines omitted]` marker for drops (not
|
||||
comment-shaped, per rtk's own finding on LLM confusion). Applied when the output isn't
|
||||
valid JSON, the tool is `bash`, and the output has ≥3 lines matching error/warn/stack-trace
|
||||
patterns. The tool restriction (added after the final whole-branch review) matters: a `grep`
|
||||
result full of matches against error-handling code trips the same ≥3-line keyword threshold
|
||||
as a real build log, but `squash_log`'s hard 20-error/10-warning cap has no byte budget and
|
||||
would silently drop legitimate matches past it — the wrong compressor for search results.
|
||||
Only `bash` (the actual log-producing tool) routes through `squash_log`; every other tool
|
||||
whose output happens to look log-shaped falls through to the gentler, byte-budgeted
|
||||
`squash_generic` instead.
|
||||
- `squash_generic(&str, budget) -> String` — importance-ranked truncation: keeps the first 10
|
||||
and last 10 lines plus any line matching a small "looks important" heuristic (non-blank,
|
||||
not a byte-for-byte repeat of the immediately preceding line), single `[N lines omitted]`
|
||||
marker for the rest, capped to `budget` bytes overall (`budget` = the 1500-byte squash
|
||||
floor doubled, i.e. 3000 bytes, chosen so the fallback path still yields a real reduction
|
||||
on anything that triggered it). Fallback for anything that isn't JSON or log-shaped.
|
||||
|
||||
Called once, at the single tool-result construction site
|
||||
(`actions/mod.rs:1420`, `let tool_msg = ChatMessage::tool_result(tool_call.id.clone(),
|
||||
output);`) — `output` is passed through `squash::apply(&tool_name, &output)` before being
|
||||
wrapped. Runs before the result is ever archived or pushed into `msgs`, so compression is
|
||||
permanent and applies uniformly whether or not compaction ever triggers.
|
||||
|
||||
### `shaping.rs`
|
||||
|
||||
Unchanged behavior from today's `shortsend.rs` (hysteresis `should_shape`, 70%-budget
|
||||
newest-first retention, LLM summarization of dropped messages), moved as-is into this file
|
||||
and updated to source token counts from `tokens.rs` instead of its own heuristic.
|
||||
|
||||
### `window.rs`
|
||||
|
||||
```rust
|
||||
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize
|
||||
```
|
||||
|
||||
Replaces the three duplicated `model_roles.values().find(...).and_then(...).unwrap_or(...)`
|
||||
blocks in `Action::Compact`, `spawn_turn`, and `view/status.rs` (×2).
|
||||
|
||||
### `mod.rs`
|
||||
|
||||
No facade function — just `pub mod dedup; pub mod shaping; pub mod squash; pub mod tokens;
|
||||
pub mod window;`. `dedup`, `shaping`, and `tokens` are called directly from each call site
|
||||
(the auto-loop and `Action::Compact`), matching CLAUDE.md's "No DI — modules call ...
|
||||
directly" convention rather than introducing an orchestration layer that only one of the two
|
||||
callers would use generically (the auto-loop already needs per-stage control today — it
|
||||
inspects `should_shape` itself to decide whether to emit `TurnEvent::Compacted` — and would
|
||||
have to unpack a facade's result anyway).
|
||||
|
||||
## Data flow (per turn)
|
||||
|
||||
1. Tool executes → raw `output: String`.
|
||||
2. `squash::apply(tool_name, &output)` — compress if over the size floor (`read` exempted).
|
||||
3. Wrapped into `ChatMessage::tool_result(...)`, archived, pushed to `msgs`.
|
||||
4. Once per loop iteration: `dedup::collapse(&msgs)` (always) → sum
|
||||
`tokens::count_message_tokens` over the result → `shaping::should_shape` → conditionally
|
||||
`shaping::shape_messages`.
|
||||
5. Result pushed as `TurnEvent::Compacted` if dedup changed anything or shaping triggered,
|
||||
consumed on the main thread to update `SessionRuntime.messages`.
|
||||
|
||||
## Fixing the manual/auto asymmetry
|
||||
|
||||
`Action::Compact` (`actions/mod.rs:547`) currently runs synchronously inside `apply_action`
|
||||
and can't block on an LLM call. Fix: make it spawn a background `std::thread::spawn` — the
|
||||
same pattern `spawn_turn` already uses (`actions/mod.rs:694`) — that runs `dedup::collapse`
|
||||
then unconditionally `shaping::shape_messages(.., force=true, Some(&client))` and reports back
|
||||
via `TurnEvent::Compacted`, identical to the automatic path. The toast sequence becomes
|
||||
"Compacting…" immediately (optimistic, non-blocking) then "History compacted" when the
|
||||
`TurnEvent` arrives. This gives manual `/compact` real LLM summarization instead of always
|
||||
falling back to the placeholder.
|
||||
|
||||
## Concise mode (separate from the `context/` module)
|
||||
|
||||
- `Settings` (`src/model/settings.rs`) gains `pub concise_output: bool`, default `false`,
|
||||
with `#[serde(default)]` for backward-compatible deserialization of existing
|
||||
`settings.json` files (matching the existing `hive_mind_node_timeout_ms` precedent in the
|
||||
same file).
|
||||
- When `true`, `run_agent_turn`'s system-prompt assembly (`actions/mod.rs:930-936`) appends a
|
||||
fourth section to `system_text`: a terse-writing instruction (persona-prompt only, no
|
||||
algorithm — drop articles/filler/hedging/pleasantries, keep code/commands/error text
|
||||
byte-exact) with an explicit carve-out disabling terseness for destructive-operation
|
||||
confirmations and security-relevant warnings, mirroring caveman's own "Auto-Clarity"
|
||||
safety exception.
|
||||
- No UI toggle is in scope for this pass — confirmed no such mechanism exists today for any
|
||||
boolean `Settings` field (`review_enabled`, `session_archive_enabled`,
|
||||
`lsp_auto_provision` are all hand-edited in `settings.json`, same as this one will be).
|
||||
|
||||
## New dependency
|
||||
|
||||
`tiktoken-rs` — pure Rust, embedded BPE vocab (`cl100k_base`/`o200k_base`), no network calls
|
||||
at runtime, MIT/Apache-2.0 dual-licensed. Added to `Cargo.toml`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per repo convention, one per new file:
|
||||
|
||||
- `dedup.rs`: same tool+args → older result replaced; different args → no-op; mutating tool
|
||||
with identical args → both kept in full; unmatched `tool_call_id` (malformed history) →
|
||||
no panic, treated as unpaired.
|
||||
- `squash.rs`: JSON input under/over the size floor; JSON with long low-entropy string values
|
||||
gets them elided while short/UUID-shaped values survive; log input with error/warn lines
|
||||
keeps highest-scored lines and emits exactly one `[N lines omitted]` marker; generic text
|
||||
keeps first/last N lines.
|
||||
- `tokens.rs`: known-string token counts against fixed expected values; empty string → 0.
|
||||
- `shaping.rs`: port the behavioral cases implied by today's hysteresis logic (85% trigger
|
||||
when not previously shaped, 95% once shaped) plus budget-drop ordering.
|
||||
- `window.rs`: role match resolves to the role's `context_window`; no match falls back to
|
||||
`default_context_window`.
|
||||
|
||||
## Migration
|
||||
|
||||
- Delete `src/app/runtime/shortsend.rs`; all three call sites (`actions/mod.rs` auto-loop,
|
||||
`Action::Compact`, and the module path itself) updated to `context::`.
|
||||
- `view/status.rs` switches its live token display to `tokens::count_tokens`, so the status
|
||||
bar finally matches what compaction measures internally.
|
||||
Reference in New Issue
Block a user