200 lines
11 KiB
Markdown
200 lines
11 KiB
Markdown
# TODO — Rewrite Zesdex Rust → TypeScript/Bun
|
||
|
||
> Plan lengkap migrasi in-place dari Rust (11 crate, clean architecture 4 lapis) ke monorepo
|
||
> TypeScript/Bun. Bahasa Indonesia, commit pakai Conventional Commits.
|
||
>
|
||
> **Status:** Fase 0–2 + 3a–3g **selesai**; Fase 4 CLI **selesai**; Fase 5 **API selesai**. Berikutnya: Fase 4 TUI + Fase 5 {daemon,ws,grpc,web}.
|
||
> Base: `bun run check` bersih, 54 test hijau.
|
||
|
||
---
|
||
|
||
## Prinsip & peta pemetaan
|
||
|
||
| Rust | TS/Bun |
|
||
|---|---|
|
||
| `serde_json` | `JSON.stringify/parse` + `zod` |
|
||
| `chrono` | `Date` / `Date.now()` (epoch ms) |
|
||
| `uuid` | `crypto.randomUUID()` |
|
||
| `sha2`, `base64 URL_SAFE_NO_PAD` | `node:crypto` / `Buffer.toString('base64url')` |
|
||
| `reqwest` | `fetch` / bun HTTP |
|
||
| `rusqlite` | `bun:sqlite` |
|
||
| `argon2` | `@node-rs/argon2` |
|
||
| `jsonwebtoken` | `node:crypto` HMAC HS256 |
|
||
| `rmcp` | `@modelcontextprotocol/sdk` |
|
||
| `ratatui`/`crossterm` | library TUI Bun / ink |
|
||
| `tiktoken-rs` | `@dqbd/tiktoken` (cl100k_base) |
|
||
| `clap` | `Bun.argv` / `commander` |
|
||
| `tokio` semaphore | Bun async + concurrency pool |
|
||
| `libc` PID lock | `node:fs` `O_EXCL` + `process.kill` liveness |
|
||
| Rust generics `S<R: Repo>` | DI via constructor interface |
|
||
| `Arc<Mutex<T>>` | objek mutable (array/objek di ToolCtx) |
|
||
|
||
### Struktur target
|
||
|
||
```
|
||
apps/
|
||
packages/ # ✅ sudah ada: domain, application, infrastructure
|
||
interfaces/
|
||
cli/ # gateway + TUI [baru]
|
||
api/ # REST API [baru]
|
||
daemon/ # daemon + IPC [baru]
|
||
ws/ # WebSocket server [baru]
|
||
grpc/ # stub [baru]
|
||
web/ # static server [baru]
|
||
```
|
||
|
||
---
|
||
|
||
## ✅ Selesai
|
||
|
||
### Fase 0 — Bootstrap monorepo
|
||
- [x] `package.json` root (workspaces), `tsconfig.json` strict, `.gitignore`
|
||
- [x] Paket `domain`, `application`, `infrastructure` (workspace `@zesdex/*`)
|
||
- [x] `bun install` + `bun test` jalan
|
||
|
||
### Fase 1 — Domain layer
|
||
- [x] `core`: ChatMessage, Conversation, ChatRequest/Response/StreamEvent, SseParser, ToolCall, repairJson, sanitizeToolArguments, ToolCallResult, UsageStats, Store
|
||
- [x] `auth`: Session, SessionId (path-traversal), OAuthToken/Config, repo/service trait
|
||
- [x] `cms`: AppConfig, ProviderConfig, ModelRole, Settings+resolveEffectiveModel, SettingsPatch, Memory, EditLog, repo/service trait
|
||
- [x] `agent`: Origin, Toast, AgentStatus, TurnEvent, SessionRuntime, AgentTurnParams, AgentProgress, ExecutionModel
|
||
- [x] `workflow`/`subagent`: WorkflowScript/NodeDirective, AccessTier, prompt builders
|
||
- [x] 27 test domain
|
||
|
||
### Fase 2 — Application layer
|
||
- [x] `ports`: ProviderService, PasswordService, TokenService, AuthService
|
||
- [x] `AgentTurnServiceImpl` (loop 50 iter, auto-compact 60k, adaptive token, temp, ErrorTracker, bounded parallel)
|
||
- [x] `OAuthUseCase` (PKCE S256 + CSRF state)
|
||
- [x] `SessionServiceImpl`, `MemoryServiceImpl`, `SettingsServiceImpl`, `ConversationServiceImpl`
|
||
- [x] 11 test application
|
||
|
||
### Fase 3a — LLM client
|
||
- [x] `LlmClient` (fetch, retry 10x backoff+jitter, abort 401/402/403)
|
||
- [x] `resolve_api_key`, `is_auth_error`, `backoff_for_error`
|
||
|
||
### Fase 3b — Tool system (`commit a291dd1`)
|
||
- [x] `Tool` interface, `ToolCtx`/`ToolCtxBuilder`, `InfrastructureToolExecutor`
|
||
- [x] Registry: `allTools` (37), `toolDefs`, `toolIsRisky`, `toolIsParallelSafe`
|
||
- [x] FS: read, write, edit, delete · **Guard**: graduated checks, resolve_path sandbox
|
||
- [x] Search: grep, glob (matcher ringan)
|
||
- [x] Shell: bash (timeout), bash_output, bash_kill · **Filter**: checkGitDestructive, checkCredentialRead
|
||
- [x] Git: git_operator, git_worktree, git_cred
|
||
- [x] Memory: remember, forget, recall
|
||
- [x] Util: cd, dir_list, dir_cache_update, pong, todowrite, todofinish
|
||
- [x] Reasoning/Plan: sequential_think, plan_enter, plan_ready
|
||
- [x] Web: web_search (async → SearXNG)
|
||
- [x] Semantic: semantic_search, rebuild_index, list_symbols (index multi-bahasa)
|
||
- [x] Best-practice: best_practice, commit_convention + BestPracticeEngine
|
||
- [x] Agent/Workflow: spawn_agents, spawn_pipeline, parallel_delegate, workflow_run, note_finding, read_findings, hive_mind (delegasi ke placeholder 3c/3d)
|
||
- [x] 16 test tool system
|
||
|
||
### Fase 3e — Persistence
|
||
- [x] File repos: settings, app_config (+ auto-detect kredensial Claude), conversation, edits.jsonl, markdown memory, rewind blob, session, oauth, session lock
|
||
- [x] `writeJsonAtomic`, `slugify`, `buildWorkspaceTree`, `runCommand`
|
||
|
||
### Fase 3c — Subagent engine
|
||
**Sumber Rust:** `apps/infrastructure/src/subagent/{engine,division,context,provider,spawn}.rs`, `subagent/auto/*`
|
||
|
||
- [x] `AccessTier.toolsFor(access)` — `tools/division.ts`: Read / Write / Full yang mem-filter registry
|
||
- [x] `run_agent` loop (`tools/engine.ts`): `MAX_ITERATIONS=25`, `TOOL_OUTPUT_MAX_CHARS=12_000`,
|
||
`MAX_CONSECUTIVE_TOOL_ERRORS=3`, `MAX_PARALLEL_TOOLS=8`; recovery note setelah 3 error beruntun
|
||
- [x] `SubagentContext` (`subagent/context.ts`) — directive, toolCtx, access, baseURL, apiKey, model
|
||
- [x] `spawn_subagent` — `spawn_tools.ts` — runAgent berbasis HTTP provider + config_resolver
|
||
- [x] `resolve_subagent_provider` (`subagent/provider.ts`) — pilih provider/model subagent dari settings + app_config
|
||
- [x] `spawn_tools.ts` — isi body `spawnAgents` (paralel) & `spawnPipeline` (berurutan)
|
||
- [x] `delegate.ts` — `runParallelDelegation` untuk `parallel_delegate`
|
||
- [x] `http_provider.ts` — shared HTTP provider service builder
|
||
- [ ] Auto-reviewer (`subagent/auto/*`) — `spawn_background_review`: git diff → LLM review → perbaiki;
|
||
wiring `review_enabled` (yang sekarang no-op di `Review` agent)
|
||
- [x] Placeholder di `apps/packages/infrastructure/src/subagent/*` diisi penuh
|
||
|
||
### Fase 3d — Workflow + hive-mind
|
||
**Sumber Rust:** `apps/infrastructure/src/workflow/{script,mod,docs}.rs`, `workflow/engine/{execution,phases,primitives}.rs`, `workflow/hive_mind/{complexity,cycle,synthesis,live,types}.rs`
|
||
|
||
- [x] `parse_workflow_script(yaml)` — parser YAML workflow (`workflow/script.ts`)
|
||
- [x] `execute_workflow(script, ctx)` — eksekusi multi-phase (`workflow/engine.ts` + `workflow/orchestrate.ts`)
|
||
- [x] Phase primitives: run sub-agent per node (`executePrimitive`)
|
||
- [x] `execute_cycle(cycle, ctx)` — hive-mind cycle, concurrency 8, kumpulkan NodeOutput (`workflow/cycle.ts`)
|
||
- [x] `synthesize_consensus(node_outputs, ctx)` — konsensus via LLM + fallback concat (`workflow/synthesis.ts`)
|
||
- [x] `complexity_heuristic` — pilih apakah cukup 1 cycle / butuh banyak (`workflow/complexity.ts`)
|
||
- [x] `docs.ts` — tulis convergence/decision doc (`workflow/docs.ts`)
|
||
- [x] Isi body `WorkflowRun` & `HiveMind` tool (ganti placeholder) di `tools/workflow.ts`
|
||
- [x] `orchestrate.ts` — top-level `executeWorkflow` + `executeHiveMind` entry points
|
||
|
||
---
|
||
|
||
## ⬜ Yang belum dikerjakan
|
||
|
||
### Fase 3f — Auth infrastructure
|
||
**Sumber Rust:** `apps/infrastructure/src/auth/{password,jwt,oauth_loopback,mod}.rs`
|
||
|
||
- [x] `password.ts` — Argon2 via `@node-rs/argon2` (hash + verify), port `PasswordService`
|
||
- [x] `jwt.ts` — HMAC HS256 via `node:crypto` (access + refresh token, typ, exp), port `TokenService`
|
||
- [x] `oauth_loopback.ts` — server TCP loopback lokal utk terima redirect OAuth; port `oauth_loopback`
|
||
- [x] `mod.ts` (`index.ts`) — composisi ke `AuthService`
|
||
- [x] Tambah dep `@node-rs/argon2` di `infrastructure/package.json`
|
||
|
||
### Fase 3g — IPC + bgbash
|
||
**Sumber Rust:** `apps/infrastructure/src/ipc/{frame,protocol,server,client,conn}.rs`, `apps/infrastructure/src/bgbash/{job,control}.rs`
|
||
|
||
- [x] `ipc/frame.ts` — length-prefixed framing via `node:net` (kirim/terima bingkai biner)
|
||
- [x] `ipc/conn.ts` — `Connection` framed JSON socket; `ipc/protocol.ts` — wire types
|
||
- [x] `ipc/server.ts` + `ipc/client.ts` — Unix socket server + attach client; protocol message (NLJSON/binary)
|
||
- [x] `bgbash/job.ts` — `BashJob` (pid, output file, log), `spawnBashJob`
|
||
- [x] `bgbash/control.ts` — registry job global: `register`, `cancel(job_id)`, `is_running`
|
||
- [x] Wire `bash` tool `run_in_background` → `spawnBashJob` + `register`; `bash_kill` → `cancel`
|
||
- [x] Update `bash_output` untuk baca dari registry/output dir
|
||
|
||
### Fase 4 — Gateway + CLI + TUI
|
||
**Sumber Rust:** `apps/gateway/src/{lib,main}.rs`, `apps/interfaces/tui/src/**`
|
||
|
||
- [x] `apps/interfaces/cli/` (paket Bun):
|
||
- [x] Parser arg: `--api/--daemon/--attach/--ws/--grpc/--web`, dispatch mode (`args.ts`)
|
||
- [x] Logging ke file (`index.ts`)
|
||
- [x] `run_single_process` — komposisi root: wire semua implementasi konkret sekali (`compose.ts`) + REPL loop
|
||
- [x] Bootstrap seed (`bootstrap.ts`) — wire-shape settings lama terbaca
|
||
- [x] Smoke: `bun run cli --help`, `bun run bootstrap` jalan; loop turn + tool read/write
|
||
- [ ] TUI mode (library TUI Bun):
|
||
- [ ] Event loop adaptive poll
|
||
- [ ] `Action` dispatcher + slash commands
|
||
- [ ] Transcript render (incremental cache), sidebar, overlays, markdown render
|
||
- [ ] Theme
|
||
- [ ] Bootstrap seed (`bootstrap`)
|
||
|
||
### Fase 5 — Server interface: api / daemon / ws / grpc / web
|
||
**Sumber Rust:** `apps/interfaces/{api,daemon,ws,grpc,web}/src/**`, `apps/infrastructure/src/middleware/*`
|
||
|
||
- [x] **API** (`apps/interfaces/api/`):
|
||
- [x] Endpoint: `auth` (login/register/refresh, rate-limited), `sessions` CRUD, `conversations`, `chat/completions` proxy, `health`
|
||
- [x] JWT middleware, CORS
|
||
- [x] DTO (dto/)
|
||
- [x] Smoke `bun run api`: health 200; register/login 200; wrong-pwd 401; protected-no-token 401; create/list/delete session; add conversation msg
|
||
- [ ] **daemon** — IPC server + attach TUI client (pakai IPC 3g)
|
||
- [x] **ws** — handler `/ws`, protokol token/connected/done/error; smoke: no-token rejected 401, with-token welcome
|
||
- [ ] **grpc** — stub (health endpoint)
|
||
- [ ] **web** — static server + proteksi traversal; frontend placeholder
|
||
|
||
### Fase 6 — Bersih-bersih + verifikasi akhir
|
||
- [ ] Hapus semua sisa crate Rust: `apps/{domain,application,infrastructure,gateway,bootstrap,interfaces/*}` file `.rs`, `Cargo.toml`, `Cargo.lock`, `.cargo/`, `lefthook.yml`
|
||
- [ ] Update `install.sh` → Bun
|
||
- [ ] Update CI workflow: `.github/workflows/{ci,deploy,release,flakehub-publish-rolling}.yml` → `bun install`/`bun test`/`bun run cli`
|
||
- [ ] Update `Dockerfile` + `flake.nix`/`shell.nix` bila dipakai (atau hapus)
|
||
- [ ] Ganti/luruskan tool release (semantic-release) ke pipeline Bun
|
||
- [ ] Berkas jadi: hapus folder Rust yang sudah kosong dari `apps/`
|
||
- [ ] Verifikasi akhir: `bun run build`, `bun test`, `tsc --noEmit`, `bun run cli --help`, smoke server api/ws/daemon
|
||
- [ ] Baca ulang file lama (`settings.json`, `app_config.json`, `conversation.json`, memory `.md`) — pastikan wire-shape terbaca
|
||
|
||
---
|
||
|
||
## Verifikasi global (tiap fase)
|
||
- [ ] `bun run build` / `tsc --noEmit` tanpa error (strict)
|
||
- [ ] `bun test` — port test Rust ke `*.test.ts` (SseParser, repairJson, resolveEffectiveModel, SessionId, turn_service, tool system, subagent, workflow)
|
||
- [ ] Smoke: `bun run cli --help`; agent mode TUI dengan model stub/lokal; pastikan loop turn + tool read/write
|
||
- [ ] Server: `bun run api` + hit health/auth; `bun run ws` + connect
|
||
- [ ] Commit tiap fase dengan message Bahasa Indonesia + Conventional Commits; test hijau sebelum lanjut
|
||
|
||
## Catatan
|
||
- Registry tool = **38** (terhitung workflow_run, note_finding, read_findings, hive_mind; ExploreCodebase tidak diport).
|
||
- `Tool.run` boleh return `string | Promise<string>`.
|
||
- `spawn_agents`/`spawn_pipeline`/`parallel_delegate` sudah terisi penuh via 3c.
|
||
- Workflow/hive-mind tools sudah terisi penuh via 3d.
|