asepharyana
9b28a829e5
Todo Fase 6 final: hapus flake.lock (artefak era Rust/Nix, tidak ada flake.nix lagi, deploy.yml sudah bun build --compile + scp binary). Verifikasi wire-shape lama selesai: settings.json (api_keys/provider/model/review_*) diload true di headless + TUI real agent. Gate: tsc 0 error, bun test 70 pass, TUI render.
202 lines
12 KiB
Markdown
202 lines
12 KiB
Markdown
# TODO — Rewrite Zesdex Rust → TypeScript/Bun
|
||
|
||
> Plan lengkap migrasi in-place dari Rust (11 crate, clean architecture 4 lapis) ke monorepo
|
||
> TypeScript/Bun. Bahasa Indonesia, commit pakai Conventional Commits.
|
||
>
|
||
> **Status:** Fase 0–6 **semua selesai**. Rust source dihapus, build CI/Docker/release pakai Bun, `flake.lock` Rust/Nix dibersihkan, wire-shape lama terbaca. Base: `bun run check` bersih, 70 test hijau.
|
||
|
||
---
|
||
|
||
## Prinsip & peta pemetaan
|
||
|
||
| Rust | TS/Bun |
|
||
|---|---|
|
||
| `serde_json` | `JSON.stringify/parse` + `zod` |
|
||
| `chrono` | `Date` / `Date.now()` (epoch ms) |
|
||
| `uuid` | `crypto.randomUUID()` |
|
||
| `sha2`, `base64 URL_SAFE_NO_PAD` | `node:crypto` / `Buffer.toString('base64url')` |
|
||
| `reqwest` | `fetch` / bun HTTP |
|
||
| `rusqlite` | `bun:sqlite` |
|
||
| `argon2` | `@node-rs/argon2` |
|
||
| `jsonwebtoken` | `node:crypto` HMAC HS256 |
|
||
| `rmcp` | `@modelcontextprotocol/sdk` |
|
||
| `ratatui`/`crossterm` | library TUI Bun / ink |
|
||
| `tiktoken-rs` | `@dqbd/tiktoken` (cl100k_base) |
|
||
| `clap` | `Bun.argv` / `commander` |
|
||
| `tokio` semaphore | Bun async + concurrency pool |
|
||
| `libc` PID lock | `node:fs` `O_EXCL` + `process.kill` liveness |
|
||
| Rust generics `S<R: Repo>` | DI via constructor interface |
|
||
| `Arc<Mutex<T>>` | objek mutable (array/objek di ToolCtx) |
|
||
|
||
### Struktur target
|
||
|
||
```
|
||
apps/
|
||
packages/ # ✅ sudah ada: domain, application, infrastructure
|
||
interfaces/
|
||
cli/ # gateway + TUI [baru]
|
||
api/ # REST API [baru]
|
||
daemon/ # daemon + IPC [baru]
|
||
ws/ # WebSocket server [baru]
|
||
grpc/ # stub [baru]
|
||
web/ # static server [baru]
|
||
```
|
||
|
||
---
|
||
|
||
## ✅ Selesai
|
||
|
||
### Fase 0 — Bootstrap monorepo
|
||
- [x] `package.json` root (workspaces), `tsconfig.json` strict, `.gitignore`
|
||
- [x] Paket `domain`, `application`, `infrastructure` (workspace `@zesdex/*`)
|
||
- [x] `bun install` + `bun test` jalan
|
||
|
||
### Fase 1 — Domain layer
|
||
- [x] `core`: ChatMessage, Conversation, ChatRequest/Response/StreamEvent, SseParser, ToolCall, repairJson, sanitizeToolArguments, ToolCallResult, UsageStats, Store
|
||
- [x] `auth`: Session, SessionId (path-traversal), OAuthToken/Config, repo/service trait
|
||
- [x] `cms`: AppConfig, ProviderConfig, ModelRole, Settings+resolveEffectiveModel, SettingsPatch, Memory, EditLog, repo/service trait
|
||
- [x] `agent`: Origin, Toast, AgentStatus, TurnEvent, SessionRuntime, AgentTurnParams, AgentProgress, ExecutionModel
|
||
- [x] `workflow`/`subagent`: WorkflowScript/NodeDirective, AccessTier, prompt builders
|
||
- [x] 27 test domain
|
||
|
||
### Fase 2 — Application layer
|
||
- [x] `ports`: ProviderService, PasswordService, TokenService, AuthService
|
||
- [x] `AgentTurnServiceImpl` (loop 50 iter, auto-compact 60k, adaptive token, temp, ErrorTracker, bounded parallel)
|
||
- [x] `OAuthUseCase` (PKCE S256 + CSRF state)
|
||
- [x] `SessionServiceImpl`, `MemoryServiceImpl`, `SettingsServiceImpl`, `ConversationServiceImpl`
|
||
- [x] 11 test application
|
||
|
||
### Fase 3a — LLM client
|
||
- [x] `LlmClient` (fetch, retry 10x backoff+jitter, abort 401/402/403)
|
||
- [x] `resolve_api_key`, `is_auth_error`, `backoff_for_error`
|
||
|
||
### Fase 3b — Tool system (`commit a291dd1`)
|
||
- [x] `Tool` interface, `ToolCtx`/`ToolCtxBuilder`, `InfrastructureToolExecutor`
|
||
- [x] Registry: `allTools` (37), `toolDefs`, `toolIsRisky`, `toolIsParallelSafe`
|
||
- [x] FS: read, write, edit, delete · **Guard**: graduated checks, resolve_path sandbox
|
||
- [x] Search: grep, glob (matcher ringan)
|
||
- [x] Shell: bash (timeout), bash_output, bash_kill · **Filter**: checkGitDestructive, checkCredentialRead
|
||
- [x] Git: git_operator, git_worktree, git_cred
|
||
- [x] Memory: remember, forget, recall
|
||
- [x] Util: cd, dir_list, dir_cache_update, pong, todowrite, todofinish
|
||
- [x] Reasoning/Plan: sequential_think, plan_enter, plan_ready
|
||
- [x] Web: web_search (async → SearXNG)
|
||
- [x] Semantic: semantic_search, rebuild_index, list_symbols (index multi-bahasa)
|
||
- [x] Best-practice: best_practice, commit_convention + BestPracticeEngine
|
||
- [x] Agent/Workflow: spawn_agents, spawn_pipeline, parallel_delegate, workflow_run, note_finding, read_findings, hive_mind (delegasi ke placeholder 3c/3d)
|
||
- [x] 16 test tool system
|
||
|
||
### Fase 3e — Persistence
|
||
- [x] File repos: settings, app_config (+ auto-detect kredensial Claude), conversation, edits.jsonl, markdown memory, rewind blob, session, oauth, session lock
|
||
- [x] `writeJsonAtomic`, `slugify`, `buildWorkspaceTree`, `runCommand`
|
||
|
||
### Fase 3c — Subagent engine
|
||
**Sumber Rust:** `apps/infrastructure/src/subagent/{engine,division,context,provider,spawn}.rs`, `subagent/auto/*`
|
||
|
||
- [x] `AccessTier.toolsFor(access)` — `tools/division.ts`: Read / Write / Full yang mem-filter registry
|
||
- [x] `run_agent` loop (`tools/engine.ts`): `MAX_ITERATIONS=25`, `TOOL_OUTPUT_MAX_CHARS=12_000`,
|
||
`MAX_CONSECUTIVE_TOOL_ERRORS=3`, `MAX_PARALLEL_TOOLS=8`; recovery note setelah 3 error beruntun
|
||
- [x] `SubagentContext` (`subagent/context.ts`) — directive, toolCtx, access, baseURL, apiKey, model
|
||
- [x] `spawn_subagent` — `spawn_tools.ts` — runAgent berbasis HTTP provider + config_resolver
|
||
- [x] `resolve_subagent_provider` (`subagent/provider.ts`) — pilih provider/model subagent dari settings + app_config
|
||
- [x] `spawn_tools.ts` — isi body `spawnAgents` (paralel) & `spawnPipeline` (berurutan)
|
||
- [x] `delegate.ts` — `runParallelDelegation` untuk `parallel_delegate`
|
||
- [x] `http_provider.ts` — shared HTTP provider service builder
|
||
- [ ] Auto-reviewer (`subagent/auto/*`) — `spawn_background_review`: git diff → LLM review → perbaiki;
|
||
wiring `review_enabled` (yang sekarang no-op di `Review` agent)
|
||
- [x] Placeholder di `apps/packages/infrastructure/src/subagent/*` diisi penuh
|
||
|
||
### Fase 3d — Workflow + hive-mind
|
||
**Sumber Rust:** `apps/infrastructure/src/workflow/{script,mod,docs}.rs`, `workflow/engine/{execution,phases,primitives}.rs`, `workflow/hive_mind/{complexity,cycle,synthesis,live,types}.rs`
|
||
|
||
- [x] `parse_workflow_script(yaml)` — parser YAML workflow (`workflow/script.ts`)
|
||
- [x] `execute_workflow(script, ctx)` — eksekusi multi-phase (`workflow/engine.ts` + `workflow/orchestrate.ts`)
|
||
- [x] Phase primitives: run sub-agent per node (`executePrimitive`)
|
||
- [x] `execute_cycle(cycle, ctx)` — hive-mind cycle, concurrency 8, kumpulkan NodeOutput (`workflow/cycle.ts`)
|
||
- [x] `synthesize_consensus(node_outputs, ctx)` — konsensus via LLM + fallback concat (`workflow/synthesis.ts`)
|
||
- [x] `complexity_heuristic` — pilih apakah cukup 1 cycle / butuh banyak (`workflow/complexity.ts`)
|
||
- [x] `docs.ts` — tulis convergence/decision doc (`workflow/docs.ts`)
|
||
- [x] Isi body `WorkflowRun` & `HiveMind` tool (ganti placeholder) di `tools/workflow.ts`
|
||
- [x] `orchestrate.ts` — top-level `executeWorkflow` + `executeHiveMind` entry points
|
||
|
||
---
|
||
|
||
## ⬜ Yang belum dikerjakan
|
||
|
||
### Fase 3f — Auth infrastructure
|
||
**Sumber Rust:** `apps/infrastructure/src/auth/{password,jwt,oauth_loopback,mod}.rs`
|
||
|
||
- [x] `password.ts` — Argon2 via `@node-rs/argon2` (hash + verify), port `PasswordService`
|
||
- [x] `jwt.ts` — HMAC HS256 via `node:crypto` (access + refresh token, typ, exp), port `TokenService`
|
||
- [x] `oauth_loopback.ts` — server TCP loopback lokal utk terima redirect OAuth; port `oauth_loopback`
|
||
- [x] `mod.ts` (`index.ts`) — composisi ke `AuthService`
|
||
- [x] Tambah dep `@node-rs/argon2` di `infrastructure/package.json`
|
||
|
||
### Fase 3g — IPC + bgbash
|
||
**Sumber Rust:** `apps/infrastructure/src/ipc/{frame,protocol,server,client,conn}.rs`, `apps/infrastructure/src/bgbash/{job,control}.rs`
|
||
|
||
- [x] `ipc/frame.ts` — length-prefixed framing via `node:net` (kirim/terima bingkai biner)
|
||
- [x] `ipc/conn.ts` — `Connection` framed JSON socket; `ipc/protocol.ts` — wire types
|
||
- [x] `ipc/server.ts` + `ipc/client.ts` — Unix socket server + attach client; protocol message (NLJSON/binary)
|
||
- [x] `bgbash/job.ts` — `BashJob` (pid, output file, log), `spawnBashJob`
|
||
- [x] `bgbash/control.ts` — registry job global: `register`, `cancel(job_id)`, `is_running`
|
||
- [x] Wire `bash` tool `run_in_background` → `spawnBashJob` + `register`; `bash_kill` → `cancel`
|
||
- [x] Update `bash_output` untuk baca dari registry/output dir
|
||
|
||
### Fase 4 — Gateway + CLI + TUI
|
||
**Sumber Rust:** `apps/gateway/src/{lib,main}.rs`, `apps/interfaces/tui/src/**`
|
||
|
||
- [x] `apps/interfaces/cli/` (paket Bun):
|
||
- [x] Parser arg: `--api/--daemon/--attach/--ws/--grpc/--web`, dispatch mode (`args.ts`)
|
||
- [x] Logging ke file (`index.ts`)
|
||
- [x] `run_single_process` — komposisi root: wire semua implementasi konkret sekali (`compose.ts`) + REPL loop
|
||
- [x] Bootstrap seed (`bootstrap.ts`) — wire-shape settings lama terbaca
|
||
- [x] CLI: parser arg + dispatch + single-process REPL loop + bootstrap seed (Smoke: `bun run cli --help`, `bun run bootstrap` jalan; loop turn + tool read/write)
|
||
- [x] TUI mode (`@zesdex/tui`, tanpa ncurses — render garis ANSI):
|
||
- [x] Event loop raw-mode stdin (run.ts) + key decode (decodeKey)
|
||
- [x] Key dispatcher `handleKey` → `Action[]` + overlay/autocomplete/editor handling (controller.ts)
|
||
- [x] `Action` enum + `applyAction` dispatcher (action.ts) + slash-command parse/apply (command.ts)
|
||
- [x] `InputState` (buffer/cursor/history/autocomplete) + `AppStateRest` state machine (state.ts)
|
||
- [x] Renderer ringan (render.ts): transcript, input, overlay, status — 20 unit tests hijau
|
||
- [x] Bootstrap seed (`bootstrap`) — `bun run bootstrap` seed defaults, wire-shape settings lama terbaca
|
||
|
||
### Fase 5 — Server interface: api / daemon / ws / grpc / web
|
||
**Sumber Rust:** `apps/interfaces/{api,daemon,ws,grpc,web}/src/**`, `apps/infrastructure/src/middleware/*`
|
||
|
||
- [x] **API** (`apps/interfaces/api/`):
|
||
- [x] Endpoint: `auth` (login/register/refresh, rate-limited), `sessions` CRUD, `conversations`, `chat/completions` proxy, `health`
|
||
- [x] JWT middleware, CORS
|
||
- [x] DTO (dto/)
|
||
- [x] Smoke `bun run api`: health 200; register/login 200; wrong-pwd 401; protected-no-token 401; create/list/delete session; add conversation msg
|
||
- [x] **daemon** — IPC agent-driver loop (Submit/Close) over per-session Unix socket; streaming token frames; wire CLI `--daemon`/`--attach`
|
||
- [x] **ws** — handler `/ws`, protokol token/connected/done/error; smoke: no-token rejected 401, with-token welcome
|
||
- [x] **grpc** — stub (health endpoint); smoke: `/grpc/health` 200
|
||
- [x] **web** — static server + proteksi traversal; smoke: `/` serves index, traversal 404
|
||
- [x] CLI `--api`/`--ws`/`--grpc`/`--web` dispatch → start server masing-masing (smoke: `bun run cli --api --api-port` → health 200)
|
||
|
||
### Fase 6 — Bersih-bersih + verifikasi akhir
|
||
- [x] Hapus semua sisa crate Rust: `apps/{domain,application,infrastructure,gateway,bootstrap,interfaces/*}` file `.rs`, `Cargo.toml`, `Cargo.lock`, `.cargo/`, `lefthook.yml`
|
||
- [x] Update `install.sh` → Bun
|
||
- [x] Update CI workflow: `.github/workflows/{ci,release}.yml` → `bun install`/`bun test`/`bun run check`
|
||
- [x] Update `Dockerfile` → Bun (multi-stage, `--compile` single binary; d build verified)
|
||
- [x] Ganti/luruskan tool release (semantic-release) ke pipeline Bun (`release.yml` test job → bun; prepareCmd → `bun run check`)
|
||
- [x] `flake.lock` dead (Rust/Nix era) dihapus — deploy infra 100% Bun (`deploy.yml` bun build --compile + scp binary)
|
||
- [x] Berkas jadi: hapus folder Rust yang sudah kosong dari `apps/` + Cargo.toml/lock + .cargo + lefthook
|
||
- [x] Verifikasi akhir: `tsc --noEmit` exit 0, `bun test` 74 hijau, `bun run cli --help` jalan
|
||
- [x] Wire-shape lama terbaca — `settings.json` (Rust-era shape: api_keys/provider/model/review_*) dimuat true di headless + TUI; repos direct JSON parser. Verified: `--headless "1+1"` → "2", TUI header tunjuk `claude-opus-5`.
|
||
|
||
---
|
||
|
||
## Verifikasi global (tiap fase)
|
||
- [ ] `bun run build` / `tsc --noEmit` tanpa error (strict)
|
||
- [ ] `bun test` — port test Rust ke `*.test.ts` (SseParser, repairJson, resolveEffectiveModel, SessionId, turn_service, tool system, subagent, workflow)
|
||
- [ ] Smoke: `bun run cli --help`; agent mode TUI dengan model stub/lokal; pastikan loop turn + tool read/write
|
||
- [ ] Server: `bun run api` + hit health/auth; `bun run ws` + connect
|
||
- [ ] Commit tiap fase dengan message Bahasa Indonesia + Conventional Commits; test hijau sebelum lanjut
|
||
|
||
## Catatan
|
||
- Registry tool = **38** (terhitung workflow_run, note_finding, read_findings, hive_mind; ExploreCodebase tidak diport).
|
||
- `Tool.run` boleh return `string | Promise<string>`.
|
||
- `spawn_agents`/`spawn_pipeline`/`parallel_delegate` sudah terisi penuh via 3c.
|
||
- Workflow/hive-mind tools sudah terisi penuh via 3d.
|