4 Commits

Include renames

Author	SHA1	Message	Date
asepharyana	f9fecfc144	chore: clean up dead barrels, duplicate config, and orphaned frontend components ... Gateway: - Remove dead barrels (ai-moderation/index, attachment-upload/index, message-capture/index) — all consumers import files directly - Remove orphaned schema/ split dir (analytics/cache/messages/meta) — schema.ts is monolithic - Merge duplicate config singleton: delete shared/config/config.ts, point all 44 imports at shared/config/index Backend: - Remove dead commandHelper.ts (voice-era fallback), ws/index.ts barrel, health.schema.ts, moderationMetrics.ts, analysis.schema.ts (0 importers; metrics/handlers route directly) Frontend: - Remove orphaned CategoryDrilldown/CoverageTiles/TopicTrends, primitives/slot, use-mobile, use-mounted - Remove unused charts donut/sparkline (TopicTrends was only consumer) Kept (verified active): shared/database/index.ts facade (11 importers), hooks/index + lib/api/index barrels (10 importers), orpc/ws.ts, charts/index.ts barrel. Verified: tsc + biome + vitest per service (backend e2e 3 failures pre-existing on main); frontend next build 8 routes.	2026-09-24 13:23:38 +07:00
asepharyana	7ef86c81ca	feat(ai): audit + harden embedding pipeline ... - Normalize text before embedding (strip mentions/URLs/emoji/markdown/control chars, lowercase, truncate) on both write and query sides so vectors aren't diluted and tokens aren't wasted - embeddingClient: retry embeddings (maxRetries 2), validate batch dimension consistency, preserve index alignment for empty-normalized texts - archiveEmbedder: store normalized text in archive payload, skip empty-normalized content - backend: normalize search queries, make archive search similarity threshold configurable (AI_LLM_EMBEDDING_ARCHIVE_MIN_SIMILARITY, default 0.6)	2026-09-01 18:01:47 +07:00
Developer	7ab9a7fd2d	fix(automod): force float encoding for embeddings — Nvidia models reject base64 ... OpenAI SDK v6 defaults to encoding_format=base64; llama-nemotron-embed (Nvidia-backed) returns 400 'do not support base64'. Semantic cache was silently disabled in prod. encoding_format: 'float' fixes it.	2026-07-31 20:14:18 +07:00
Developer	1249ae81d8	perf(automod): compress prompts ~40% + semantic cache via AI_LLM_EMBEDDING_MODEL ... Prompt overhaul (token-frugal, same quality): - rules.ts 28KB -> 10.3KB: every normative rule kept (safe lists, SARA 6 kategori, LGBT/Israel zero tolerance, anti-evasion, decision tree, evasi hierarchy, image rules) with duplicated phrasing removed - examples.ts 24.7KB -> 20KB: all 31 teaching examples kept; analysis strings shortened, redundant categories/policy_version dropped from example outputs (both optional in the response schema) - output.ts 13.8KB -> 6.8KB: compressed schema + personality + format rules; CRITICAL bans on generic analysis and reply-context requirement retained - system.ts: MEDIA_INSTRUCTIONS compressed, key rules kept Semantic moderation cache (AI_LLM_EMBEDDING_MODEL): - New embeddingClient.ts: OpenAI-compatible embeddings + cosine similarity; degrades gracefully when model/key unset - textCacheStore: stores embedding JSON per verdict, findSimilarTextModeration reuses near-duplicate verdicts (min 0.97 cosine, processing locks skipped) - moderationOrchestrator: after exact-hash miss, embed text-only targets and reuse stored verdict for near-duplicates -> skips expensive chat completion for spam variants; fresh verdicts written back with embedding - Config: AI_LLM_EMBEDDING_MODEL / MIN_SIMILARITY (0.97) / MAX_CANDIDATES (30) - Migration 0012: ADD COLUMN embedding to text_analysis_cache (idempotent) - .env.example documents the new vars	2026-07-31 19:37:53 +07:00