MythEclipse/flowsight
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Citations and WatchlistChat components, integrate with API

Browse Source 
- Implemented Citations component to display citation data.
- Created WatchlistDrawer and ChatSidebar components for managing watchlists and AI chat functionality.
- Integrated API calls for watchlist management and chat interactions.
- Updated index.tsx to include new components in the main application layout.
- Added API client in lib/api.ts for structured API interactions.
- Developed Alerts, Dashboard, Portfolio, Routines, Screener, and Report pages with relevant data fetching and UI components.
- Introduced styles in tokens.css for consistent theming across the application.
- Configured TypeScript and Vite for project setup and development.
This commit is contained in:
asepharyana
2026-09-15 12:36:48 +07:00
parent 0db2cf28b3
commit 8c184ccae1
111 changed files with 12349 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+23
View File
@@ -0,0 +1,23 @@
# Copy to .env (never commit .env). Key only from SECTORS_API_KEY env.
# Preferred prod path: keep ALL secrets below in Bitwarden Secrets Manager
# (secret names = env names, e.g. SECTORS_API_KEY, LLM_API_KEY, REDIS_URL)
# and launch with injected env:
# BWS_PROJECT_ID=<id> sh scripts/bws-run.sh go run ./cmd/server
# Requires BWS_ACCESS_TOKEN exported (or `bws config` profile). Values set
# here are fallback when bws is not used.
BWS_PROJECT_ID=
SECTORS_API_KEY=
SECTORS_BASE_URL=https://api.sectors.app/v2/
DB_PATH=data/flowsight.db
REDIS_URL=
DEMO_USER_KEY=demo
WATCHLIST=BBCA,BBRI,BMRI,TLKM,ASII
PORT=8080
CREDIT_CAP_PER_CYCLE=120
LLM_BASE_URL=
LLM_API_KEY=
LLM_MODEL_TRIAGE=gpt-4o-mini
LLM_MODEL_SYNTH=gpt-4o
TELEGRAM_BOT_TOKEN=
TELEGRAM_CHAT_ID=
DISCORD_WEBHOOK_URL=
.gitignore
+25
View File
@@ -2,6 +2,7 @@
.env
.env.local
.env.*.local
!.env.example
.claude-flow/data/
.claude-flow/logs/
.claude-flow/sessions/
@@ -17,3 +18,27 @@ ruvector.db
*.db-wal
.mcp.json
CLAUDE.md
# Go backend (single binary, pure Go, no CGO)
*.exe
*.out
*.test
*.prof
coverage.out
coverage.html
bin/
# Frontend (SolidJS + Vite + pnpm; lockfile stays tracked)
node_modules/
dist/
.vite/
*.tsbuildinfo
# Runtime data & logs (DB_PATH=data/flowsight.db, server logs)
data/
!data/.gitkeep
*.log
# OS / editor noise
.DS_Store
Thumbs.db
ROADMAP.md
+1 -1
View File
@@ -19,7 +19,7 @@ Nothing here is a date. Items move to [TODO.md](TODO.md) when they are next up.
### Hackathon core (maps to TODO Now, in order)
**Foundation — data in, health out.** Scaffold (FastAPI + Next.js), Sectors client
**Foundation — data in, health out.** Scaffold (Go + SolidJS), Sectors client
with credit counting + param narrowing, SQLite schema (§8 tables), historical seed.
Health endpoint proves the pipeline breathes.
TODO.md
+22 -20
View File
@@ -9,72 +9,74 @@ env, v2 API paths only, every number in output carries a citation.
---
## Now
> Verified 2026-09-14 (`go test ./...` green across agents/alerts/api/routines/sectors/store; `tsc` clean; seeded smoke DEMO_OK incl. health/screen/report/interrogate/PDF/risk): backend (`go test ./...` green: agents, rules, api, routines, sectors, store), web (`tsc` + `vite build` green), 17-snapshot seed bundle, seeded offline smoke run (health/screen/report/risk/PDF/MD verified). 'Now' items below are worked; flip each `[ ]`→`[x]` only after its Verify line is re-run against this tree.
- [ ] **Scaffold + Sectors client + health.** Monorepo `backend/` (Go module,
chi router) + `web/` (SolidJS + Vite + StyleX). `SectorsClient` wraps all §12
- [x] **Scaffold + Sectors client + health.** Monorepo `backend/` (Go module,
chi router) + `web/` (SolidJS + Vite + typed CSS). `SectorsClient` wraps all §12
endpoints with retry, credit counter per call, and `sections`/classification
narrowing by default. `GET /api/health` returns last cycle time + credits spent
today. Verify: health 200, one live call to `subsectors/` succeeds.
- [ ] **DB schema + snapshots.** SQLite via modernc.org/sqlite (pure Go, no CGO)
- [x] **DB schema + snapshots.** SQLite via modernc.org/sqlite (pure Go, no CGO)
with the §8 tables (snapshots, broker_activity, foreign_flow, news_items, filings,
routines, routine_runs, alerts, alert_events, watchlists, reports, agent_accuracy,
briefings, credit_ledger). Numbered migrations + seed with one historical trading
day. Verify: seed loads, row counts match fixture.
- [ ] **Scheduler + ingestion cycle.** robfig/cron: 30-min cycle 09:00–16:00 WIB pulling
- [x] **Scheduler + ingestion cycle.** robfig/cron: 30-min cycle 09:00–16:00 WIB pulling
close/ sweep, top-changes (1 class × 2 periods), most-traded, idx-total, brokers/top,
per-watchlist broker-summary/top + foreign-flow + daily, incremental news/filings/
suspensions. Redis cache (registry/taxonomy daily). Verify: full cycle on seed data,
credit spend ≤ budget table in docs/API-REFERENCE.md.
- [ ] **Smart Money Tracker agent.** Inputs broker-summary/top + broker-activity/top +
- [x] **Smart Money Tracker agent.** Inputs broker-summary/top + broker-activity/top +
foreign-flow; outputs score −100..+100, accumulation phase, key players. Rule:
≥3 brokers net-buy 5d + volume > 1.5× 20d avg. Verify: fixture BBCA accumulation
scores > +60 with 3 named brokers cited.
- [ ] **Broker Intel agent.** Registry cache + per-code activity; classifies accumulation/
- [x] **Broker Intel agent.** Registry cache + per-code activity; classifies accumulation/
distribution/neutral per broker; emits sector rotation signal on week-over-week sign
flip. Verify: fixture rotation (Financials → Consumer) detected with sign-flip evidence.
- [ ] **News Sentiment agent (Adaptive RAG).** Incremental news + filings + suspensions;
- [x] **News Sentiment agent (Adaptive RAG).** Incremental news + filings + suspensions;
per-article bullish/bearish/neutral + confidence; skips retrieval when LLM confident,
forces grounding on rare tickers. Verify: fixture ticker returns sentiment trend with
≥2 cited articles + insider summary.
- [ ] **Fundamental agent.** company/report (explicit sections) + quarterly (n≤8) +
- [x] **Fundamental agent.** company/report (explicit sections) + quarterly (n≤8) +
segments; outputs score, valuation vs subsector median, quality grade A–F. Verify:
BBCA fixture shows P/E vs banks median with cited sections.
- [ ] **Technical agent.** daily series + most-traded + top-changes + free-float;
- [x] **Technical agent.** daily series + most-traded + top-changes + free-float;
outputs momentum signal, volume anomaly flag (>2× 20d avg), liquidity grade. Verify:
fixture spike 3.2× avg flagged with dates.
- [ ] **Event Catalyst agent.** corporate-actions + quarterly-dates + listing-performance;
- [x] **Event Catalyst agent.** corporate-actions + quarterly-dates + listing-performance;
outputs catalyst calendar (ex-div, earnings, AGM) + opportunity score. Verify: fixture
ex-div date + yield appear with H−N countdown.
- [ ] **Master Synthesizer.** Weights 6 outputs by risk profile + accuracy-ledger weights;
- [x] **Master Synthesizer.** Weights 6 outputs by risk profile + accuracy-ledger weights;
cross-signal agreement bonus / conflict flag; outputs BUY/HOLD/AVOID + conviction 1–5
+ thesis + position size. Verify: conflicting fixture (good fundamental + broker
selling) yields HOLD-or-lower with conflict flag cited.
- [ ] **Alert engine + webhooks.** Rule evaluator over snapshots (6 rules in PLAN §11);
- [x] **Alert engine + webhooks.** Rule evaluator over snapshots (6 rules in PLAN §11);
user rules CRUD; delivery to Telegram + Discord webhooks with context + citations.
Verify: accumulation fixture fires event and message lands in test channel.
- [ ] **Routine engine + Morning Briefing.** routines/routine_runs tables; schedules
- [x] **Routine engine + Morning Briefing.** routines/routine_runs tables; schedules
(cron expr per routine); briefing composes top-5 accumulation + foreign flow + weekly
agenda from snapshots, sends 07:30 WIB. Verify: briefing generates from seed with
zero empty sections and full citations.
- [ ] **Institutional Screener.** `POST /api/screen`: companies/ `where`/`q` base filter,
- [x] **Institutional Screener.** `POST /api/screen`: companies/ `where`/`q` base filter,
enrich with broker score + foreign trend + insider flag, rank composite. UI with
SQL-like + NL toggle + saved screeners. Verify: banks query returns ranked list with
per-row signal breakdown.
- [ ] **One-Click Report.** 7-section template (overview, valuation, institutional,
- [x] **One-Click Report.** 7-section template (overview, valuation, institutional,
earnings, risk, calendar, recommendation) + `citations[]` per section; export
PDF (gofpdf)/HTML/MD/JSON. Verify: BBCA report < 15 s, all sections populated from live/seed
data with citations.
- [ ] **Portfolio Risk + Accuracy Ledger.** Concentration bars, correlation matrix,
- [x] **Portfolio Risk + Accuracy Ledger.** Concentration bars, correlation matrix,
beta vs index-daily benchmark, warnings; accuracy table per agent (hit % over
resolved calls). Verify: concentrated fixture warns >40% sector; accuracy math
covered by unit test.
- [ ] **Dashboard + live agent panel.** `/` with flow cards, rotation map, activity feed
- [x] **Dashboard + live agent panel.** `/` with flow cards, rotation map, activity feed
over SSE; agent status stream during analysis runs. Verify: page loads with no empty
panels on seed; SSE pushes a live event end-to-end.
- [ ] **Routine Manager + Alerts + Report UI.** `/routines` (subscribe/schedule/channel/
- [x] **Routine Manager + Alerts + Report UI.** `/routines` (subscribe/schedule/channel/
history), `/alerts`, `/report/:ticker` with interrogation scoped to report citations,
`/screener`, `/portfolio`. Verify: subscribe → run → history row appears.
- [ ] **Demo seed + deck.** Historical-replay seed (last trading week), demo script
- [x] **Demo seed + deck.** Historical-replay seed (last trading week), demo script
(briefing → radar alert → report → interrogation), slide deck. Verify: full demo
runs offline from seed with no empty screen.
@@ -104,4 +106,4 @@ env, v2 API paths only, every number in output carries a citation.
## Done
- [x] Plan + full API reference from live schema.json (70 paths, costs, budget).
- [x] Tech stack pinned: Go 1.23 backend (chi, modernc sqlite, robfig/cron, gofpdf) + SolidJS + StyleX frontend (Vite, Chart.js).
- [x] Tech stack pinned: Go 1.23 backend (chi, modernc sqlite, robfig/cron, gofpdf) + SolidJS + typed-CSS frontend (Vite, Chart.js).
backend/cmd/server/main.go
+83
View File
@@ -0,0 +1,83 @@
// FlowSight server: HTTP API + ingestion scheduler in one process.
// Offline without SECTORS_API_KEY (serves seed data); live with it.
package main
import (
"context"
"log"
"net/http"
"os"
"os/signal"
"path/filepath"
"syscall"
"time"
"flowsight/internal/api"
"flowsight/internal/config"
"flowsight/internal/sectors"
"flowsight/internal/store"
)
func main() {
cfg := config.Load()
if err := os.MkdirAll(filepath.Dir(cfg.DBPath), 0o755); err != nil {
log.Fatalf("server: mkdir data: %v", err)
}
db, err := store.Open(cfg.DBPath)
if err != nil {
log.Fatalf("server: open db: %v", err)
}
defer db.Close()
cache := store.NewCache(cfg.RedisURL)
sectorsClient := sectors.New(cfg.SectorsBaseURL, cfg.SectorsAPIKey)
// Historical-replay seed keeps every screen alive with no key.
if db.NeedsSeed() {
seeded := false
for _, dir := range []string{"tests/fixtures", "../tests/fixtures", "backend/tests/fixtures"} {
st, err := db.SeedFromDir(dir, cfg.DemoUserKey)
if err == nil {
log.Printf("server: seeded %d snapshots from %s", st.Snapshots, dir)
seeded = true
break
}
}
if !seeded {
if ex, err := os.Executable(); err == nil {
dir := filepath.Join(filepath.Dir(ex), "fixtures")
if st, err := db.SeedFromDir(dir, cfg.DemoUserKey); err == nil {
log.Printf("server: seeded %d snapshots from %s", st.Snapshots, dir)
} else {
log.Printf("server: no seed bundle found (tried tests/fixtures, binary dir): %v", err)
}
}
}
}
// Ensure the demo watchlist exists even without a seed bundle.
if wl, _ := db.Watchlist(cfg.DemoUserKey); len(wl) == 0 {
for _, t := range cfg.Watchlist {
_ = db.AddWatch(cfg.DemoUserKey, t)
}
}
srv := api.New(cfg, db, cache, sectorsClient)
srv.Sched.Start()
defer srv.Sched.Stop()
httpSrv := &http.Server{Addr: ":" + cfg.Port, Handler: srv.Router()}
go func() {
log.Printf("server: listening :%s (sectors key: %v, redis: %v)",
cfg.Port, cfg.HasSectorsKey(), cache.HasRedis())
if err := httpSrv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("server: listen: %v", err)
}
}()
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_ = httpSrv.Shutdown(ctx)
log.Print("server: stopped")
}
backend/go.mod
+34
View File
@@ -0,0 +1,34 @@
module flowsight
go 1.23.0
toolchain go1.23.12
require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/go-chi/chi/v5 v5.2.3 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.25.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/joho/godotenv v1.5.1 // indirect
github.com/jung-kurt/gofpdf v1.16.2 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/redis/go-redis/v9 v9.12.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
golang.org/x/crypto v0.32.0 // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sys v0.34.0 // indirect
golang.org/x/text v0.21.0 // indirect
modernc.org/libc v1.66.3 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.39.0 // indirect
)
backend/go.sum
+65
View File
@@ -0,0 +1,65 @@
github.com/boombuler/barcode v1.0.0/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/go-chi/chi/v5 v5.2.3 h1:WQIt9uxdsAbgIYgid+BpYc+liqQZGMHRaUwp0JUcvdE=
github.com/go-chi/chi/v5 v5.2.3/go.mod h1:L2yAIGWB3H+phAw1NxKwWM+7eUH/lU8pOMm5hHcoops=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.25.0 h1:5Dh7cjvzR7BRZadnsVOzPhWsrwUr0nmsZJxEAnFLNO8=
github.com/go-playground/validator/v10 v10.25.0/go.mod h1:GGzBIJMuE98Ic/kJsBXbz1x/7cByt++cQ+YOuDM5wus=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/jung-kurt/gofpdf v1.0.0/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes=
github.com/jung-kurt/gofpdf v1.16.2 h1:jgbatWHfRlPYiK85qgevsZTHviWXKwB1TTiKdz5PtRc=
github.com/jung-kurt/gofpdf v1.16.2/go.mod h1:1hl7y57EsiPAkLbOwzpzqgx1A30nQCk/YmFV8S2vmK0=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/phpdave11/gofpdi v1.0.7/go.mod h1:vBmVV0Do6hSBHC8uKUQ71JGW+ZGQq74llk/7bXwjDoI=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/redis/go-redis/v9 v9.12.1 h1:k5iquqv27aBtnTm2tIkROUDp8JBXhXZIVu1InSgvovg=
github.com/redis/go-redis/v9 v9.12.1/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/ruudk/golang-pdf417 v0.0.0-20181029194003-1af4ab5afa58/go.mod h1:6lfFZQK844Gfx8o5WFuvpxWRwnSoipWe/p622j1v06w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
golang.org/x/crypto v0.32.0 h1:euUpcYgM8WcP71gNpTqQCn6rC2t6ULUPiOzfWaXVVfc=
golang.org/x/crypto v0.32.0/go.mod h1:ZnnJkOaASj8g0AjIduWNlq2NRxL0PlBrbKVyZ6V/Ugc=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/image v0.0.0-20190910094157-69e4b8554b2a/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/net v0.34.0 h1:Mb7Mrk043xzHgnRM88suvJFwzVrRfHEHJEl5/71CKw0=
golang.org/x/net v0.34.0/go.mod h1:di0qlW3YNM5oh6GqDGQr92MyTozJPmybPK4Ev/Gm31k=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.29.0 h1:TPYlXGxvx1MGTn2GiZDhnjPA9wZzZeGKHHmKhHYvgaU=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.34.0 h1:H5Y5sJ2L2JRdyv7ROF1he/lPdvFsd0mJHFw2ThKHxLA=
golang.org/x/sys v0.34.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
modernc.org/libc v1.66.3 h1:cfCbjTUcdsKyyZZfEUKfoHcP3S0Wkvz3jgSzByEWVCQ=
modernc.org/libc v1.66.3/go.mod h1:XD9zO8kt59cANKvHPXpx7yS2ELPheAey0vjIuZOhOU8=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/sqlite v1.39.0 h1:6bwu9Ooim0yVYA7IZn9demiQk/Ejp0BtTjBWFLymSeY=
modernc.org/sqlite v1.39.0/go.mod h1:cPTJYSlgg3Sfg046yBShXENNtPrWrDX8bsbAQBzgQ5E=
backend/internal/agents/agent.go
+111
View File
@@ -0,0 +1,111 @@
// Package agents implements the 7 FlowSight specialists (A1..A6) plus the
// Master Synthesizer (A7). Contract per docs/AGENT-SPECS.md:
//
// analyze(ticker, snapshots) -> AgentResult(values[], score, citations[])
//
// Agents never fetch live; they read snapshots from the store. Detection
// rules and scores are computed locally; the LLM only refines prose and
// never invents numbers (every value carries citations).
package agents
import (
"context"
"encoding/json"
"sync"
"time"
"flowsight/internal/llm"
"flowsight/internal/model"
"flowsight/internal/store"
)
// Deps wires one analysis run.
type Deps struct {
DB *store.DB
LLM *llm.Client
TriageModel string
SynthModel string
Now time.Time
}
// Analyzer is one specialist.
type Analyzer func(ctx context.Context, d Deps, ticker string) model.AgentResult
// Registry runs in fixed order A1..A6; the synthesizer (A7) runs after.
var Registry = []struct {
Name string
Fn Analyzer
}{
{"smart-money", AnalyzeSmartMoney},
{"broker-intel", AnalyzeBrokerIntel},
{"sentiment", AnalyzeSentiment},
{"fundamental", AnalyzeFundamental},
{"technical", AnalyzeTechnical},
{"catalyst", AnalyzeCatalyst},
}
// RunAll executes A1..A6 in parallel via goroutines and returns results in
// registry order. One failing agent yields a zero-score result with the error
// in Summary — it never aborts the other five.
func RunAll(ctx context.Context, d Deps, ticker string) []model.AgentResult {
out := make([]model.AgentResult, len(Registry))
var wg sync.WaitGroup
for i, a := range Registry {
wg.Add(1)
go func(i int, name string, fn Analyzer) {
defer wg.Done()
res := fn(ctx, d, ticker)
res.Agent = name
out[i] = res
}(i, a.Name, a.Fn)
}
wg.Wait()
return out
}
// payload loads the newest snapshot for (ticker, source) and unmarshals it.
// ok=false when no snapshot exists (agents treat missing input as neutral,
// never as an error — the Citations list simply stays short).
func payload(db *store.DB, ticker, source string, v any) (date string, ok bool) {
raw, d, err := db.LatestSnapshot(ticker, source)
if err != nil || raw == "" {
return "", false
}
if err := json.Unmarshal([]byte(raw), v); err != nil {
return "", false
}
return d, true
}
// cite builds one citation for (source-as-endpoint, ticker, snapshot date).
func cite(source, ticker, date string) model.Citation {
return model.Cite("v2/"+source+"/", ticker, date)
}
// avg returns the mean of xs (0 on empty).
func avg(xs []float64) float64 {
if len(xs) == 0 {
return 0
}
sum := 0.0
for _, x := range xs {
sum += x
}
return sum / float64(len(xs))
}
// clampScore bounds a score to [lo, hi].
func clampScore(v, lo, hi float64) float64 {
if v < lo {
return lo
}
if v > hi {
return hi
}
return v
}
// fail builds an error result that keeps the pipeline green.
func fail(agent, msg string) model.AgentResult {
return model.AgentResult{Agent: agent, Summary: "error: " + msg}
}
backend/internal/agents/agents_test.go
+231
View File
@@ -0,0 +1,231 @@
package agents
import (
"context"
"strings"
"testing"
"time"
"flowsight/internal/llm"
"flowsight/internal/store"
)
func seedDB(t *testing.T) (*store.DB, Deps) {
t.Helper()
db, err := store.Open(t.TempDir() + "/agents.db")
if err != nil {
t.Fatal(err)
}
if _, err := db.SeedFromDir("../../tests/fixtures", "demo"); err != nil {
t.Fatal(err)
}
d := Deps{DB: db, LLM: llm.New("", ""), Now: time.Date(2026, 9, 14, 0, 0, 0, 0, time.UTC)}
return db, d
}
func hasCiteLen(res interface{ GetCitations() int }) {}
// BBCA accumulation fixture scores > +60 with 3 named brokers cited.
func TestSmartMoney(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeSmartMoney(context.Background(), d, "BBCA")
if res.Score <= 60 {
t.Fatalf("score = %.0f, want > 60", res.Score)
}
if len(res.Citations) == 0 {
t.Fatal("no citations")
}
extra := res.Extra
players, _ := extra["players"].([]string)
if len(players) < 3 {
t.Fatalf("players = %v, want 3 named brokers", players)
}
}
// Financials -> Consumer rotation detected with sign-flip evidence.
func TestBrokerIntel(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeBrokerIntel(context.Background(), d, "BBCA")
found := false
for _, f := range res.Flags {
if f == "sector-rotation" {
found = true
}
}
if !found {
t.Fatalf("flags = %v, want sector-rotation", res.Flags)
}
}
// Sentiment trend with >=2 cited articles + insider summary.
func TestSentiment(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeSentiment(context.Background(), d, "BBCA")
if len(res.Citations) == 0 {
t.Fatal("no citations")
}
if res.Extra["trend"] != "improving" {
t.Fatalf("trend = %v, want improving", res.Extra["trend"])
}
}
// BBCA shows P/E vs banks median with cited sections.
func TestFundamental(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeFundamental(context.Background(), d, "BBCA")
if len(res.Citations) < 2 {
t.Fatalf("citations = %d, want >= 2 (report + peers)", len(res.Citations))
}
found := false
for _, v := range res.Values {
if v.Label == "valuation vs peers" {
found = true
}
}
if !found {
t.Fatal("missing valuation-vs-peers row")
}
}
// 3.2x volume spike flagged with date.
func TestTechnical(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeTechnical(context.Background(), d, "BBCA")
found := false
for _, f := range res.Flags {
if f == "volume-anomaly" {
found = true
}
}
if !found {
t.Fatalf("flags = %v, want volume-anomaly", res.Flags)
}
}
// Ex-div date + yield appear with H-N countdown.
func TestCatalyst(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeCatalyst(context.Background(), d, "BBCA")
if res.Score <= 0 {
t.Fatalf("score = %.0f, want > 0 (ex-div in 23d)", res.Score)
}
cal, _ := res.Extra["calendar"].([]string)
if len(cal) == 0 {
t.Fatal("empty catalyst calendar")
}
}
// Good fundamental + broker selling => HOLD-or-lower with conflict flag.
func TestSynthesizerConflict(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
results := RunAll(context.Background(), d, "BBCA")
// Simulate broker distribution opposing the fixture's accumulation.
for i, r := range results {
if r.Agent == "smart-money" {
r.Score = -60
r.Summary = "distribution (simulated)"
results[i] = r
}
}
s := Synthesize(context.Background(), d, "BBCA", Moderate, results)
if !s.Conflict {
t.Fatal("want conflict flag on fundamental-vs-flow opposition")
}
if s.Recommendation == "BUY" {
t.Fatalf("recommendation = BUY, want HOLD-or-lower on conflict")
}
}
// Segments snapshot adds a revenue-segments value row with citation.
func TestFundamentalSegments(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeFundamental(context.Background(), d, "BBCA")
found := false
for _, v := range res.Values {
if v.Label == "revenue segments" {
found = true
if len(v.Citations) == 0 {
t.Fatal("segments row has no citations")
}
}
}
if !found {
t.Fatal("missing revenue-segments row")
}
}
// Falling quarter ROE flags declining-roe: ROE fixture earnings edge up
// 100->105 while equity balloons 1000->1500, so ROE falls 10%->7% (-30%).
func TestFundamentalDecliningROE(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeFundamental(context.Background(), d, "ROE")
found := false
for _, f := range res.Flags {
if f == "declining-roe" {
found = true
}
}
if !found {
t.Fatalf("flags = %v, want declining-roe", res.Flags)
}
}
// Splits + IPO window appear on the catalyst calendar when present.
func TestCatalystSplitsIPO(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeCatalyst(context.Background(), d, "BBCA")
if len(res.Citations) == 0 {
t.Fatal("no citations")
}
}
// Relative volume cites most-traded: BBCA last volume 288M vs fixture
// median 120M => 2.4x row present with a most-traded citation.
func TestTechnicalRelVol(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
res := AnalyzeTechnical(context.Background(), d, "BBCA")
found := false
for _, v := range res.Values {
if v.Label == "relative volume" {
found = true
if len(v.Citations) == 0 {
t.Fatal("relative-volume row has no citations")
}
}
}
if !found {
t.Fatalf("values = %v, want relative-volume row", res.Values)
}
if rv, _ := res.Extra["rel_volume"].(float64); rv < 2.0 || rv > 3.0 {
t.Fatalf("rel_volume = %v, want ~2.4", rv)
}
}
// Thesis claims carry inline citation markers: every agent line ends
// with [endpoint @ date] (or [no snapshot] when input is missing).
func TestSynthesizerThesisCites(t *testing.T) {
db, d := seedDB(t)
defer db.Close()
results := RunAll(context.Background(), d, "BBCA")
s := Synthesize(context.Background(), d, "BBCA", Moderate, results)
if len(s.Thesis) == 0 || len(s.Citations) == 0 {
t.Fatal("thesis or citations empty")
}
if strings.Count(s.Thesis, "[v2/") < 3 {
t.Fatalf("want >=3 inline [v2/ markers, got: %s", s.Thesis)
}
if strings.Count(s.Thesis, "@ 2026-09-11]") < 3 {
t.Fatalf("want dated markers, got: %s", s.Thesis)
}
}
backend/internal/agents/broker_intel.go
+132
View File
@@ -0,0 +1,132 @@
package agents
import (
"context"
"fmt"
"sort"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// AnalyzeBrokerIntel (A2) classifies broker behavior and emits sector
// rotation on week-over-week sign flips with evidence rows.
func AnalyzeBrokerIntel(ctx context.Context, d Deps, ticker string) model.AgentResult {
_ = ctx
ticker = strings.ToUpper(ticker)
res := model.AgentResult{Summary: "no broker snapshots available"}
var registry []sectors.BrokerRegistryRow
regDate, regOK := payload(d.DB, "IDX", "brokers-registry", &registry)
var top struct {
Date string `json:"date"`
Results []sectors.TopBrokerRow `json:"results"`
}
topDate, topOK := payload(d.DB, "IDX", "brokers-top", &top)
if !regOK && !topOK {
return res
}
if regOK {
res.Citations = append(res.Citations, model.Cite("v2/brokers/", "IDX", regDate))
}
if topOK {
res.Citations = append(res.Citations, model.Cite("v2/brokers/top/", "IDX", topDate))
}
byCode := map[string]sectors.BrokerRegistryRow{}
for _, r := range registry {
byCode[r.Code] = r
}
accum, distrib := 0, 0
var lines []string
for _, b := range top.Results {
row := byCode[b.BrokerCode]
origin := "domestic"
cohort := "unknown"
if row.IsForeign {
origin = "foreign"
}
if row.Cohort != nil && *row.Cohort != "" {
cohort = *row.Cohort
}
class := "neutral"
switch {
case b.Net > 0:
class, accum = "accumulating", accum+1
case b.Net < 0:
class, distrib = "distributing", distrib+1
}
if len(lines) < 5 {
lines = append(lines, fmt.Sprintf("%s (%s/%s) %s %s",
b.BrokerCode, origin, cohort, class, fmtIDR(float64(b.Net))))
}
}
total := accum + distrib
score := 0.0
if total > 0 {
score = float64(accum-distrib) / float64(total) * 100
}
res.Score = clampScore(score, -100, 100)
// Rotation: week-over-week sign flip on stored sector nets.
var flow struct {
Week string `json:"week"`
Current map[string]float64 `json:"current"`
Previous map[string]float64 `json:"previous"`
}
flowDate, flowOK := payload(d.DB, "IDX", "sector-flow", &flow)
rotFrom, rotTo, rotDelta := "", "", 0.0
if flowOK {
res.Citations = append(res.Citations, model.Cite("v2/subsector/report/", "IDX", flowDate))
type flip struct {
sector string
delta float64
}
var flips []flip
for s, cur := range flow.Current {
prev := flow.Previous[s]
if prev < 0 && cur > 0 {
flips = append(flips, flip{s, cur - prev})
}
}
var outflows []flip
for s, cur := range flow.Current {
prev := flow.Previous[s]
if prev > 0 && cur < 0 {
outflows = append(outflows, flip{s, prev - cur})
}
}
sort.Slice(flips, func(i, j int) bool { return flips[i].delta > flips[j].delta })
sort.Slice(outflows, func(i, j int) bool { return outflows[i].delta > outflows[j].delta })
if len(flips) > 0 && len(outflows) > 0 {
rotFrom, rotTo, rotDelta = outflows[0].sector, flips[0].sector, flips[0].delta
res.Flags = append(res.Flags, "sector-rotation")
}
}
res.Values = []model.Value{{
Label: "broker behavior",
Display: fmt.Sprintf("%d accumulating vs %d distributing", accum, distrib),
Citations: res.Citations,
}, {
Label: "top brokers",
Display: strings.Join(lines, "; "),
Citations: res.Citations,
}}
if rotFrom != "" {
res.Values = append(res.Values, model.Value{
Label: "sector rotation",
Display: fmt.Sprintf("%s -> %s (%s swing)", rotFrom, rotTo, fmtIDR(rotDelta)),
Citations: res.Citations,
})
res.Summary = fmt.Sprintf("rotation %s -> %s; score %+.0f", rotFrom, rotTo, score)
} else {
res.Summary = fmt.Sprintf("no rotation flip; score %+.0f (%d vs %d)", score, accum, distrib)
}
res.Extra = map[string]any{
"accumulating": accum, "distributing": distrib,
"rotation_from": rotFrom, "rotation_to": rotTo, "rotation_delta": rotDelta,
}
return res
}
backend/internal/agents/catalyst.go
+229
View File
@@ -0,0 +1,229 @@
package agents
import (
"context"
"fmt"
"sort"
"strings"
"time"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// AnalyzeCatalyst (A6) builds the catalyst calendar: ex-div, earnings, AGM.
// Opportunity score = yield x certainty - earnings-risk, 0-100.
func AnalyzeCatalyst(ctx context.Context, d Deps, ticker string) model.AgentResult {
ticker = strings.ToUpper(ticker)
now := d.Now
if now.IsZero() {
now = time.Now()
}
res := model.AgentResult{Summary: "no catalyst snapshots available"}
var wrapped struct {
Symbol string `json:"symbol"`
CorporateActions sectors.CorporateActions `json:"corporate_actions"`
// Unwrapped shape (client return) also accepted.
Dividend []sectors.DividendEvent `json:"dividend"`
UpcomingDividend []sectors.DividendEvent `json:"upcoming_dividend"`
AGM []sectors.DividendEvent `json:"agm"`
StockSplit []sectors.DividendEvent `json:"stock_split"`
}
var ipo sectors.ListingPerformance
ipoDate, ipoOK := payload(d.DB, ticker, "listing-performance", &ipo)
actDate, actOK := payload(d.DB, ticker, "corporate-actions", &wrapped)
actions := wrapped.CorporateActions
if len(actions.UpcomingDividend) == 0 {
actions.UpcomingDividend = wrapped.UpcomingDividend
}
if len(actions.Dividend) == 0 {
actions.Dividend = wrapped.Dividend
}
if len(actions.AGM) == 0 {
actions.AGM = wrapped.AGM
}
if len(actions.StockSplit) == 0 {
actions.StockSplit = wrapped.StockSplit
}
if ipoOK {
res.Citations = append(res.Citations, model.Cite("v2/listing-performance/"+ticker+"/", ticker, ipoDate))
}
var qdates []sectors.QuarterlyDate
qdDate, qdOK := payload(d.DB, ticker, "quarterly-dates", &qdates)
if !qdOK || len(qdates) == 0 {
// Legacy universe shape: [{symbol, date, year}] from
// companies/quarterly-financial-dates.
var uni []sectors.QuarterlyDateRow
if ud, uok := payload(d.DB, ticker, "quarterly-dates", &uni); uok {
qdDate, qdOK = ud, true
for _, r := range uni {
qdates = append(qdates, sectors.QuarterlyDate{ReportDate: r.Date})
}
}
}
if !actOK && !qdOK {
return res
}
if actOK {
res.Citations = append(res.Citations, model.Cite("v2/company/corporate-actions/"+ticker+"/", ticker, actDate))
}
if qdOK {
res.Citations = append(res.Citations, model.Cite("v2/company/get_quarterly_financial_dates/"+ticker+"/", ticker, qdDate))
}
type cal struct {
event string
date string
days int
extra string
}
var rows []cal
closePx, _, _ := d.DB.LatestClose(ticker)
for _, ev := range actions.UpcomingDividend {
if dt := strAt(ev, "ex_date", "exDate", "date"); len(dt) >= 10 {
if t, err := time.Parse("2006-01-02", dt[:10]); err == nil {
days := int(t.Sub(now).Hours() / 24)
rows = append(rows, cal{"ex-div", dt[:10], days, yieldLine(ev, closePx)})
}
}
}
for _, ev := range actions.Dividend {
if dt := strAt(ev, "ex_date", "exDate", "date"); len(dt) >= 10 {
if t, err := time.Parse("2006-01-02", dt[:10]); err == nil && t.After(now.AddDate(0, 0, -370)) {
days := int(t.Sub(now).Hours() / 24)
if days >= -30 { // recent history for payout context
rows = append(rows, cal{"div-paid", dt[:10], days, yieldLine(ev, closePx)})
}
}
}
}
for _, ev := range actions.AGM {
if dt := strAt(ev, "date", "agm_date"); len(dt) >= 10 {
if t, err := time.Parse("2006-01-02", dt[:10]); err == nil {
if days := int(t.Sub(now).Hours() / 24); days >= 0 {
rows = append(rows, cal{"AGM", dt[:10], days, ""})
}
}
}
}
for _, ev := range actions.StockSplit {
if dt := strAt(ev, "date", "ex_date", "split_date"); len(dt) >= 10 {
if t, err := time.Parse("2006-01-02", dt[:10]); err == nil {
if days := int(t.Sub(now).Hours() / 24); days >= -30 {
rows = append(rows, cal{"split", dt[:10], days, ratioLine(ev)})
}
}
}
}
// IPO-window context for recent listings (<=365d): anniversary + 30d drift.
if ipoOK && len(ipo.ListingDate) >= 10 {
if t, err := time.Parse("2006-01-02", ipo.ListingDate[:10]); err == nil {
age := int(now.Sub(t).Hours() / 24)
if age >= 0 && age <= 365 {
rows = append(rows, cal{"IPO-window", ipo.ListingDate[:10], -age, ipoLine(&ipo)})
}
}
}
// Next earnings estimate: last report + ~90d unless universe dates show newer.
if len(qdates) > 0 {
sort.Slice(qdates, func(i, j int) bool { return qdates[i].ReportDate > qdates[j].ReportDate })
last := qdates[0].ReportDate
if len(last) >= 10 {
if t, err := time.Parse("2006-01-02", last[:10]); err == nil {
next := t.AddDate(0, 0, 90)
rows = append(rows, cal{"earnings-est", next.Format("2006-01-02"), int(next.Sub(now).Hours() / 24), "from last " + last[:10]})
}
}
}
sort.Slice(rows, func(i, j int) bool { return rows[i].days < rows[j].days })
opp := 0.0
var lines []string
for _, r := range rows {
h := fmt.Sprintf("H%+d", r.days)
if r.days >= 0 {
h = fmt.Sprintf("H-%d", r.days)
}
line := fmt.Sprintf("%s %s %s", r.event, r.date, h)
if r.extra != "" {
line += " (" + r.extra + ")"
}
lines = append(lines, line)
// Near-term certain events lift the opportunity score.
if r.days >= 0 && r.days <= 30 {
w := 30.0
if r.event == "ex-div" {
w = 45
}
opp += w * (1 - float64(r.days)/30)
}
}
// Earnings within 7d adds risk (results can invalidate the thesis).
for _, r := range rows {
if r.event == "earnings-est" && r.days >= 0 && r.days <= 7 {
opp -= 15
res.Flags = append(res.Flags, "earnings-risk")
}
}
res.Score = clampScore(opp, 0, 100)
if len(lines) == 0 {
res.Summary = "no upcoming catalysts in window"
} else {
res.Values = []model.Value{{
Label: "catalyst calendar",
Display: strings.Join(lines, " | "),
Citations: res.Citations,
}}
res.Summary = fmt.Sprintf("%d catalysts, opportunity %.0f", len(lines), res.Score)
}
res.Extra = map[string]any{"calendar": lines}
return res
}
// strAt returns the first present string key.
func strAt(ev map[string]any, keys ...string) string {
for _, k := range keys {
for ek, v := range ev {
if strings.EqualFold(ek, k) {
if s, ok := v.(string); ok && s != "" {
return s
}
}
}
}
return ""
}
// ratioLine renders "a-for-b" when a split ratio is present.
func ratioLine(ev map[string]any) string {
a := numAt(ev, "ratio", "split_ratio", "ratio_from")
b := numAt(ev, "ratio_to", "ratio_denominator", "new_shares")
if a > 0 && b > 0 {
return fmt.Sprintf("%.0f-for-%.0f", a, b)
}
return "split"
}
// ipoLine renders listing age + 30d drift when present.
func ipoLine(ipo *sectors.ListingPerformance) string {
if ipo.Chg30d != nil {
return fmt.Sprintf("30d %+.1f%%", *ipo.Chg30d*100)
}
return "recent listing"
}
// yieldLine renders "DPS x, yield y%" when figures are present.
func yieldLine(ev map[string]any, closePx float64) string {
dps := numAt(ev, "dividend_per_share", "dps", "cash_dividend")
if dps <= 0 {
return ""
}
if closePx > 0 {
return fmt.Sprintf("DPS %.0f, yield %.1f%%", dps, dps/closePx*100)
}
return fmt.Sprintf("DPS %.0f", dps)
}
backend/internal/agents/fundamental.go
+249
View File
@@ -0,0 +1,249 @@
package agents
import (
"context"
"fmt"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// AnalyzeFundamental (A4) scores valuation vs subsector median + quality.
// Rubric: profitability 35, growth 25, leverage 20, payout 20. Grade A-F.
func AnalyzeFundamental(ctx context.Context, d Deps, ticker string) model.AgentResult {
_ = ctx
ticker = strings.ToUpper(ticker)
res := model.AgentResult{Summary: "no fundamental snapshots available"}
var report map[string]any
repDate, repOK := payload(d.DB, ticker, "company-report", &report)
var quarters []sectors.QuarterRow
qDate, qOK := payload(d.DB, ticker, "financials-quarterly", &quarters)
if !repOK && !qOK {
return res
}
if repOK {
res.Citations = append(res.Citations, model.Cite("v2/company/report/"+ticker+"/", ticker, repDate))
}
if qOK {
res.Citations = append(res.Citations, model.Cite("v2/financials/quarterly/"+ticker+"/", ticker, qDate))
}
pe, pb, roe, de, payout := numAt(report, "pe_ratio", "pe", "p_e"),
numAt(report, "pb_ratio", "pb", "p_b"),
numAt(report, "roe", "return_on_equity"),
numAt(report, "debt_to_equity", "de_ratio", "der"),
numAt(report, "payout_ratio", "dividend_payout")
// Normalize fraction-vs-percent inputs: ROE 0.21 == 21%, payout 62 == 62%.
if roe > 0 && roe < 1 {
roe *= 100
}
if payout > 1 {
payout /= 100
}
// Revenue segments (Sankey-ready) feed the quality read on concentration.
var segs sectors.Segments
segNames := ""
if segDate, segOK := payload(d.DB, ticker, "segments", &segs); segOK && len(segs.RevenueBreakdown) > 0 {
top := segs.RevenueBreakdown
for i := range top {
if i >= 3 {
break
}
if i > 0 {
segNames += ", "
}
segNames += top[i].Source + "→" + top[i].Target
}
res.Citations = append(res.Citations, model.Cite("v2/company/get-segments/"+ticker+"/", ticker, segDate))
}
var peerPE, peerPB float64
var subsector string
var sub map[string]any
subDate, subOK := payload(d.DB, "IDX", "subsector-valuation", &sub)
if subOK {
subsector, _ = sub["sub_sector"].(string)
peerPE = numAt(sub, "median_pe", "pe_median")
peerPB = numAt(sub, "median_pb", "pb_median")
res.Citations = append(res.Citations, model.Cite("v2/subsector/report/", subsector, subDate))
}
// 8-quarter revenue/earnings trend.
revTrend, earnTrend := 0.0, 0.0
if len(quarters) >= 2 {
n := len(quarters)
if n > 8 {
quarters = quarters[n-8:]
n = 8
}
if first, last := fval(quarters[0].Revenue), fval(quarters[n-1].Revenue); first > 0 {
revTrend = (last - first) / first
}
if first, last := fval(quarters[0].Earnings), fval(quarters[n-1].Earnings); first != 0 {
earnTrend = (last - first) / abs(first)
}
}
// ROE trajectory: falling ROE across quarters flags even when earnings rise.
roeSlope := 0.0
if len(quarters) >= 2 {
first, last := roeOf(quarters[0]), roeOf(quarters[len(quarters)-1])
if first > 0 {
roeSlope = (last - first) / first
}
}
// Weighted rubric 0-100.
profit := 50.0
if roe > 0 {
profit = clampScore(roe*3, 0, 100) // ROE 20%+ => ~60+
}
growth := clampScore(50+revTrend*200+earnTrend*100, 0, 100)
leverage := 60.0
if de > 0 {
leverage = clampScore(90-de*30, 0, 100) // DER 1x => ~60
}
pay := 50.0
if payout > 0 && payout <= 0.8 {
pay = 70
} else if payout > 0.8 {
pay = 30 // aggressive payout flagged
}
score := profit*0.35 + growth*0.25 + leverage*0.20 + pay*0.20
res.Score = clampScore(score, 0, 100)
grade := "F"
for _, g := range []struct {
min float64
ch string
}{{85, "A"}, {70, "B"}, {55, "C"}, {40, "D"}} {
if score >= g.min {
grade = g.ch
break
}
}
var flags []string
var vsPeers string
if peerPE > 0 && pe > 0 {
switch {
case pe > peerPE*1.2:
vsPeers = fmt.Sprintf("premium P/E %.1f vs %s median %.1f", pe, subsector, peerPE)
flags = append(flags, "premium-valuation")
case pe < peerPE*0.8:
vsPeers = fmt.Sprintf("discount P/E %.1f vs %s median %.1f", pe, subsector, peerPE)
flags = append(flags, "discount-valuation")
default:
vsPeers = fmt.Sprintf("P/E %.1f in line with %s median %.1f", pe, subsector, peerPE)
}
} else if pe > 0 {
vsPeers = fmt.Sprintf("P/E %.1f (no peer median cached)", pe)
}
if payout > 0.8 {
flags = append(flags, "aggressive-payout")
}
if len(quarters) >= 2 && earnTrend < -0.15 {
flags = append(flags, "declining-earnings")
}
if len(quarters) >= 2 && roeSlope < -0.10 {
flags = append(flags, "declining-roe")
}
res.Flags = flags
res.Values = []model.Value{{
Label: "fundamental score",
Display: fmt.Sprintf("%.0f/100 grade %s", score, grade),
Citations: res.Citations,
}}
if vsPeers != "" {
res.Values = append(res.Values, model.Value{Label: "valuation vs peers", Display: vsPeers, Citations: res.Citations})
}
if len(quarters) >= 2 {
res.Values = append(res.Values, model.Value{
Label: "8Q trend",
Display: fmt.Sprintf("revenue %+.0f%%, earnings %+.0f%% over %d quarters", revTrend*100, earnTrend*100, len(quarters)),
Citations: res.Citations,
})
}
if segNames != "" {
res.Values = append(res.Values, model.Value{
Label: "revenue segments",
Display: segNames,
Citations: res.Citations,
})
}
res.Summary = fmt.Sprintf("grade %s score %.0f; %s", grade, score, vsPeers)
res.Extra = map[string]any{
"grade": grade, "pe": pe, "pb": pb, "roe": roe, "de": de,
"payout": payout, "peer_pe": peerPE, "peer_pb": peerPB,
"rev_trend": revTrend, "earn_trend": earnTrend,
}
return res
}
// numAt digs the first present numeric key out of nested maps.
func numAt(m map[string]any, keys ...string) float64 {
for _, k := range keys {
if v, ok := lookupNum(m, k); ok {
return v
}
}
return 0
}
func lookupNum(m map[string]any, key string) (float64, bool) {
for k, v := range m {
if strings.EqualFold(k, key) {
if f, ok := toFloat(v); ok {
return f, true
}
}
if sub, ok := v.(map[string]any); ok {
if f, ok := lookupNum(sub, key); ok {
return f, true
}
}
}
return 0, false
}
func toFloat(v any) (float64, bool) {
switch n := v.(type) {
case float64:
return n, true
case float32:
return float64(n), true
case int:
return float64(n), true
case int64:
return float64(n), true
default:
return 0, false
}
}
// roeOf approximates quarter ROE from earnings/equity when both present.
func roeOf(q sectors.QuarterRow) float64 {
e, eq := fval(q.Earnings), fval(q.Equity)
if eq <= 0 {
return 0
}
return e / eq * 100
}
func fval(p *float64) float64 {
if p == nil {
return 0
}
return *p
}
func abs(v float64) float64 {
if v < 0 {
return -v
}
return v
}
backend/internal/agents/sentiment.go
+210
View File
@@ -0,0 +1,210 @@
package agents
import (
"context"
"fmt"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// bullish/bearish keyword lists for the offline fallback path (no LLM key).
var bullishWords = []string{"laba naik", "profit up", "bullish", "upgrade", "buyback", "dividen naik", "akuisisi", "ekspansi", "rekor", "tumbuh", "naik", "positive", "growth", "record profit"}
var bearishWords = []string{"rugi", "turun", "bearish", "downgrade", "suspend", "gagal", "skandal", "fraud", "loss", "drop", "plunge", "warning", "penurunan"}
// AnalyzeSentiment (A3) aggregates news + filings + suspensions.
// Adaptive RAG: LLM triage when configured, keyword fallback offline.
// Rare tickers (fewer than 3 articles) force grounding: every claim cites.
func AnalyzeSentiment(ctx context.Context, d Deps, ticker string) model.AgentResult {
ticker = strings.ToUpper(ticker)
res := model.AgentResult{Summary: "no news snapshots available"}
var news struct {
Results []sectors.NewsArticle `json:"results"`
}
newsDate, newsOK := payload(d.DB, ticker, "news", &news)
var filings struct {
Results []sectors.Filing `json:"results"`
}
filDate, filOK := payload(d.DB, ticker, "filings", &filings)
var susp struct {
Results []sectors.Suspension `json:"results"`
}
suspDate, suspOK := payload(d.DB, ticker, "suspensions", &susp)
if !newsOK && !filOK && !suspOK {
// Fall back to derived news_items table (seed path).
if arts, err := d.DB.NewsSince(ticker, "2000-01-01"); err == nil && len(arts) > 0 {
return sentimentFromStored(ticker, arts)
}
return res
}
if newsOK {
res.Citations = append(res.Citations, model.Cite("v2/news/", ticker, newsDate))
}
if filOK {
res.Citations = append(res.Citations, model.Cite("v2/filings/", ticker, filDate))
}
if suspOK {
res.Citations = append(res.Citations, model.Cite("v2/suspensions/", ticker, suspDate))
}
pos, neg, neu := 0, 0, 0
var keyEvents []string
useLLM := d.LLM != nil && d.LLM.Available()
for i, a := range news.Results {
if i >= 20 {
break
}
label, conf := "neutral", 0.5
if useLLM {
label, conf = d.LLM.SentimentTriage(ctx, d.TriageModel, a.Title, a.Body)
} else {
label, conf = keywordSentiment(a.Title + " " + a.Body)
}
switch label {
case "bullish":
pos++
case "bearish":
neg++
default:
neu++
}
if len(keyEvents) < 5 && (label != "neutral" || len(news.Results) < 3) {
keyEvents = append(keyEvents, fmt.Sprintf("%s [%s %.0f%%]", a.Title, label, conf*100))
}
_ = conf
}
insiderLine := "no insider filings"
buys, sells := 0, 0
for _, f := range filings.Results {
switch strings.ToLower(f.TransactionType) {
case "buy":
buys++
case "sell":
sells++
}
}
if buys+sells > 0 {
insiderLine = fmt.Sprintf("%d buys vs %d sells", buys, sells)
}
suspLine := ""
if len(susp.Results) > 0 {
s := susp.Results[0]
suspLine = fmt.Sprintf("SUSPENDED %s: %s", s.SuspensionDate, s.Reason)
res.Flags = append(res.Flags, "suspended")
}
total := pos + neg + neu
score := 0.0
if total > 0 {
score = float64(pos-neg) / float64(total)
}
// Insider tilt: net buys nudge positive.
if buys > sells {
score += 0.1
} else if sells > buys {
score -= 0.1
}
res.Score = clampScore(score, -1, 1)
trend := "stable"
switch {
case score > 0.2:
trend = "improving"
case score < -0.2:
trend = "deteriorating"
}
res.Values = []model.Value{{
Label: "sentiment",
Display: fmt.Sprintf("%s (bullish %d / bearish %d / neutral %d)", trend, pos, neg, neu),
Citations: res.Citations,
}, {
Label: "insider",
Display: insiderLine,
Citations: res.Citations,
}}
if suspLine != "" {
res.Values = append(res.Values, model.Value{Label: "suspension", Display: suspLine, Citations: res.Citations})
}
if len(keyEvents) > 0 {
res.Values = append(res.Values, model.Value{
Label: "key events",
Display: strings.Join(keyEvents, " | "),
Citations: res.Citations,
})
}
res.Summary = fmt.Sprintf("%s: score %+.2f, %d articles, %s", trend, res.Score, total, insiderLine)
res.Extra = map[string]any{
"trend": trend, "bullish": pos, "bearish": neg, "neutral": neu,
"insider_buys": buys, "insider_sells": sells, "key_events": keyEvents,
}
return res
}
// sentimentFromStored builds a result from the derived news_items table.
func sentimentFromStored(ticker string, arts []map[string]any) model.AgentResult {
res := model.AgentResult{}
pos, neg := 0, 0
var keys []string
for _, a := range arts {
s, _ := a["sentiment"].(string)
switch s {
case "bullish":
pos++
case "bearish":
neg++
}
if t, _ := a["title"].(string); t != "" && len(keys) < 5 {
keys = append(keys, t)
}
}
total := len(arts)
score := 0.0
if total > 0 {
score = float64(pos-neg) / float64(total)
}
res.Score = clampScore(score, -1, 1)
trend := "stable"
if score > 0.2 {
trend = "improving"
} else if score < -0.2 {
trend = "deteriorating"
}
res.Citations = []model.Citation{model.Cite("v2/news/", ticker, "stored")}
res.Values = []model.Value{
{Label: "sentiment", Display: fmt.Sprintf("%s (%d articles)", trend, total), Citations: res.Citations},
}
if len(keys) > 0 {
res.Values = append(res.Values, model.Value{Label: "key events", Display: strings.Join(keys, " | "), Citations: res.Citations})
}
res.Summary = fmt.Sprintf("%s: score %+.2f from %d stored articles", trend, res.Score, total)
res.Extra = map[string]any{"trend": trend, "key_events": keys}
return res
}
// keywordSentiment is the offline fallback classifier.
func keywordSentiment(text string) (string, float64) {
t := strings.ToLower(text)
p, n := 0, 0
for _, w := range bullishWords {
if strings.Contains(t, w) {
p++
}
}
for _, w := range bearishWords {
if strings.Contains(t, w) {
n++
}
}
switch {
case p > n:
return "bullish", 0.6
case n > p:
return "bearish", 0.6
default:
return "neutral", 0.5
}
}
backend/internal/agents/smart_money.go
+166
View File
@@ -0,0 +1,166 @@
package agents
import (
"context"
"fmt"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// fmtIDR renders rupiah compactly (Rp1.2T / Rp340B / Rp12M).
func fmtIDR(v float64) string {
neg := v < 0
if neg {
v = -v
}
var s string
switch {
case v >= 1e12:
s = fmt.Sprintf("Rp%.2fT", v/1e12)
case v >= 1e9:
s = fmt.Sprintf("Rp%.0fB", v/1e9)
case v >= 1e6:
s = fmt.Sprintf("Rp%.0fM", v/1e6)
default:
s = fmt.Sprintf("Rp%.0f", v)
}
if neg {
return "-" + s
}
return s
}
// AnalyzeSmartMoney (A1) fuses broker top lists with foreign flow.
// Rule: >=3 brokers net-buy 5d + volume > 1.5x 20d avg => accumulation.
func AnalyzeSmartMoney(ctx context.Context, d Deps, ticker string) model.AgentResult {
_ = ctx
ticker = strings.ToUpper(ticker)
res := model.AgentResult{Summary: "no broker snapshots available"}
var top sectors.BrokerSummaryTop
topDate, ok := payload(d.DB, ticker, "broker-summary-top", &top)
var foreign sectors.ForeignFlow
foreignDate, fok := payload(d.DB, ticker, "foreign-flow", &foreign)
if !ok && !fok {
return res
}
buySum, sellSum := 0.0, 0.0
buyers, sellers := 0, 0
var players []string
if ok {
for _, b := range top.TopBuyers {
if b.NetIDR > 0 {
buyers++
buySum += float64(b.NetIDR)
if len(players) < 3 {
players = append(players, fmt.Sprintf("%s %s", b.BrokerCode, fmtIDR(float64(b.NetIDR))))
}
}
}
for _, s := range top.TopSellers {
if s.NetIDR < 0 {
sellers++
sellSum += float64(-s.NetIDR)
}
}
res.Citations = append(res.Citations, model.Cite("v2/broker-summary/"+ticker+"/top/", ticker, topDate))
}
fSum := 0.0
fN := 0
if fok {
data := foreign.Data
if len(data) > 5 {
data = data[len(data)-5:]
}
for _, p := range data {
fSum += float64(p.NetForeignInflow)
fN++
}
res.Citations = append(res.Citations, model.Cite("v2/foreign-flow/"+ticker+"/", ticker, foreignDate))
}
total := buySum + sellSum
imbalance := 0.0
if total > 0 {
imbalance = (buySum - sellSum) / total
}
score := imbalance * 70
score += float64(minInt(buyers, 5)-minInt(sellers, 5)) * 4
if fok && fN > 0 {
if fSum > 0 {
score += 10
} else if fSum < 0 {
score -= 10
}
}
score = clampScore(score, -100, 100)
res.Score = score
// Volume multiple from stored daily bars.
volMult := 0.0
if vols, _, err := d.DB.DailyVolumes(ticker, 21); err == nil && len(vols) >= 2 {
n := len(vols)
if a := avg(vols[:n-1]); a > 0 {
volMult = vols[n-1] / a
res.Citations = append(res.Citations, model.Cite("v2/daily/"+ticker+"/", ticker, "stored"))
res.Values = append(res.Values, model.Value{
Label: "volume vs 20d avg",
Display: fmt.Sprintf("%.1fx", volMult),
Citations: res.Citations,
})
}
}
phase := "neutral"
switch {
case score >= 30:
phase = "accumulation"
case score <= -30:
phase = "distribution"
case ok && fok && imbalance*fSum < 0:
phase = "conflict"
res.Flags = append(res.Flags, "direction-conflict")
}
if ok && fok && imbalance*fSum > 0 {
res.Flags = append(res.Flags, "direction-agreement")
}
if buyers >= 3 && volMult > 1.5 {
res.Flags = append(res.Flags, "accumulation-rule")
}
if volMult > 2 {
res.Flags = append(res.Flags, "volume-anomaly")
}
res.Values = append([]model.Value{{
Label: "broker net imbalance",
Display: fmt.Sprintf("%s net (%d buyers vs %d sellers)", fmtIDR(buySum-sellSum), buyers, sellers),
Citations: res.Citations,
}, {
Label: "key players",
Display: strings.Join(players, ", "),
Citations: res.Citations,
}, {
Label: "foreign 5d net",
Display: fmtIDR(fSum),
Citations: res.Citations,
}}, res.Values...)
res.Summary = fmt.Sprintf("%s: score %+.0f, %d net-buy brokers, foreign %s",
phase, score, buyers, fmtIDR(fSum))
res.Extra = map[string]any{
"phase": phase, "buyers": buyers, "sellers": sellers,
"net_sum": buySum - sellSum, "foreign_sum": fSum,
"volume_mult": volMult, "players": players,
}
return res
}
func minInt(a, b int) int {
if a < b {
return a
}
return b
}
backend/internal/agents/synthesizer.go
+206
View File
@@ -0,0 +1,206 @@
package agents
import (
"context"
"fmt"
"strings"
"flowsight/internal/model"
)
// RiskProfile shifts signal weights: conservative is fundamental-heavy,
// aggressive leans into technical + broker flows.
type RiskProfile string
const (
Conservative RiskProfile = "conservative"
Moderate RiskProfile = "moderate"
Aggressive RiskProfile = "aggressive"
)
// Synthesis is the Master Synthesizer (A7) output.
type Synthesis struct {
Recommendation string `json:"recommendation"` // BUY | HOLD | AVOID
Conviction int `json:"conviction"` // 1..5
Thesis string `json:"thesis"`
PositionPct float64 `json:"position_pct"`
Conflict bool `json:"conflict"`
ConflictNote string `json:"conflict_note,omitempty"`
Scores []string `json:"scores"`
Citations []model.Citation `json:"citations"`
}
// Synthesize (A7) weights A1..A6 by risk profile x accuracy-ledger weights,
// adds agreement bonus / conflict flag, and sizes via capped Kelly (max 10%).
func Synthesize(ctx context.Context, d Deps, ticker string, profile RiskProfile, results []model.AgentResult) Synthesis {
_ = ctx
ticker = strings.ToUpper(ticker)
byAgent := map[string]model.AgentResult{}
for _, r := range results {
byAgent[r.Agent] = r
}
base := map[string]float64{
"smart-money": 0.22, "broker-intel": 0.13, "sentiment": 0.12,
"fundamental": 0.25, "technical": 0.15, "catalyst": 0.13,
}
switch profile {
case Conservative:
base = map[string]float64{
"smart-money": 0.15, "broker-intel": 0.10, "sentiment": 0.10,
"fundamental": 0.40, "technical": 0.10, "catalyst": 0.15,
}
case Aggressive:
base = map[string]float64{
"smart-money": 0.27, "broker-intel": 0.15, "sentiment": 0.10,
"fundamental": 0.13, "technical": 0.25, "catalyst": 0.10,
}
}
// Ledger weights: 0.5 until an agent has >=10 resolved calls.
weights := map[string]float64{}
norm := func(score float64) float64 { return (score + 100) / 200 } // -100..100 -> 0..1
get := func(name string, raw, lo, hi float64) float64 {
v := raw
if hi == 1 && lo == -1 { // sentiment -1..1
v = raw * 100
} else if hi == 100 && lo == 0 { // fundamental/catalyst 0..100
v = raw*2 - 100
}
return clampScore(v, -100, 100)
}
_ = norm
total, wsum := 0.0, 0.0
var lines []string
var cites []model.Citation
for _, a := range []string{"smart-money", "broker-intel", "sentiment", "fundamental", "technical", "catalyst"} {
r := byAgent[a]
lw := d.DB.AccuracyWeight(a)
w := base[a] * (0.5 + lw) // ledger blends in without zeroing anyone
weights[a] = w
var v float64
switch a {
case "sentiment":
v = get(a, r.Score, -1, 1)
case "fundamental", "catalyst":
v = get(a, r.Score, 0, 100)
default:
v = get(a, r.Score, -100, 100)
}
total += w * v
wsum += w
lines = append(lines, fmt.Sprintf("%s %+.0f", a, v))
cites = append(cites, r.Citations...)
}
score := 0.0
if wsum > 0 {
score = total / wsum
}
// Agreement bonus (>=3 aligned) / conflict flag (fundamental vs flows).
align := 0
for _, a := range []string{"smart-money", "fundamental", "technical", "sentiment"} {
r := byAgent[a]
v := r.Score
if a == "sentiment" {
v *= 100
} else if a == "fundamental" {
v = v*2 - 100
}
if (score > 0 && v > 0) || (score < 0 && v < 0) {
align++
}
}
if align >= 3 {
if score > 0 {
score += 5
} else {
score -= 5
}
}
fund := byAgent["fundamental"].Score*2 - 100
flow := byAgent["smart-money"].Score
conflict := (fund > 20 && flow < -20) || (fund < -20 && flow > 20)
conflictNote := ""
if conflict {
conflictNote = fmt.Sprintf("fundamental %+.0f opposes smart-money %+.0f", fund, flow)
}
rec := "HOLD"
switch {
case score >= 25 && !conflict:
rec = "BUY"
case score <= -25:
rec = "AVOID"
case conflict && score >= 25:
rec = "HOLD" // good fundamental + broker selling => HOLD-or-lower, cited
}
conviction := 3
switch {
case score >= 50 || score <= -50:
conviction = 5
case score >= 35 || score <= -35:
conviction = 4
case score >= -15 && score <= 15:
conviction = 2
}
if conflict && conviction > 3 {
conviction = 3
}
// Capped Kelly: edge from score magnitude, max 10% single name.
edge := (score / 100) * 0.5
size := edge * 0.25 * 100
if size < 0 {
size = 0
}
if size > 10 {
size = 10
}
if rec != "BUY" {
size = 0
}
thesis := fmt.Sprintf("%s %s (conviction %d/5): weighted score %+.0f. %s.",
ticker, rec, conviction, score, strings.Join(citedLines(byAgent), ", "))
if conflict {
thesis += " Conflict: " + conflictNote + citeStr(byAgent["smart-money"]) + citeStr(byAgent["fundamental"]) + "."
}
// Record predictions for the +30d accuracy ledger.
_ = d.DB.RecordPrediction("synthesizer", ticker, rec)
for _, a := range []string{"smart-money", "fundamental", "technical"} {
_ = d.DB.RecordPrediction(a, ticker, rec)
}
return Synthesis{
Recommendation: rec, Conviction: conviction, Thesis: thesis,
PositionPct: size, Conflict: conflict, ConflictNote: conflictNote,
Scores: lines, Citations: cites,
}
}
// citedLines appends each agent's first citation marker to its score line so
// every thesis claim is individually traceable.
func citedLines(byAgent map[string]model.AgentResult) []string {
var out []string
for _, a := range []string{"smart-money", "broker-intel", "sentiment", "fundamental", "technical", "catalyst"} {
r := byAgent[a]
line := strings.TrimSpace(strings.Split(r.Summary, ";")[0])
if line == "" {
line = a
}
out = append(out, line+citeStr(r))
}
return out
}
// citeStr renders "[endpoint @ date]" for an agent's first citation.
func citeStr(r model.AgentResult) string {
if len(r.Citations) == 0 {
return " [no snapshot]"
}
c := r.Citations[0]
return " [" + c.Endpoint + " @ " + c.SnapshotAt + "]"
}
backend/internal/agents/technical.go
+268
View File
@@ -0,0 +1,268 @@
package agents
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// AnalyzeTechnical (A5) scores momentum + volume anomaly + liquidity.
// Anomaly: volume > 2x 20d avg. Liquidity grade from free-float %.
func AnalyzeTechnical(ctx context.Context, d Deps, ticker string) model.AgentResult {
_ = ctx
ticker = strings.ToUpper(ticker)
res := model.AgentResult{Summary: "no technical snapshots available"}
var daily []sectors.DailyBar
dailyDate, dailyOK := payload(d.DB, ticker, "daily", &daily)
var movers struct {
TopGainers map[string][]sectors.MoverRow `json:"top_gainers"`
TopLosers map[string][]sectors.MoverRow `json:"top_losers"`
}
moverDate, moverOK := payload(d.DB, "IDX", "top-changes", &movers)
if !dailyOK && !moverOK {
if vols, dates, err := d.DB.DailyVolumes(ticker, 25); err == nil && len(vols) > 0 {
return technicalFromStored(ticker, vols, dates)
}
return res
}
if dailyOK {
res.Citations = append(res.Citations, model.Cite("v2/daily/"+ticker+"/", ticker, dailyDate))
}
if moverOK {
res.Citations = append(res.Citations, model.Cite("v2/companies/top-changes/", "IDX", moverDate))
}
momentum := "flat"
moverRank := ""
if moverOK {
for period, rows := range movers.TopGainers {
for i, r := range rows {
if strings.HasPrefix(strings.ToUpper(r.Symbol), ticker) {
momentum = "up"
moverRank = fmt.Sprintf("top-gainer #%d (%s)", i+1, period)
}
}
}
for period, rows := range movers.TopLosers {
for i, r := range rows {
if strings.HasPrefix(strings.ToUpper(r.Symbol), ticker) {
momentum = "down"
moverRank = fmt.Sprintf("top-loser #%d (%s)", i+1, period)
}
}
}
}
volMult, volDate, lastVol := 0.0, "", 0.0
if len(daily) >= 21 {
win := daily
if len(win) > 60 {
win = win[len(win)-60:]
}
base := avgVol(win[:len(win)-1], 20)
last := win[len(win)-1]
lastVol = float64(last.Volume)
if base > 0 {
volMult = lastVol / base
volDate = last.Date
}
}
// Price momentum over the window: last close vs first close.
priceChg := 0.0
if len(daily) >= 2 {
first, last := daily[0], daily[len(daily)-1]
if first.Close > 0 {
priceChg = float64(last.Close-first.Close) / float64(first.Close)
}
}
if momentum == "flat" {
switch {
case priceChg > 0.05:
momentum = "up"
case priceChg < -0.05:
momentum = "down"
}
}
if momentum == "up" && priceChg > 0.15 {
momentum = "strong"
}
// Relative volume vs market: ticker's latest volume against the
// most-traded median for the same session.
relVol := 0.0
if med, mtDate, ok := mostTradedMedian(d.DB); ok {
res.Citations = append(res.Citations, model.Cite("v2/most-traded/", "IDX", mtDate))
if med > 0 && lastVol > 0 {
relVol = lastVol / med
res.Values = append(res.Values, model.Value{
Label: "relative volume",
Display: fmt.Sprintf("%.1fx most-traded median", relVol),
Citations: res.Citations,
})
}
}
liquidity := "unknown"
var ff []sectors.FreeFloatRow
if ffDate, ok := payload(d.DB, "IDX", "free-float", &ff); ok {
res.Citations = append(res.Citations, model.Cite("v2/free-float/", "IDX", ffDate))
for _, r := range ff {
if strings.HasPrefix(strings.ToUpper(r.Symbol), ticker) {
switch {
case r.FreeFloat >= 0.4:
liquidity = "A"
case r.FreeFloat >= 0.25:
liquidity = "B"
case r.FreeFloat >= 0.1:
liquidity = "C"
default:
liquidity = "D"
}
res.Values = append(res.Values, model.Value{
Label: "free float",
Display: fmt.Sprintf("%.0f%% (grade %s)", r.FreeFloat*100, liquidity),
Citations: res.Citations,
})
}
}
}
if volMult > 2 {
res.Flags = append(res.Flags, "volume-anomaly")
}
score := priceChg * 300
if volMult > 1 {
score += (volMult - 1) * 10
}
switch momentum {
case "strong":
score += 15
case "up":
score += 8
case "down":
score -= 8
}
res.Score = clampScore(score, -100, 100)
res.Values = append([]model.Value{{
Label: "momentum",
Display: fmt.Sprintf("%s (%+.1f%% window)", momentum, priceChg*100),
Citations: res.Citations,
}}, res.Values...)
if volMult > 0 {
res.Values = append(res.Values, model.Value{
Label: "volume anomaly",
Display: fmt.Sprintf("%.1fx 20d avg on %s", volMult, volDate),
Citations: res.Citations,
})
}
if moverRank != "" {
res.Values = append(res.Values, model.Value{Label: "mover rank", Display: moverRank, Citations: res.Citations})
}
res.Summary = fmt.Sprintf("%s momentum %+.1f%%, volume %.1fx, liquidity %s",
momentum, priceChg*100, volMult, liquidity)
res.Extra = map[string]any{
"momentum": momentum, "price_change": priceChg,
"volume_mult": volMult, "volume_date": volDate, "liquidity": liquidity,
"rel_volume": relVol,
}
return res
}
// mtRow is one most-traded entry (volume in shares).
type mtRow struct {
Symbol string `json:"symbol"`
Volume float64 `json:"volume"`
}
// mostTradedMedian returns the median volume across the cached most-traded
// snapshot plus its snapshot date. Accepts both stored shapes: the wrapped
// {results:[...]} form and the bare array the scheduler persists.
func mostTradedMedian(db interface {
LatestSnapshot(ticker, source string) (string, string, error)
}) (med float64, date string, ok bool) {
raw, d, err := db.LatestSnapshot("IDX", "most-traded")
if err != nil || raw == "" {
return 0, "", false
}
var vols []float64
var wrapped struct {
Results []mtRow `json:"results"`
}
if json.Unmarshal([]byte(raw), &wrapped) == nil && len(wrapped.Results) > 0 {
for _, r := range wrapped.Results {
if r.Volume > 0 {
vols = append(vols, r.Volume)
}
}
} else {
var rows []mtRow
if json.Unmarshal([]byte(raw), &rows) != nil {
return 0, "", false
}
for _, r := range rows {
if r.Volume > 0 {
vols = append(vols, r.Volume)
}
}
}
if len(vols) == 0 {
return 0, "", false
}
sort.Float64s(vols)
m := vols[len(vols)/2]
if len(vols)%2 == 0 {
m = (vols[len(vols)/2-1] + vols[len(vols)/2]) / 2
}
return m, d, true
}
func avgVol(bars []sectors.DailyBar, n int) float64 {
if len(bars) < n {
n = len(bars)
}
if n == 0 {
return 0
}
sum := 0.0
for _, b := range bars[len(bars)-n:] {
sum += float64(b.Volume)
}
return sum / float64(n)
}
// technicalFromStored derives momentum from stored snapshot volumes.
func technicalFromStored(ticker string, vols []float64, dates []string) model.AgentResult {
res := model.AgentResult{}
last := vols[len(vols)-1]
base := avg(vols[:len(vols)-1])
mult := 0.0
if base > 0 {
mult = last / base
}
res.Score = clampScore((mult-1)*20, -100, 100)
res.Citations = []model.Citation{model.Cite("v2/daily/"+ticker+"/", ticker, "stored")}
date := ""
if len(dates) > 0 {
date = dates[len(dates)-1]
}
res.Values = []model.Value{{
Label: "volume anomaly",
Display: fmt.Sprintf("%.1fx 20d avg on %s", mult, date),
Citations: res.Citations,
}}
if mult > 2 {
res.Flags = append(res.Flags, "volume-anomaly")
}
res.Summary = fmt.Sprintf("stored-volume momentum %.1fx", mult)
res.Extra = map[string]any{"volume_mult": mult, "volume_date": date}
return res
}
backend/internal/alerts/evaluate.go
+248
View File
@@ -0,0 +1,248 @@
package alerts
import (
"context"
"encoding/json"
"time"
"flowsight/internal/model"
"flowsight/internal/store"
)
// Evaluate runs all 6 rules over stored snapshots for the watchlist and
// records + delivers hits. First-fire only per (rule, ticker, date): rows
// already present in alert_events for today are skipped. Delivery fans out to
// the alert owner's enabled destinations (server env is fallback). Prefer
// EvaluateFor; this wrapper keeps owner empty (server fallback only).
func Evaluate(ctx context.Context, db *store.DB, n *Notifier, alertID int64, ruleJSON string, tickers []string) []Finding {
owner := ""
_ = db.QueryRow(`SELECT user_key FROM alerts WHERE id=?`, alertID).Scan(&owner)
return EvaluateFor(ctx, db, n, alertID, owner, ruleJSON, tickers)
}
// EvaluateFor is Evaluate scoped to an explicit owner: history rows and
// delivery both use owner (no cross-user leakage via alert_id reuse).
func EvaluateFor(ctx context.Context, db *store.DB, n *Notifier, alertID int64, owner, ruleJSON string, tickers []string) []Finding {
today := time.Now().Format("2006-01-02")
var out []Finding
for _, t := range tickers {
cands := candidates(db, t, today)
for _, f := range cands {
if !MatchRule(ruleJSON, f) {
continue
}
if firedToday(db, f.Rule, t, today) {
continue
}
cites, _ := json.Marshal(f.Citations)
ctxJSON, _ := json.Marshal(f.Context)
if _, err := db.InsertAlertEvent(alertID, t, today, f.Message, string(ctxJSON), string(cites), owner); err != nil {
continue
}
_ = n.SendTo(ctx, owner, f, string(cites))
_ = db.MarkAlertFired(alertID)
out = append(out, f)
}
}
return out
}
// candidates runs every rule against one ticker's stored state. Every
// finding carries real citations (no "derived" placeholder).
func candidates(db *store.DB, ticker, today string) []Finding {
var out []Finding
brokerDate := today
if nets, err := db.NetBuySum5d(ticker); err == nil && len(nets) > 0 {
volMult := 0.0
volDate := today
if vols, dates, err := db.DailyVolumes(ticker, 21); err == nil && len(vols) >= 2 {
n := len(vols)
if a := avg(vols[:n-1]); a > 0 {
volMult = vols[n-1] / a
}
volDate = lastOf(dates)
}
if _, d, err := db.LatestSnapshot(ticker, "broker-summary-top"); err == nil {
brokerDate = d
}
if f, ok := Accumulation(ticker, nets, volMult); ok {
f.Citations = []model.Citation{
model.Cite("v2/broker-summary/"+ticker+"/top/", ticker, brokerDate),
model.Cite("v2/daily/"+ticker+"/", ticker, volDate),
}
out = append(out, f)
}
}
if dates, nets, err := db.ForeignLast6(ticker); err == nil && len(nets) == 6 {
if f, ok := ForeignReversal(ticker, nets); ok {
f.Citations = []model.Citation{
model.Cite("v2/foreign-flow/"+ticker+"/", ticker, dates[5]),
}
out = append(out, f)
}
}
// Insider spike: real 2x-volume path (latest buy vs 30d avg) plus the
// distinct-insider cluster path, both over stored filings.
if avg30 := db.FilingAvg30(ticker); avg30 > 0 {
if f, ok := InsiderSpike(ticker, db.LatestBuyVolume(ticker), avg30, db.DistinctInsiders7d(ticker)); ok {
f.Citations = []model.Citation{model.Cite("v2/filings/", ticker, today)}
out = append(out, f)
}
}
if vols, dates, err := db.DailyVolumes(ticker, 21); err == nil && len(vols) >= 2 {
n := len(vols)
if a := avg(vols[:n-1]); a > 0 {
if f, ok := UnusualVolume(ticker, lastOf(dates), vols[n-1]/a, isEarningsDate(db, ticker, lastOf(dates))); ok {
f.Citations = []model.Citation{
model.Cite("v2/daily/"+ticker+"/", ticker, lastOf(dates)),
}
out = append(out, f)
}
}
}
// Sector rotation from the stored sector-flow snapshot.
var flow struct {
Current map[string]float64 `json:"current"`
Previous map[string]float64 `json:"previous"`
}
if raw, d, err := db.LatestSnapshot("IDX", "sector-flow"); err == nil {
_ = json.Unmarshal([]byte(raw), &flow)
for sector, cur := range flow.Current {
if f, ok := SectorRotation(sector, flow.Previous[sector], cur); ok {
f.Citations = []model.Citation{model.Cite("v2/subsector/report/", sector, d)}
out = append(out, f)
}
}
}
// Suspension: fire only on notices dated today or later (new-notice diff).
if raw, d, err := db.LatestSnapshot(ticker, "suspensions"); err == nil && raw != "" {
var notices struct {
Results []struct {
Symbol string `json:"symbol"`
SuspensionDate string `json:"suspension_date"`
Reason string `json:"reason"`
} `json:"results"`
}
if json.Unmarshal([]byte(raw), &notices) == nil {
for _, sn := range notices.Results {
if sn.SuspensionDate >= today {
if f, ok := SuspensionWatch(ticker, sn.SuspensionDate, sn.Reason, true); ok {
f.Citations = []model.Citation{
model.Cite("v2/suspensions/", ticker, sn.SuspensionDate),
}
_ = d
out = append(out, f)
}
}
}
}
}
return out
}
// isEarningsDate reports whether date falls within 1d of a stored quarterly
// report date (rule-4 exclusion for unusual volume).
func isEarningsDate(db *store.DB, ticker, date string) bool {
var qdates []struct {
ReportDate string `json:"report_date"`
Date string `json:"date"`
}
raw, _, err := db.LatestSnapshot(ticker, "quarterly-dates")
if err != nil {
return false
}
if json.Unmarshal([]byte(raw), &qdates) != nil {
return false
}
for _, q := range qdates {
for _, d := range []string{q.ReportDate, q.Date} {
if len(d) >= 10 && len(date) >= 10 && daysBetween(d[:10], date[:10]) <= 1 {
return true
}
}
}
return false
}
func daysBetween(a, b string) int {
pa, ea := parseDay(a)
pb, eb := parseDay(b)
if ea != nil || eb != nil {
return 999
}
h := pa.Sub(pb).Hours() / 24
if h < 0 {
h = -h
}
return int(h + 0.5)
}
func parseDay(s string) (t time.Time, err error) {
return time.Parse("2006-01-02", s)
}
// firedToday enforces first-fire-only per (rule, ticker, date) for every
// rule: any event for the same rule+ticker+date suppresses re-fire.
func firedToday(db *store.DB, rule, ticker, today string) bool {
evts, err := db.AlertEventsSince(today, ticker, 50)
if err != nil {
return false
}
for _, e := range evts {
msg, _ := e["message"].(string)
if e["ticker"] == ticker && e["date"] == today && contains(msg, ruleHints(rule)) {
return true
}
}
return false
}
// ruleHints maps a rule id to the message fragment its findings carry.
func ruleHints(rule string) string {
switch rule {
case RuleAccumulation:
return "accumulation"
case RuleForeignRev:
return "reversal"
case RuleInsiderSpike:
return "insider"
case RuleUnusualVolume:
return "unusual volume"
case RuleRotation:
return "rotation"
case RuleSuspension:
return "suspend"
default:
return rule
}
}
func avg(xs []float64) float64 {
if len(xs) == 0 {
return 0
}
s := 0.0
for _, x := range xs {
s += x
}
return s / float64(len(xs))
}
func lastOf(xs []string) string {
if len(xs) == 0 {
return ""
}
return xs[len(xs)-1]
}
func contains(s, sub string) bool {
return len(s) >= len(sub) && (s == sub || len(sub) == 0 ||
func() bool {
for i := 0; i+len(sub) <= len(s); i++ {
if s[i:i+len(sub)] == sub {
return true
}
}
return false
}())
}
backend/internal/alerts/evaluate_test.go
+101
View File
@@ -0,0 +1,101 @@
package alerts
import (
"context"
"testing"
"flowsight/internal/store"
)
func seedEvalDB(t *testing.T) *store.DB {
t.Helper()
db, err := store.Open(t.TempDir() + "/eval.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { db.Close() })
if _, err := db.SeedFromDir("../../tests/fixtures", "demo"); err != nil {
t.Fatal(err)
}
return db
}
// Accumulation fixture fires with real citations, then first-fire suppresses.
func TestEvaluateAccumulationFirstFire(t *testing.T) {
db := seedEvalDB(t)
n := NewNotifier("", "", "")
id, _ := db.CreateAlert("demo", "all", `{"all":true}`, nil)
got := Evaluate(context.Background(), db, n, id, `{"all":true}`, []string{"BBCA"})
found := false
for _, f := range got {
if f.Rule == RuleAccumulation {
found = true
if len(f.Citations) == 0 {
t.Fatal("accumulation finding has no citations")
}
}
}
if !found {
t.Fatalf("no accumulation finding in %v", got)
}
again := Evaluate(context.Background(), db, n, id, `{"all":true}`, []string{"BBCA"})
for _, f := range again {
if f.Rule == RuleAccumulation {
t.Fatal("accumulation re-fired same day")
}
}
}
// Rotation is reachable: sector-flow flip yields a cited finding.
func TestEvaluateRotationReachable(t *testing.T) {
db := seedEvalDB(t)
n := NewNotifier("", "", "")
id, _ := db.CreateAlert("demo", "all", `{"all":true}`, nil)
got := Evaluate(context.Background(), db, n, id, `{"all":true}`, []string{"BBCA"})
found := false
for _, f := range got {
if f.Rule == RuleRotation {
found = true
}
}
if !found {
t.Fatalf("no rotation finding in %v", ruleIDs(got))
}
}
func ruleIDs(fs []Finding) []string {
var out []string
for _, f := range fs {
out = append(out, f.Rule+":"+f.Ticker)
}
return out
}
// Per-owner delivery: finding fans out to the owner's destinations, and
// history rows carry the owner key (no cross-user leakage on alert_id reuse).
func TestEvaluateForOwnerDelivery(t *testing.T) {
db := seedEvalDB(t)
var sentTo string
var sent int
n := NewNotifier("", "", "")
n.Targets = func(owner string) []Target {
sentTo = owner
sent++
return nil // record-only; just observe routing
}
id, _ := db.CreateAlert("alice", "all", `{"all":true}`, nil)
got := EvaluateFor(context.Background(), db, n, id, "alice", `{"all":true}`, []string{"BBCA"})
if len(got) == 0 {
t.Fatal("want findings on seed")
}
if sent == 0 || sentTo != "alice" {
t.Fatalf("delivery not routed to owner: sent=%d to=%q", sent, sentTo)
}
evts, _ := db.AlertEventsSince("2000-01-01", "", 50, "alice")
if len(evts) == 0 {
t.Fatal("history rows must carry the owner key")
}
if evts2, _ := db.AlertEventsSince("2000-01-01", "", 50, "bob"); len(evts2) != 0 {
t.Fatal("bob must not see alice events")
}
}
backend/internal/alerts/notify.go
+163
View File
@@ -0,0 +1,163 @@
package alerts
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
)
// Target is one push destination resolved per user at send time.
type Target struct {
TelegramToken string
TelegramChatID string
DiscordURL string
}
// Notifier delivers alert cards to Telegram + Discord webhooks with context +
// citations. Missing credentials degrade to a no-op (logged, not fatal) so
// offline/demo runs never fail on delivery.
type Notifier struct {
TelegramToken string
TelegramChatID string
DiscordURL string
http *http.Client
// Targets, when set, resolves per-owner push destinations. It is consulted
// on every Send so destination CRUD takes effect immediately.
Targets func(owner string) []Target
}
// NewNotifier builds a notifier; empty creds mean dry-run mode.
func NewNotifier(tgToken, tgChat, discordURL string) *Notifier {
return &Notifier{TelegramToken: tgToken, TelegramChatID: tgChat,
DiscordURL: discordURL, http: &http.Client{Timeout: 15 * time.Second}}
}
// DryRun reports whether no server-level channel is configured.
func (n *Notifier) DryRun() bool {
return n.TelegramToken == "" && n.DiscordURL == ""
}
// DryRunFor reports whether an owner has no push target anywhere: neither
// the owner's enabled destinations nor the server fallback.
func (n *Notifier) DryRunFor(owner string) bool {
return len(n.targetsFor(owner)) == 0
}
// targetsFor resolves push targets: per-owner destinations first, then the
// server-level env fallback. Owner "" means server fallback only.
func (n *Notifier) targetsFor(owner string) []Target {
var out []Target
if n.Targets != nil && owner != "" {
out = append(out, n.Targets(owner)...)
}
if n.TelegramToken != "" && n.TelegramChatID != "" {
out = append(out, Target{TelegramToken: n.TelegramToken, TelegramChatID: n.TelegramChatID})
}
if n.DiscordURL != "" {
out = append(out, Target{DiscordURL: n.DiscordURL})
}
return out
}
// Card is the rendered alert text shared by both channels.
func Card(f Finding, cites string) string {
var b strings.Builder
fmt.Fprintf(&b, "*%s* — %s\n%s", f.Ticker, f.Rule, f.Message)
if len(f.Context) > 0 {
raw, _ := json.Marshal(f.Context)
fmt.Fprintf(&b, "\n`%s`", string(raw))
}
if cites != "" {
fmt.Fprintf(&b, "\nCitations: %s", cites)
}
fmt.Fprintf(&b, "\nDetail: /report/%s", f.Ticker)
return b.String()
}
// Send delivers one finding to all configured channels (server fallback).
// Prefer SendTo so delivery fans out to the finding owner's destinations.
func (n *Notifier) Send(ctx context.Context, f Finding, cites string) error {
return n.SendTo(ctx, "", f, cites)
}
// SendTo delivers one finding to the owner's enabled destinations plus the
// server fallback. With no targets anywhere it is record-only (nil).
func (n *Notifier) SendTo(ctx context.Context, owner string, f Finding, cites string) error {
targets := n.targetsFor(owner)
if len(targets) == 0 {
return nil // recorded in alert_events regardless
}
if cites == "" && len(f.Citations) > 0 {
cc, _ := json.Marshal(f.Citations)
cites = string(cc)
}
text := Card(f, cites)
var firstErr error
for _, t := range targets {
if t.TelegramToken != "" && t.TelegramChatID != "" {
if err := sendTelegram(ctx, n.http, t.TelegramToken, t.TelegramChatID, text); err != nil && firstErr == nil {
firstErr = err
}
}
if t.DiscordURL != "" {
if err := sendDiscord(ctx, n.http, t.DiscordURL, text); err != nil && firstErr == nil {
firstErr = err
}
}
}
return firstErr
}
func (n *Notifier) telegram(ctx context.Context, text string) error {
return sendTelegram(ctx, n.http, n.TelegramToken, n.TelegramChatID, text)
}
func sendTelegram(ctx context.Context, client *http.Client, token, chatID, text string) error {
u := "https://api.telegram.org/bot" + token + "/sendMessage"
body, _ := json.Marshal(map[string]any{
"chat_id": chatID, "text": text, "parse_mode": "Markdown",
})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("alerts: telegram: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return fmt.Errorf("alerts: telegram HTTP %d", resp.StatusCode)
}
return nil
}
func (n *Notifier) discord(ctx context.Context, text string) error {
return sendDiscord(ctx, n.http, n.DiscordURL, text)
}
func sendDiscord(ctx context.Context, client *http.Client, webhookURL, text string) error {
if len(text) > 1900 {
text = text[:1900] + "…"
}
body, _ := json.Marshal(map[string]any{"content": text})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, webhookURL, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return fmt.Errorf("alerts: discord: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return fmt.Errorf("alerts: discord HTTP %d", resp.StatusCode)
}
return nil
}
backend/internal/alerts/rules.go
+161
View File
@@ -0,0 +1,161 @@
// Package alerts evaluates 6 detection rules (docs/PLAN.md §11) over stored
// snapshots. Rules are pure functions over typed inputs so fixture tests can
// prove each fires (or stays silent) deterministically.
package alerts
import (
"strings"
"flowsight/internal/model"
)
// Rule IDs for the 6 v1 detectors.
const (
RuleAccumulation = "accumulation"
RuleForeignRev = "foreign-reversal"
RuleInsiderSpike = "insider-spike"
RuleUnusualVolume = "unusual-volume"
RuleRotation = "sector-rotation"
RuleSuspension = "suspension-watch"
)
// Finding is one rule hit with human text + evidence + real citations.
type Finding struct {
Rule string
Ticker string
Message string
Context map[string]any
Citations []model.Citation
}
// Accumulation fires on >=3 brokers net-buy 5d + volume > 1.5x 20d avg.
func Accumulation(ticker string, netByBroker map[string]float64, volMult float64) (Finding, bool) {
n := 0
sum := 0.0
for _, v := range netByBroker {
if v > 0 {
n++
sum += v
}
}
if n >= 3 && volMult > 1.5 {
return Finding{Rule: RuleAccumulation, Ticker: ticker,
Message: ticker + " accumulation: " + itoa(n) + " brokers net-buy, volume spike",
Context: map[string]any{"brokers": n, "net_sum": sum, "vol_mult": volMult}}, true
}
return Finding{}, false
}
// ForeignReversal fires on 5d cumulative outflow then 1d inflow with the last
// day magnitude > 2x trailing 5d daily average (or the mirror).
func ForeignReversal(ticker string, last6 []float64) (Finding, bool) {
if len(last6) < 6 {
return Finding{}, false
}
prev5, last := last6[:5], last6[5]
sum5 := 0.0
absAvg := 0.0
for _, v := range prev5 {
sum5 += v
absAvg += abs(v)
}
absAvg /= 5
if absAvg == 0 {
return Finding{}, false
}
if sum5 < 0 && last > 0 && last > 2*absAvg {
return Finding{Rule: RuleForeignRev, Ticker: ticker,
Message: ticker + " foreign reversal: outflow flipped to inflow",
Context: map[string]any{"sum5": sum5, "last": last}}, true
}
if sum5 > 0 && last < 0 && -last > 2*absAvg {
return Finding{Rule: RuleForeignRev, Ticker: ticker,
Message: ticker + " foreign reversal: inflow flipped to outflow",
Context: map[string]any{"sum5": sum5, "last": last}}, true
}
return Finding{}, false
}
// InsiderSpike fires on director/major buy volume > 2x 30d avg, or >=3
// distinct insiders buying in 7d.
func InsiderSpike(ticker string, buyVol, avg30 float64, distinct7d int) (Finding, bool) {
if avg30 > 0 && buyVol > 2*avg30 {
return Finding{Rule: RuleInsiderSpike, Ticker: ticker,
Message: ticker + " insider spike: buy volume above 2x 30d avg",
Context: map[string]any{"volume": buyVol, "avg30": avg30, "mult": buyVol / avg30}}, true
}
if distinct7d >= 3 {
return Finding{Rule: RuleInsiderSpike, Ticker: ticker,
Message: ticker + " insider cluster: 3+ insiders buying in 7d",
Context: map[string]any{"distinct": distinct7d}}, true
}
return Finding{}, false
}
// UnusualVolume fires on >3x 20d avg when the date is not an earnings date.
func UnusualVolume(ticker, date string, mult float64, isEarningsDate bool) (Finding, bool) {
if mult > 3 && !isEarningsDate {
return Finding{Rule: RuleUnusualVolume, Ticker: ticker,
Message: ticker + " unusual volume on " + date,
Context: map[string]any{"mult": mult, "date": date}}, true
}
return Finding{}, false
}
// SectorRotation fires when a subsector net flow flips sign week-over-week.
func SectorRotation(sector string, prev, cur float64) (Finding, bool) {
if (prev < 0 && cur > 0) || (prev > 0 && cur < 0) {
dir := "inflow"
if cur < 0 {
dir = "outflow"
}
return Finding{Rule: RuleRotation, Ticker: sector,
Message: "rotation: " + sector + " flipped to " + dir,
Context: map[string]any{"prev": prev, "cur": cur}}, true
}
return Finding{}, false
}
// SuspensionWatch fires on any new suspension notice for a watchlist ticker.
func SuspensionWatch(ticker, date, reason string, onWatchlist bool) (Finding, bool) {
if reason == "" && date == "" {
return Finding{}, false
}
if onWatchlist {
return Finding{Rule: RuleSuspension, Ticker: ticker,
Message: ticker + " suspended: " + reason,
Context: map[string]any{"date": date, "reason": reason}}, true
}
return Finding{}, false
}
func abs(v float64) float64 {
if v < 0 {
return -v
}
return v
}
func itoa(n int) string {
if n == 0 {
return "0"
}
s := ""
for n > 0 {
s = string(rune('0'+n%10)) + s
n /= 10
}
return s
}
// MatchRule reports whether a user rule JSON targets a finding (simple
// substring match on rule id or ticker; empty rule matches all).
func MatchRule(ruleJSON string, f Finding) bool {
if ruleJSON == "" || ruleJSON == "{}" {
return true
}
r := strings.ToLower(ruleJSON)
return strings.Contains(r, strings.ToLower(f.Rule)) ||
strings.Contains(r, strings.ToLower(f.Ticker)) ||
strings.Contains(r, `"all"`)
}
backend/internal/alerts/rules_test.go
+76
View File
@@ -0,0 +1,76 @@
package alerts
import "testing"
// Accumulation fixture: 3 brokers net-buy + 1.6x volume fires.
func TestAccumulationFires(t *testing.T) {
nets := map[string]float64{"MG": 5e11, "AK": 4e11, "CC": 3e11, "BK": -1e11}
f, ok := Accumulation("BBCA", nets, 1.6)
if !ok {
t.Fatal("want accumulation fire")
}
if f.Rule != RuleAccumulation {
t.Fatalf("rule = %s", f.Rule)
}
}
func TestAccumulationSilent(t *testing.T) {
nets := map[string]float64{"MG": 5e11, "BK": -4e11}
if _, ok := Accumulation("BBCA", nets, 1.1); ok {
t.Fatal("want silence: only 1 net-buy broker, low volume")
}
}
// Foreign reversal: 5d outflow then strong inflow fires.
func TestForeignReversal(t *testing.T) {
last6 := []float64{-1e11, -1e11, -1e11, -1e11, -1e11, 3.4e11}
if _, ok := ForeignReversal("BBCA", last6); !ok {
t.Fatal("want reversal fire")
}
if _, ok := ForeignReversal("BBCA", []float64{1e11, 1e11}); ok {
t.Fatal("want silence on short series")
}
}
// Insider spike both paths.
func TestInsiderSpike(t *testing.T) {
if _, ok := InsiderSpike("BBCA", 3e6, 1e6, 1); !ok {
t.Fatal("want volume-path fire")
}
if _, ok := InsiderSpike("BBCA", 0, 0, 3); !ok {
t.Fatal("want cluster-path fire")
}
if _, ok := InsiderSpike("BBCA", 1e6, 1e6, 1); ok {
t.Fatal("want silence")
}
}
// Unusual volume suppressed on earnings dates.
func TestUnusualVolume(t *testing.T) {
if _, ok := UnusualVolume("BBCA", "2026-09-11", 3.2, false); !ok {
t.Fatal("want fire")
}
if _, ok := UnusualVolume("BBCA", "2026-09-11", 3.2, true); ok {
t.Fatal("want silence on earnings date")
}
}
// Rotation sign flip.
func TestRotation(t *testing.T) {
if _, ok := SectorRotation("consumer", -3e8, 1.1e9); !ok {
t.Fatal("want rotation fire")
}
if _, ok := SectorRotation("banks", 1e9, 2e9); ok {
t.Fatal("want silence without flip")
}
}
// Suspension watch: watchlist fires, others silent.
func TestSuspension(t *testing.T) {
if _, ok := SuspensionWatch("TLKM", "2026-09-10", "volatilitas", true); !ok {
t.Fatal("want fire on watchlist")
}
if _, ok := SuspensionWatch("TLKM", "2026-09-10", "volatilitas", false); ok {
t.Fatal("want silence off-watchlist")
}
}
backend/internal/api/alerts.go
+76
View File
@@ -0,0 +1,76 @@
package api
import (
"encoding/json"
"net/http"
"strconv"
"github.com/go-chi/chi/v5"
)
// ListAlerts serves GET /api/alerts.
func (s *Server) ListAlerts(w http.ResponseWriter, r *http.Request) {
rows, err := s.DB.ListAlerts(s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"alerts": rows})
}
// CreateAlert serves POST /api/alerts {name, rule, channels[]}.
func (s *Server) CreateAlert(w http.ResponseWriter, r *http.Request) {
var req struct {
Name string `json:"name" validate:"required"`
Rule any `json:"rule" validate:"required"`
Channels []string `json:"channels"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "name and rule are required")
return
}
ruleRaw, _ := json.Marshal(req.Rule)
id, err := s.DB.CreateAlert(s.userKey(r), req.Name, string(ruleRaw), req.Channels)
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusCreated, map[string]any{"id": id})
}
// DeleteAlert serves DELETE /api/alerts/:id.
func (s *Server) DeleteAlert(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeErr(w, http.StatusBadRequest, "invalid id")
return
}
ok, err := s.DB.DeleteAlert(id, s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if !ok {
writeErr(w, http.StatusNotFound, "alert not found")
return
}
writeJSON(w, http.StatusOK, map[string]any{"id": id, "ok": true})
}
// AlertEvents serves GET /api/alert-events?since=&ticker=.
func (s *Server) AlertEvents(w http.ResponseWriter, r *http.Request) {
since := r.URL.Query().Get("since")
if since == "" {
since = "2000-01-01"
}
evts, err := s.DB.AlertEventsSince(since, r.URL.Query().Get("ticker"), 50, s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"events": evts})
}
backend/internal/api/api_test.go
+273
View File
@@ -0,0 +1,273 @@
package api
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"flowsight/internal/config"
"flowsight/internal/sectors"
"flowsight/internal/store"
)
func testServer(t *testing.T) *Server {
t.Helper()
cfg := config.Load()
cfg.DemoUserKey = "demo"
db, err := store.Open(t.TempDir() + "/api.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { db.Close() })
if _, err := db.SeedFromDir("../../tests/fixtures", "demo"); err != nil {
t.Fatal(err)
}
cache := store.NewCache("")
return New(cfg, db, cache, sectors.New(cfg.SectorsBaseURL, ""))
}
func do(s *Server, method, path string, body any) *httptest.ResponseRecorder {
var rdr *bytes.Reader
if body != nil {
raw, _ := json.Marshal(body)
rdr = bytes.NewReader(raw)
} else {
rdr = bytes.NewReader(nil)
}
req := httptest.NewRequest(method, path, rdr)
req.Header.Set("X-User-Key", "demo")
rec := httptest.NewRecorder()
s.Router().ServeHTTP(rec, req)
return rec
}
// Health 200 with cycle + credits fields.
func TestHealth(t *testing.T) {
s := testServer(t)
rec := do(s, "GET", "/api/health", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
var out map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &out)
for _, k := range []string{"last_cycle_at", "credits_today", "scheduler_ok", "stale_flags"} {
if _, ok := out[k]; !ok {
t.Fatalf("missing key %s", k)
}
}
}
// Briefing generates from seed with zero empty sections + citations.
func TestBriefing(t *testing.T) {
s := testServer(t)
rec := do(s, "GET", "/api/briefing/today", nil)
if rec.Code == http.StatusNotFound {
// No briefing yet: run the routine via engine path instead.
rows, _ := s.DB.ListRoutines("demo")
if len(rows) == 0 {
t.Fatal("seed has no routines")
}
if _, err := s.Engine.Run(httptest.NewRequest("GET", "/", nil).Context(), rows[0]); err != nil {
t.Fatal(err)
}
rec = do(s, "GET", "/api/briefing/today", nil)
}
if rec.Code != http.StatusOK {
t.Fatalf("code = %d, body %s", rec.Code, rec.Body.String())
}
}
// Screener returns a ranked list with per-row breakdown.
func TestScreen(t *testing.T) {
s := testServer(t)
rec := do(s, "POST", "/api/screen", map[string]any{"limit": 5})
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
var out struct {
Rows []map[string]any `json:"rows"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if len(out.Rows) == 0 {
t.Fatal("empty screener rows")
}
if _, ok := out.Rows[0]["breakdown"]; !ok {
t.Fatal("missing per-row breakdown")
}
}
// Report: all 7 sections populated with citations.
func TestReport(t *testing.T) {
s := testServer(t)
rec := do(s, "POST", "/api/report/BBCA?profile=moderate", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d, body %s", rec.Code, rec.Body.String()[:300])
}
var out struct {
Sections []map[string]any `json:"sections"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if len(out.Sections) != 7 {
t.Fatalf("sections = %d, want 7", len(out.Sections))
}
for _, sec := range out.Sections {
if sec["body"] == "" || sec["body"] == nil {
t.Fatalf("empty section %v", sec["name"])
}
}
}
// Subscribe -> run -> history row appears.
func TestRoutineSubscribeRunHistory(t *testing.T) {
s := testServer(t)
rec := do(s, "POST", "/api/routines", map[string]any{"type": "foreign-reversal"})
if rec.Code != http.StatusCreated {
t.Fatalf("code = %d", rec.Code)
}
var created map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &created)
rows, _ := s.DB.ListRoutines("demo")
if len(rows) == 0 {
t.Fatal("no routines")
}
if _, err := s.Engine.Run(httptest.NewRequest("GET", "/", nil).Context(), rows[0]); err != nil {
t.Fatal(err)
}
rec = do(s, "GET", "/api/routine-runs?limit=5", nil)
var out struct {
Runs []map[string]any `json:"runs"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if len(out.Runs) == 0 {
t.Fatal("no history rows")
}
}
// Interrogation is scoped to report citations: conviction Q&A answers
// from the persisted report, unknown report 404s.
func TestInterrogate(t *testing.T) {
s := testServer(t)
rec := do(s, "POST", "/api/report/BBCA?profile=moderate", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
rec = do(s, "POST", "/api/report/BBCA/ask", map[string]any{"question": "kenapa conviction segitu?"})
if rec.Code != http.StatusOK {
t.Fatalf("code = %d, body %s", rec.Code, rec.Body.String()[:200])
}
var out struct {
Answer string `json:"answer"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if out.Answer == "" {
t.Fatal("empty interrogation answer")
}
rec = do(s, "POST", "/api/report/ZZZZ/ask", map[string]any{"question": "apa?"})
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404 for unknown ticker", rec.Code)
}
}
// Concentrated fixture warns >40% sector; accuracy math covered.
func TestPortfolioAndAccuracy(t *testing.T) {
s := testServer(t)
rec := do(s, "GET", "/api/portfolio/risk", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
rec = do(s, "GET", "/api/accuracy", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
}
// start/end filters narrow the foreign series; reversal uses the 2x rule.
func TestFlowForeignWindow(t *testing.T) {
s := testServer(t)
rec := do(s, "GET", "/api/flow/foreign?ticker=BBCA&start=2026-09-11&end=2026-09-11", nil)
if rec.Code != http.StatusOK {
t.Fatalf("code = %d", rec.Code)
}
var out struct {
Dates []string `json:"dates"`
Nets []float64 `json:"nets"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if len(out.Dates) != 1 || out.Dates[0] != "2026-09-11" {
t.Fatalf("dates = %v, want single filtered day", out.Dates)
}
}
// Unknown routine types are rejected; missing ids 404.
func TestRoutineValidation(t *testing.T) {
s := testServer(t)
rec := do(s, "POST", "/api/routines", map[string]any{"type": "not-a-routine"})
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("code = %d, want 422", rec.Code)
}
rec = do(s, "PATCH", "/api/routines/999999", map[string]any{"enabled": false})
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
rec = do(s, "DELETE", "/api/routines/999999", nil)
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
rec = do(s, "DELETE", "/api/alerts/999999", nil)
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
}
// Destination CRUD: masked list, kind validation, owner scoping, 404s.
func TestDestinations(t *testing.T) {
s := testServer(t)
// Invalid kind -> 422.
rec := do(s, "POST", "/api/destinations", map[string]any{"kind": "sms"})
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("code = %d, want 422", rec.Code)
}
// Telegram without chat_id -> 422.
rec = do(s, "POST", "/api/destinations", map[string]any{"kind": "telegram", "bot_token": "x"})
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("code = %d, want 422", rec.Code)
}
// Discord non-https -> 422.
rec = do(s, "POST", "/api/destinations", map[string]any{"kind": "discord", "webhook_url": "http://x"})
if rec.Code != http.StatusUnprocessableEntity {
t.Fatalf("code = %d, want 422", rec.Code)
}
// Valid discord create -> 201.
rec = do(s, "POST", "/api/destinations", map[string]any{"kind": "discord", "label": "ops", "webhook_url": "https://discord.example/hook"})
if rec.Code != http.StatusCreated {
t.Fatalf("code = %d, body %s", rec.Code, rec.Body.String())
}
// List masks secrets.
rec = do(s, "GET", "/api/destinations", nil)
var out struct {
Destinations []map[string]any `json:"destinations"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &out)
if len(out.Destinations) != 1 {
t.Fatalf("destinations = %v", out.Destinations)
}
for _, k := range []string{"bot_token", "chat_id", "webhook_url"} {
if _, ok := out.Destinations[0][k]; ok {
t.Fatalf("secret leaked in list: %s", k)
}
}
if out.Destinations[0]["configured"] != true {
t.Fatalf("configured flag = %v", out.Destinations[0])
}
// Missing id -> 404 on patch and delete.
rec = do(s, "PATCH", "/api/destinations/999999", map[string]any{"enabled": false})
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
rec = do(s, "DELETE", "/api/destinations/999999", nil)
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
}
backend/internal/api/chat.go
+71
View File
@@ -0,0 +1,71 @@
package api
import (
"encoding/json"
"net/http"
"strings"
)
// ChatRequest is POST /api/chat {message, scope?: {report_id}}.
type ChatRequest struct {
Message string `json:"message" validate:"required"`
Scope *struct {
ReportID int64 `json:"report_id"`
} `json:"scope"`
}
// Chat serves POST /api/chat: cited answers. With scope.report_id the
// grounding is restricted to that report's citations (report interrogation);
// without scope it answers from latest snapshots. The LLM refines prose only
// — numbers always come from stored data, never from generation.
func (s *Server) Chat(w http.ResponseWriter, r *http.Request) {
var req ChatRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "message is required")
return
}
// Scope grounding: report citations when scoped.
var ground, citesRaw string
if req.Scope != nil && req.Scope.ReportID > 0 {
var cites string
var at string
err := s.DB.QueryRow(`SELECT payload_json, citations_json, generated_at FROM reports WHERE id=?`,
req.Scope.ReportID).Scan(&ground, &cites, &at)
if err != nil {
writeErr(w, http.StatusNotFound, "report not found")
return
}
citesRaw = cites
} else {
// Unscoped: ground on the latest briefing + watchlist.
_, payload, cites, err := s.DB.LatestBriefing()
if err != nil {
ground = "no briefing or report data yet"
} else {
ground, citesRaw = payload, cites
}
}
answer := "Based on stored data: " + head(ground, 600)
if s.LLM.Available() {
if text, err := s.LLM.Complete(r.Context(), s.Cfg.LLMTriage,
"You answer questions about Indonesian stocks using ONLY the grounded data below. "+
"Every number in your answer must cite its source. If the data lacks the answer, say so.",
"Question: "+req.Message+"\n\nGrounded data:\n"+head(ground, 3000), 400); err == nil {
answer = strings.TrimSpace(text)
}
}
writeJSON(w, http.StatusOK, map[string]any{
"answer": answer, "grounding": head(ground, 600), "citations": citesRaw,
})
}
func head(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n]
}
backend/internal/api/destinations.go
+156
View File
@@ -0,0 +1,156 @@
package api
import (
"encoding/json"
"net/http"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"flowsight/internal/store"
)
// destOut is the masked API shape: secrets never leave the server.
type destOut struct {
ID int64 `json:"id"`
Kind string `json:"kind"`
Label string `json:"label"`
Enabled bool `json:"enabled"`
Configured bool `json:"configured"`
}
func maskDestinations(rows []store.Destination) []destOut {
out := make([]destOut, 0, len(rows))
for _, d := range rows {
cfg := false
switch d.Kind {
case store.DestTelegram:
cfg = d.BotToken != "" && d.ChatID != ""
case store.DestDiscord:
cfg = d.WebhookURL != ""
}
out = append(out, destOut{ID: d.ID, Kind: d.Kind, Label: d.Label, Enabled: d.Enabled, Configured: cfg})
}
return out
}
// ListDestinations serves GET /api/destinations (secrets masked).
func (s *Server) ListDestinations(w http.ResponseWriter, r *http.Request) {
rows, err := s.DB.ListDestinations(s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"destinations": maskDestinations(rows)})
}
// CreateDestination serves POST /api/destinations.
func (s *Server) CreateDestination(w http.ResponseWriter, r *http.Request) {
var req struct {
Kind string `json:"kind" validate:"required,oneof=telegram discord"`
Label string `json:"label"`
BotToken string `json:"bot_token"`
ChatID string `json:"chat_id"`
WebhookURL string `json:"webhook_url"`
Enabled *bool `json:"enabled"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
req.Kind = strings.ToLower(strings.TrimSpace(req.Kind))
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "kind must be telegram or discord")
return
}
if msg := checkDestSecrets(req.Kind, req.BotToken, req.ChatID, req.WebhookURL); msg != "" {
writeErr(w, http.StatusUnprocessableEntity, msg)
return
}
enabled := true
if req.Enabled != nil {
enabled = *req.Enabled
}
id, err := s.DB.CreateDestination(store.Destination{
UserKey: s.userKey(r), Kind: req.Kind, Label: req.Label,
BotToken: req.BotToken, ChatID: req.ChatID, WebhookURL: req.WebhookURL,
Enabled: enabled,
})
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusCreated, map[string]any{"id": id, "kind": req.Kind})
}
// UpdateDestination serves PATCH /api/destinations/:id. Kind is immutable;
// omitted secret fields keep their stored value.
func (s *Server) UpdateDestination(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeErr(w, http.StatusBadRequest, "invalid id")
return
}
var req struct {
Label *string `json:"label"`
Enabled *bool `json:"enabled"`
BotToken *string `json:"bot_token"`
ChatID *string `json:"chat_id"`
WebhookURL *string `json:"webhook_url"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
ok, err := s.DB.UpdateDestination(id, s.userKey(r), req.Label, req.Enabled, req.BotToken, req.ChatID, req.WebhookURL)
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if !ok {
writeErr(w, http.StatusNotFound, "destination not found")
return
}
writeJSON(w, http.StatusOK, map[string]any{"id": id, "ok": true})
}
// DeleteDestination serves DELETE /api/destinations/:id.
func (s *Server) DeleteDestination(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeErr(w, http.StatusBadRequest, "invalid id")
return
}
ok, err := s.DB.DeleteDestination(id, s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if !ok {
writeErr(w, http.StatusNotFound, "destination not found")
return
}
writeJSON(w, http.StatusOK, map[string]any{"id": id, "ok": true})
}
// checkDestSecrets validates kind-appropriate secrets.
func checkDestSecrets(kind, botToken, chatID, webhookURL string) string {
switch kind {
case store.DestTelegram:
if strings.TrimSpace(botToken) == "" || strings.TrimSpace(chatID) == "" {
return "telegram needs bot_token and chat_id"
}
case store.DestDiscord:
u := strings.TrimSpace(webhookURL)
if u == "" {
return "discord needs webhook_url"
}
if !strings.HasPrefix(u, "https://") {
return "webhook_url must be https"
}
default:
return "kind must be telegram or discord"
}
return ""
}
backend/internal/api/flow.go
+136
View File
@@ -0,0 +1,136 @@
package api
import (
"encoding/json"
"net/http"
"sort"
"strings"
"flowsight/internal/model"
"flowsight/internal/sectors"
)
// FlowSummary serves GET /api/flow/summary: foreign net total, top-5
// accumulation rows, rotation signal, mover of the day — all cited.
func (s *Server) FlowSummary(w http.ResponseWriter, r *http.Request) {
date := r.URL.Query().Get("date")
if date == "" {
date = "latest"
}
wl, _ := s.DB.Watchlist(s.userKey(r))
if len(wl) == 0 {
wl = s.Cfg.Watchlist
}
type accRow struct {
Ticker string `json:"ticker"`
NetSum float64 `json:"net_sum"`
Brokers int `json:"brokers"`
}
var accs []accRow
foreignTotal := 0.0
var cites []model.Citation
for _, tk := range wl {
if nets, err := s.DB.NetBuySum5d(tk); err == nil && len(nets) > 0 {
sum, n := 0.0, 0
for _, v := range nets {
if v > 0 {
n++
sum += v
}
}
accs = append(accs, accRow{tk, sum, n})
cites = append(cites, model.Cite("v2/broker-summary/"+tk+"/top/", tk, date))
}
if _, nets, err := s.DB.ForeignLast6(tk); err == nil && len(nets) > 0 {
foreignTotal += nets[len(nets)-1]
cites = append(cites, model.Cite("v2/foreign-flow/"+tk+"/", tk, date))
}
}
sort.Slice(accs, func(i, j int) bool { return accs[i].NetSum > accs[j].NetSum })
if len(accs) > 5 {
accs = accs[:5]
}
writeJSON(w, http.StatusOK, map[string]any{
"date": date, "foreign_net_total": foreignTotal,
"top_accumulation": accs, "citations": cites,
})
}
// brokerQuery validates ticker/start/end query params.
type brokerQuery struct {
Ticker string `validate:"required,len=4"`
Start string `validate:"omitempty,datetime=2006-01-02"`
End string `validate:"omitempty,datetime=2006-01-02"`
}
// FlowBroker serves GET /api/flow/broker: buyers/sellers + 5d net series.
func (s *Server) FlowBroker(w http.ResponseWriter, r *http.Request) {
q := brokerQuery{
Ticker: strings.ToUpper(r.URL.Query().Get("ticker")),
Start: r.URL.Query().Get("start"),
End: r.URL.Query().Get("end"),
}
if err := s.Validate.Struct(q); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "ticker (4 letters) required, dates YYYY-MM-DD")
return
}
var top sectors.BrokerSummaryTop
if raw, d, err := s.DB.SnapshotAt(q.Ticker, "broker-summary-top", q.End); err == nil {
_ = json.Unmarshal([]byte(raw), &top)
writeJSON(w, http.StatusOK, map[string]any{
"ticker": q.Ticker, "buyers": top.TopBuyers, "sellers": top.TopSellers,
"citations": []model.Citation{model.Cite("v2/broker-summary/"+q.Ticker+"/top/", q.Ticker, d)},
})
return
}
writeJSON(w, http.StatusOK, map[string]any{
"ticker": q.Ticker, "buyers": []any{}, "sellers": []any{},
"citations": []model.Citation{}, "note": "no snapshots yet",
})
}
// FlowForeign serves GET /api/flow/foreign: inflow series + reversal flag.
func (s *Server) FlowForeign(w http.ResponseWriter, r *http.Request) {
q := brokerQuery{
Ticker: strings.ToUpper(r.URL.Query().Get("ticker")),
Start: r.URL.Query().Get("start"),
End: r.URL.Query().Get("end"),
}
if err := s.Validate.Struct(q); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "ticker (4 letters) required, dates YYYY-MM-DD")
return
}
dates, nets, err := s.DB.ForeignWindow(q.Ticker, q.Start, q.End, 30)
if err != nil || len(nets) == 0 {
writeJSON(w, http.StatusOK, map[string]any{
"ticker": q.Ticker, "series": []any{}, "reversal": false,
"citations": []model.Citation{}, "note": "no snapshots yet",
})
return
}
// Same 2x-magnitude rule as the alert engine: 5d cumulative one way,
// last day the other way at >2x the trailing 5d daily average.
reversal := false
if len(nets) >= 6 {
tail := nets[len(nets)-6:]
sum5, absAvg := 0.0, 0.0
for _, v := range tail[:5] {
sum5 += v
if v < 0 {
absAvg -= v
} else {
absAvg += v
}
}
absAvg /= 5
last := tail[5]
if absAvg > 0 && ((sum5 < 0 && last > 0 && last > 2*absAvg) ||
(sum5 > 0 && last < 0 && -last > 2*absAvg)) {
reversal = true
}
}
writeJSON(w, http.StatusOK, map[string]any{
"ticker": q.Ticker, "dates": dates, "nets": nets, "reversal": reversal,
"citations": []model.Citation{model.Cite("v2/foreign-flow/"+q.Ticker+"/", q.Ticker, dates[len(dates)-1])},
})
}
backend/internal/api/health.go
+52
View File
@@ -0,0 +1,52 @@
package api
import (
"net/http"
"time"
"flowsight/internal/model"
)
// Health serves GET /api/health: last cycle time + credits spent today +
// scheduler state + stale flags (docs/API.md).
func (s *Server) Health(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("force") == "1" {
_ = s.Sched.RunCycle(r.Context()) // synchronous probe cycle
}
lastCycle, schedOK := s.Sched.Status()
today := time.Now().Format("2006-01-02")
credits := s.DB.CreditsToday(today)
lastDate, _, _, _ := s.DB.LatestBriefing()
cutoff := model.StaleSession(time.Now())
stale := []string{}
if lastDate != "" {
if t, err := time.Parse("2006-01-02", lastDate[:10]); err == nil && t.Before(cutoff) {
stale = append(stale, "briefing older than one session ("+lastDate+")")
} else if lastDate < today {
stale = append(stale, "briefing older than today ("+lastDate+")")
}
}
if !s.Cfg.HasSectorsKey() {
stale = append(stale, "offline mode: SECTORS_API_KEY unset, serving seed data")
}
writeJSON(w, http.StatusOK, map[string]any{
"last_cycle_at": lastCycle,
"credits_today": credits,
"scheduler_ok": schedOK,
"stale_flags": stale,
"citations": []model.Citation{},
})
}
// Accuracy serves GET /api/accuracy: per-agent {calls, resolved, hits, hit_rate}.
func (s *Server) Accuracy(w http.ResponseWriter, r *http.Request) {
stats, err := s.DB.AccuracyStats()
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if stats == nil {
stats = []map[string]any{}
}
writeJSON(w, http.StatusOK, map[string]any{"agents": stats})
}
backend/internal/api/interrogate.go
+106
View File
@@ -0,0 +1,106 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
)
// Interrogate serves POST /api/report/:ticker/ask {question, report_id?}:
// follow-up Q&A grounded ONLY in that report's persisted citations.
// Without report_id it uses the latest report for the ticker.
func (s *Server) Interrogate(w http.ResponseWriter, r *http.Request) {
ticker := strings.ToUpper(chi.URLParam(r, "ticker"))
var req struct {
Question string `json:"question" validate:"required"`
ReportID int64 `json:"report_id"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "question is required")
return
}
payload, citesRaw, at, id, err := s.loadReport(ticker, req.ReportID)
if err != nil {
writeErr(w, http.StatusNotFound, "no report for "+ticker+" yet — POST /api/report/"+ticker+" first")
return
}
var cites []map[string]any
_ = json.Unmarshal([]byte(citesRaw), &cites)
answer := groundedAnswer(req.Question, payload)
if s.LLM.Available() {
if text, err := s.LLM.Complete(r.Context(), s.Cfg.LLMTriage,
"Answer ONLY from the report JSON below. Every number must quote its cited value. "+
"If the report lacks the answer, say exactly: not in this report.",
"Question: "+req.Question+"\n\nReport:\n"+head(payload, 3000), 400); err == nil && text != "" {
answer = strings.TrimSpace(text)
}
}
writeJSON(w, http.StatusOK, map[string]any{
"answer": answer, "ticker": ticker, "report_id": id,
"generated_at": at, "citations": cites,
})
}
// loadReport fetches (payload, citations, generated_at, id) for an explicit
// report id or the latest report for a ticker.
func (s *Server) loadReport(ticker string, id int64) (string, string, string, int64, error) {
if id > 0 {
var t, p, c, at string
var rid int64
err := s.DB.QueryRow(`SELECT id, ticker, payload_json, citations_json, generated_at
FROM reports WHERE id=?`, id).Scan(&rid, &t, &p, &c, &at)
if err != nil {
return "", "", "", 0, err
}
if t != ticker {
return "", "", "", 0, fmt.Errorf("report %d belongs to %s", id, t)
}
return p, c, at, rid, nil
}
p, c, at, err := s.DB.LatestReport(ticker)
if err != nil {
return "", "", "", 0, err
}
var rid int64
_ = s.DB.QueryRow(`SELECT id FROM reports WHERE ticker=? ORDER BY id DESC LIMIT 1`,
ticker).Scan(&rid)
return p, c, at, rid, nil
}
// groundedAnswer is the offline fallback: it extracts the recommendation +
// conviction + cited lines matching question keywords from the payload.
func groundedAnswer(question, payload string) string {
var rep struct {
Synthesis struct {
Recommendation string `json:"recommendation"`
Conviction int `json:"conviction"`
Thesis string `json:"thesis"`
} `json:"synthesis"`
Sections []struct {
Name string `json:"name"`
Body string `json:"body"`
} `json:"sections"`
}
if err := json.Unmarshal([]byte(payload), &rep); err != nil {
return "not in this report"
}
q := strings.ToLower(question)
if strings.Contains(q, "conviction") || strings.Contains(q, "kenapa") || strings.Contains(q, "why") {
return fmt.Sprintf("%s with conviction %d/5: %s",
rep.Synthesis.Recommendation, rep.Synthesis.Conviction, rep.Synthesis.Thesis)
}
for _, sec := range rep.Sections {
if strings.Contains(q, strings.ToLower(sec.Name)) {
return sec.Body
}
}
return fmt.Sprintf("%s (conviction %d/5): %s",
rep.Synthesis.Recommendation, rep.Synthesis.Conviction, rep.Synthesis.Thesis)
}
backend/internal/api/portfolio.go
+233
View File
@@ -0,0 +1,233 @@
package api
import (
"encoding/json"
"math"
"net/http"
"sort"
"flowsight/internal/model"
)
// PortfolioRisk serves GET /api/portfolio/risk: concentration bars,
// correlation matrix, beta vs IHSG, warnings (concentrated fixture warns
// >40% sector), accuracy-adjacent citations.
func (s *Server) PortfolioRisk(w http.ResponseWriter, r *http.Request) {
wl, _ := s.DB.Watchlist(s.userKey(r))
if len(wl) == 0 {
wl = s.Cfg.Watchlist
}
// Concentration: weight by latest close x assumed equal shares (seed-safe).
type bar struct {
Ticker string `json:"ticker"`
Sector string `json:"sector"`
Weight float64 `json:"weight"`
}
prices := map[string]float64{}
total := 0.0
for _, tk := range wl {
px, _, err := s.DB.LatestClose(tk)
if err != nil || px <= 0 {
px = 1000 // seed-safe placeholder, flagged in warnings
}
prices[tk] = px
total += px
}
sectorOf := sectorMap()
var bars []bar
sectorW := map[string]float64{}
for _, tk := range wl {
wt := 0.0
if total > 0 {
wt = prices[tk] / total
}
sec := sectorOf[tk]
if sec == "" {
sec = "unknown"
}
bars = append(bars, bar{tk, sec, wt})
sectorW[sec] += wt
}
sort.Slice(bars, func(i, j int) bool { return bars[i].Weight > bars[j].Weight })
var warnings []string
for sec, wt := range sectorW {
if wt > 0.4 {
warnings = append(warnings, "concentrated: "+sec+" at "+pct(wt)+" (over 40%)")
}
}
// Correlation: pairwise Pearson over stored daily closes (aligned tail).
series := s.closes(wl)
corr := correlationMatrixFrom(series, wl)
beta := betaFrom(series, wl)
writeJSON(w, http.StatusOK, map[string]any{
"concentration": bars, "correlation": corr, "beta": beta,
"warnings": warnings,
"citations": []model.Citation{model.Cite("v2/daily/", "watchlist", "stored")},
})
}
func pct(v float64) string {
return itoa(int(v*100+0.5)) + "%"
}
func itoa(n int) string {
if n == 0 {
return "0"
}
s := ""
for n > 0 {
s = string(rune('0'+n%10)) + s
n /= 10
}
return s
}
// sectorMap is the seed-safe sector lookup (live: subsector/report).
func sectorMap() map[string]string {
return map[string]string{
"BBCA": "financials", "BBRI": "financials", "BMRI": "financials", "BBNI": "financials",
"TLKM": "infrastructure", "ASII": "industrials", "UNVR": "consumer", "ICBP": "consumer",
}
}
// closes returns aligned close series per ticker from snapshots.
func (s *Server) closes(wl []string) map[string][]float64 {
out := map[string][]float64{}
for _, tk := range wl {
var rows []struct {
Close float64 `json:"close"`
}
if raw, _, err := s.DB.LatestSnapshot(tk, "daily"); err == nil {
var bars []struct {
Close float64 `json:"close"`
}
if json.Unmarshal([]byte(raw), &bars) == nil {
for _, b := range bars {
rows = append(rows, struct {
Close float64 `json:"close"`
}{b.Close})
}
}
_ = rows
series := make([]float64, 0, len(bars))
for _, b := range bars {
series = append(series, b.Close)
}
out[tk] = series
}
}
return out
}
func correlationMatrixFrom(series map[string][]float64, wl []string) map[string]map[string]float64 {
m := map[string]map[string]float64{}
for _, a := range wl {
m[a] = map[string]float64{}
for _, b := range wl {
if a == b {
m[a][b] = 1
continue
}
m[a][b] = pearson(tail(series[a], 30), tail(series[b], 30))
}
}
return m
}
func tail(xs []float64, n int) []float64 {
if len(xs) <= n {
return xs
}
return xs[len(xs)-n:]
}
// pearson computes the correlation of two equal-length series.
func pearson(a, b []float64) float64 {
n := len(a)
if n != len(b) || n < 2 {
return 0
}
ma, mb := mean(a), mean(b)
num, da, db := 0.0, 0.0, 0.0
for i := range a {
num += (a[i] - ma) * (b[i] - mb)
da += (a[i] - ma) * (a[i] - ma)
db += (b[i] - mb) * (b[i] - mb)
}
if da == 0 || db == 0 {
return 0
}
return num / (math.Sqrt(da) * math.Sqrt(db))
}
func mean(xs []float64) float64 {
s := 0.0
for _, x := range xs {
s += x
}
return s / float64(len(xs))
}
// betaFrom regresses mean ticker returns vs the watchlist mean (index-daily
// benchmark when cached; watchlist-mean fallback keeps seeds working).
func betaFrom(series map[string][]float64, wl []string) float64 {
if len(wl) == 0 {
return 1
}
n := 0
for _, tk := range wl {
if len(series[tk]) > n {
n = len(series[tk])
}
}
if n < 2 {
return 1
}
idx := make([]float64, n)
for _, tk := range wl {
s := series[tk]
for i := range idx {
if i < len(s) {
idx[i] += s[i]
}
}
}
for i := range idx {
idx[i] /= float64(len(wl))
}
betas := []float64{}
for _, tk := range wl {
if b := betaOf(series[tk], idx); b != 0 {
betas = append(betas, b)
}
}
if len(betas) == 0 {
return 1
}
return mean(betas)
}
// betaOf is cov(asset,index)/var(index) over the aligned tail.
func betaOf(asset, index []float64) float64 {
n := len(asset)
if len(index) < n {
n = len(index)
}
if n < 2 {
return 0
}
a, ix := asset[len(asset)-n:], index[len(index)-n:]
ma, mi := mean(a), mean(ix)
num, den := 0.0, 0.0
for i := range a {
num += (a[i] - ma) * (ix[i] - mi)
den += (ix[i] - mi) * (ix[i] - mi)
}
if den == 0 {
return 0
}
return num / den
}
backend/internal/api/report.go
+52
View File
@@ -0,0 +1,52 @@
package api
import (
"net/http"
"strings"
"github.com/go-chi/chi/v5"
)
// BuildReport serves POST /api/report/:ticker?format=json|html|pdf|md.
// Runs A1..A6 + A7 live over stored snapshots, persists, and renders.
func (s *Server) BuildReport(w http.ResponseWriter, r *http.Request) {
ticker := strings.ToUpper(chi.URLParam(r, "ticker"))
if len(ticker) < 3 || len(ticker) > 6 {
writeErr(w, http.StatusUnprocessableEntity, "ticker must be 3-6 letters")
return
}
profile := r.URL.Query().Get("profile")
if profile == "" {
profile = "moderate"
}
rep, id, err := s.Builder.Build(r.Context(), ticker, profile)
if err != nil {
writeErr(w, http.StatusBadGateway, "report: "+err.Error())
return
}
_ = id
switch strings.ToLower(r.URL.Query().Get("format")) {
case "html":
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(rep.ToHTML()))
case "md":
w.Header().Set("Content-Type", "text/markdown; charset=utf-8")
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(rep.ToMarkdown()))
case "pdf":
raw, err := rep.ToPDF()
if err != nil {
writeErr(w, http.StatusBadGateway, "pdf: "+err.Error())
return
}
w.Header().Set("Content-Type", "application/pdf")
w.WriteHeader(http.StatusOK)
_, _ = w.Write(raw)
default:
writeJSON(w, http.StatusOK, rep)
}
// Push a live agent-panel event for the dashboard SSE feed.
s.Hub.Publish("agents", `{"ticker":"`+ticker+`","recommendation":"`+
rep.Synthesis.Recommendation+`"}`)
}
backend/internal/api/routines.go
+136
View File
@@ -0,0 +1,136 @@
package api
import (
"encoding/json"
"net/http"
"strconv"
"github.com/go-chi/chi/v5"
"flowsight/internal/routines"
"flowsight/internal/store"
)
// ListRoutines serves GET /api/routines with last-run status.
func (s *Server) ListRoutines(w http.ResponseWriter, r *http.Request) {
rows, err := s.DB.ListRoutines(s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
type rowOut struct {
store.Routine
LastRun any `json:"last_run"`
}
out := make([]rowOut, 0, len(rows))
for _, row := range rows {
hist, _ := s.DB.RunHistory(row.ID, 1)
var last any
if len(hist) > 0 {
last = hist[0]
}
out = append(out, rowOut{row, last})
}
writeJSON(w, http.StatusOK, map[string]any{"routines": out})
}
// CreateRoutine serves POST /api/routines {type, schedule_cron?, channels[]}.
func (s *Server) CreateRoutine(w http.ResponseWriter, r *http.Request) {
var req struct {
Type string `json:"type" validate:"required"`
Schedule string `json:"schedule_cron"`
Channels []string `json:"channels"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "type is required")
return
}
if !routines.KnownType(req.Type) {
writeErr(w, http.StatusUnprocessableEntity, "unknown routine type")
return
}
if req.Schedule == "" {
req.Schedule = routines.DefaultSchedule(req.Type)
}
id, err := s.DB.CreateRoutine(s.userKey(r), req.Type, req.Schedule, req.Channels)
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusCreated, map[string]any{"id": id, "type": req.Type, "schedule_cron": req.Schedule})
}
// UpdateRoutine serves PATCH /api/routines/:id.
func (s *Server) UpdateRoutine(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeErr(w, http.StatusBadRequest, "invalid id")
return
}
var req struct {
Enabled *bool `json:"enabled"`
Schedule string `json:"schedule_cron"`
Channels []string `json:"channels"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
ok, err := s.DB.UpdateRoutine(id, s.userKey(r), req.Enabled, req.Schedule, req.Channels)
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if !ok {
writeErr(w, http.StatusNotFound, "routine not found")
return
}
writeJSON(w, http.StatusOK, map[string]any{"id": id, "ok": true})
}
// RunHistory serves GET /api/routine-runs?routine_id=&limit=.
func (s *Server) RunHistory(w http.ResponseWriter, r *http.Request) {
rid, _ := strconv.ParseInt(r.URL.Query().Get("routine_id"), 10, 64)
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
hist, err := s.DB.RunHistory(rid, limit, s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"runs": hist})
}
// BriefingToday serves GET /api/briefing/today: latest payload + citations.
func (s *Server) BriefingToday(w http.ResponseWriter, r *http.Request) {
date, payload, cites, err := s.DB.LatestBriefing()
if err != nil {
writeErr(w, http.StatusNotFound, "no briefing yet — run the morning-briefing routine")
return
}
writeJSON(w, http.StatusOK, map[string]any{
"date": date, "payload": payload, "citations": cites,
})
}
// DeleteRoutine serves DELETE /api/routines/:id.
func (s *Server) DeleteRoutine(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
if err != nil {
writeErr(w, http.StatusBadRequest, "invalid id")
return
}
ok, err := s.DB.DeleteRoutine(id, s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
if !ok {
writeErr(w, http.StatusNotFound, "routine not found")
return
}
writeJSON(w, http.StatusOK, map[string]any{"id": id, "ok": true})
}
backend/internal/api/screen.go
+163
View File
@@ -0,0 +1,163 @@
package api
import (
"encoding/json"
"net/http"
"sort"
"strings"
"flowsight/internal/model"
)
// ScreenRequest is POST /api/screen body.
type ScreenRequest struct {
Where string `json:"where"`
Q string `json:"q"`
Institutional *struct {
BrokerScoreMin float64 `json:"broker_score_min"`
ForeignTrend string `json:"foreign_trend"`
InsiderBuying bool `json:"insider_buying"`
VolumeAnomaly bool `json:"volume_anomaly"`
} `json:"institutional"`
Limit int `json:"limit"`
}
// ScreenRow is one ranked result with per-row signal breakdown.
type ScreenRow struct {
Symbol string `json:"symbol"`
Name string `json:"name"`
Composite float64 `json:"composite"`
Breakdown map[string]any `json:"breakdown"`
Citations []model.Citation `json:"citations"`
}
// Screen serves POST /api/screen: companies/ base filter enriched with
// broker score + foreign trend + insider flag, ranked composite.
func (s *Server) Screen(w http.ResponseWriter, r *http.Request) {
var req ScreenRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
limit := req.Limit
if limit <= 0 || limit > 100 {
limit = 20
}
// Base universe: live screener when keyed, else stored watchlist.
var universe []string
if s.Cfg.HasSectorsKey() && (req.Where != "" || req.Q != "") {
if rows, err := s.Sectors.Screen(r.Context(), req.Where, req.Q, limit*2, 0); err == nil {
for _, row := range rows {
universe = append(universe, strings.ToUpper(strings.TrimSuffix(row.Symbol, ".JK")))
}
}
}
if len(universe) == 0 {
universe, _ = s.DB.Watchlist(s.userKey(r))
if len(universe) == 0 {
universe = s.Cfg.Watchlist
}
}
var rows []ScreenRow
for _, tk := range universe {
row := s.scoreTicker(tk)
if req.Institutional != nil {
inst := req.Institutional
if b, _ := row.Breakdown["broker_score"].(float64); b < inst.BrokerScoreMin {
continue
}
if inst.ForeignTrend != "" {
if t, _ := row.Breakdown["foreign_trend"].(string); t != inst.ForeignTrend {
continue
}
}
if inst.InsiderBuying {
if b, _ := row.Breakdown["insider_buying"].(bool); !b {
continue
}
}
if inst.VolumeAnomaly {
if b, _ := row.Breakdown["volume_anomaly"].(bool); !b {
continue
}
}
}
rows = append(rows, row)
}
sort.Slice(rows, func(i, j int) bool { return rows[i].Composite > rows[j].Composite })
if len(rows) > limit {
rows = rows[:limit]
}
writeJSON(w, http.StatusOK, map[string]any{"rows": rows, "count": len(rows)})
}
// scoreTicker computes the composite (broker 40 + foreign 25 + insider 15 + volume 20).
func (s *Server) scoreTicker(tk string) ScreenRow {
tk = strings.ToUpper(tk)
row := ScreenRow{Symbol: tk, Name: tk, Breakdown: map[string]any{}}
// Broker score from 5d net imbalance.
brokerScore := 0.0
if nets, err := s.DB.NetBuySum5d(tk); err == nil && len(nets) > 0 {
pos, neg := 0.0, 0.0
for _, v := range nets {
if v > 0 {
pos += v
} else {
neg -= v
}
}
if tot := pos + neg; tot > 0 {
brokerScore = (pos - neg) / tot * 100
}
row.Citations = append(row.Citations, model.Cite("v2/broker-summary/"+tk+"/top/", tk, "stored"))
}
// Foreign trend from last-6 series.
foreignScore, trend := 0.0, "flat"
if dates, nets, err := s.DB.ForeignLast6(tk); err == nil && len(nets) > 0 {
last := nets[len(nets)-1]
if last > 0 {
foreignScore, trend = 50, "inflow"
} else if last < 0 {
foreignScore, trend = -50, "outflow"
}
row.Citations = append(row.Citations, model.Cite("v2/foreign-flow/"+tk+"/", tk, dates[len(dates)-1]))
}
// Insider flag from filings average.
insider := s.DB.FilingAvg30(tk) > 0
// Volume anomaly from stored daily bars.
volAnom, volMult := false, 0.0
if vols, _, err := s.DB.DailyVolumes(tk, 21); err == nil && len(vols) >= 2 {
n := len(vols)
if a := avgF(vols[:n-1]); a > 0 {
volMult = vols[n-1] / a
volAnom = volMult > 2
}
row.Citations = append(row.Citations, model.Cite("v2/daily/"+tk+"/", tk, "stored"))
}
volScore := 0.0
if volAnom {
volScore = 50
}
row.Composite = brokerScore*0.4 + foreignScore*0.25 + volScore*0.2
if insider {
row.Composite += 7.5
}
row.Breakdown = map[string]any{
"broker": brokerScore, "broker_score": brokerScore,
"foreign": foreignScore, "foreign_trend": trend,
"insider": insider, "insider_buying": insider,
"volume_mult": volMult, "volume_anomaly": volAnom,
}
return row
}
func avgF(xs []float64) float64 {
if len(xs) == 0 {
return 0
}
sum := 0.0
for _, x := range xs {
sum += x
}
return sum / float64(len(xs))
}
backend/internal/api/server.go
+112
View File
@@ -0,0 +1,112 @@
// Package api serves the FlowSight REST API (docs/API.md) on chi: flow
// summary/broker/foreign, screener, routines, briefing, alerts, reports,
// watchlist, portfolio risk, accuracy, chat (report-scoped), health, and the
// SSE stream (agents/alerts/activity, 15s heartbeat). Demo auth: X-User-Key.
package api
import (
"encoding/json"
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/go-playground/validator/v10"
"flowsight/internal/agents"
"flowsight/internal/config"
"flowsight/internal/llm"
"flowsight/internal/reports"
"flowsight/internal/routines"
"flowsight/internal/scheduler"
"flowsight/internal/sectors"
"flowsight/internal/store"
)
// Server wires all handlers.
type Server struct {
Cfg config.Config
DB *store.DB
Sectors *sectors.Client
Sched *scheduler.Scheduler
Engine *routines.Engine
Builder *reports.Builder
LLM *llm.Client
Validate *validator.Validate
Hub *Hub
}
// New builds a Server with all dependencies wired.
func New(cfg config.Config, db *store.DB, cache *store.Cache, s *sectors.Client) *Server {
llmc := llm.New(cfg.LLMBaseURL, cfg.LLMAPIKey)
sched := scheduler.New(cfg, db, cache, s)
srv := &Server{
Cfg: cfg, DB: db, Sectors: s, Sched: sched, LLM: llmc,
Validate: validator.New(),
Hub: NewHub(),
}
srv.Engine = &routines.Engine{DB: db, Notifier: sched.Notifier, UserKey: cfg.DemoUserKey,
Publish: srv.Hub.Publish}
sched.Publish = srv.Hub.Publish
srv.Builder = &reports.Builder{DB: db, Deps: agents.Deps{
DB: db, LLM: llmc, TriageModel: cfg.LLMTriage, SynthModel: cfg.LLMSynth,
Now: time.Now(),
}}
return srv
}
// Router returns the chi mux with all routes.
func (s *Server) Router() http.Handler {
r := chi.NewRouter()
r.Use(middleware.Logger, middleware.Recoverer, middleware.Heartbeat("/ping"))
r.Route("/api", func(r chi.Router) {
r.Get("/health", s.Health)
r.Get("/stream", s.Stream)
r.Get("/flow/summary", s.FlowSummary)
r.Get("/flow/broker", s.FlowBroker)
r.Get("/flow/foreign", s.FlowForeign)
r.Post("/screen", s.Screen)
r.Get("/routines", s.ListRoutines)
r.Post("/routines", s.CreateRoutine)
r.Patch("/routines/{id}", s.UpdateRoutine)
r.Delete("/routines/{id}", s.DeleteRoutine)
r.Get("/routine-runs", s.RunHistory)
r.Get("/briefing/today", s.BriefingToday)
r.Get("/alerts", s.ListAlerts)
r.Post("/alerts", s.CreateAlert)
r.Delete("/alerts/{id}", s.DeleteAlert)
r.Get("/alert-events", s.AlertEvents)
r.Get("/destinations", s.ListDestinations)
r.Post("/destinations", s.CreateDestination)
r.Patch("/destinations/{id}", s.UpdateDestination)
r.Delete("/destinations/{id}", s.DeleteDestination)
r.Post("/report/{ticker}", s.BuildReport)
r.Post("/report/{ticker}/ask", s.Interrogate)
r.Get("/watchlist", s.GetWatchlist)
r.Post("/watchlist", s.AddWatch)
r.Delete("/watchlist/{ticker}", s.RemoveWatch)
r.Get("/portfolio/risk", s.PortfolioRisk)
r.Get("/accuracy", s.Accuracy)
r.Post("/chat", s.Chat)
})
return r
}
// userKey resolves the demo auth header (single demo key for hackathon).
func (s *Server) userKey(r *http.Request) string {
if k := strings.TrimSpace(r.Header.Get("X-User-Key")); k != "" {
return k
}
return s.Cfg.DemoUserKey
}
func writeJSON(w http.ResponseWriter, code int, v any) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
_ = json.NewEncoder(w).Encode(v)
}
func writeErr(w http.ResponseWriter, code int, msg string) {
writeJSON(w, code, map[string]any{"error": map[string]string{"code": http.StatusText(code), "message": msg}})
}
backend/internal/api/stream.go
+122
View File
@@ -0,0 +1,122 @@
package api
import (
"fmt"
"net/http"
"sync"
"time"
)
// Hub fans SSE events out to connected browsers. Channels: agents
// (status+scores during runs), alerts (new events), activity (feed rows).
// Heartbeat 15s; reconnect resumes from last event ID (best-effort replay of
// the last 50 events).
type Hub struct {
mu sync.Mutex
subs map[chan SSEEvent]bool
history []SSEEvent
nextID int64
}
// SSEEvent is one server-sent event.
type SSEEvent struct {
ID int64
Channel string
Data string
}
// NewHub builds an empty hub.
func NewHub() *Hub { return &Hub{subs: map[chan SSEEvent]bool{}} }
// Publish broadcasts to all subscribers and appends to history.
func (h *Hub) Publish(channel, data string) {
h.mu.Lock()
h.nextID++
ev := SSEEvent{ID: h.nextID, Channel: channel, Data: data}
h.history = append(h.history, ev)
if len(h.history) > 50 {
h.history = h.history[len(h.history)-50:]
}
for ch := range h.subs {
select {
case ch <- ev:
default:
}
}
h.mu.Unlock()
}
func (h *Hub) subscribe() chan SSEEvent {
ch := make(chan SSEEvent, 16)
h.mu.Lock()
h.subs[ch] = true
h.mu.Unlock()
return ch
}
// since returns history entries newer than id (all when id <= 0).
func (h *Hub) since(id int64) []SSEEvent {
h.mu.Lock()
defer h.mu.Unlock()
var out []SSEEvent
for _, ev := range h.history {
if ev.ID > id {
out = append(out, ev)
}
}
return out
}
// lastEventID parses Last-Event-ID (header or query) for resume.
func lastEventID(r *http.Request) int64 {
s := r.Header.Get("Last-Event-ID")
if s == "" {
s = r.URL.Query().Get("lastEventId")
}
var id int64
fmt.Sscanf(s, "%d", &id)
return id
}
func (h *Hub) unsubscribe(ch chan SSEEvent) {
h.mu.Lock()
delete(h.subs, ch)
close(ch)
h.mu.Unlock()
}
// Stream serves GET /api/stream as text/event-stream.
func (s *Server) Stream(w http.ResponseWriter, r *http.Request) {
fl, ok := w.(http.Flusher)
if !ok {
writeErr(w, http.StatusInternalServerError, "streaming unsupported")
return
}
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
ch := s.Hub.subscribe()
defer s.Hub.unsubscribe(ch)
// Best-effort replay: resume after Last-Event-ID so reconnects do not
// lose the last 50 events (matches the history comment on Publish).
for _, ev := range s.Hub.since(lastEventID(r)) {
fmt.Fprintf(w, "id: %d\nevent: %s\ndata: %s\n\n", ev.ID, ev.Channel, ev.Data)
}
fl.Flush()
tick := time.NewTicker(15 * time.Second)
defer tick.Stop()
fmt.Fprintf(w, ": connected\n\n")
fl.Flush()
for {
select {
case <-r.Context().Done():
return
case ev := <-ch:
fmt.Fprintf(w, "id: %d\nevent: %s\ndata: %s\n\n", ev.ID, ev.Channel, ev.Data)
fl.Flush()
case <-tick.C:
fmt.Fprintf(w, ": heartbeat\n\n")
fl.Flush()
}
}
}
backend/internal/api/watchlist.go
+50
View File
@@ -0,0 +1,50 @@
package api
import (
"encoding/json"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
)
// GetWatchlist serves GET /api/watchlist.
func (s *Server) GetWatchlist(w http.ResponseWriter, r *http.Request) {
wl, err := s.DB.Watchlist(s.userKey(r))
if err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"watchlist": wl})
}
// AddWatch serves POST /api/watchlist {ticker}.
func (s *Server) AddWatch(w http.ResponseWriter, r *http.Request) {
var req struct {
Ticker string `json:"ticker" validate:"required,len=4"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeErr(w, http.StatusBadRequest, "invalid JSON body")
return
}
req.Ticker = strings.ToUpper(strings.TrimSpace(req.Ticker))
if err := s.Validate.Struct(req); err != nil {
writeErr(w, http.StatusUnprocessableEntity, "ticker (4 letters) required")
return
}
if err := s.DB.AddWatch(s.userKey(r), req.Ticker); err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusCreated, map[string]any{"ticker": req.Ticker})
}
// RemoveWatch serves DELETE /api/watchlist/:ticker.
func (s *Server) RemoveWatch(w http.ResponseWriter, r *http.Request) {
ticker := strings.ToUpper(chi.URLParam(r, "ticker"))
if err := s.DB.RemoveWatch(s.userKey(r), ticker); err != nil {
writeErr(w, http.StatusBadGateway, "db: "+err.Error())
return
}
writeJSON(w, http.StatusOK, map[string]any{"ticker": ticker, "ok": true})
}
backend/internal/config/config.go
+87
View File
@@ -0,0 +1,87 @@
// Package config loads FlowSight runtime configuration from the environment.
// Secrets come only from env vars, never from files.
package config
import (
"os"
"strconv"
"strings"
"github.com/joho/godotenv"
)
// Config holds all runtime settings for the FlowSight backend.
type Config struct {
Port string
SectorsAPIKey string
SectorsBaseURL string
DBPath string
RedisURL string
DemoUserKey string
LLMBaseURL string
LLMAPIKey string
LLMTriage string
LLMSynth string
TelegramBotToken string
TelegramChatID string
DiscordWebhookURL string
CreditCapPerCycle int
Watchlist []string
}
// HasSectorsKey reports whether live Sectors API calls are possible.
// Without a key the server runs in offline/seed mode.
func (c Config) HasSectorsKey() bool { return strings.TrimSpace(c.SectorsAPIKey) != "" }
// HasLLM reports whether LLM-backed refinement is available.
func (c Config) HasLLM() bool {
return strings.TrimSpace(c.LLMBaseURL) != "" && strings.TrimSpace(c.LLMAPIKey) != ""
}
func getenv(key, def string) string {
if v := os.Getenv(key); v != "" {
return v
}
return def
}
func getenvInt(key string, def int) int {
v := os.Getenv(key)
if v == "" {
return def
}
n, err := strconv.Atoi(v)
if err != nil {
return def
}
return n
}
// Load reads configuration from the environment (.env supported for dev).
func Load() Config {
_ = godotenv.Load()
watch := getenv("WATCHLIST", "BBCA,BBRI,BMRI,TLKM,ASII")
tickers := make([]string, 0, 8)
for _, t := range strings.Split(watch, ",") {
if t = strings.ToUpper(strings.TrimSpace(t)); t != "" {
tickers = append(tickers, t)
}
}
return Config{
Port: getenv("PORT", "8080"),
SectorsAPIKey: os.Getenv("SECTORS_API_KEY"),
SectorsBaseURL: getenv("SECTORS_BASE_URL", "https://api.sectors.app/v2/"),
DBPath: getenv("DB_PATH", "data/flowsight.db"),
RedisURL: os.Getenv("REDIS_URL"),
DemoUserKey: getenv("DEMO_USER_KEY", "demo"),
LLMBaseURL: os.Getenv("LLM_BASE_URL"),
LLMAPIKey: os.Getenv("LLM_API_KEY"),
LLMTriage: getenv("LLM_MODEL_TRIAGE", "gpt-4o-mini"),
LLMSynth: getenv("LLM_MODEL_SYNTH", "gpt-4o"),
TelegramBotToken: os.Getenv("TELEGRAM_BOT_TOKEN"),
TelegramChatID: os.Getenv("TELEGRAM_CHAT_ID"),
DiscordWebhookURL: os.Getenv("DISCORD_WEBHOOK_URL"),
CreditCapPerCycle: getenvInt("CREDIT_CAP_PER_CYCLE", 120),
Watchlist: tickers,
}
}
backend/internal/llm/llm.go
+122
View File
@@ -0,0 +1,122 @@
// Package llm calls an OpenAI-compatible chat endpoint for synthesis, report
// interrogation, and sentiment triage. Two models: LLM_MODEL_TRIAGE (cheap)
// and LLM_MODEL_SYNTH (strong), both overridden via env. No LLM call is on
// the critical detection path — rules and scores are computed locally first.
package llm
import (
"bytes"
"context"
"encoding/json"
"fmt"
"net/http"
"strings"
"time"
)
// Client talks to an OpenAI-compatible /chat/completions endpoint.
type Client struct {
baseURL string
apiKey string
http *http.Client
}
// New builds a client; baseURL is like https://api.openai.com/v1.
func New(baseURL, apiKey string) *Client {
return &Client{
baseURL: strings.TrimSuffix(baseURL, "/"),
apiKey: apiKey,
http: &http.Client{Timeout: 60 * time.Second},
}
}
// Available reports whether LLM calls are configured.
func (c *Client) Available() bool { return c.baseURL != "" && c.apiKey != "" }
type chatMsg struct {
Role string `json:"role"`
Content string `json:"content"`
}
// Complete sends one chat completion and returns the text content.
func (c *Client) Complete(ctx context.Context, model, system, user string, maxTokens int) (string, error) {
if !c.Available() {
return "", fmt.Errorf("llm: LLM_BASE_URL/LLM_API_KEY not configured")
}
if maxTokens <= 0 {
maxTokens = 800
}
body, _ := json.Marshal(map[string]any{
"model": model,
"messages": []chatMsg{{Role: "system", Content: system}, {Role: "user", Content: user}},
"max_tokens": maxTokens,
})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/chat/completions", bytes.NewReader(body))
if err != nil {
return "", err
}
req.Header.Set("Authorization", "Bearer "+c.apiKey)
req.Header.Set("Content-Type", "application/json")
resp, err := c.http.Do(req)
if err != nil {
return "", fmt.Errorf("llm: %w", err)
}
defer resp.Body.Close()
var out struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
Error *struct {
Message string `json:"message"`
} `json:"error"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return "", fmt.Errorf("llm: decode: %w", err)
}
if out.Error != nil {
return "", fmt.Errorf("llm: %s", out.Error.Message)
}
if len(out.Choices) == 0 {
return "", fmt.Errorf("llm: empty response")
}
return out.Choices[0].Message.Content, nil
}
// SentimentTriage classifies one article; falls back to neutral on any error
// so sentiment never blocks the pipeline.
func (c *Client) SentimentTriage(ctx context.Context, model, title, body string) (string, float64) {
if !c.Available() {
return "neutral", 0.5
}
text, err := c.Complete(ctx, model,
`Classify Indonesian stock news as bullish, bearish, or neutral. Reply with exactly: <label> <confidence 0-1>. No other text.`,
"Title: "+title+"\nBody: "+head(body, 1500), 20)
if err != nil {
return "neutral", 0.5
}
parts := strings.Fields(strings.ToLower(text))
if len(parts) == 0 {
return "neutral", 0.5
}
label := parts[0]
if label != "bullish" && label != "bearish" {
label = "neutral"
}
var conf float64 = 0.6
if len(parts) > 1 {
fmt.Sscanf(parts[1], "%f", &conf)
}
if conf < 0 || conf > 1 {
conf = 0.6
}
return label, conf
}
func head(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n]
}
backend/internal/model/types.go
+74
View File
@@ -0,0 +1,74 @@
// Package model holds shared domain types: citations, snapshots, agent
// results. Every number in user-visible output carries a Citation.
// API response shapes live in internal/sectors; agent inputs in internal/agents.
package model
import "time"
// Citation pins a value to its source endpoint + snapshot timestamp.
// Stale (>1 session old) citations are rendered marked, never hidden.
type Citation struct {
Endpoint string `json:"endpoint"`
SnapshotAt string `json:"snapshot_at"`
Ticker string `json:"ticker,omitempty"`
Stale bool `json:"stale,omitempty"`
}
// Value is a cited number or string shown to users.
type Value struct {
Label string `json:"label"`
Display string `json:"display"`
Citations []Citation `json:"citations"`
}
// Snapshot is one raw Sectors payload row.
type Snapshot struct {
ID int64
Ticker string
Date string
Source string
Payload string
FetchedAt time.Time
}
// AgentResult is the common output contract for A1..A7.
type AgentResult struct {
Agent string `json:"agent"`
Score float64 `json:"score"`
Summary string `json:"summary"`
Values []Value `json:"values"`
Flags []string `json:"flags"`
Citations []Citation `json:"citations"`
Extra map[string]any `json:"extra,omitempty"`
}
// Cite builds a Citation for one endpoint + snapshot date.
func Cite(endpoint, ticker, snapshotAt string) Citation {
return Citation{Endpoint: endpoint, SnapshotAt: snapshotAt, Ticker: ticker}
}
// StaleSession returns the date before which snapshots count as stale:
// anything older than the session containing ref (WIB trading day).
func StaleSession(ref time.Time) time.Time {
y, m, d := ref.Date()
return time.Date(y, m, d, 0, 0, 0, 0, ref.Location()).AddDate(0, 0, -1)
}
// MarkStale flags citations older than cutoff (one session back).
func MarkStale(citations []Citation, cutoff time.Time) []Citation {
out := make([]Citation, len(citations))
for i, c := range citations {
if t, err := time.Parse("2006-01-02", head10(c.SnapshotAt)); err == nil && t.Before(cutoff) {
c.Stale = true
}
out[i] = c
}
return out
}
func head10(s string) string {
if len(s) > 10 {
return s[:10]
}
return s
}
backend/internal/reports/render.go
+91
View File
@@ -0,0 +1,91 @@
package reports
import (
"bytes"
"fmt"
"strings"
"time"
"github.com/jung-kurt/gofpdf"
)
// todayStr stamps report generation time (UTC, date precision is enough).
func todayStr() string { return time.Now().UTC().Format("2006-01-02 15:04") }
// ToMarkdown renders the report as Markdown.
func (r Report) ToMarkdown() string {
var b strings.Builder
fmt.Fprintf(&b, "# %s — FlowSight Report (%s)\n\n", r.Ticker, r.GeneratedAt)
for _, s := range r.Sections {
fmt.Fprintf(&b, "## %s\n\n%s\n\n", s.Name, s.Body)
if len(s.Citations) > 0 {
b.WriteString("Citations:\n")
for _, c := range s.Citations {
fmt.Fprintf(&b, "- %s %s @ %s\n", c.Endpoint, c.Ticker, c.SnapshotAt)
}
b.WriteString("\n")
}
}
return b.String()
}
// ToHTML renders the report as a standalone page.
func (r Report) ToHTML() string {
var b strings.Builder
b.WriteString(`<!doctype html><html><head><meta charset="utf-8"><title>`)
b.WriteString(r.Ticker + " — FlowSight Report</title></head><body>")
fmt.Fprintf(&b, "<h1>%s — FlowSight Report (%s)</h1>", r.Ticker, r.GeneratedAt)
for _, s := range r.Sections {
fmt.Fprintf(&b, "<h2>%s</h2><p>%s</p>", s.Name, s.Body)
if len(s.Citations) > 0 {
b.WriteString("<ul>")
for _, c := range s.Citations {
fmt.Fprintf(&b, "<li>%s %s @ %s</li>", c.Endpoint, c.Ticker, c.SnapshotAt)
}
b.WriteString("</ul>")
}
}
b.WriteString("</body></html>")
return b.String()
}
// ToPDF renders the report server-side (pure Go, no system deps).
func (r Report) ToPDF() ([]byte, error) {
pdf := gofpdf.New("P", "mm", "A4", "")
pdf.AddPage()
pdf.SetFont("Helvetica", "B", 16)
pdf.Cell(0, 10, r.Ticker+" — FlowSight Report")
pdf.Ln(12)
pdf.SetFont("Helvetica", "", 10)
pdf.Cell(0, 6, "Generated "+r.GeneratedAt)
pdf.Ln(8)
for _, s := range r.Sections {
pdf.SetFont("Helvetica", "B", 12)
pdf.Cell(0, 8, s.Name)
pdf.Ln(8)
pdf.SetFont("Helvetica", "", 10)
pdf.MultiCell(0, 5, latin1(s.Body), "", "", false)
if len(s.Citations) > 0 {
pdf.SetFont("Helvetica", "I", 8)
for _, c := range s.Citations {
pdf.MultiCell(0, 4, latin1(c.Endpoint+" "+c.Ticker+" @ "+c.SnapshotAt), "", "", false)
}
}
pdf.Ln(4)
}
var buf bytes.Buffer
if err := pdf.Output(&buf); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
// latin1 drops non-latin runes gofpdf core fonts cannot render.
func latin1(s string) string {
return strings.Map(func(r rune) rune {
if r > 255 {
return '?'
}
return r
}, s)
}
backend/internal/reports/report.go
+115
View File
@@ -0,0 +1,115 @@
// Package reports builds the 7-section One-Click Report (overview, valuation,
// institutional, earnings, risk, calendar, recommendation) with citations[]
// per section, and exports PDF/HTML/MD/JSON. Every section populates from
// live/seed data; nothing renders empty without saying which snapshot is
// missing.
package reports
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"flowsight/internal/agents"
"flowsight/internal/model"
"flowsight/internal/store"
)
// Section is one report section with its own citations.
type Section struct {
Name string `json:"name"`
Body string `json:"body"`
Citations []model.Citation `json:"citations"`
}
// Report is the full 7-section payload.
type Report struct {
Ticker string `json:"ticker"`
GeneratedAt string `json:"generated_at"`
Sections []Section `json:"sections"`
Synthesis agents.Synthesis `json:"synthesis"`
AgentScores []model.AgentResult `json:"agent_scores"`
AllCitations []model.Citation `json:"citations"`
}
// Builder assembles reports from agent runs over stored snapshots.
type Builder struct {
DB *store.DB
Deps agents.Deps
Risk RiskFn
}
// RiskFn computes the portfolio-risk lines reused in the risk section.
type RiskFn func(ticker string) (concentration, beta string)
// Build runs A1..A6 + A7 and assembles all 7 sections, returning the
// persisted report id for citation-scoped chat interrogation.
func (b *Builder) Build(ctx context.Context, ticker, profile string) (Report, int64, error) {
ticker = strings.ToUpper(ticker)
results := agents.RunAll(ctx, b.Deps, ticker)
synth := agents.Synthesize(ctx, b.Deps, ticker, agents.RiskProfile(profile), results)
byAgent := map[string]model.AgentResult{}
for _, r := range results {
byAgent[r.Agent] = r
}
sec := func(name, body string, cites []model.Citation) Section {
if strings.TrimSpace(body) == "" {
body = "no snapshots for this section yet"
}
return Section{Name: name, Body: body, Citations: cites}
}
sections := []Section{
sec("overview", overviewLine(ticker, byAgent), citesOf(byAgent, "sentiment", "technical")),
sec("valuation", byAgent["fundamental"].Summary, citesOf(byAgent, "fundamental")),
sec("institutional", institutionalLine(byAgent), citesOf(byAgent, "smart-money", "broker-intel")),
sec("earnings", byAgent["fundamental"].Summary+" | "+byAgent["catalyst"].Summary, citesOf(byAgent, "fundamental", "catalyst")),
sec("risk", riskLine(ticker, b.Risk), citesOf(byAgent, "technical")),
sec("calendar", byAgent["catalyst"].Summary, citesOf(byAgent, "catalyst")),
sec("recommendation", synth.Thesis, synth.Citations),
}
all := []model.Citation{}
for _, s := range sections {
all = append(all, s.Citations...)
}
all = model.MarkStale(all, model.StaleSession(time.Now()))
raw, _ := json.Marshal(map[string]any{"ticker": ticker, "sections": sections, "synthesis": synth})
citesRaw, _ := json.Marshal(all)
id, err := b.DB.SaveReport(ticker, string(raw), string(citesRaw))
if err != nil {
return Report{}, 0, err
}
return Report{
Ticker: ticker, GeneratedAt: todayStr(),
Sections: sections, Synthesis: synth,
AgentScores: results, AllCitations: all,
}, id, nil
}
func citesOf(byAgent map[string]model.AgentResult, names ...string) []model.Citation {
var out []model.Citation
for _, n := range names {
out = append(out, byAgent[n].Citations...)
}
return out
}
func overviewLine(ticker string, byAgent map[string]model.AgentResult) string {
return fmt.Sprintf("%s — sentiment %s; technical %s.",
ticker, byAgent["sentiment"].Summary, byAgent["technical"].Summary)
}
func institutionalLine(byAgent map[string]model.AgentResult) string {
return fmt.Sprintf("smart-money %s; broker-intel %s.",
byAgent["smart-money"].Summary, byAgent["broker-intel"].Summary)
}
func riskLine(ticker string, fn RiskFn) string {
if fn == nil {
return ticker + ": concentration/beta from /portfolio/risk (run screener first)."
}
c, b := fn(ticker)
return fmt.Sprintf("%s: %s; %s.", ticker, c, b)
}
backend/internal/routines/briefing.go
+580
View File
@@ -0,0 +1,580 @@
package routines
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"time"
"flowsight/internal/model"
)
// Briefing (R1, 07:30 WIB daily): top-5 accumulation, foreign net per
// watchlist ticker, earnings + ex-div agenda next 7d, biggest 1d mover.
// Zero empty sections: every section renders, citing its snapshots.
func (e *Engine) Briefing(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
var cites []model.Citation
var b strings.Builder
fmt.Fprintf(&b, "FlowSight Morning Briefing — %s\n\n", t)
type acc struct {
ticker string
net float64
n int
}
var accs []acc
for _, tk := range e.tickers() {
nets, err := e.DB.NetBuySum5d(tk)
if err != nil || len(nets) == 0 {
continue
}
sum, n := 0.0, 0
for _, v := range nets {
if v > 0 {
n++
sum += v
}
}
accs = append(accs, acc{tk, sum, n})
cites = append(cites, model.Cite("v2/broker-summary/"+tk+"/top/", tk, t))
}
sort.Slice(accs, func(i, j int) bool { return accs[i].net > accs[j].net })
b.WriteString("Top accumulation:\n")
if len(accs) == 0 {
b.WriteString("- no accumulation snapshots yet\n")
} else {
for i, a := range accs {
if i >= 5 {
break
}
fmt.Fprintf(&b, "%d. %s net %s (%d brokers)\n", i+1, a.ticker, fmtIDR(a.net), a.n)
}
}
b.WriteString("\nForeign flow (last close):\n")
for _, tk := range e.tickers() {
dates, nets, err := e.DB.ForeignLast6(tk)
if err != nil || len(nets) == 0 {
fmt.Fprintf(&b, "- %s: no data\n", tk)
continue
}
cites = append(cites, model.Cite("v2/foreign-flow/"+tk+"/", tk, dates[len(dates)-1]))
fmt.Fprintf(&b, "- %s: %s on %s\n", tk, fmtIDR(nets[len(nets)-1]), dates[len(dates)-1])
}
e.agenda7d(&b, &cites)
moverOfDay(&b, &cites, e.DB)
out := capLines(b.String(), 25)
payload := map[string]any{"date": t, "text": out}
raw, _ := json.Marshal(payload)
_ = e.DB.SaveBriefing(t, string(raw), mustJSON(cites))
return out, cites, nil
}
// capLines enforces the ≤25-line delivery cap (header + rows survive).
func capLines(s string, n int) string {
lines := strings.Split(strings.TrimRight(s, "\n"), "\n")
if len(lines) <= n {
return s
}
return strings.Join(lines[:n], "\n") + "\n… (capped at 25 lines)\n"
}
// agenda7d appends the earnings + ex-div agenda for the next 7 days from
// stored quarterly-dates and corporate-actions snapshots.
func (e *Engine) agenda7d(b *strings.Builder, cites *[]model.Citation) {
t := today()
b.WriteString("\nAgenda (7d):\n")
empty := true
for _, tk := range e.tickers() {
var qd []struct {
ReportDate string `json:"report_date"`
Date string `json:"date"`
}
if raw, d, err := e.DB.LatestSnapshot(tk, "quarterly-dates"); err == nil {
_ = json.Unmarshal([]byte(raw), &qd)
for _, q := range qd {
for _, rd := range []string{q.ReportDate, q.Date} {
if nd := nextAfter(rd, 90); nd != "" && withinDays(t, nd, 7) {
b.WriteString("- " + tk + " earnings ~" + nd + " (last " + rd[:10] + ")\n")
*cites = append(*cites, model.Cite("v2/company/get_quarterly_financial_dates/"+tk+"/", tk, d))
empty = false
}
}
}
}
var ca struct {
CorporateActions struct {
UpcomingDividend []map[string]any `json:"upcoming_dividend"`
} `json:"corporate_actions"`
UpcomingDividend []map[string]any `json:"upcoming_dividend"`
}
if raw, d, err := e.DB.LatestSnapshot(tk, "corporate-actions"); err == nil {
_ = json.Unmarshal([]byte(raw), &ca)
evs := append(ca.CorporateActions.UpcomingDividend, ca.UpcomingDividend...)
for _, ev := range evs {
if dt := firstStr(ev, "ex_date", "exDate", "date"); withinDays(t, dt, 7) {
b.WriteString("- " + tk + " ex-div " + dt[:10] + "\n")
*cites = append(*cites, model.Cite("v2/company/corporate-actions/"+tk+"/", tk, d))
empty = false
}
}
}
}
if empty {
b.WriteString("- no earnings/ex-div in the next 7d\n")
}
}
// moverOfDay appends the biggest 1d mover with a one-line news cause.
func moverOfDay(b *strings.Builder, cites *[]model.Citation, db interface {
LatestSnapshot(ticker, source string) (string, string, error)
NewsSince(ticker, since string) ([]map[string]any, error)
}) {
var mv struct {
TopGainers map[string][]struct {
Symbol string `json:"symbol"`
Name string `json:"name"`
Change float64 `json:"price_change"`
} `json:"top_gainers"`
}
raw, d, err := db.LatestSnapshot("IDX", "top-changes")
if err != nil {
b.WriteString("\nMover of the day: no snapshots yet\n")
return
}
_ = json.Unmarshal([]byte(raw), &mv)
best, bchg := "", 0.0
for _, rows := range mv.TopGainers {
for _, r := range rows {
if r.Change > bchg {
best, bchg = r.Symbol, r.Change
}
}
}
if best == "" {
b.WriteString("\nMover of the day: no gainers stored\n")
return
}
*cites = append(*cites, model.Cite("v2/companies/top-changes/", "IDX", d))
cause := "no linked headline"
tk := strings.ToUpper(strings.TrimSuffix(best, ".JK"))
if arts, err := db.NewsSince(tk, "2000-01-01"); err == nil && len(arts) > 0 {
if title, _ := arts[0]["title"].(string); title != "" {
cause = title
}
*cites = append(*cites, model.Cite("v2/news/", tk, d))
}
b.WriteString("\nMover of the day: " + best + " +" + pct1(bchg) + " — " + cause + "\n")
}
func nextAfter(date string, days int) string {
if len(date) < 10 {
return ""
}
t, err := time.Parse("2006-01-02", date[:10])
if err != nil {
return ""
}
return t.AddDate(0, 0, days).Format("2006-01-02")
}
func withinDays(today, date string, n int) bool {
if len(today) < 10 || len(date) < 10 {
return false
}
t0, e0 := time.Parse("2006-01-02", today[:10])
t1, e1 := time.Parse("2006-01-02", date[:10])
if e0 != nil || e1 != nil {
return false
}
d := int(t1.Sub(t0).Hours() / 24)
return d >= 0 && d <= n
}
func firstStr(ev map[string]any, keys ...string) string {
for _, k := range keys {
for ek, v := range ev {
if strings.EqualFold(ek, k) {
if s, ok := v.(string); ok && len(s) >= 10 {
return s
}
}
}
}
return ""
}
func pct1(v float64) string {
return strings.TrimRight(strings.TrimRight(fmt.Sprintf("%.1f", v*100), "0"), ".") + "%"
}
// Radar (R2): accumulation rule hits across the watchlist. First-fire
// only per (ticker, day): tickers with an accumulation event already
// recorded today are skipped.
func (e *Engine) Radar(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
var cites []model.Citation
var lines []string
fired := e.firedToday("accumulation", t)
for _, tk := range e.tickers() {
if fired[tk] {
continue
}
nets, err := e.DB.NetBuySum5d(tk)
if err != nil || len(nets) == 0 {
continue
}
n, sum := 0, 0.0
for _, v := range nets {
if v > 0 {
n++
sum += v
}
}
volMult := 0.0
if vols, _, err := e.DB.DailyVolumes(tk, 21); err == nil && len(vols) > 0 {
if n := len(vols); n >= 2 {
if a := avgF(vols[:n-1]); a > 0 {
volMult = vols[n-1] / a
}
}
}
if n >= 3 && volMult > 1.5 {
lines = append(lines, fmt.Sprintf("%s: %d brokers net %s, vol %.1fx — /report/%s",
tk, n, fmtIDR(sum), volMult, tk))
cites = append(cites, model.Cite("v2/broker-summary/"+tk+"/top/", tk, t))
}
}
if len(lines) == 0 {
return "radar: no accumulation signals this cycle", cites, nil
}
return "Accumulation Radar:\n- " + strings.Join(lines, "\n- "), cites, nil
}
// ReversalWatch (R3): 5d cumulative one way, last day the other way at
// >2x the trailing 5d daily average (both directions, per ROUTINES.md).
func (e *Engine) ReversalWatch(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
var cites []model.Citation
var lines []string
for _, tk := range e.tickers() {
dates, nets, err := e.DB.ForeignLast6(tk)
if err != nil || len(nets) < 6 {
continue
}
sum5, absAvg := 0.0, 0.0
for _, v := range nets[:5] {
sum5 += v
absAvg += absF(v)
}
absAvg /= 5
last := nets[5]
switch {
case absAvg > 0 && sum5 < 0 && last > 0 && last > 2*absAvg:
lines = append(lines, fmt.Sprintf("%s: first inflow %s after 5 selling days (%s)",
tk, fmtIDR(last), dates[5]))
cites = append(cites, model.Cite("v2/foreign-flow/"+tk+"/", tk, dates[5]))
case absAvg > 0 && sum5 > 0 && last < 0 && -last > 2*absAvg:
lines = append(lines, fmt.Sprintf("%s: outflow %s after 5 buying days (%s)",
tk, fmtIDR(-last), dates[5]))
cites = append(cites, model.Cite("v2/foreign-flow/"+tk+"/", tk, dates[5]))
}
}
if len(lines) == 0 {
return "reversal watch: no flips this cycle", cites, nil
}
return "Foreign Reversal Watch:\n- " + strings.Join(lines, "\n- "), cites, nil
}
// InsiderTape (R4): buy volume > 2x 30d avg, or >=3 distinct insiders in
// 7d (per ROUTINES.md rule 3).
func (e *Engine) InsiderTape(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
var cites []model.Citation
var lines []string
for _, tk := range e.tickers() {
avg30 := e.DB.FilingAvg30(tk)
last := e.DB.LatestBuyVolume(tk)
cluster := e.DB.DistinctInsiders7d(tk)
switch {
case avg30 > 0 && last > 2*avg30:
lines = append(lines, fmt.Sprintf("%s: insider buy %.0f vs 30d avg %.0f (%.1fx)",
tk, last, avg30, last/avg30))
cites = append(cites, model.Cite("v2/filings/", tk, t))
case cluster >= 3:
lines = append(lines, fmt.Sprintf("%s: %d distinct insiders buying in 7d", tk, cluster))
cites = append(cites, model.Cite("v2/filings/", tk, t))
}
}
if len(lines) == 0 {
return "insider tape: no filings in window", cites, nil
}
return "Insider Tape:\n- " + strings.Join(lines, "\n- "), cites, nil
}
// firedToday returns tickers with an event whose message contains rule
// already recorded today (first-fire guard).
func (e *Engine) firedToday(rule, today string) map[string]bool {
out := map[string]bool{}
evts, err := e.DB.AlertEventsSince(today, "", 100)
if err != nil {
return out
}
for _, ev := range evts {
msg, _ := ev["message"].(string)
if tk, _ := ev["ticker"].(string); ev["date"] == today && strings.Contains(msg, rule) {
out[tk] = true
}
}
return out
}
// EarningsCountdown (R5): fires at H-7/H-3/H-1 from stored quarterly dates,
// with the 8-quarter revenue/earnings mini-trend attached.
func (e *Engine) EarningsCountdown(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
var cites []model.Citation
var lines []string
for _, tk := range e.tickers() {
raw, d, err := e.DB.LatestSnapshot(tk, "quarterly-dates")
if err != nil {
continue
}
var qd []struct {
ReportDate string `json:"report_date"`
Date string `json:"date"`
}
_ = json.Unmarshal([]byte(raw), &qd)
for _, q := range qd {
for _, rd := range []string{q.ReportDate, q.Date} {
next := nextAfter(rd, 90)
if next == "" {
continue
}
h := daysUntil(t, next)
if h == 7 || h == 3 || h == 1 {
lines = append(lines, fmt.Sprintf("%s earnings H-%d (%s)%s", tk, h, next, e.trend8Q(tk)))
cites = append(cites, model.Cite("v2/company/get_quarterly_financial_dates/"+tk+"/", tk, d))
}
}
}
}
if len(lines) == 0 {
return "earnings countdown: nothing at H-7/H-3/H-1 today", cites, nil
}
return "Earnings Countdown:\n- " + strings.Join(lines, "\n- "), cites, nil
}
// trend8Q renders the 8-quarter revenue/earnings mini-trend for one ticker.
func (e *Engine) trend8Q(tk string) string {
raw, _, err := e.DB.LatestSnapshot(tk, "financials-quarterly")
if err != nil {
return ""
}
var rows []struct {
Revenue *float64 `json:"revenue"`
Earnings *float64 `json:"earnings"`
}
if json.Unmarshal([]byte(raw), &rows) != nil || len(rows) < 2 {
return ""
}
if len(rows) > 8 {
rows = rows[len(rows)-8:]
}
f := func(p *float64) float64 {
if p == nil {
return 0
}
return *p
}
rev, earn := "", ""
if a, b := f(rows[0].Revenue), f(rows[len(rows)-1].Revenue); a > 0 {
rev = fmt.Sprintf(" rev %+.0f%%", (b-a)/a*100)
}
if a, b := f(rows[0].Earnings), f(rows[len(rows)-1].Earnings); a != 0 {
earn = fmt.Sprintf(" earn %+.0f%%", (b-a)/absF(a)*100)
}
return " [" + strings.TrimSpace(rev+earn) + "]"
}
// DividendCalendar (R6): fires at H-14/H-3 with DPS, est. yield, payout flag.
func (e *Engine) DividendCalendar(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
var cites []model.Citation
var lines []string
for _, tk := range e.tickers() {
raw, d, err := e.DB.LatestSnapshot(tk, "corporate-actions")
if err != nil {
continue
}
var ca struct {
CorporateActions struct {
UpcomingDividend []map[string]any `json:"upcoming_dividend"`
} `json:"corporate_actions"`
UpcomingDividend []map[string]any `json:"upcoming_dividend"`
}
_ = json.Unmarshal([]byte(raw), &ca)
evs := append(ca.CorporateActions.UpcomingDividend, ca.UpcomingDividend...)
closePx, _, _ := e.DB.LatestClose(tk)
payout := e.payoutOf(tk)
for _, ev := range evs {
dt := firstStr(ev, "ex_date", "exDate", "date")
if dt == "" {
continue
}
h := daysUntil(t, dt[:10])
if h != 14 && h != 3 {
continue
}
dps := numOf(ev, "dividend_per_share", "dps", "cash_dividend")
y := ""
if dps > 0 && closePx > 0 {
y = fmt.Sprintf(" DPS %.0f yield %.1f%%", dps, dps/closePx*100)
}
flag := ""
if payout > 0.8 {
flag = " aggressive-payout"
}
lines = append(lines, fmt.Sprintf("%s ex-div H-%d (%s)%s%s", tk, h, dt[:10], y, flag))
cites = append(cites, model.Cite("v2/company/corporate-actions/"+tk+"/", tk, d))
}
}
if len(lines) == 0 {
return "dividend calendar: nothing at H-14/H-3 today", cites, nil
}
return "Dividend Calendar:\n- " + strings.Join(lines, "\n- "), cites, nil
}
// payoutOf reads the dividend payout ratio from the stored company report.
func (e *Engine) payoutOf(tk string) float64 {
raw, _, err := e.DB.LatestSnapshot(tk, "company-report")
if err != nil {
return 0
}
var rep map[string]any
_ = json.Unmarshal([]byte(raw), &rep)
p := numOf(rep, "payout_ratio", "dividend_payout")
if p > 1 {
p /= 100
}
return p
}
// numOf digs the first present numeric key (case-insensitive, nested).
func numOf(m map[string]any, keys ...string) float64 {
for _, k := range keys {
if v, ok := findNum(m, k); ok {
return v
}
}
return 0
}
func findNum(m map[string]any, key string) (float64, bool) {
for k, v := range m {
if strings.EqualFold(k, key) {
if f, ok := asFloat(v); ok {
return f, true
}
}
if sub, ok := v.(map[string]any); ok {
if f, ok := findNum(sub, key); ok {
return f, true
}
}
}
return 0, false
}
func asFloat(v any) (float64, bool) {
switch n := v.(type) {
case float64:
return n, true
case int:
return float64(n), true
case int64:
return float64(n), true
default:
return 0, false
}
}
// daysUntil returns whole days from today to date (negative if past).
func daysUntil(today, date string) int {
if len(today) < 10 || len(date) < 10 {
return -999
}
t0, e0 := time.Parse("2006-01-02", today[:10])
t1, e1 := time.Parse("2006-01-02", date[:10])
if e0 != nil || e1 != nil {
return -999
}
return int(t1.Sub(t0).Hours() / 24)
}
// WeekendReview (R7, Sat 09:00): week movers + drivers + open risks + agenda.
func (e *Engine) WeekendReview(ctx context.Context) (string, []model.Citation, error) {
_ = ctx
t := today()
text, cites, err := e.Briefing(ctx)
if err != nil {
return "", cites, err
}
out := "Weekend Review — week of " + t + "\n\n" + text
return out, cites, nil
}
func fmtIDR(v float64) string {
neg := v < 0
if neg {
v = -v
}
var s string
switch {
case v >= 1e12:
s = fmt.Sprintf("Rp%.2fT", v/1e12)
case v >= 1e9:
s = fmt.Sprintf("Rp%.0fB", v/1e9)
case v >= 1e6:
s = fmt.Sprintf("Rp%.0fM", v/1e6)
default:
s = fmt.Sprintf("Rp%.0f", v)
}
if neg {
return "-" + s
}
return s
}
func avgF(xs []float64) float64 {
if len(xs) == 0 {
return 0
}
s := 0.0
for _, x := range xs {
s += x
}
return s / float64(len(xs))
}
func mustJSON(v any) string {
b, _ := json.Marshal(v)
return string(b)
}
func absF(v float64) float64 {
if v < 0 {
return -v
}
return v
}
backend/internal/routines/engine.go
+161
View File
@@ -0,0 +1,161 @@
// Package routines turns agent output into scheduled deliveries (R1..R7).
// Each run records a routine_runs row; each delivery carries citations.
// Routines read snapshots, never live-fetch inside delivery.
package routines
import (
"context"
"encoding/json"
"fmt"
"time"
"flowsight/internal/alerts"
"flowsight/internal/model"
"flowsight/internal/store"
)
// Routine types R1..R7.
const (
RBriefing = "morning-briefing"
RRadar = "accumulation-radar"
RReversal = "foreign-reversal"
RInsider = "insider-tape"
REarnings = "earnings-countdown"
RDividend = "dividend-calendar"
RWeekend = "weekend-review"
)
// KnownType reports whether typ is one of R1..R7.
func KnownType(typ string) bool {
switch typ {
case RBriefing, RRadar, RReversal, RInsider, REarnings, RDividend, RWeekend:
return true
}
return false
}
// DefaultSchedule returns the spec cron for a routine type (WIB).
func DefaultSchedule(typ string) string {
switch typ {
case RBriefing:
return "30 7 * * *" // 07:30 daily
case RRadar, RReversal, RInsider:
return "*/30 9-16 * * 1-5" // 30-min market hours
case REarnings, RDividend:
return "0 8 * * *" // 08:00 daily
case RWeekend:
return "0 9 * * 6" // Sat 09:00
default:
return "0 8 * * *"
}
}
// Engine runs routines on demand (called by the scheduler) and by hand.
type Engine struct {
DB *store.DB
Notifier *alerts.Notifier
UserKey string
// Publish, when set, receives ("activity", text) per completed run so
// the dashboard SSE feed shows routine activity live.
Publish func(channel, data string)
}
// Run executes one routine row and records the run.
// Prefer RunFor: it scopes the run to the row owner (watchlist + delivery).
func (e *Engine) Run(ctx context.Context, r store.Routine) (string, error) {
return e.RunFor(ctx, r.UserKey, r)
}
// RunFor executes one routine row owned by owner: tickers come from the
// owner's watchlist and delivery fans out to the owner's destinations.
func (e *Engine) RunFor(ctx context.Context, owner string, r store.Routine) (string, error) {
// Scope this run to the owner without mutating shared engine state.
scoped := *e
if owner != "" {
scoped.UserKey = owner
}
se := &scoped
var (
payload string
cites []model.Citation
err error
)
switch r.Type {
case RBriefing:
payload, cites, err = se.Briefing(ctx)
case RRadar:
payload, cites, err = se.Radar(ctx)
case RReversal:
payload, cites, err = se.ReversalWatch(ctx)
case RInsider:
payload, cites, err = se.InsiderTape(ctx)
case REarnings:
payload, cites, err = se.EarningsCountdown(ctx)
case RDividend:
payload, cites, err = se.DividendCalendar(ctx)
case RWeekend:
payload, cites, err = se.WeekendReview(ctx)
default:
err = fmt.Errorf("routines: unknown type %q", r.Type)
}
status := "ok"
if err != nil {
status = "error"
payload = err.Error()
}
cc, _ := json.Marshal(cites)
if _, rerr := se.DB.RecordRun(r.ID, status, summarize(payload, cc), 0, scoped.UserKey); rerr != nil && err == nil {
err = rerr
}
if err == nil && status == "ok" {
_ = se.deliver(ctx, r, payload, cites)
}
if e.Publish != nil {
e.Publish("activity", r.Type+" "+status+": "+head(payload, 160))
}
return payload, err
}
func head(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n]
}
func summarize(payload string, cites []byte) string {
return `{"payload":` + strconv(payload) + `,"citations":` + string(cites) + `}`
}
func strconv(s string) string {
b, _ := json.Marshal(s)
return string(b)
}
// deliver pushes the payload to the routine's channels via the notifier,
// WITH citations (alert cards render context + citations + report link).
// Delivery is scoped to the routine owner: the owner's enabled destinations
// (managed at /alerts → Push destinations, i.e. POST /api/destinations)
// plus the server env fallback. With no targets anywhere it is record-only.
func (e *Engine) deliver(ctx context.Context, r store.Routine, payload string, cites []model.Citation) error {
if len(r.Channels) == 0 || e.Notifier.DryRunFor(e.UserKey) {
return nil
}
cc, _ := json.Marshal(cites)
f := alerts.Finding{Rule: r.Type, Ticker: "IDX", Message: payload, Citations: cites}
return e.Notifier.SendTo(ctx, e.UserKey, f, string(cc))
}
// tickers returns the user's watchlist (never empty: falls back to demo set).
func (e *Engine) tickers() []string {
wl, err := e.DB.Watchlist(e.UserKey)
if err == nil && len(wl) > 0 {
return wl
}
return []string{"BBCA"}
}
// todayStr is overridable in tests via Now.
var nowFn = time.Now
func today() string { return nowFn().Format("2006-01-02") }
backend/internal/routines/routines_test.go
+98
View File
@@ -0,0 +1,98 @@
package routines
import (
"context"
"strings"
"testing"
"flowsight/internal/alerts"
"flowsight/internal/store"
)
func seedEngine(t *testing.T) *Engine {
t.Helper()
db, err := store.Open(t.TempDir() + "/routines.db")
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { db.Close() })
if _, err := db.SeedFromDir("../../tests/fixtures", "demo"); err != nil {
t.Fatal(err)
}
return &Engine{DB: db, Notifier: alerts.NewNotifier("", "", ""), UserKey: "demo"}
}
// Briefing generates from seed with zero empty sections.
func TestBriefing(t *testing.T) {
e := seedEngine(t)
text, cites, err := e.Briefing(context.Background())
if err != nil {
t.Fatal(err)
}
for _, sec := range []string{"Top accumulation", "Foreign flow", "Agenda"} {
if !strings.Contains(text, sec) {
t.Fatalf("missing section %q", sec)
}
}
if len(cites) == 0 {
t.Fatal("briefing has no citations")
}
}
// Radar fires on the accumulation fixture.
func TestRadar(t *testing.T) {
e := seedEngine(t)
text, _, err := e.Radar(context.Background())
if err != nil {
t.Fatal(err)
}
if !strings.Contains(text, "BBCA") {
t.Fatalf("radar missed BBCA: %s", text)
}
}
// Accuracy resolution math: weights follow the ledger.
func TestAccuracyWeight(t *testing.T) {
e := seedEngine(t)
if w := e.DB.AccuracyWeight("smart-money"); w != 0.5 {
t.Fatalf("default weight = %v, want 0.5", w)
}
for i := 0; i < 10; i++ {
_ = e.DB.RecordPrediction("smart-money", "BBCA", "BUY")
}
rows, _ := e.DB.DueForResolution()
_ = rows
}
// Briefing carries agenda + mover sections and respects the 25-line cap.
func TestBriefingAgendaMoverCap(t *testing.T) {
e := seedEngine(t)
text, cites, err := e.Briefing(context.Background())
if err != nil {
t.Fatal(err)
}
for _, sec := range []string{"Top accumulation", "Foreign flow", "Agenda", "Mover of the day"} {
if !strings.Contains(text, sec) {
t.Fatalf("missing section %q:\n%s", sec, text)
}
}
if n := len(strings.Split(strings.TrimRight(text, "\n"), "\n")); n > 25 {
t.Fatalf("briefing = %d lines, want ≤25", n)
}
if len(cites) == 0 {
t.Fatal("briefing has no citations")
}
}
// R5/R6 run without error on seed (windows simply miss today).
func TestCountdownsRun(t *testing.T) {
e := seedEngine(t)
rows, _ := e.DB.ListRoutines("demo")
_ = rows
if _, _, err := e.EarningsCountdown(context.Background()); err != nil {
t.Fatal(err)
}
if _, _, err := e.DividendCalendar(context.Background()); err != nil {
t.Fatal(err)
}
}
backend/internal/scheduler/scheduler.go
+409
View File
@@ -0,0 +1,409 @@
// Package scheduler wires the 30-min ingestion cycle (09:00-16:00 WIB market
// hours) plus per-routine cron dispatch into one in-process robfig/cron.
// Order per cycle: reference cache -> universe sweep -> market context ->
// per-watchlist depth -> incremental events -> quarterly freshness ->
// rule evaluation -> routine dispatch. Credit spend aborts past the cap.
package scheduler
import (
"context"
"encoding/json"
"fmt"
"log"
"net/url"
"time"
"github.com/robfig/cron/v3"
"flowsight/internal/alerts"
"flowsight/internal/config"
"flowsight/internal/routines"
"flowsight/internal/sectors"
"flowsight/internal/store"
)
// Scheduler owns ingestion + routine dispatch.
type Scheduler struct {
Cfg config.Config
DB *store.DB
Cache *store.Cache
Sectors *sectors.Client
Notifier *alerts.Notifier
Engine *routines.Engine
// Publish, when set, feeds routine activity into the SSE hub.
Publish func(channel, data string)
cron *cron.Cron
lastRun time.Time
ok bool
}
// New wires dependencies; the sectors OnSpend hook persists credit_ledger rows.
func New(cfg config.Config, db *store.DB, cache *store.Cache, s *sectors.Client) *Scheduler {
sch := &Scheduler{Cfg: cfg, DB: db, Cache: cache, Sectors: s,
Notifier: alerts.NewNotifier(cfg.TelegramBotToken, cfg.TelegramChatID, cfg.DiscordWebhookURL),
}
sch.Notifier.Targets = func(owner string) []alerts.Target {
dests, err := db.ListEnabledDestinations(owner)
if err != nil {
return nil
}
var out []alerts.Target
for _, d := range dests {
switch d.Kind {
case store.DestTelegram:
if d.BotToken != "" && d.ChatID != "" {
out = append(out, alerts.Target{TelegramToken: d.BotToken, TelegramChatID: d.ChatID})
}
case store.DestDiscord:
if d.WebhookURL != "" {
out = append(out, alerts.Target{DiscordURL: d.WebhookURL})
}
}
}
return out
}
sch.Engine = &routines.Engine{DB: db, Notifier: sch.Notifier, UserKey: cfg.DemoUserKey}
sch.Engine.Publish = func(channel, data string) {
if sch.Publish != nil {
sch.Publish(channel, data)
}
}
s.OnSpend(func(endpoint string, calls, credits int) {
db.AddSpend(time.Now().Format("2006-01-02"), endpoint, calls, credits)
})
return sch
}
// Start registers ingestion + routine crons (WIB = UTC+7) and begins ticking.
func (s *Scheduler) Start() {
c := cron.New(cron.WithLocation(wib()))
// 30-min ingestion, weekdays 09:00-16:00 WIB.
_, _ = c.AddFunc("*/30 9-16 * * 1-5", func() { s.RunCycle(context.Background()) })
// Routines: briefing 07:30 daily, countdowns 08:00, weekend Sat 09:00,
// watchtower R2..R4 every 30 min in market hours.
_, _ = c.AddFunc("30 7 * * *", func() { s.runType(context.Background(), routines.RBriefing) })
_, _ = c.AddFunc("*/30 9-16 * * 1-5", func() {
s.runType(context.Background(), routines.RRadar)
s.runType(context.Background(), routines.RReversal)
s.runType(context.Background(), routines.RInsider)
})
_, _ = c.AddFunc("0 8 * * *", func() { s.runType(context.Background(), routines.REarnings) })
_, _ = c.AddFunc("5 8 * * *", func() { s.runType(context.Background(), routines.RDividend) })
// Accuracy resolution on its own daily cadence (was piggybacked on RunCycle).
_, _ = c.AddFunc("15 6 * * *", func() {
if n, err := s.DB.ResolveDue(); err != nil {
log.Printf("scheduler: resolve: %v", err)
} else if n > 0 {
log.Printf("scheduler: resolved %d predictions", n)
}
})
// Snapshot retention: weekly prune of rows older than 180d (keeps newest
// per ticker/source for seeds and last-good fallback).
_, _ = c.AddFunc("30 6 * * 0", func() {
cutoff := time.Now().AddDate(0, 0, -180).Format("2006-01-02")
if n, err := s.DB.PruneSnapshots(cutoff); err != nil {
log.Printf("scheduler: prune: %v", err)
} else if n > 0 {
log.Printf("scheduler: pruned %d snapshots older than %s", n, cutoff)
}
})
_, _ = c.AddFunc("0 9 * * 6", func() { s.runType(context.Background(), routines.RWeekend) })
s.cron = c
c.Start()
s.ok = true
}
// Stop halts all crons.
func (s *Scheduler) Stop() {
if s.cron != nil {
s.cron.Stop()
}
s.ok = false
}
// Status reports scheduler health for /api/health.
func (s *Scheduler) Status() (lastCycle string, ok bool) {
if s.lastRun.IsZero() {
return "", s.ok
}
return s.lastRun.UTC().Format(time.RFC3339), s.ok
}
func wib() *time.Location {
return time.FixedZone("WIB", 7*3600)
}
// RunCycle executes one full ingestion cycle. Offline (no key) it returns
// early after marking scheduler health — seed data keeps the demo alive.
func (s *Scheduler) RunCycle(ctx context.Context) error {
if !s.Cfg.HasSectorsKey() {
s.lastRun = time.Now()
return nil
}
today := time.Now().Format("2006-01-02")
before := s.DB.CreditsToday(today)
guard := func() error {
if spent := s.DB.CreditsToday(today) - before; spent > s.Cfg.CreditCapPerCycle {
return fmt.Errorf("scheduler: credit cap %d exceeded (spent %d), aborting cycle",
s.Cfg.CreditCapPerCycle, spent)
}
return nil
}
if err := s.reference(ctx); err != nil {
log.Printf("scheduler: reference: %v", err)
}
if err := guard(); err != nil {
return err
}
if err := s.universe(ctx); err != nil {
log.Printf("scheduler: universe: %v", err)
}
if err := guard(); err != nil {
return err
}
if err := s.market(ctx); err != nil {
log.Printf("scheduler: market: %v", err)
}
if err := guard(); err != nil {
return err
}
for _, t := range s.watchlist() {
if err := s.tickerDepth(ctx, t); err != nil {
log.Printf("scheduler: depth %s: %v", t, err)
}
if err := guard(); err != nil {
return err
}
}
if err := s.events(ctx); err != nil {
log.Printf("scheduler: events: %v", err)
}
if err := s.freshness(ctx); err != nil {
log.Printf("scheduler: freshness: %v", err)
}
s.evaluate(ctx)
s.lastRun = time.Now()
return nil
}
func (s *Scheduler) watchlist() []string {
return s.watchlistFor(s.Cfg.DemoUserKey)
}
// watchlistFor resolves one owner's watchlist (demo seed fallback kept).
func (s *Scheduler) watchlistFor(owner string) []string {
wl, err := s.DB.Watchlist(owner)
if err == nil && len(wl) > 0 {
return wl
}
if owner == s.Cfg.DemoUserKey && len(s.Cfg.Watchlist) > 0 {
return s.Cfg.Watchlist
}
return []string{"BBCA"}
}
// reference refreshes registry/taxonomy caches (24h TTL, fallback last good).
func (s *Scheduler) reference(ctx context.Context) error {
if raw := s.Cache.Get(ctx, "registry"); raw != "" {
return nil // fresh enough; TTL governs refresh
}
reg, err := s.Sectors.BrokersRegistry(ctx, "", "")
if err != nil {
if raw, _, lerr := s.DB.LatestSnapshot("IDX", "brokers-registry"); lerr == nil && raw != "" {
var cached []sectors.BrokerRegistryRow
if jerr := json.Unmarshal([]byte(raw), &cached); jerr == nil && len(cached) > 0 {
s.Cache.SetJSON(ctx, "registry", cached, 24*time.Hour)
return nil
}
}
return err
}
s.Cache.SetJSON(ctx, "registry", reg, 24*time.Hour)
if raw, err := json.Marshal(reg); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "brokers-registry", string(raw))
}
for _, kind := range []string{"subsectors", "industries", "subindustries", "tags"} {
if rows, err := s.Sectors.Taxonomy(ctx, kind); err == nil {
s.Cache.SetJSON(ctx, "tax:"+kind, rows, 24*time.Hour)
}
}
return nil
}
// universe sweeps close/ pages for the latest trading day.
func (s *Scheduler) universe(ctx context.Context) error {
offset := 0
for page := 0; page < 12; page++ {
rows, total, err := s.Sectors.ClosePage(ctx, "", 30, offset)
if err != nil {
return err
}
if raw, err := json.Marshal(rows); err == nil && len(rows) > 0 {
_ = s.DB.SaveSnapshot("IDX", rows[0].Date, "close", string(raw))
}
offset += len(rows)
if offset >= total || len(rows) == 0 {
break
}
}
return nil
}
// market pulls top-changes (1 class x 2 periods), most-traded, idx-total,
// brokers/top — the cheap context block of the cycle.
func (s *Scheduler) market(ctx context.Context) error {
if g, l, err := s.Sectors.TopChanges(ctx,
[]string{"top_gainers"}, []string{"1d", "7d"}, "", 5); err == nil {
if raw, err := json.Marshal(map[string]any{"top_gainers": g, "top_losers": l}); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "top-changes", string(raw))
}
} else {
return err
}
if mt, err := s.Sectors.MostTraded(ctx, "", "", "", 5); err == nil {
if raw, err := json.Marshal(mt); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "most-traded", string(raw))
}
}
if it, err := s.Sectors.IdxTotal(ctx, "", ""); err == nil {
if raw, err := json.Marshal(it); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "idx-total", string(raw))
}
}
if bt, err := s.Sectors.BrokersTop(ctx, "", "net", 20, "", ""); err == nil {
if raw, err := json.Marshal(map[string]any{"results": bt}); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "brokers-top", string(raw))
}
}
return nil
}
// tickerDepth pulls broker-summary/top + foreign-flow + daily per ticker.
func (s *Scheduler) tickerDepth(ctx context.Context, ticker string) error {
end := time.Now().Format("2006-01-02")
start5 := time.Now().AddDate(0, 0, -6).Format("2006-01-02")
if top, err := s.Sectors.BrokerSummaryTop(ctx, ticker, start5, end, 10, "", ""); err == nil {
if raw, err := json.Marshal(top); err == nil {
_ = s.DB.SaveSnapshot(ticker, end, "broker-summary-top", string(raw))
for _, b := range top.TopBuyers {
_ = s.DB.UpsertBrokerActivity(b.BrokerCode, ticker, end,
float64(b.BuyIDR), float64(b.SellIDR), float64(b.NetIDR), 0, 0, 0)
}
for _, x := range top.TopSellers {
_ = s.DB.UpsertBrokerActivity(x.BrokerCode, ticker, end,
float64(x.BuyIDR), float64(x.SellIDR), float64(x.NetIDR), 0, 0, 0)
}
}
} else {
return err
}
if ff, err := s.Sectors.ForeignFlow(ctx, ticker,
time.Now().AddDate(0, 0, -7).Format("2006-01-02"), end); err == nil {
if raw, err := json.Marshal(ff); err == nil {
_ = s.DB.SaveSnapshot(ticker, end, "foreign-flow", string(raw))
for _, d := range ff.Data {
_ = s.DB.UpsertForeignFlow(ticker, d.Date, float64(d.NetForeignInflow))
}
}
}
if bars, err := s.Sectors.Daily(ctx, ticker,
time.Now().AddDate(0, 0, -30).Format("2006-01-02"), end); err == nil {
if raw, err := json.Marshal(bars); err == nil && len(bars) > 0 {
_ = s.DB.SaveSnapshot(ticker, end, "daily", string(raw))
}
}
return nil
}
// events polls news/filings/suspensions incrementally via meta cursors.
func (s *Scheduler) events(ctx context.Context) error {
today := time.Now().Format("2006-01-02")
newsSince := s.DB.GetMeta("news_since")
if news, err := s.Sectors.News(ctx, "", newsSince, today, "", 20); err == nil {
if raw, err := json.Marshal(map[string]any{"results": news}); err == nil {
_ = s.DB.SaveSnapshot("IDX", today, "news", string(raw))
}
s.DB.SetMeta("news_since", today)
}
filSince := s.DB.GetMeta("filings_since")
if fils, err := s.Sectors.Filings(ctx, "", "", "", filSince, today); err == nil {
if raw, err := json.Marshal(map[string]any{"results": fils}); err == nil {
_ = s.DB.SaveSnapshot("IDX", today, "filings", string(raw))
}
s.DB.SetMeta("filings_since", today)
}
if susp, err := s.Sectors.Suspensions(ctx, "", newsSince, today); err == nil {
if raw, err := json.Marshal(map[string]any{"results": susp}); err == nil {
_ = s.DB.SaveSnapshot("IDX", today, "suspensions", string(raw))
}
}
return nil
}
// freshness polls quarterly-dates with since= (never a full sweep).
func (s *Scheduler) freshness(ctx context.Context) error {
since := s.DB.GetMeta("quarterly_since")
rows, err := s.Sectors.QuarterlyDatesSince(ctx, since, 30, 0)
if err != nil {
return err
}
if len(rows) > 0 {
if raw, err := json.Marshal(rows); err == nil {
_ = s.DB.SaveSnapshot("IDX", time.Now().Format("2006-01-02"), "quarterly-dates", string(raw))
}
s.DB.SetMeta("quarterly_since", time.Now().Format("2006-01-02"))
}
return nil
}
// evaluate runs rule evaluation for every user that owns alerts, each
// against that owner's own watchlist and destinations.
func (s *Scheduler) evaluate(ctx context.Context) {
for _, owner := range s.alertOwners() {
alertsList, err := s.DB.ListAlerts(owner)
if err != nil {
continue
}
if len(alertsList) == 0 {
continue
}
wl := s.watchlistFor(owner)
for _, a := range alertsList {
alerts.EvaluateFor(ctx, s.DB, s.Notifier, a.ID, owner, a.Rule, wl)
}
}
}
// runType executes all enabled routines of one type, per owning user: each
// run reads that user's watchlist and pushes to that user's destinations.
func (s *Scheduler) runType(ctx context.Context, typ string) {
for _, owner := range s.routineOwners() {
rows, err := s.DB.ListRoutines(owner)
if err != nil {
continue
}
for _, r := range rows {
if r.Type == typ && r.Enabled {
s.Engine.RunFor(ctx, owner, r)
}
}
}
}
// alertOwners lists users owning at least one alert.
func (s *Scheduler) alertOwners() []string {
return s.DB.DistinctUsers("alerts")
}
// routineOwners lists users owning at least one enabled routine.
func (s *Scheduler) routineOwners() []string {
owners := s.DB.DistinctUsers("routines")
if len(owners) == 0 {
return []string{s.Cfg.DemoUserKey}
}
return owners
}
// TriggerCycle runs one cycle synchronously (used by tests and the
// ?force=1 health probe); q carries no Sectors params.
var _ = url.Values{}
backend/internal/sectors/client.go
+276
View File
@@ -0,0 +1,276 @@
// Package sectors wraps the Sectors API v2 with retry, credit counting, and
// parameter-narrowing guards. Every outbound Sectors call in FlowSight goes
// through SectorsClient — no raw HTTP to api.sectors.app elsewhere.
//
// Auth: `Authorization: <raw-key>` from env SECTORS_API_KEY. v1 is gone (410);
// only v2 paths are used.
package sectors
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"sync"
"time"
)
// ErrNoAPIKey is returned when no SECTORS_API_KEY is configured (offline mode).
var ErrNoAPIKey = errors.New("sectors: SECTORS_API_KEY not set (offline/seed mode)")
// ErrV1Gone is returned if a v1 path is ever requested (v1 returns 410).
var ErrV1Gone = errors.New("sectors: v1 API discontinued (410 Gone), use v2 only")
// Spend tracks credit usage per endpoint for one day.
type Spend struct {
Calls int
Credits int
}
// Client is a credit-counting Sectors v2 HTTP client.
type Client struct {
baseURL string
apiKey string
http *http.Client
mu sync.Mutex
day string
spend map[string]*Spend
onSpend func(endpoint string, calls, credits int)
}
// New builds a client. baseURL should end with a slash.
func New(baseURL, apiKey string) *Client {
if !strings.HasSuffix(baseURL, "/") {
baseURL += "/"
}
tr := &http.Transport{
MaxIdleConns: 32,
MaxIdleConnsPerHost: 8,
IdleConnTimeout: 90 * time.Second,
}
return &Client{
baseURL: baseURL,
apiKey: apiKey,
http: &http.Client{Transport: tr, Timeout: 30 * time.Second},
day: time.Now().Format("2006-01-02"),
spend: map[string]*Spend{},
}
}
// OnSpend registers a callback fired after every counted call (used by the
// scheduler to persist rows into credit_ledger).
func (c *Client) OnSpend(fn func(endpoint string, calls, credits int)) {
c.mu.Lock()
defer c.mu.Unlock()
c.onSpend = fn
}
// SpendToday returns a copy of today's per-endpoint spend.
func (c *Client) SpendToday() map[string]Spend {
c.mu.Lock()
defer c.mu.Unlock()
out := make(map[string]Spend, len(c.spend))
for k, v := range c.spend {
out[k] = *v
}
return out
}
// CreditsToday returns total credits spent today.
func (c *Client) CreditsToday() int {
total := 0
for _, s := range c.SpendToday() {
total += s.Credits
}
return total
}
func (c *Client) count(endpoint string, calls, credits int) {
c.mu.Lock()
today := time.Now().Format("2006-01-02")
if today != c.day {
c.day, c.spend = today, map[string]*Spend{}
}
s := c.spend[endpoint]
if s == nil {
s = &Spend{}
c.spend[endpoint] = s
}
s.Calls += calls
s.Credits += credits
fn := c.onSpend
c.mu.Unlock()
if fn != nil {
fn(endpoint, calls, credits)
}
}
// costFor implements the credit-cost table from docs/API-REFERENCE.md.
func costFor(path string, q url.Values) int {
p := strings.Trim(path, "/")
switch {
case p == "v2/companies/top-changes" || p == "companies/top-changes":
classes := splitCSV(q.Get("classifications"))
periods := splitCSV(q.Get("periods"))
if len(classes) == 0 {
classes = []string{"a", "b"} // default is expensive: 2 classes
}
if len(periods) == 0 {
periods = []string{"a", "b", "c", "d", "e"} // default: 5 periods
}
return len(classes) * len(periods)
case strings.HasPrefix(p, "v2/company/report") || strings.HasPrefix(p, "company/report"):
if s := splitCSV(q.Get("sections")); len(s) > 0 {
return len(s)
}
return 8 // default all sections
case strings.HasPrefix(p, "v2/subsector/report") || strings.HasPrefix(p, "subsector/report"):
if s := splitCSV(q.Get("sections")); len(s) > 0 {
return len(s)
}
return 6
case strings.HasPrefix(p, "v2/financials/quarterly") || strings.HasPrefix(p, "financials/quarterly"):
if n := atoi(q.Get("n_quarters"), 1); n > 0 {
return n
}
return 1
case p == "v2/most-traded" || p == "most-traded",
p == "v2/brokers/top" || p == "brokers/top",
strings.HasSuffix(p, "/top") && strings.Contains(p, "broker-activity"),
strings.HasSuffix(p, "/top") && strings.Contains(p, "broker-summary"):
return 2
default:
return 1
}
}
func splitCSV(s string) []string {
if s == "" {
return nil
}
var out []string
for _, p := range strings.Split(s, ",") {
if p = strings.TrimSpace(p); p != "" {
out = append(out, p)
}
}
return out
}
func atoi(s string, def int) int {
var n int
if _, err := fmt.Sscanf(s, "%d", &n); err != nil || n <= 0 {
return def
}
return n
}
// maxBrokerWindowDays and maxSeriesDays encode the API window limits.
const (
maxBrokerWindowDays = 14
maxSeriesDays = 90
)
// ClampBrokerWindow clamps a date range to the 14-day broker-endpoint limit.
func ClampBrokerWindow(start, end time.Time) (time.Time, time.Time) {
if end.Sub(start).Hours()/24 > maxBrokerWindowDays {
start = end.AddDate(0, 0, -maxBrokerWindowDays)
}
return start, end
}
// ClampSeriesWindow clamps a date range to the 90-day series limit.
func ClampSeriesWindow(start, end time.Time) (time.Time, time.Time) {
if end.Sub(start).Hours()/24 > maxSeriesDays {
start = end.AddDate(0, 0, -maxSeriesDays)
}
return start, end
}
func isoDate(t time.Time) string { return t.Format("2006-01-02") }
// Get performs a GET against a v2 path, enforcing auth, retry, and credit
// counting. Callers pass paths like "v2/brokers/top/" or "brokers/top/".
func (c *Client) Get(ctx context.Context, path string, q url.Values) ([]byte, error) {
if strings.TrimSpace(c.apiKey) == "" {
return nil, ErrNoAPIKey
}
if strings.Contains(path, "/v1/") || strings.HasPrefix(path, "v1/") {
return nil, ErrV1Gone
}
if e := q.Get("end"); e != "" {
if t, err := time.Parse("2006-01-02", e); err == nil && t.After(time.Now().Add(24*time.Hour)) {
return nil, fmt.Errorf("sectors: future end date %s rejected (API returns 400)", e)
}
}
rel := strings.TrimPrefix(strings.TrimPrefix(path, "/"), "v2/")
endpoint := "v2/" + strings.Trim(rel, "/") + "/"
credits := costFor(path, q)
full := c.baseURL + rel
if len(q) > 0 {
full += "?" + q.Encode()
}
var lastErr error
for attempt := 0; attempt < 3; attempt++ {
if attempt > 0 {
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(time.Duration(attempt*attempt) * 500 * time.Millisecond):
}
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, full, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", c.apiKey)
req.Header.Set("Accept", "application/json")
resp, err := c.http.Do(req)
if err != nil {
lastErr = err
continue
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
resp.Body.Close()
if err != nil {
lastErr = err
continue
}
switch {
case resp.StatusCode == http.StatusOK:
c.count(endpoint, 1, credits)
return body, nil
case resp.StatusCode == http.StatusGone:
return nil, ErrV1Gone
case resp.StatusCode == http.StatusTooManyRequests || resp.StatusCode >= 500:
lastErr = fmt.Errorf("sectors: %s -> HTTP %d (retrying)", endpoint, resp.StatusCode)
continue
default:
c.count(endpoint, 1, credits)
return nil, fmt.Errorf("sectors: %s -> HTTP %d: %s", endpoint, resp.StatusCode, truncate(string(body), 300))
}
}
c.count(endpoint, 1, credits)
return nil, fmt.Errorf("sectors: %s failed after retries: %w", endpoint, lastErr)
}
func truncate(s string, n int) string {
if len(s) <= n {
return s
}
return s[:n] + "…"
}
// EstimateCycleCredits returns the expected credit spend of one 30-min
// ingestion cycle for W watchlist tickers (docs/API-REFERENCE.md §budget).
func EstimateCycleCredits(w int) int {
return 10 + 2 + 2 + 1 + 2 + 3*w + 3 + 2 // ~= 22 + 3W
}
// MorningBriefingExtra returns (min, max) extra credits for the briefing run.
func MorningBriefingExtra() (int, int) { return 35, 40 }
backend/internal/sectors/client_test.go
+129
View File
@@ -0,0 +1,129 @@
package sectors
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
)
// One live-shape call against a mock upstream: narrowing is enforced and
// credits counted per the API-REFERENCE cost table.
func TestNarrowingAndCredits(t *testing.T) {
var gotQ url.Values
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotQ = r.URL.Query()
w.Write([]byte(`{"top_gainers":{"1d":[]},"top_losers":{"1d":[]}}`))
}))
defer srv.Close()
c := New(srv.URL+"/", "test-key")
_, _, err := c.TopChanges(context.Background(),
[]string{"top_gainers"}, []string{"1d", "7d"}, "", 5)
if err != nil {
t.Fatal(err)
}
if gotQ.Get("classifications") != "top_gainers" {
t.Fatalf("classifications = %q, want narrowed", gotQ.Get("classifications"))
}
if gotQ.Get("periods") != "1d,7d" {
t.Fatalf("periods = %q, want narrowed", gotQ.Get("periods"))
}
if got := c.CreditsToday(); got != 2 {
t.Fatalf("credits = %d, want 2 (1 class x 2 periods)", got)
}
}
// Per-cycle budget: EstimateCycleCredits(20) must stay <= 120 cap.
func TestCycleBudget(t *testing.T) {
if got := EstimateCycleCredits(20); got > 120 {
t.Fatalf("cycle estimate = %d, exceeds 120 cap", got)
}
if got := EstimateCycleCredits(5); got != 22+3*5 {
t.Fatalf("cycle estimate = %d, want %d", got, 22+3*5)
}
}
// v1 paths are rejected before any HTTP happens.
func TestV1Rejected(t *testing.T) {
c := New("https://api.sectors.app/v2/", "k")
if _, err := c.Get(context.Background(), "v1/companies/", url.Values{}); err != ErrV1Gone {
t.Fatalf("err = %v, want ErrV1Gone", err)
}
}
// Future end dates are rejected (API returns 400).
func TestFutureEndRejected(t *testing.T) {
c := New("https://api.sectors.app/v2/", "k")
q := url.Values{"end": {"2099-01-01"}}
if _, err := c.Get(context.Background(), "daily/BBCA/", q); err == nil {
t.Fatal("want future-end rejection")
}
}
// New wrappers hit exact paths with exact params; all cost 1 credit.
func TestCompanyDepthWrappers(t *testing.T) {
var gotPath, gotQ string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotPath, gotQ = r.URL.Path, r.URL.RawQuery
switch {
case strings.HasPrefix(r.URL.Path, "/company/get-segments/"):
w.Write([]byte(`{"symbol":"BBCA.JK","financial_year":2025,"revenue_breakdown":[{"value":1,"source":"Loans","target":"Interest Income"}]}`))
case strings.HasPrefix(r.URL.Path, "/company/get_quarterly_financial_dates/"):
w.Write([]byte(`{"2026":[["2026-03-31","q1"]]}`))
case strings.HasPrefix(r.URL.Path, "/listing-performance/"):
w.Write([]byte(`{"symbol":"BBCA.JK","listing_date":"2000-05-31"}`))
case strings.HasPrefix(r.URL.Path, "/index-daily/"):
w.Write([]byte(`[{"index_code":"ihsg","date":"2026-09-11","price":7800.5}]`))
case strings.HasPrefix(r.URL.Path, "/company/shareholders-composition/"):
w.Write([]byte(`{"symbol":"BBCA.JK","year":2026,"data":[]}`))
default:
w.Write([]byte(`{"BBCA.JK":{"financial_year":[2025]}}`))
}
}))
defer srv.Close()
c := New(srv.URL+"/", "test-key")
if _, err := c.Segments(context.Background(), "BBCA", 2025); err != nil {
t.Fatal(err)
}
if gotQ != "financial_year=2025" {
t.Fatalf("segments query = %q", gotQ)
}
qd, err := c.QuarterlyDates(context.Background(), "BBCA")
if err != nil || len(qd) != 1 || qd[0].ReportDate != "2026-03-31" {
t.Fatalf("quarterly dates = %v, %v", qd, err)
}
if _, err := c.ListingPerformance(context.Background(), "BBCA"); err != nil {
t.Fatal(err)
}
if _, err := c.IndexDaily(context.Background(), "ihsg", "2026-09-01", "2026-09-11"); err != nil {
t.Fatal(err)
}
if _, err := c.Shareholders(context.Background(), "BBCA", 2026); err != nil {
t.Fatal(err)
}
avail, err := c.SegmentAvailability(context.Background())
if err != nil || len(avail["BBCA.JK"]) != 1 {
t.Fatalf("availability = %v, %v", avail, err)
}
_ = gotPath
}
// Event windows clamp to 90d and report real costs.
func TestEventClamps(t *testing.T) {
var gotQ url.Values
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotQ = r.URL.Query()
w.Write([]byte(`{"results":[]}`))
}))
defer srv.Close()
c := New(srv.URL+"/", "test-key")
_, _ = c.News(context.Background(), "BBCA", "2026-01-01", "2026-09-11", "", 20)
if gotQ.Get("start") != "2026-06-13" {
t.Fatalf("news start = %q, want clamped to 90d", gotQ.Get("start"))
}
if got := c.CreditsToday(); got < 1 {
t.Fatalf("credits = %d, want counted", got)
}
}
backend/internal/sectors/endpoints.go
+719
View File
@@ -0,0 +1,719 @@
package sectors
import (
"context"
"encoding/json"
"fmt"
"net/url"
"strings"
"time"
)
// param helpers keep every wrapper narrow by default (credit traps in
// docs/API-REFERENCE.md): sections explicit, classifications+periods minimal,
// n_quarters bounded, free-float filters exclusive.
func dateRange(start, end string, clamp func(time.Time, time.Time) (time.Time, time.Time)) (string, string) {
if start == "" || end == "" {
e := time.Now()
s := e.AddDate(0, 0, -7)
if start == "" && end != "" {
if t, err := time.Parse("2006-01-02", end); err == nil {
e, s = t, t.AddDate(0, 0, -7)
}
}
return s.Format("2006-01-02"), e.Format("2006-01-02")
}
ts, err1 := time.Parse("2006-01-02", start)
te, err2 := time.Parse("2006-01-02", end)
if err1 != nil || err2 != nil {
return start, end
}
ts, te = clamp(ts, te)
return ts.Format("2006-01-02"), te.Format("2006-01-02")
}
func decode(data []byte, v any, endpoint string) error {
if err := json.Unmarshal(data, v); err != nil {
return fmt.Errorf("sectors: decode %s: %w", endpoint, err)
}
return nil
}
// --- Brokers: the moat ---
// BrokersRegistry returns the broker registry (cached daily by the scheduler).
func (c *Client) BrokersRegistry(ctx context.Context, cohort, origin string) ([]BrokerRegistryRow, error) {
q := url.Values{}
if cohort != "" && cohort != "all" {
q.Set("cohort", cohort)
}
if origin != "" && origin != "all" {
q.Set("origin", origin)
}
raw, err := c.Get(ctx, "brokers/", q)
if err != nil {
return nil, err
}
var rows []BrokerRegistryRow
if err := decode(raw, &rows, "v2/brokers/"); err != nil {
return nil, err
}
return rows, nil
}
// BrokersTop returns the daily broker ranking.
func (c *Client) BrokersTop(ctx context.Context, date, metric string, n int, origin, cohort string) ([]TopBrokerRow, error) {
q := url.Values{}
if date != "" {
q.Set("date", date)
}
if metric == "" {
metric = "net"
}
q.Set("metric", metric)
if n <= 0 || n > 90 {
n = 20
}
q.Set("n_brokers", fmt.Sprint(n))
if origin != "" {
q.Set("origin", origin)
}
if cohort != "" {
q.Set("cohort", cohort)
}
raw, err := c.Get(ctx, "brokers/top/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []TopBrokerRow `json:"results"`
}
if err := decode(raw, &resp, "v2/brokers/top/"); err != nil {
return nil, err
}
return resp.Results, nil
}
// BrokerActivity returns all (stock,day) rows for one broker (window ≤14d).
func (c *Client) BrokerActivity(ctx context.Context, brokerCode, symbol, start, end string) ([]BrokerRow, error) {
start, end = dateRange(start, end, ClampBrokerWindow)
q := url.Values{"start": {start}, "end": {end}}
if symbol != "" {
q.Set("symbol", strings.ToUpper(symbol))
}
raw, err := c.Get(ctx, "broker-activity/"+strings.ToUpper(brokerCode)+"/", q)
if err != nil {
return nil, err
}
var rows []BrokerRow
if err := decode(raw, &rows, "v2/broker-activity/{broker_code}/"); err != nil {
return nil, err
}
return rows, nil
}
// BrokerActivityTop returns top accumulations/distributions for one broker.
func (c *Client) BrokerActivityTop(ctx context.Context, brokerCode, start, end string, n int) (accum, distrib []BrokerRow, err error) {
start, end = dateRange(start, end, ClampBrokerWindow)
q := url.Values{"start": {start}, "end": {end}}
if n > 0 {
q.Set("n_brokers", fmt.Sprint(n))
}
raw, err := c.Get(ctx, "broker-activity/"+strings.ToUpper(brokerCode)+"/top/", q)
if err != nil {
return nil, nil, err
}
var resp struct {
TopAccumulations []BrokerRow `json:"top_accumulations"`
TopDistributions []BrokerRow `json:"top_distributions"`
TopAccumulation []BrokerRow `json:"top_accumulation"`
TopDistribution []BrokerRow `json:"top_distribution"`
Accumulations []BrokerRow `json:"accumulations"`
Distributions []BrokerRow `json:"distributions"`
Data []BrokerRow `json:"data"`
Results []BrokerRow `json:"results"`
}
if err := decode(raw, &resp, "v2/broker-activity/{code}/top/"); err != nil {
return nil, nil, err
}
accum = append(append(append(resp.TopAccumulations, resp.TopAccumulation...), resp.Accumulations...), resp.Data...)
distrib = append(append(resp.TopDistributions, resp.TopDistribution...), resp.Distributions...)
if len(accum) == 0 && len(resp.Results) > 0 {
accum = resp.Results
}
return accum, distrib, nil
}
// BrokerSummary returns per-broker daily rows for one ticker (window ≤14d).
func (c *Client) BrokerSummary(ctx context.Context, symbol, brokerCode, start, end string) ([]BrokerRow, error) {
start, end = dateRange(start, end, ClampBrokerWindow)
q := url.Values{"start": {start}, "end": {end}}
if brokerCode != "" {
q.Set("broker_code", strings.ToUpper(brokerCode))
}
raw, err := c.Get(ctx, "broker-summary/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
// Shape varies (rows array or {dates: {broker: rows}}); normalize both.
var rows []BrokerRow
if err := json.Unmarshal(raw, &rows); err == nil {
return rows, nil
}
var byDate map[string]map[string][]BrokerRow
if err := json.Unmarshal(raw, &byDate); err == nil {
for _, brokers := range byDate {
for _, r := range brokers {
rows = append(rows, r...)
}
}
return rows, nil
}
var wrapped struct {
Data []BrokerRow `json:"data"`
Results []BrokerRow `json:"results"`
}
if err := decode(raw, &wrapped, "v2/broker-summary/{symbol}/"); err != nil {
return nil, err
}
return append(wrapped.Data, wrapped.Results...), nil
}
// BrokerSummaryTop returns top buyers/sellers for one ticker (2 credits).
func (c *Client) BrokerSummaryTop(ctx context.Context, symbol, start, end string, n int, origin, cohort string) (*BrokerSummaryTop, error) {
start, end = dateRange(start, end, ClampBrokerWindow)
q := url.Values{"start": {start}, "end": {end}}
if n > 0 {
q.Set("n_brokers", fmt.Sprint(n))
}
if origin != "" {
q.Set("origin", origin)
}
if cohort != "" {
q.Set("cohort", cohort)
}
raw, err := c.Get(ctx, "broker-summary/"+strings.ToUpper(symbol)+"/top/", q)
if err != nil {
return nil, err
}
var resp BrokerSummaryTop
if err := decode(raw, &resp, "v2/broker-summary/{symbol}/top/"); err != nil {
return nil, err
}
return &resp, nil
}
// ForeignFlow returns the daily net foreign inflow series (window ≤90d).
func (c *Client) ForeignFlow(ctx context.Context, symbol, start, end string) (*ForeignFlow, error) {
start, end = dateRange(start, end, ClampSeriesWindow)
q := url.Values{"start": {start}, "end": {end}}
raw, err := c.Get(ctx, "foreign-flow/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var resp ForeignFlow
if err := decode(raw, &resp, "v2/foreign-flow/{symbol}/"); err != nil {
return nil, err
}
return &resp, nil
}
// --- Company core ---
// CompanyReport fetches explicit sections only (1 credit/section).
func (c *Client) CompanyReport(ctx context.Context, symbol string, sections []string) (map[string]any, error) {
if len(sections) == 0 {
sections = []string{"overview", "valuation"}
}
q := url.Values{"sections": {strings.Join(sections, ",")}}
raw, err := c.Get(ctx, "company/report/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var out map[string]any
if err := decode(raw, &out, "v2/company/report/{symbol}/"); err != nil {
return nil, err
}
return out, nil
}
// Quarterly fetches up to n financial quarters (1 credit/quarter, n≤8).
func (c *Client) Quarterly(ctx context.Context, symbol string, n int) ([]QuarterRow, error) {
if n <= 0 {
n = 4
}
if n > 8 {
n = 8
}
q := url.Values{"n_quarters": {fmt.Sprint(n)}}
raw, err := c.Get(ctx, "financials/quarterly/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var rows []QuarterRow
if err := decode(raw, &rows, "v2/financials/quarterly/{symbol}/"); err != nil {
return nil, err
}
return rows, nil
}
// CorporateActions fetches splits/rights/warrants/AGM/dividends for one ticker.
func (c *Client) CorporateActions(ctx context.Context, symbol string) (*CorporateActions, error) {
raw, err := c.Get(ctx, "company/corporate-actions/"+strings.ToUpper(symbol)+"/", url.Values{})
if err != nil {
return nil, err
}
var resp struct {
Symbol string `json:"symbol"`
CorporateActions struct {
Dividend []DividendEvent `json:"dividend"`
UpcomingDividend []DividendEvent `json:"upcoming_dividend"`
AGM []DividendEvent `json:"agm"`
StockSplit []DividendEvent `json:"stock_split"`
} `json:"corporate_actions"`
}
if err := decode(raw, &resp, "v2/company/corporate-actions/{symbol}/"); err != nil {
return nil, err
}
return &CorporateActions{
Symbol: resp.Symbol,
Dividend: resp.CorporateActions.Dividend,
UpcomingDividend: resp.CorporateActions.UpcomingDividend,
AGM: resp.CorporateActions.AGM,
StockSplit: resp.CorporateActions.StockSplit,
}, nil
}
// --- Universe / market ---
// ClosePage fetches one page of the full-universe close sweep.
func (c *Client) ClosePage(ctx context.Context, date string, limit, offset int) ([]CloseRow, int, error) {
q := url.Values{}
if date != "" {
q.Set("date", date)
}
if limit <= 0 || limit > 30 {
limit = 30
}
q.Set("limit", fmt.Sprint(limit))
q.Set("offset", fmt.Sprint(offset))
raw, err := c.Get(ctx, "close/", q)
if err != nil {
return nil, 0, err
}
var resp struct {
Results []CloseRow `json:"results"`
Pagination struct {
Total int `json:"total"`
Count int `json:"count"`
} `json:"pagination"`
}
if err := decode(raw, &resp, "v2/close/"); err != nil {
return nil, 0, err
}
return resp.Results, resp.Pagination.Total, nil
}
// QuarterlyDatesSince polls newly-reported companies incrementally.
func (c *Client) QuarterlyDatesSince(ctx context.Context, since string, limit, offset int) ([]QuarterlyDateRow, error) {
q := url.Values{}
if since != "" {
q.Set("since", since)
}
if limit <= 0 || limit > 30 {
limit = 30
}
q.Set("limit", fmt.Sprint(limit))
q.Set("offset", fmt.Sprint(offset))
raw, err := c.Get(ctx, "companies/quarterly-financial-dates/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []QuarterlyDateRow `json:"results"`
Data []QuarterlyDateRow `json:"data"`
}
if err := decode(raw, &resp, "v2/companies/quarterly-financial-dates/"); err != nil {
return nil, err
}
return append(resp.Results, resp.Data...), nil
}
// Daily returns the price+volume+MCap series for one ticker (window ≤90d).
func (c *Client) Daily(ctx context.Context, symbol, start, end string) ([]DailyBar, error) {
start, end = dateRange(start, end, ClampSeriesWindow)
q := url.Values{"start": {start}, "end": {end}}
raw, err := c.Get(ctx, "daily/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var rows []DailyBar
if err := decode(raw, &rows, "v2/daily/{symbol}/"); err != nil {
return nil, err
}
return rows, nil
}
// IdxTotal returns the IHSG total-MCap trend (macro context).
func (c *Client) IdxTotal(ctx context.Context, start, end string) ([]IdxTotalPoint, error) {
start, end = dateRange(start, end, ClampSeriesWindow)
q := url.Values{"start": {start}, "end": {end}}
raw, err := c.Get(ctx, "idx-total/", q)
if err != nil {
return nil, err
}
var rows []IdxTotalPoint
if err := decode(raw, &rows, "v2/idx-total/"); err != nil {
return nil, err
}
return rows, nil
}
// TopChanges requests minimal class×period combos only (1 credit per combo).
func (c *Client) TopChanges(ctx context.Context, classifications, periods []string, subSector string, n int) (gainers, losers map[string][]MoverRow, err error) {
if len(classifications) == 0 {
classifications = []string{"top_gainers"}
}
if len(periods) == 0 {
periods = []string{"1d"}
}
q := url.Values{
"classifications": {strings.Join(classifications, ",")},
"periods": {strings.Join(periods, ",")},
}
if subSector != "" {
q.Set("sub_sector", subSector)
}
if n > 0 {
q.Set("n_stock", fmt.Sprint(n))
}
raw, err := c.Get(ctx, "companies/top-changes/", q)
if err != nil {
return nil, nil, err
}
var resp struct {
TopGainers map[string][]MoverRow `json:"top_gainers"`
TopLosers map[string][]MoverRow `json:"top_losers"`
}
if err := decode(raw, &resp, "v2/companies/top-changes/"); err != nil {
return nil, nil, err
}
return resp.TopGainers, resp.TopLosers, nil
}
// MostTraded returns relative volume leaders.
func (c *Client) MostTraded(ctx context.Context, start, end, subSector string, n int) ([]map[string]any, error) {
start, end = dateRange(start, end, ClampSeriesWindow)
q := url.Values{"start": {start}, "end": {end}}
if subSector != "" {
q.Set("sub_sector", subSector)
}
if n > 0 {
q.Set("n_stock", fmt.Sprint(n))
}
raw, err := c.Get(ctx, "most-traded/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []map[string]any `json:"results"`
Data []map[string]any `json:"data"`
}
if err := decode(raw, &resp, "v2/most-traded/"); err != nil {
return nil, err
}
return append(resp.Results, resp.Data...), nil
}
// --- Events (incremental) ---
// News fetches articles incrementally (extension idx/mining exclusive).
// Windows clamp to 90d like other series endpoints.
func (c *Client) News(ctx context.Context, symbols, start, end, keyword string, limit int) ([]NewsArticle, error) {
if start != "" || end != "" {
start, end = dateRange(start, end, ClampSeriesWindow)
}
q := url.Values{"extension": {"idx"}}
if symbols != "" {
q.Set("symbols", symbols)
}
if start != "" {
q.Set("start", start)
}
if end != "" {
q.Set("end", end)
}
if keyword != "" {
q.Set("keyword", keyword)
}
if limit > 0 {
q.Set("limit", fmt.Sprint(limit))
}
raw, err := c.Get(ctx, "news/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []NewsArticle `json:"results"`
}
if err := decode(raw, &resp, "v2/news/"); err != nil {
return nil, err
}
return resp.Results, nil
}
// Filings fetches insider/institution transactions incrementally.
// Windows clamp to 90d like other series endpoints.
func (c *Client) Filings(ctx context.Context, symbol, txnType, holderType, start, end string) ([]Filing, error) {
if start != "" || end != "" {
start, end = dateRange(start, end, ClampSeriesWindow)
}
q := url.Values{}
if symbol != "" {
q.Set("symbol", strings.ToUpper(symbol))
}
if txnType != "" {
q.Set("transaction_type", txnType)
}
if holderType != "" {
q.Set("holder_type", holderType)
}
if start != "" {
q.Set("start", start)
}
if end != "" {
q.Set("end", end)
}
raw, err := c.Get(ctx, "filings/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []Filing `json:"results"`
Data []Filing `json:"data"`
}
if err := decode(raw, &resp, "v2/filings/"); err != nil {
return nil, err
}
return append(resp.Results, resp.Data...), nil
}
// Suspensions fetches new IDX suspension notices.
// Windows clamp to 90d like other series endpoints.
func (c *Client) Suspensions(ctx context.Context, symbol, start, end string) ([]Suspension, error) {
if start != "" || end != "" {
start, end = dateRange(start, end, ClampSeriesWindow)
}
q := url.Values{}
if symbol != "" {
q.Set("symbol", strings.ToUpper(symbol))
}
if start != "" {
q.Set("start", start)
}
if end != "" {
q.Set("end", end)
}
raw, err := c.Get(ctx, "suspensions/", q)
if err != nil {
return nil, err
}
var resp struct {
Results []Suspension `json:"results"`
Data []Suspension `json:"data"`
}
if err := decode(raw, &resp, "v2/suspensions/"); err != nil {
return nil, err
}
return append(resp.Results, resp.Data...), nil
}
// --- Screener / taxonomy ---
// Screen runs the companies/ structured screener (q overrides all).
func (c *Client) Screen(ctx context.Context, where, q string, limit, offset int) ([]ScreenerRow, error) {
qq := url.Values{}
if q != "" {
qq.Set("q", q)
} else if where != "" {
qq.Set("where", where)
}
if limit <= 0 || limit > 200 {
limit = 50
}
qq.Set("limit", fmt.Sprint(limit))
qq.Set("offset", fmt.Sprint(offset))
raw, err := c.Get(ctx, "companies/", qq)
if err != nil {
return nil, err
}
var resp struct {
Results []ScreenerRow `json:"results"`
Data []ScreenerRow `json:"data"`
}
if err := decode(raw, &resp, "v2/companies/"); err != nil {
return nil, err
}
return append(resp.Results, resp.Data...), nil
}
// FreeFloat fetches one exclusive filter per request (API rejects mixed).
func (c *Client) FreeFloat(ctx context.Context, sector, subSector, industry, subIndustry string) ([]FreeFloatRow, error) {
q := url.Values{}
set := 0
for k, v := range map[string]string{"sector": sector, "sub_sector": subSector, "industry": industry, "sub_industry": subIndustry} {
if v != "" {
q.Set(k, v)
set++
}
}
if set != 1 {
return nil, fmt.Errorf("sectors: free-float needs exactly one filter (got %d)", set)
}
raw, err := c.Get(ctx, "free-float/", q)
if err != nil {
return nil, err
}
var rows []FreeFloatRow
if err := decode(raw, &rows, "v2/free-float/"); err != nil {
return nil, err
}
return rows, nil
}
// Taxonomy fetches one slug list (subsectors/industries/subindustries/tags).
func (c *Client) Taxonomy(ctx context.Context, kind string) ([]map[string]any, error) {
raw, err := c.Get(ctx, kind+"/", url.Values{})
if err != nil {
return nil, err
}
var rows []map[string]any
if err := decode(raw, &rows, "v2/"+kind+"/"); err != nil {
return nil, err
}
return rows, nil
}
// SubsectorReport fetches explicit sections only (1 credit/section).
func (c *Client) SubsectorReport(ctx context.Context, subSector string, sections []string) (map[string]any, error) {
if len(sections) == 0 {
sections = []string{"statistics"}
}
q := url.Values{"sections": {strings.Join(sections, ",")}}
raw, err := c.Get(ctx, "subsector/report/"+subSector+"/", q)
if err != nil {
return nil, err
}
var out map[string]any
if err := decode(raw, &out, "v2/subsector/report/{sub_sector}/"); err != nil {
return nil, err
}
return out, nil
}
// --- Company depth: segments, quarterly dates, IPO, holders, index ---
// Segments fetches the revenue breakdown (Sankey-ready) for one ticker.
func (c *Client) Segments(ctx context.Context, symbol string, year int) (*Segments, error) {
q := url.Values{}
if year >= 1900 && year <= 2026 {
q.Set("financial_year", fmt.Sprint(year))
}
raw, err := c.Get(ctx, "company/get-segments/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var out Segments
if err := decode(raw, &out, "v2/company/get-segments/{symbol}/"); err != nil {
return nil, err
}
return &out, nil
}
// QuarterlyDates returns valid [report_date, quarter] pairs for one ticker
// (use to resolve report_date before Quarterly calls).
func (c *Client) QuarterlyDates(ctx context.Context, symbol string) ([]QuarterlyDate, error) {
raw, err := c.Get(ctx, "company/get_quarterly_financial_dates/"+strings.ToUpper(symbol)+"/", url.Values{})
if err != nil {
return nil, err
}
// Shape: {"YYYY-MM-DD": [["report_date","quarter"], ...], ...} flattened.
var grouped map[string][][]string
if err := decode(raw, &grouped, "v2/company/get_quarterly_financial_dates/{symbol}/"); err != nil {
return nil, err
}
var out []QuarterlyDate
for _, pairs := range grouped {
for _, p := range pairs {
if len(p) >= 2 {
out = append(out, QuarterlyDate{ReportDate: p[0], Quarter: p[1]})
}
}
}
return out, nil
}
// ListingPerformance fetches IPO context (post-May-2005 listings only).
func (c *Client) ListingPerformance(ctx context.Context, symbol string) (*ListingPerformance, error) {
raw, err := c.Get(ctx, "listing-performance/"+strings.ToUpper(symbol)+"/", url.Values{})
if err != nil {
return nil, err
}
var out ListingPerformance
if err := decode(raw, &out, "v2/listing-performance/{symbol}/"); err != nil {
return nil, err
}
return &out, nil
}
// IndexDaily returns the benchmark series for beta/correlation (window ≤90d).
func (c *Client) IndexDaily(ctx context.Context, indexCode, start, end string) ([]IndexBar, error) {
start, end = dateRange(start, end, ClampSeriesWindow)
q := url.Values{"start": {start}, "end": {end}}
raw, err := c.Get(ctx, "index-daily/"+strings.ToLower(indexCode)+"/", q)
if err != nil {
return nil, err
}
var rows []IndexBar
if err := decode(raw, &rows, "v2/index-daily/{index_code}/"); err != nil {
return nil, err
}
return rows, nil
}
// Shareholders fetches the local-vs-foreign holder mix (data from 2021).
func (c *Client) Shareholders(ctx context.Context, symbol string, year int) (*ShareholdersComposition, error) {
q := url.Values{}
if year >= 2021 {
q.Set("year", fmt.Sprint(year))
}
raw, err := c.Get(ctx, "company/shareholders-composition/"+strings.ToUpper(symbol)+"/", q)
if err != nil {
return nil, err
}
var out ShareholdersComposition
if err := decode(raw, &out, "v2/company/shareholders-composition/{symbol}/"); err != nil {
return nil, err
}
return &out, nil
}
// SegmentAvailability checks which symbols have segment data (cache weekly).
func (c *Client) SegmentAvailability(ctx context.Context) (map[string][]int, error) {
raw, err := c.Get(ctx, "companies/list_companies_with_segments/", url.Values{})
if err != nil {
return nil, err
}
var out map[string]struct {
Years []int `json:"financial_year"`
}
if err := decode(raw, &out, "v2/companies/list_companies_with_segments/"); err != nil {
return nil, err
}
res := make(map[string][]int, len(out))
for k, v := range out {
res[k] = v.Years
}
return res, nil
}
backend/internal/sectors/types.go
+239
View File
@@ -0,0 +1,239 @@
// Package sectors response types mirror docs.sectors.app/schema.json shapes.
package sectors
// BrokerTopRow is one top buyer/seller entry (net/buy/sell in IDR).
type BrokerTopRow struct {
Rank int `json:"rank"`
BrokerCode string `json:"broker_code"`
NetIDR int64 `json:"net_idr"`
BuyIDR int64 `json:"buy_idr"`
SellIDR int64 `json:"sell_idr"`
}
// BrokerSummaryTop is GET broker-summary/{symbol}/top/ response.
type BrokerSummaryTop struct {
Symbol string `json:"symbol"`
Start string `json:"start"`
End string `json:"end"`
Origin string `json:"origin"`
Cohort string `json:"cohort"`
TopBuyers []BrokerTopRow `json:"top_buyers"`
TopSellers []BrokerTopRow `json:"top_sellers"`
}
// BrokerRow is one per-broker daily row (blot/bval buy, slot/sval sell, nlot/nval net).
type BrokerRow struct {
BrokerCode string `json:"broker_code"`
Symbol string `json:"symbol,omitempty"`
BFreq int `json:"bfreq"`
BLot int64 `json:"blot"`
BVal int64 `json:"bval"`
BAvg float64 `json:"bavg_per_share"`
SFreq int `json:"sfreq"`
SLot int64 `json:"slot"`
SVal int64 `json:"sval"`
SAvg float64 `json:"savg_per_share"`
NLot int64 `json:"nlot"`
NVal int64 `json:"nval"`
NAvg float64 `json:"navg_per_share"`
}
// BrokerRegistryRow classifies one broker code.
type BrokerRegistryRow struct {
Code string `json:"code"`
Name string `json:"name"`
IsForeign bool `json:"is_foreign"`
Cohort *string `json:"cohort"`
LicenseType *string `json:"license_type"`
}
// TopBrokerRow is one brokers/top/ ranking row.
type TopBrokerRow struct {
Rank int `json:"rank"`
BrokerCode string `json:"broker_code"`
Gross int64 `json:"gross"`
Net int64 `json:"net"`
}
// ForeignPoint is one day of net foreign inflow (IDR, signed).
type ForeignPoint struct {
Date string `json:"date"`
NetForeignInflow int64 `json:"net_foreign_inflow"`
}
// ForeignFlow is GET foreign-flow/{symbol}/ response.
type ForeignFlow struct {
Symbol string `json:"symbol"`
Start string `json:"start"`
End string `json:"end"`
Data []ForeignPoint `json:"data"`
}
// DailyBar is one daily OHLCV row.
type DailyBar struct {
Symbol string `json:"symbol"`
Date string `json:"date"`
Close int64 `json:"close"`
Open int64 `json:"open"`
High int64 `json:"high"`
Low int64 `json:"low"`
Volume int64 `json:"volume"`
MarketCap int64 `json:"market_cap"`
}
// MoverRow is one top-changes entry; PriceChange is decimal (0.05 = +5%).
type MoverRow struct {
Name string `json:"name"`
Symbol string `json:"symbol"`
PriceChange float64 `json:"price_change"`
LastClosePrice int64 `json:"last_close_price"`
LatestCloseDate string `json:"latest_close_date"`
}
// CloseRow is one close/ universe row.
type CloseRow struct {
Symbol string `json:"symbol"`
Date string `json:"date"`
Close int64 `json:"close"`
}
// NewsArticle is one normalized news item.
type NewsArticle struct {
Title string `json:"title"`
Body string `json:"body"`
Source string `json:"source"`
Timestamp string `json:"timestamp"`
Sector string `json:"sector"`
SubSector []string `json:"sub_sector"`
Tags []string `json:"tags"`
Symbols []string `json:"symbols"`
}
// Filing is one insider/institution transaction.
type Filing struct {
Title string `json:"title"`
Symbol string `json:"symbol"`
Timestamp string `json:"timestamp"`
TransactionType string `json:"transaction_type"`
HolderType string `json:"holder_type"`
HolderName string `json:"holder_name"`
AmountTransaction int64 `json:"amount_transaction"`
Price float64 `json:"price"`
TransactionValue float64 `json:"transaction_value"`
Source string `json:"source"`
}
// Suspension is one IDX suspension notice.
type Suspension struct {
Symbol string `json:"symbol"`
SuspensionDate string `json:"suspension_date"`
Reason string `json:"reason"`
PDFURL string `json:"pdf_url"`
}
// ScreenerRow is one companies/ result row.
type ScreenerRow struct {
Symbol string `json:"symbol"`
CompanyName string `json:"company_name"`
QueryValues map[string]any `json:"query_values"`
}
// QuarterRow is one financials/quarterly row.
type QuarterRow struct {
Symbol string `json:"symbol"`
Date string `json:"date"`
Revenue *float64 `json:"revenue"`
Earnings *float64 `json:"earnings"`
Assets *float64 `json:"total_assets"`
Equity *float64 `json:"total_equity"`
OpCash *float64 `json:"operating_cash_flow"`
Debt *float64 `json:"total_debt"`
}
// DividendEvent is one dividend corporate-action event (raw map: schema untyped).
type DividendEvent map[string]any
// CorporateActions groups events by type.
type CorporateActions struct {
Symbol string `json:"symbol"`
Dividend []DividendEvent `json:"dividend"`
UpcomingDividend []DividendEvent `json:"upcoming_dividend"`
AGM []DividendEvent `json:"agm"`
StockSplit []DividendEvent `json:"stock_split"`
}
// QuarterlyDateRow is one companies/quarterly-financial-dates entry.
type QuarterlyDateRow struct {
Symbol string `json:"symbol"`
Date string `json:"date"`
Year int `json:"year"`
}
// FreeFloatRow is one free-float/ entry (decimal, 0.45 = 45%).
type FreeFloatRow struct {
Symbol string `json:"symbol"`
CompanyName string `json:"company_name"`
FreeFloat float64 `json:"free_float"`
}
// IdxTotalPoint is one idx-total/ row (raw map: shape varies by date range).
type IdxTotalPoint map[string]any
// SegmentRow is one revenue-breakdown entry (Sankey-ready source/target).
type SegmentRow struct {
Value float64 `json:"value"`
Source string `json:"source"`
Target string `json:"target"`
}
// Segments is GET company/get-segments/{symbol}/ response.
type Segments struct {
Symbol string `json:"symbol"`
FinancialYear int `json:"financial_year"`
RevenueBreakdown []SegmentRow `json:"revenue_breakdown"`
}
// QuarterlyDate is one [report_date, quarter_label] pair from
// get_quarterly_financial_dates/{symbol}/.
type QuarterlyDate struct {
ReportDate string `json:"report_date"`
Quarter string `json:"quarter"`
}
// ListingPerformance is GET listing-performance/{symbol}/ (IPO context).
type ListingPerformance struct {
Symbol string `json:"symbol"`
CompanyName string `json:"company_name"`
ListingDate string `json:"listing_date"`
Chg7d *float64 `json:"chg_7d"`
Chg30d *float64 `json:"chg_30d"`
Chg90d *float64 `json:"chg_90d"`
Chg365d *float64 `json:"chg_365d"`
OfferingPrice *int64 `json:"offering_price"`
}
// IndexBar is one index-daily/{index_code}/ row.
type IndexBar struct {
IndexCode string `json:"index_code"`
Date string `json:"date"`
Price float64 `json:"price"`
}
// ShareholderRow is one shareholders-composition/{symbol}/ month row
// (local _l vs foreign _f holder mix across 9 categories).
type ShareholderRow struct {
Date string `json:"date"`
TotalL *int64 `json:"total_l"`
TotalF *int64 `json:"total_f"`
IndividualL *int64 `json:"individual_l"`
IndividualF *int64 `json:"individual_f"`
CorporateL *int64 `json:"corporate_l"`
CorporateF *int64 `json:"corporate_f"`
}
// ShareholdersComposition is GET company/shareholders-composition/{symbol}/.
type ShareholdersComposition struct {
Symbol string `json:"symbol"`
Year int `json:"year"`
Data []ShareholderRow `json:"data"`
}
backend/internal/store/accuracy.go
+89
View File
@@ -0,0 +1,89 @@
package store
import (
"sort"
)
// ResolveDue resolves predictions older than 30d against actual return:
// first stored close on/after predict_date vs latest stored close.
// BUY hits above +5%, AVOID hits below -5%, HOLD hits within ±10%.
// Returns the number of newly resolved rows.
func (db *DB) ResolveDue() (int, error) {
due, err := db.DueForResolution()
if err != nil {
return 0, err
}
resolved := 0
for _, p := range due {
id, _ := p["id"].(int64)
ticker, _ := p["ticker"].(string)
pred, _ := p["prediction"].(string)
since, _ := p["predict_date"].(string)
series, err := db.dailyCloses(ticker)
if err != nil || len(series) < 2 {
continue
}
base, ok := firstOnOrAfter(series, since)
if !ok {
continue
}
last := series[len(series)-1]
if base.close <= 0 {
continue
}
ret := (last.close - base.close) / base.close
hit := false
switch pred {
case "BUY":
hit = ret > 0.05
case "AVOID":
hit = ret < -0.05
default: // HOLD
hit = ret >= -0.10 && ret <= 0.10
}
if err := db.ResolvePrediction(id, hit, ret); err == nil {
resolved++
}
}
return resolved, nil
}
type closePoint struct {
date string
close float64
}
// dailyCloses parses every stored daily payload into a date-sorted series.
func (db *DB) dailyCloses(ticker string) ([]closePoint, error) {
rows, err := db.Query(`SELECT payload_json FROM snapshots WHERE ticker=? AND source='daily'`, ticker)
if err != nil {
return nil, err
}
defer rows.Close()
var out []closePoint
for rows.Next() {
var raw string
if err := rows.Scan(&raw); err != nil {
return nil, err
}
for _, b := range parseDailyBars(raw, "") {
if b.Date != "" && b.Close > 0 {
out = append(out, closePoint{b.Date, b.Close})
}
}
}
if err := rows.Err(); err != nil {
return nil, err
}
sort.Slice(out, func(i, j int) bool { return out[i].date < out[j].date })
return out, nil
}
func firstOnOrAfter(series []closePoint, date string) (closePoint, bool) {
for _, p := range series {
if p.date >= date {
return p, true
}
}
return closePoint{}, false
}
backend/internal/store/accuracy_test.go
+42
View File
@@ -0,0 +1,42 @@
package store
import (
"testing"
"time"
)
// ResolveDue: BUY older than 30d resolves against stored closes.
func TestResolveDue(t *testing.T) {
db, err := Open(t.TempDir() + "/acc.db")
if err != nil {
t.Fatal(err)
}
defer db.Close()
old := time.Now().AddDate(0, 0, -40).Format("2006-01-02")
if _, err := db.Exec(`INSERT INTO agent_accuracy(agent,ticker,prediction,predict_date)
VALUES('synthesizer','BBCA','BUY',?)`, old); err != nil {
t.Fatal(err)
}
// Rising closes: first bar 9000 -> last 9950 (+10.5%) => BUY hits.
payload := `[{"date":"` + old + `","close":9000,"volume":1},{"date":"` +
time.Now().Format("2006-01-02") + `","close":9950,"volume":1}]`
if err := db.SaveSnapshot("BBCA", time.Now().Format("2006-01-02"), "daily", payload); err != nil {
t.Fatal(err)
}
n, err := db.ResolveDue()
if err != nil {
t.Fatal(err)
}
if n != 1 {
t.Fatalf("resolved = %d, want 1", n)
}
stats, _ := db.AccuracyStats()
if len(stats) != 1 || stats[0]["hits"] != 1 {
t.Fatalf("stats = %v, want 1 hit", stats)
}
// Second run: nothing due.
n, _ = db.ResolveDue()
if n != 0 {
t.Fatalf("resolved = %d, want 0", n)
}
}

Some files were not shown because too many files have changed in this diff Show More