Commit Graph
14 Commits
Author SHA1 Message Date
asepharyana 6bfffa5364 fix(anime): parse otakudesu search results from .chivsrc li
The search parser targeted the index layout (.venz ul li, .thumbz h2.jdlflm,
img, .epz, genre-tag/status/rating classes) but otakudesu search results live
in <ul class=chivsrc><li> with <h2><a>Title</a></h2> plus .set label/value
rows. Search silently returned 0 items.

Rewrote to parse .chivsrc li, extracting title + url from h2 a and
Status/Rating/Genre from .set rows. No poster on the search page (empty).
2026-08-31 20:22:39 +07:00
asepharyana 7a12f0e9c8 fix(cache): self-heal Redis connection + don't 500 on cache write failure
Deploy Scraper / build-and-deploy (push) Canceled after 0s
The scraper cached a single RedisCache (one multiplexed connection) in a
OnceCell forever. When that one connection broke (Redis restart, idle
timeout, network blip), every cache op failed with 'cache io: broken pipe',
and because Cache::get_or_set propagated the post-compute write error,
EVERY API (anime, anime2, komik) returned 500 until a process restart.

Fix:
- Build a fresh RedisCache from a freshly checked-out deadpool connection
  per call, so a broken connection self-heals without a process restart
  (deadpool recycles/drops dead connections and reconnects on checkout).
- Make Cache::get_or_set treat a cache-write failure as non-fatal: return
  the freshly computed value (cache is best-effort), so a transient Redis
  outage degrades to cache-less instead of 500.

Pre-existing clippy warnings (repositories/parsers) untouched — out of scope.
2026-08-31 20:11:04 +07:00
asepharyana 3d38008af3 build: use published mytheclipse crates for queue + tracing (1.21.2)
Deploy Scraper / build-and-deploy (push) Canceled after 0s
mytheclipse-queue and mytheclipse-tracing are now published on crates.io
(v1.21.2). Switch both from local path deps to version deps so the whole
mytheclipse family resolves from crates.io — no more path deps in the
dependency graph.
2026-08-30 20:02:01 +07:00
asepharyana e681bbbe51 build: use published mytheclipse crates (1.21.2) via version deps
Deploy Scraper / build-and-deploy (push) Canceled after 0s
The user published all mytheclipse crates to crates.io at 1.21.2
(including the mytheclipse-cache redis-0.32 fix). Switch the scraper to
version deps for the released crates (mytheclipse, -cache, -config,
-event) so it tracks the published releases instead of local path deps.
mytheclipse-queue + mytheclipse-tracing are not yet on crates.io, so they
remain path deps to the local workspace.
2026-08-30 19:45:05 +07:00
asepharyana 966d86ff67 refactor: migrate thread/async/queue/scheduler infra to mytheclipse
Deploy Scraper / build-and-deploy (push) Canceled after 0s
- bootstrap: TracingLayer from mytheclipse-tracing (composed with scraper
  env filter), RuntimeConfig::auto() thread logging, init job queue + cron.
- proxy_fetch: leader task via ::mytheclipse::spawn_io, gzip decompression
  offloaded to ::mytheclipse::compute (sized rayon pool, panic-isolated),
  bounded by new async fetch limiter (tokio Semaphore bridge).
- queue: new infrastructure/queue module over mytheclipse-queue
  (InMemoryQueue + WorkerPool + BackpressureEnforcer) for repair jobs.
- scheduler: new infrastructure/scheduler.rs using mytheclipse::cron for
  the daily 02:00 UTC image-cache cleanup.
- deps: add mytheclipse-queue + mytheclipse-tracing path deps; mytheclipse
  -> full feature; rayon 1.12; keep path deps for unpublished crates.
- tests: infra_round2 runtime smoke tests (compute panic isolation,
  spawn_io, backpressure admission, cron parse, queue roundtrip).
- Fix: local cache::mytheclipse bridge module shadowed the mytheclipse
  crate name; use leading :: at spawn_io/compute call sites.
2026-08-30 19:40:01 +07:00
asepharyana c7ffd29e79 refactor: migrate infra to mytheclipse crates (retry, cache, event, ratelimit, config)
Deploy Scraper / build-and-deploy (push) Canceled after 0s
Replace hand-rolled infrastructure with the custom mytheclipse library:

- retry (src/infrastructure/scraping/retry.rs): backoff crate -> mytheclipse
  RetryConfig + retry() with retry_all predicate; same call-site helpers.
- cache (src/infrastructure/cache): deadpool raw AsyncCommands ->
  mytheclipse_cache::RedisCache bridge (src/infrastructure/cache/mytheclipse.rs),
  typed JSON wrapper keeps the Cache<'a> API used by use cases.
- events (src/events/bus.rs): custom broadcast pub/sub ->
  mytheclipse_event InMemoryEventBus + TypedEventBus alias.
- ratelimit (src/presentation/middleware/ratelimit.rs): hand-rolled window ->
  mytheclipse::RateLimiter token bucket.
- config (src/config/mod.rs): config crate -> mytheclipse_config ConfigLoader,
  preserving env-only fallback (config files optional).

No route/API/Redis-key changes. mytheclipse-cache path-dep points at local
source (redis 0.32 aligned with deadpool). Removed backoff/config deps.
2026-08-30 18:53:00 +07:00
asepharyana 6ebff90d5b ci: deploy via Attic binary cache (attic.asepharyana.my.id) instead of ssh copy
Deploy Scraper / build-and-deploy (push) Canceled after 0s
Push build result to attic cache directly from runner, then VPS
substitutes it via nix-store --realise. ssh copy retained only as fallback.
2026-08-27 20:38:36 +07:00
asepharyana d1b24f8f85 ci: add standalone Nix build + deploy workflow; remove parent notify
Deploy Scraper / build-and-deploy (push) Canceled after 0s
Repo is now self-contained: flake.nix (cargo build) + deploy.yml
(nix build -> nix copy -> nix-env profile -> systemctl restart scraper).
No longer dispatches submodule-updated to asepharyana-hub.
2026-08-27 19:46:35 +07:00
asepharyana ad0d4f4992 chore: update dependabot-auto-merge.yml
Notify Parent Repo / dispatch (push) Canceled after 0s
2026-08-26 08:53:39 +07:00
asepharyana 7ff47a79bc chore: update dependabot.yml 2026-08-26 08:53:38 +07:00
asepharyana c709c84a4d feat: add dependabot auto-merge workflow 2026-08-25 20:32:55 +07:00
asepharyana 55cfd513e3 feat: update dependabot config 2026-08-25 18:16:42 +07:00
asepharyana 380ecfcd9b feat: update dependabot config 2026-08-25 18:12:56 +07:00
asepharyana a23698a6c4 feat: add dependabot config (npm auto-deps) 2026-08-25 18:08:40 +07:00