Scrape https://www.tiktok.com/embed/v2/{id} HTML and extract the direct
v16m.tiktokcdn.com MP4 from the <video data-testid=play-video> tag.
Works server-side (no auth) while main site/API are Cloudflare-blocked.
Verimplemented before tikwm/yt-dlp/Playwright fallbacks.
- Nix flake now installs scrape_media.py to $out/bin so CI deploys it with the binary
- run_playwright_scraper locates the script alongside the running binary (Nix store),
the Cargo manifest dir (dev), /home/code/scraper, or $SCRAPER_SCRIPT_DIR
- Fix var_os() Option match
- Add scrape_media.py using Playwright+Chromium to bypass Cloudflare/anti-bot
- Add run_playwright_scraper() Rust helper (spawns Python, probes venv)
- Add playwright_to_download_result() JSON->DownloadResult converter
- Instagram/Facebook: try downr.org, fall back to Playwright (returns real cdninstagram/fbcdn URLs)
- Twitter: scope scraper::Html/Selector parsing in a block so the future stays Send, then Playwright fallback
- Revert --impersonate chrome (unsupported on Linux) to --user-agent
- Install Chromium browsers to /usr/local/share/ms-playwright for all users
The current otakudesu site changed paths and card layout, breaking two APIs:
1. genre pages: singular /genre/{slug}/page/{n}/ 301-redirects to a dead
otakudesu.io placeholder → 0 items. Fix: plural /genres/{slug}/page/{n}/.
Also re-target parse_genre_anime_document from the retired .venz ul li /
.thumbz h2.jdlflm layout to the new .col-anime card layout
(.col-anime-title a, .col-anime-eps, .col-anime-rating, .col-anime-cover img).
2. latest: /latest-anime/ was removed (301 → otakudesu.io). The homepage IS
the latest-episodes feed (.venz ul li with .thumbz h2.jdlflm/.epz), so
fetch_latest_anime_page now uses base_url() for page 1 and /page/N/ for
later pages instead of the dead path.
The search parser targeted the index layout (.venz ul li, .thumbz h2.jdlflm,
img, .epz, genre-tag/status/rating classes) but otakudesu search results live
in <ul class=chivsrc><li> with <h2><a>Title</a></h2> plus .set label/value
rows. Search silently returned 0 items.
Rewrote to parse .chivsrc li, extracting title + url from h2 a and
Status/Rating/Genre from .set rows. No poster on the search page (empty).
- Add AlqEpisode type with episode, title, url, date, download_url
- Add parse_episode_list() to extract episodes from .eplister
- Add parse_episode_download() to extract download URL from episode page
- Add episodes field to AlqDetailData
- Fetch download URLs for latest 5 episodes in parallel
alqanime.net is behind Cloudflare challenge - direct fetch returns
challenge page and proxy relays are down. alqanime.si works with
direct fetch.
Also change repository to use fetch_with_proxy (direct+relay fallback)
instead of fetch_with_proxy_only (relay only).
- Change BASE_URL from https://alqanime.si to https://alqanime.net
- Remove BASE_DETAIL_URL (now same as BASE_URL)
- Remove detail_image_url method (no longer needed)
- Simplify detail() use case: remove dual-fetch logic that merged
results from both domains (now same URL)
- Fix CLAUDE.md stale comment
- Delete infrastructure/browser/ directory entirely (browser pool is gone)
- Remove browser pool initialization from bootstrap
- Remove fetch_via_browserless fallback from proxy_fetch
- perform_proxy_chain now only uses relay endpoints
- Clean up log messages, README, and compose env var
The proxy-bun service at https://git.imrnes.team/MythEclipse/proxy-bun
handles all proxy needs now.
- Restructure from Modular MVC to Clean Architecture with 4 strict layers
- Domain: entities (anime/komik/proxy), Repository traits, typed errors
- Application: use case classes with proper error propagation
- Infrastructure: repository impls, parsers, Redis cache, HTTP/scraping, browser
- Presentation: Axum handlers, DTOs, AppState, router, AppError+IntoResponse
- Migrate all parsers (otakudesu, alqanime, komik) to native infra implementations
- Replace once_cell::sync::Lazy/OnceCell with std::sync::LazyLock/OnceLock
- Remove 150+ old files in modules/ and shared/ directories
- Remove once_cell from Cargo.toml dependencies
- Fix test/debug binaries to use new import paths
- Zero new clippy warnings
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
asepharyana
asepharyana
