asepharyana
a41425a887
Deploy Scraper / build-and-deploy (push) Canceled after 0s
- YouTube MP3/MP4: replaced dead savetube.media + ytdlpyton.nvlgroup.my.id with yt-dlp subprocess
- Added find_ytdlp() + run_ytdlp_json() helpers with spawn_blocking for >64KB stdout
- Fixed all resp['key'] -> resp.get('key') panics in Twitter, Spotify, KrakenFiles, Danbooru
- Removed dead fetch_youtube_v2_mp3/mp4 + decrypt_savetube functions
- ScraperError::Http -> AppError::ScraperError -> HTTP 502 instead of 500
- All endpoints now return graceful 502 with error message instead of crashing 500
Tested with real URLs:
- YouTube MP3: 200, success, real audio URL extracted
- YouTube MP4: 200, success, 48 format URLs extracted
- TikTok: 200 (tikwm provider)
- MediaFire: 200 (success)
Scraper API
Backend service berbasis Axum untuk scraping, image proxy/cache, dan endpoint API inti.
Stack
- Rust + Axum
- SeaORM (MySQL)
- Redis (deadpool-redis)
- Utoipa + Swagger UI
Quick Start
cargo run
Server bind ke 0.0.0.0:${PORT} dengan default port 4091.
Required Environment Variables
DATABASE_URL=mysql://asephs:hunterz@localhost:3306/sosmed
JWT_SECRET=change-me
REDIS_URL=redis://localhost:6379
Optional yang sering dipakai:
RUST_LOG=info
MINIO_ENDPOINT=
MINIO_BUCKET_NAME=
MINIO_ACCESS_KEY=
MINIO_SECRET_KEY=
Useful Endpoints
GET /docs- Swagger UIGET /api-docs/openapi.json- OpenAPI JSONGET /api/anime2/*GET /api/komik/*POST /api/proxy/image-cachePOST /api/proxy/image-cache/audit
Notes
- Database akan dicek/dibuat saat startup jika
DATABASE_URLbertipe MySQL. - Service ini juga menginisialisasi browser pool dan scheduler saat boot.
License
MIT