refactor: migrate infra to mytheclipse crates (retry, cache, event, ratelimit, config)
Deploy Scraper / build-and-deploy (push) Canceled after 0s

Replace hand-rolled infrastructure with the custom mytheclipse library:

- retry (src/infrastructure/scraping/retry.rs): backoff crate -> mytheclipse
  RetryConfig + retry() with retry_all predicate; same call-site helpers.
- cache (src/infrastructure/cache): deadpool raw AsyncCommands ->
  mytheclipse_cache::RedisCache bridge (src/infrastructure/cache/mytheclipse.rs),
  typed JSON wrapper keeps the Cache<'a> API used by use cases.
- events (src/events/bus.rs): custom broadcast pub/sub ->
  mytheclipse_event InMemoryEventBus + TypedEventBus alias.
- ratelimit (src/presentation/middleware/ratelimit.rs): hand-rolled window ->
  mytheclipse::RateLimiter token bucket.
- config (src/config/mod.rs): config crate -> mytheclipse_config ConfigLoader,
  preserving env-only fallback (config files optional).

No route/API/Redis-key changes. mytheclipse-cache path-dep points at local
source (redis 0.32 aligned with deadpool). Removed backoff/config deps.
This commit is contained in:
asepharyana
2026-08-30 18:53:00 +07:00
parent 6ebff90d5b
commit c7ffd29e79
14 changed files with 703 additions and 564 deletions
+3 -7
View File
@@ -1,7 +1,6 @@
//! Otakudesu anime scraping repository.
use async_trait::async_trait;
use backoff::future::retry;
use tracing::{info, warn};
use crate::domain::entity::anime::{
@@ -13,7 +12,7 @@ use crate::domain::repository::ScrapingRepository;
use crate::infrastructure::repository::parsers::otakudesu_parser;
use crate::infrastructure::scraping::html_fetcher::fetch_html_with_retry;
use crate::infrastructure::scraping::proxy_fetch::fetch_with_proxy;
use crate::infrastructure::scraping::retry::{default_backoff, transient};
use crate::infrastructure::scraping::retry::{default_backoff, retry, retry_all};
const OTAKUDESU_BASE_URL: &str = "https://otakudesu.cloud";
@@ -208,14 +207,11 @@ impl OtakudesuRepository {
}
Err(e) => {
warn!("Failed to fetch URL: {}, error: {:?}", url_owned, e);
Err(transient(ScrapingError::Http(format!(
"Proxy fetch failed: {}",
e
))))
Err(ScrapingError::Http(format!("Proxy fetch failed: {}", e)))
}
}
};
retry(backoff, fetch_op)
retry(backoff, retry_all, fetch_op)
.await
.map_err(|e| ScrapingError::Http(e.to_string()))
}