refactor: migrate infra to mytheclipse crates (retry, cache, event, ratelimit, config)
Deploy Scraper / build-and-deploy (push) Canceled after 0s
Deploy Scraper / build-and-deploy (push) Canceled after 0s
Replace hand-rolled infrastructure with the custom mytheclipse library: - retry (src/infrastructure/scraping/retry.rs): backoff crate -> mytheclipse RetryConfig + retry() with retry_all predicate; same call-site helpers. - cache (src/infrastructure/cache): deadpool raw AsyncCommands -> mytheclipse_cache::RedisCache bridge (src/infrastructure/cache/mytheclipse.rs), typed JSON wrapper keeps the Cache<'a> API used by use cases. - events (src/events/bus.rs): custom broadcast pub/sub -> mytheclipse_event InMemoryEventBus + TypedEventBus alias. - ratelimit (src/presentation/middleware/ratelimit.rs): hand-rolled window -> mytheclipse::RateLimiter token bucket. - config (src/config/mod.rs): config crate -> mytheclipse_config ConfigLoader, preserving env-only fallback (config files optional). No route/API/Redis-key changes. mytheclipse-cache path-dep points at local source (redis 0.32 aligned with deadpool). Removed backoff/config deps.
This commit is contained in:
@@ -1,7 +1,6 @@
|
||||
//! Otakudesu anime scraping repository.
|
||||
|
||||
use async_trait::async_trait;
|
||||
use backoff::future::retry;
|
||||
use tracing::{info, warn};
|
||||
|
||||
use crate::domain::entity::anime::{
|
||||
@@ -13,7 +12,7 @@ use crate::domain::repository::ScrapingRepository;
|
||||
use crate::infrastructure::repository::parsers::otakudesu_parser;
|
||||
use crate::infrastructure::scraping::html_fetcher::fetch_html_with_retry;
|
||||
use crate::infrastructure::scraping::proxy_fetch::fetch_with_proxy;
|
||||
use crate::infrastructure::scraping::retry::{default_backoff, transient};
|
||||
use crate::infrastructure::scraping::retry::{default_backoff, retry, retry_all};
|
||||
|
||||
const OTAKUDESU_BASE_URL: &str = "https://otakudesu.cloud";
|
||||
|
||||
@@ -208,14 +207,11 @@ impl OtakudesuRepository {
|
||||
}
|
||||
Err(e) => {
|
||||
warn!("Failed to fetch URL: {}, error: {:?}", url_owned, e);
|
||||
Err(transient(ScrapingError::Http(format!(
|
||||
"Proxy fetch failed: {}",
|
||||
e
|
||||
))))
|
||||
Err(ScrapingError::Http(format!("Proxy fetch failed: {}", e)))
|
||||
}
|
||||
}
|
||||
};
|
||||
retry(backoff, fetch_op)
|
||||
retry(backoff, retry_all, fetch_op)
|
||||
.await
|
||||
.map_err(|e| ScrapingError::Http(e.to_string()))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user