refactor: migrate infra to mytheclipse crates (retry, cache, event, ratelimit, config)
Deploy Scraper / build-and-deploy (push) Canceled after 0s

Replace hand-rolled infrastructure with the custom mytheclipse library:

- retry (src/infrastructure/scraping/retry.rs): backoff crate -> mytheclipse
  RetryConfig + retry() with retry_all predicate; same call-site helpers.
- cache (src/infrastructure/cache): deadpool raw AsyncCommands ->
  mytheclipse_cache::RedisCache bridge (src/infrastructure/cache/mytheclipse.rs),
  typed JSON wrapper keeps the Cache<'a> API used by use cases.
- events (src/events/bus.rs): custom broadcast pub/sub ->
  mytheclipse_event InMemoryEventBus + TypedEventBus alias.
- ratelimit (src/presentation/middleware/ratelimit.rs): hand-rolled window ->
  mytheclipse::RateLimiter token bucket.
- config (src/config/mod.rs): config crate -> mytheclipse_config ConfigLoader,
  preserving env-only fallback (config files optional).

No route/API/Redis-key changes. mytheclipse-cache path-dep points at local
source (redis 0.32 aligned with deadpool). Removed backoff/config deps.
This commit is contained in:
asepharyana
2026-08-30 18:53:00 +07:00
parent 6ebff90d5b
commit c7ffd29e79
14 changed files with 703 additions and 564 deletions
+19 -33
View File
@@ -1,4 +1,8 @@
//! Rate limiting middleware.
//! Rate limiting middleware — backed by `mytheclipse::RateLimiter`.
//!
//! The limiter is a token bucket from the mytheclipse core crate. The
//! middleware keeps the same axum shape (State<Arc<RateLimiter>>) and
//! `check()` semantics, but delegates token accounting to the library.
use axum::{
extract::Request,
@@ -7,42 +11,24 @@ use axum::{
response::{IntoResponse, Response},
Json,
};
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Instant;
use mytheclipse::RateLimiter;
use std::sync::Arc;
use crate::presentation::dto::common::ApiResponse;
/// Simple in-memory rate limiter.
pub struct RateLimiter {
max_requests: u64,
window_secs: u64,
counter: AtomicU64,
window_start: Mutex<Instant>,
/// Convenience alias so callers don't need the mytheclipse import.
pub type AppRateLimiter = RateLimiter;
/// Build a rate limiter (rate = requests/sec, burst = max burst capacity).
pub fn new_rate_limiter(rate_per_sec: f64, burst: u64) -> Arc<RateLimiter> {
Arc::new(RateLimiter::new(rate_per_sec, burst))
}
impl RateLimiter {
pub fn new(max_requests: u64, window_secs: u64) -> Arc<Self> {
Arc::new(Self {
max_requests,
window_secs,
counter: AtomicU64::new(0),
window_start: Mutex::new(Instant::now()),
})
}
pub fn check(&self) -> bool {
let Ok(mut window_guard) = self.window_start.lock() else {
return false;
};
let window = &mut *window_guard;
if window.elapsed().as_secs() >= self.window_secs {
*window = Instant::now();
self.counter.store(0, Ordering::SeqCst);
}
let count = self.counter.fetch_add(1, Ordering::SeqCst);
count < self.max_requests
}
/// Compatibility constructor matching the old (max_requests, window_secs) API.
/// Converts a fixed window into an equivalent token-bucket rate.
pub fn new_window_rate_limiter(max_requests: u64, window_secs: u64) -> Arc<RateLimiter> {
let rate_per_sec = max_requests as f64 / window_secs.max(1) as f64;
new_rate_limiter(rate_per_sec, max_requests)
}
pub async fn rate_limit_middleware(
@@ -50,7 +36,7 @@ pub async fn rate_limit_middleware(
request: Request,
next: Next,
) -> Response {
if state.check() {
if state.try_acquire().is_ok() {
next.run(request).await
} else {
(