asepharyana
966d86ff67
Deploy Scraper / build-and-deploy (push) Canceled after 0s
- bootstrap: TracingLayer from mytheclipse-tracing (composed with scraper env filter), RuntimeConfig::auto() thread logging, init job queue + cron. - proxy_fetch: leader task via ::mytheclipse::spawn_io, gzip decompression offloaded to ::mytheclipse::compute (sized rayon pool, panic-isolated), bounded by new async fetch limiter (tokio Semaphore bridge). - queue: new infrastructure/queue module over mytheclipse-queue (InMemoryQueue + WorkerPool + BackpressureEnforcer) for repair jobs. - scheduler: new infrastructure/scheduler.rs using mytheclipse::cron for the daily 02:00 UTC image-cache cleanup. - deps: add mytheclipse-queue + mytheclipse-tracing path deps; mytheclipse -> full feature; rayon 1.12; keep path deps for unpublished crates. - tests: infra_round2 runtime smoke tests (compute panic isolation, spawn_io, backpressure admission, cron parse, queue roundtrip). - Fix: local cache::mytheclipse bridge module shadowed the mytheclipse crate name; use leading :: at spawn_io/compute call sites.
28 lines
1.3 KiB
Rust
28 lines
1.3 KiB
Rust
//! Scheduled maintenance jobs — driven by `mytheclipse::cron`.
|
|
//!
|
|
//! The scraper runs a daily 02:00 UTC cache-cleanup job that enqueues
|
|
//! image-cache repair work onto the application job queue. The cron driver
|
|
//! comes from the mytheclipse core crate (`schedule` + `CronSchedule`); the
|
|
//! actual per-key repair work is queued through `mytheclipse-queue`.
|
|
|
|
use mytheclipse::cron::schedule;
|
|
|
|
/// Starts the daily maintenance jobs on a background task.
|
|
///
|
|
/// Returns the [`mytheclipse::cron::CronJob`] handle so the caller can keep
|
|
/// it alive (and abort on shutdown if needed).
|
|
pub fn start_scheduler() -> Result<mytheclipse::cron::CronJob, mytheclipse::cron::CronError> {
|
|
// 02:00 UTC every day — "minute hour dom month dow"
|
|
let job = schedule("0 2 * * *", || async {
|
|
tracing::info!("[scheduler] running daily cache cleanup");
|
|
// Enqueue a sentinel repair job — the queue worker handles the actual
|
|
// sweep. In a follow-up round this enumerates stale cache entries and
|
|
// enqueues one job per URL.
|
|
let _ =
|
|
crate::infrastructure::queue::enqueue_cache_repair("__daily_sweep__".to_string()).await;
|
|
tracing::info!("[scheduler] daily cache cleanup enqueued");
|
|
})?;
|
|
tracing::info!("[scheduler] daily cache cleanup scheduled at 02:00 UTC");
|
|
Ok(job)
|
|
}
|