feat: add corex-storage crate for unified storage abstraction

- Introduced corex-storage crate with support for local disk, S3-compatible, and Google Cloud Storage backends.
- Implemented StorageDriver trait for various storage backends.
- Added LocalFileStorage for local disk operations.
- Added S3Storage for S3-compatible object storage with multipart upload support.
- Added GcsStorage for Google Cloud Storage operations.
- Included error handling for storage operations.
- Added tests for each storage backend to ensure functionality.
- Created README.md for documentation and usage examples.
- Added Apache and MIT licenses for open-source compliance.
This commit is contained in:
asepharyana committed 2026-08-28 22:24:18 +07:00
1 parent 6a4b2b8f54
commit db4f277336
47 files changed

No files matched your search

+51
View File
@@ -0,0 +1,51 @@
//! # corex-storage
//!
//! A unified storage & file system abstraction so file handling doesn't get
//! locked to one physical storage location.
//!
//! - **Local disk** ([`local::LocalFileStorage`], feature `local`, default).
//! - **S3-compatible object storage** ([`s3::S3Storage`], feature `s3`) —
//! works with Amazon S3 and any S3-compatible endpoint, including MinIO
//! (pass a custom endpoint via [`s3::S3Storage::connect_with_endpoint`]).
//! - **Google Cloud Storage** ([`gcs::GcsStorage`], feature `gcs`).
//!
//! All operations are stream-based (an [`traits::ObjectStream`] is a boxed
//! [`tokio::io::AsyncRead`]), so uploading/downloading a huge object never
//! requires holding it entirely in memory.
//!
//! ## Example
//!
//! ```
//! use corex_storage::{LocalFileStorage, StorageDriver, bytes_stream, read_to_vec};
//! # #[tokio::main] async fn main() {
//! # let dir = tempfile::tempdir().unwrap();
//! let storage = LocalFileStorage::new(dir.path());
//! storage.put("hello.txt", bytes_stream(b"hi".to_vec())).await.unwrap();
//! let data = read_to_vec(storage.get("hello.txt").await.unwrap()).await.unwrap();
//! assert_eq!(data, b"hi");
//! # }
//! ```
pub mod traits;
#[cfg(feature = "local")]
pub mod local;
#[cfg(feature = "s3")]
pub mod s3;
#[cfg(feature = "gcs")]
pub mod gcs;
pub use traits::{
bytes_stream, read_to_vec, ObjectMeta, ObjectStream, StorageDriver, StorageError,
};
#[cfg(feature = "local")]
pub use local::LocalFileStorage;
#[cfg(feature = "s3")]
pub use s3::S3Storage;
#[cfg(feature = "gcs")]
pub use gcs::GcsStorage;