MythEclipse/asepharyana-hub
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor: update architecture and documentation for scraper service, remove deprecated services

Browse Source
This commit is contained in:
asepharyana
2026-07-21 12:54:57 +07:00
parent cdad9ae720
commit 5d6bdf6ce3
5 changed files with 47 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+8 -41
View File
@@ -100,22 +100,11 @@ graph TB
subgraph "orange Containers" subgraph "orange Containers"
TRAEFIK[Traefik :443] TRAEFIK[Traefik :443]
RUST_AUTH[rust-auth :3000]
ELYSIA[elysia-api :4092]
REACT[react-web :80]
SCRAPER[scraper-api :4091] SCRAPER[scraper-api :4091]
end end
TRAEFIK --> RUST_AUTH
TRAEFIK --> ELYSIA
TRAEFIK --> REACT
TRAEFIK --> SCRAPER TRAEFIK --> SCRAPER
RUST_AUTH -.->|Tailscale IP| PG
ELYSIA -.->|Tailscale IP| PG
RUST_AUTH -.->|Tailscale IP| REDIS
ELYSIA -.->|Tailscale IP| REDIS
style IMRNES fill:#3a7,color:#fff style IMRNES fill:#3a7,color:#fff
style ORANGE fill:#37a,color:#fff style ORANGE fill:#37a,color:#fff
style ARCH fill:#773,color:#fff style ARCH fill:#773,color:#fff
@@ -154,8 +143,8 @@ sequenceDiagram
App->>DB: sqlx/Drizzle query via Tailscale App->>DB: sqlx/Drizzle query via Tailscale
DB-->>App: Result set DB-->>App: Result set
else Cache lookup else Cache lookup
App->>Redis: GET/SET via Tailscale App->>Cache: GET/SET via Tailscale
Redis-->>App: Cached value Cache-->>App: Cached value
end end
App-->>Traefik: HTTP response App-->>Traefik: HTTP response
@@ -217,7 +206,7 @@ The `orange` VPS (Tailscale `100.96.248.86`) hosts all application containers:
### Selective Deployment ### Selective Deployment
The deploy workflow supports selective updates — if only `infra/compose/elysia.yml` changed, only `elysia-api` is pulled and recreated, avoiding disruption to other services. The deploy workflow supports selective updates — if only one compose file changed, only the corresponding service is pulled and recreated, avoiding disruption to other services.
```mermaid ```mermaid
graph TB graph TB
@@ -254,7 +243,7 @@ Each application lives in its own Git repository and is imported as a submodule
### Submodule Lifecycle ### Submodule Lifecycle
1. Developer pushes to a submodule (e.g., `apps/elysia`) 1. Developer pushes to a submodule (e.g., `apps/scraper`)
2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA 2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA
3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service 3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service
4. The compose manifest is updated and committed with the new SHA tag 4. The compose manifest is updated and committed with the new SHA tag
@@ -264,15 +253,12 @@ Each application lives in its own Git repository and is imported as a submodule
```bash ```bash
# Update a single submodule to latest # Update a single submodule to latest
cd apps/elysia cd apps/scraper
git checkout main git checkout main
git pull git pull
cd ../.. cd ../..
git add apps/elysia git add apps/scraper
git commit -m "chore(elysia): update submodule to latest" git commit -m "chore(scraper): update submodule to latest"
# Update all submodules
git submodule update --remote --merge
``` ```
## Service Mesh & Inter-Service Communication ## Service Mesh & Inter-Service Communication
@@ -287,35 +273,16 @@ graph LR
TRAEFIK[Traefik :443] TRAEFIK[Traefik :443]
subgraph "app-shared-net" subgraph "app-shared-net"
REACT[react-web<br/>:80]
ELYSIA[elysia-api<br/>:4092]
RUST_AUTH[rust-auth<br/>:3000]
SCRAPER[scraper-api<br/>:4091] SCRAPER[scraper-api<br/>:4091]
REDIS[redis<br/>:6379]
end end
end end
subgraph "Imrnes VPS"
PG[(PostgreSQL<br/>:5432)]
REDIS_IMR[Redis<br/>:6379]
end
WWW -->|HTTPS| TRAEFIK WWW -->|HTTPS| TRAEFIK
TRAEFIK --> REACT
TRAEFIK --> ELYSIA
TRAEFIK --> RUST_AUTH
TRAEFIK --> SCRAPER TRAEFIK --> SCRAPER
ELYSIA -->|Tailscale| PG
RUST_AUTH -->|Tailscale| PG
ELYSIA -->|internal| REDIS
RUST_AUTH -->|internal| REDIS
ELYSIA -->|Tailscale| REDIS_IMR
RUST_AUTH -->|Tailscale| REDIS_IMR
``` ```
## Observability ## Observability
- **Prometheus metrics**: Available on rust-auth via `axum-prometheus` - **Traefik access logs**: JSON format, logged at INFO level
- **Traefik access logs**: JSON format, logged at INFO level - **Traefik access logs**: JSON format, logged at INFO level
- **Dashboard**: Traefik dashboard at `traefik.asepharyana.my.id` (secured) - **Dashboard**: Traefik dashboard at `traefik.asepharyana.my.id` (secured)
CONTRIBUTING.md
+13 -65
View File
@@ -16,7 +16,6 @@
- **Git** with LFS support - **Git** with LFS support
- **Node.js** >= 22.11.0 (via `.node-version` or `.nvmrc`) - **Node.js** >= 22.11.0 (via `.node-version` or `.nvmrc`)
- **Bun** >= 1.3.11 (package manager) - **Bun** >= 1.3.11 (package manager)
- **Rust** >= 1.89.0 (for Rust services)
- **Docker** and **Docker Compose** (for shared infrastructure) - **Docker** and **Docker Compose** (for shared infrastructure)
## Local Setup ## Local Setup
@@ -40,18 +39,14 @@ This checks out all submodules at the pinned commit (not `main`). The submodules
| Path | Remote | | Path | Remote |
| ---------------- | --------------------------------------- | | ---------------- | --------------------------------------- |
| `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| `apps/scraper` | `asepharyana/asepharyana-hub-scraper` | | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
| `apps/react` | `asepharyana/asepharyana-hub-react` |
| `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
### 3. Install Dependencies per Service ### 3. Install Dependencies per Service
Install dependencies for TypeScript/Bun services: Install dependencies for TypeScript/Bun services:
```bash ```bash
cd apps/elysia && bun install && cd ../.. cd apps/scraper && bun install && cd ../..
cd apps/react && npm install && cd ../..
``` ```
### 4. Start Shared Infrastructure ### 4. Start Shared Infrastructure
@@ -75,76 +70,42 @@ Key variables to configure:
| Variable | Description | | Variable | Description |
| -------------- | ----------------------------------------------------- | | -------------- | ----------------------------------------------------- |
| `DATABASE_URL` | PostgreSQL connection (Tailscale IP to `imrnes` VPS) | | `DATABASE_URL` | PostgreSQL connection (Tailscale IP to `imrnes` VPS) |
| `REDIS_URL` | Redis connection (`redis://localhost:6379` for local) |
| `JWT_SECRET` | JWT signing secret |
| `GITHUB_TOKEN` | GitHub personal access token | | `GITHUB_TOKEN` | GitHub personal access token |
## Development Workflow ## Development Workflow
### Running Services ### Running Services
**Rust API (rust-auth):** Refer to each service's own documentation for setup and development instructions.
```bash
cd apps/rust-auth
cargo run
```
**Elysia API (elysia):**
```bash
cd apps/elysia
bun run dev
```
**React Frontend (react):**
```bash
cd apps/react
npm run dev
```
### API Documentation ### API Documentation
- Rust OpenAPI: `http://localhost:4091/docs` Refer to each service's own documentation for API docs and endpoints.
- Elysia Swagger: `http://localhost:4092/docs`
- Elysia AsyncAPI: `http://localhost:4092/docs-ws`
## Coding Standards ## Coding Standards
### Linting ### Linting
- **ESLint** with `@antfu/eslint-config` for TypeScript/JavaScript - **Biome** for TypeScript/JavaScript formatting and linting
- **Cargo Clippy** for Rust
Run linting: Run linting:
```bash ```bash
# TypeScript/JavaScript # TypeScript/JavaScript
eslint . --no-error-on-unmatched-pattern bun run check
# Rust specific
cd apps/rust-auth && cargo clippy -- -D warnings
``` ```
### Formatting ### Formatting
- **Prettier** for TypeScript/JavaScript/Markdown (config in `.prettierrc`) - **Biome** for TypeScript/JavaScript
- Single quotes, 100 print width, 2-space indent, trailing commas
- **Cargo fmt** for Rust
- **EditorConfig** for general formatting (`.editorconfig`) - **EditorConfig** for general formatting (`.editorconfig`)
```bash ```bash
# Prettier # Format all
prettier --write . bun run format
# Rust
cd apps/rust-auth && cargo fmt
``` ```
### Rust Configuration
Rust services use edition `2024` with stable toolchain (nightly features may be used).
## Commit Message Format ## Commit Message Format
@@ -177,18 +138,15 @@ This project enforces **Conventional Commits** for all commit messages.
### Examples ### Examples
``` ```
feat(rust-auth): add OAuth2 Google login flow feat(scraper): add new data source integration
fix(elysia): handle null JWT payload in auth middleware chore: update biome config to v10
chore: update eslint config to v10
docs: add API endpoint documentation for scraper docs: add API endpoint documentation for scraper
refactor(react): extract Header component from App
test(elysia): add unit tests for rate limiter
ci: migrate to CodeQL v3 ci: migrate to CodeQL v3
``` ```
### Scopes ### Scopes
Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps` Common scopes: `scraper`, `infra`, `ci`, `deps`
## Pull Request Process ## Pull Request Process
@@ -202,8 +160,7 @@ Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
3. **Run checks locally** before pushing: 3. **Run checks locally** before pushing:
```bash ```bash
cd apps/react && npx tsc --noEmit bun run check
eslint . --no-error-on-unmatched-pattern
``` ```
4. **Push and open a PR** against `main`. CI will automatically run: 4. **Push and open a PR** against `main`. CI will automatically run:
@@ -227,13 +184,4 @@ Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
## Adding a New Service ## Adding a New Service
See `docs/add-new-app.md` for the complete step-by-step guide. In summary: See `docs/add-new-app.md` for the complete step-by-step guide.
1. Create the app in `apps/<name>`
2. Add it as a Git submodule in `.gitmodules`
3. Register it in `infra/compose/<name>.yml`
4. Add a Dockerfile at `infra/docker/<name>.Dockerfile`
5. Add Traefik routing config in `infra/traefik/dynamic/apps.yaml`
6. Add CI entries in `.github/workflows/docker-build-push.yml`
7. Add compose file to the deploy script in `deploy-docker.yml`
8. Add any required GitHub secrets for the service
README.md
+8 -31
View File
@@ -5,12 +5,9 @@ Aplikasi dipisah sebagai submodule agar frontend, API, dan service pendukung bis
## Services ## Services
| Service | Path | Default Local Port | Notes | | Service | Path | Notes |
| :---------- | :--------------- | :----------------- | :---------------------------------------------------------------------------- | | :------ | :------------- | :------------------ |
| Rust API | `apps/rust-auth` | `4091` | API utama (Axum + SeaORM), scraping, image proxy/cache, metrics, OpenAPI docs | | Scraper | `apps/scraper` | Web scraper service |
| Elysia API | `apps/elysia` | `4092` | API realtime/auth/chat/quiz (Elysia + Bun + Drizzle + Redis) |
| React Web | `apps/react` | `3000` | Frontend React/Vite |
| Scraper | `apps/scraper` | — | Web scraper service |
## Infrastructure ## Infrastructure
@@ -18,7 +15,7 @@ File compose berada di `infra/compose/`:
- `traefik.yml`: reverse proxy Traefik untuk semua layanan. - `traefik.yml`: reverse proxy Traefik untuk semua layanan.
- `shared.yml`: Redis. - `shared.yml`: Redis.
- `rust-auth.yml`, `elysia.yml`, `react.yml`, `scraper.yml`: manifest deploy per service (image GHCR bertag SHA). - `scraper.yml`: manifest deploy per service (image GHCR bertag SHA).
Dockerfile per service berada di `infra/docker/`. Dockerfile per service berada di `infra/docker/`.
@@ -27,9 +24,6 @@ Dockerfile per service berada di `infra/docker/`.
Build image via Dockerfile: Build image via Dockerfile:
```bash ```bash
docker build -f infra/docker/rust.Dockerfile -t rust-auth:latest .
docker build -f infra/docker/elysia.Dockerfile -t elysia-api:latest .
docker build -f infra/docker/react.Dockerfile -t react-web:latest .
docker build -f infra/docker/scraper.Dockerfile -t scraper-api:latest . docker build -f infra/docker/scraper.Dockerfile -t scraper-api:latest .
``` ```
@@ -38,9 +32,8 @@ Tag and push:
```bash ```bash
SHORT_SHA=$(git rev-parse --short HEAD) SHORT_SHA=$(git rev-parse --short HEAD)
docker tag rust-auth:latest ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA docker tag scraper-api:latest ghcr.io/asepharyana/asepharyana-hub/scraper-api:sha-$SHORT_SHA
docker push ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA docker push ghcr.io/asepharyana/asepharyana-hub/scraper-api:sha-$SHORT_SHA
# repeat for elysia-api, react-web, scraper-api
``` ```
## Local Development ## Local Development
@@ -53,27 +46,11 @@ docker compose -f infra/compose/shared.yml up -d
### 2) Jalankan service yang dibutuhkan ### 2) Jalankan service yang dibutuhkan
```bash Refer to each service's own documentation for development setup.
# Rust API
cd apps/rust-auth
cargo run
# Elysia API
cd apps/elysia
bun install
bun run dev
# React web
cd apps/react
npm install
npm run dev
```
## API Docs and Monitoring ## API Docs and Monitoring
- Rust OpenAPI: `/docs` Refer to each service's own documentation for API docs.
- Elysia Swagger: `/docs`
- Elysia AsyncAPI viewer: `/docs-ws`
## Deployment Notes ## Deployment Notes
docs/adr/0001-use-hub-repo-with-submodules.md
+1 -4
View File
@@ -23,10 +23,7 @@ Use `asepharyana-hub` as the root hub repository.
Current app submodules: Current app submodules:
| Service | Path | Remote | | Service | Path | Remote |
| -------------- | ---------------- | --------------------------------------- | | ----------- | -------------- | --------------------------------------- |
| Elysia API | `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| React frontend | `apps/react` | `asepharyana/asepharyana-hub-react` |
| Rust auth API | `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
| Scraper API | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` | | Scraper API | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
## Consequences ## Consequences
docs/superpowers/specs/2026-07-09-monorepo-restructure-design.md
+13 -23
View File
@@ -21,10 +21,7 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
| Service | Path | Git remote baru | | Service | Path | Git remote baru |
|---------|------|----------------| |---------|------|----------------|
| Elysia API | `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| React Frontend | `apps/react` | `asepharyana/asepharyana-hub-react` |
| Scraper | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` | | Scraper | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
| Rust Auth | `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
## Service yang dihapus ## Service yang dihapus
@@ -32,8 +29,11 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
|---------|------| |---------|------|
| Docker Manager | `apps/docker-manager/` | | Docker Manager | `apps/docker-manager/` |
| TeleUploader | `apps/teleuploader/` | | TeleUploader | `apps/teleuploader/` |
| Elysia API | `apps/elysia/` |
| React Frontend | `apps/react/` |
| Rust Auth | `apps/rust-auth/` |
## File yang akan dihapus ## File yang akan dihapus (gelombang 1)
- `apps/docker-manager/` (seluruh direktori) - `apps/docker-manager/` (seluruh direktori)
- `apps/teleuploader/` (seluruh direktori) - `apps/teleuploader/` (seluruh direktori)
@@ -68,10 +68,10 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
| Lama | Baru | | Lama | Baru |
|------|------| |------|------|
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/elysia-api:*` | `ghcr.io/asepharyana/asepharyana-hub/elysia-api:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/react-web:*` | `ghcr.io/asepharyana/asepharyana-hub/react-web:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/scraper-api:*` | `ghcr.io/asepharyana/asepharyana-hub/scraper-api:*` | | `ghcr.io/mytheclipse/ultimate-asepharyana.tech/scraper-api:*` | `ghcr.io/asepharyana/asepharyana-hub/scraper-api:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/rust-auth:*` | `ghcr.io/asepharyana/asepharyana-hub/rust-auth:*` | | `ghcr.io/mytheclipse/ultimate-asepharyana.tech/elysia-api:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/react-web:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/rust-auth:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/docker-manager:*` | — (dihapus) | | `ghcr.io/mytheclipse/ultimate-asepharyana.tech/docker-manager:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/teleuploader:*` | — (dihapus) | | `ghcr.io/mytheclipse/ultimate-asepharyana.tech/teleuploader:*` | — (dihapus) |
@@ -79,35 +79,25 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
### `docker-build-push.yml` ### `docker-build-push.yml`
- `IMAGE_NAME_PREFIX`: `mytheclipse/ultimate-asepharyana.tech` → `asepharyana/asepharyana-hub` - `IMAGE_NAME_PREFIX`: `mytheclipse/ultimate-asepharyana.tech` → `asepharyana/asepharyana-hub`
- Hapus service entries: `docker-manager`, `teleuploader` - Hapus service entries: `docker-manager`, `teleuploader`, `elysia-api`, `react-web`, `rust-auth`
- Update repo URLs di `wait-submodule-ref` dari `MythEclipse/*` ke `asepharyana/*` - Update repo URLs di `wait-submodule-ref` dari `MythEclipse/*` ke `asepharyana/*`
- Update `update-manifest` phase — hapus service docker-manager & teleuploader - Update `update-manifest` phase — hapus service docker-manager, teleuploader, elysia-api, react-web, rust-auth
### `deploy-docker.yml` ### `deploy-docker.yml`
- Update `git remote add origin` - Update `git remote add origin`
- Hapus docker-manager & teleuploader dari compose list dan checkout - Hapus docker-manager, teleuploader, elysia-api, react-web, rust-auth dari compose list dan checkout
### `update-submodule.yml` ### `update-submodule.yml`
- Hapus service docker-manager & teleuploader - Hapus service docker-manager, teleuploader, elysia-api, react-web, rust-auth
- Update nama workflow
## `.gitmodules` ## `.gitmodules`
Hanya berisi 4 apps dengan remote baru: Hanya berisi 1 app scraper:
```ini ```ini
[submodule "apps/elysia"]
path = apps/elysia
url = https://github.com/asepharyana/asepharyana-hub-elysia.git
[submodule "apps/react"]
path = apps/react
url = https://github.com/asepharyana/asepharyana-hub-react.git
[submodule "apps/scraper"] [submodule "apps/scraper"]
path = apps/scraper path = apps/scraper
url = https://github.com/asepharyana/asepharyana-hub-scraper.git url = https://github.com/asepharyana/asepharyana-hub-scraper.git
[submodule "apps/rust-auth"]
path = apps/rust-auth
url = https://github.com/asepharyana/asepharyana-hub-rust-auth.git
``` ```
## Execution plan ## Execution plan
@@ -129,7 +119,7 @@ Hanya berisi 4 apps dengan remote baru:
## Post-execution state ## Post-execution state
- Root direktori `asepharyana-hub/` dengan source code apps utuh (tanpa git) - Root direktori `asepharyana-hub/` dengan source code apps utuh (tanpa git)
- 4 app submodule terdaftar di `.gitmodules` dengan remote baru - 1 app submodule (scraper) terdaftar di `.gitmodules` dengan remote baru
- Infra/docs/scripts tetap menyatu di root - Infra/docs/scripts tetap menyatu di root
- 0 references ke `MythEclipse/ultimate-asepharyana.tech` di file konfigurasi - 0 references ke `MythEclipse/ultimate-asepharyana.tech` di file konfigurasi
- Siap untuk `git init && git add && git commit` kapan saja - Siap untuk `git init && git add && git commit` kapan saja