feat(observability): add OpenTelemetry Collector, Jaeger, and custom dashboard for monitoring
fix(infra): update NATS configuration for OpenTelemetry tracing and JetStream support refactor(dapr): modify Dapr sidecar configuration to use new resources path and config file chore(docs): update deployment documentation to include observability services docs(commit-convention): establish commit message guidelines for Asepharyana Hub docs(deploy-workflow): outline CI/CD pipeline and manual deployment steps for Asepharyana Hub feat(event-driven): implement event-driven architecture patterns with Dapr and NATS docs(hub-rules): define repository structure, submodule strategy, and infrastructure patterns
This commit is contained in:
asepharyana
@@ -0,0 +1,65 @@
|
||||
---
|
||||
name: commit-convention
|
||||
description: Commit message convention — type(scope): description for Asepharyana Hub
|
||||
---
|
||||
|
||||
# Commit Convention — Asepharyana Hub
|
||||
|
||||
## Format
|
||||
|
||||
```
|
||||
<type>(<scope>): <description>
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer]
|
||||
```
|
||||
|
||||
## Types
|
||||
|
||||
| Type | Usage |
|
||||
| ---------- | ------------------------------------ |
|
||||
| `feat` | Fitur baru |
|
||||
| `fix` | Bug fix |
|
||||
| `chore` | Maintenance, config, tooling |
|
||||
| `docs` | Dokumentasi |
|
||||
| `refactor` | Perubahan kode tanpa fungsional baru |
|
||||
| `test` | Nambah/update test |
|
||||
| `ci` | CI/CD workflows |
|
||||
| `perf` | Optimasi performa |
|
||||
| `style` | Formatting (tanda kutip, dll) |
|
||||
|
||||
## Scopes
|
||||
|
||||
| Scope | Area |
|
||||
| ------------- | --------------------------------- |
|
||||
| `scraper` | apps/scraper submodule |
|
||||
| `infra` | infra/ (compose, traefik, docker) |
|
||||
| `ci` | .github/workflows/ |
|
||||
| `dapr` | Dapr config & sidecar |
|
||||
| `nats` | NATS message bus |
|
||||
| `docs` | Dokumentasi |
|
||||
| `deps` | Dependencies |
|
||||
| `scripts` | Utility scripts |
|
||||
| `root` | Root config files |
|
||||
|
||||
## Contoh
|
||||
|
||||
```
|
||||
feat(scraper): add anime detail caching via Dapr pubsub
|
||||
fix(infra): correct NATS CLI flags for JetStream
|
||||
chore(deps): update biome to v2.5.3
|
||||
docs(infra): add deployment order for Dapr services
|
||||
ci(deploy): add nats.yml to ALL_COMPOSE_FILES
|
||||
refactor(scraper): migrate EventBus from tokio broadcast to Dapr pubsub
|
||||
```
|
||||
|
||||
## Aturan
|
||||
|
||||
1. **Wajib** menyertakan scope dalam tanda kurung
|
||||
2. **Wajib** `Co-Authored-By` untuk commit yang digenerate AI
|
||||
3. **Gunakan imperative mood**: "add" bukan "added" / "adds"
|
||||
4. **Jangan capitalize** type: `feat:` bukan `Feat:`
|
||||
5. **No period** di akhir subject baris
|
||||
6. Body explain **why** dan **what**, bukan **how**
|
||||
7. Refer issue dengan `Closes #123` atau `Fixes #123` di footer
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
name: deploy-workflow
|
||||
description: CI/CD pipeline, Docker build patterns, manual deploy steps, and troubleshooting for Asepharyana Hub
|
||||
---
|
||||
|
||||
# Deploy & Workflow — Asepharyana Hub
|
||||
|
||||
## CI/CD Pipeline
|
||||
|
||||
### Build Pipeline (`docker-build-push.yml`)
|
||||
Trigger: push ke `main` yang touch `apps/**`, `infra/**`, `infra/docker/**`
|
||||
|
||||
1. **changes** — detect service mana yg berubah via git diff
|
||||
2. **wait-submodule-ref** — (repository_dispatch only) tunggu SHA commit fetchable
|
||||
3. **build** — matrix build per service, push ke GHCR (`sha-<short>` + `latest`)
|
||||
4. **update-manifest** — update image tag di compose file, commit + push
|
||||
|
||||
### Deploy Pipeline (`deploy-docker.yml`)
|
||||
Trigger: build selesai, atau push ke `main` touch `infra/**`
|
||||
|
||||
1. SSH ke `orangevps` (via `secrets.VPS_HOST`)
|
||||
2. Sync repo (`git fetch --depth=1 + reset`)
|
||||
3. Login ke GHCR
|
||||
4. Deteksi compose file yg berubah
|
||||
5. Pull images + restart container selektif
|
||||
|
||||
### Secrets Required
|
||||
| Secret | Untuk |
|
||||
|--------|-------|
|
||||
| `SSH_PRIVATE_KEY` | SSH ke VPS |
|
||||
| `VPS_HOST` | IP/host VPS (tailscale IP) |
|
||||
| `VPS_USER` | SSH user, biasanya `root` |
|
||||
| `VPS_TARGET_DIR` | Lokasi repo di VPS |
|
||||
| `ENV_FILE_PRODUCTION` | .env content untuk production |
|
||||
|
||||
### Selective Deployment
|
||||
- Hanya compose file yg berubah yang di-redeploy
|
||||
- Selective: `UP_FLAGS="-d"` (tanpa `--remove-orphans`)
|
||||
- Full deploy: `UP_FLAGS="-d --remove-orphans"`
|
||||
|
||||
## Docker Patterns
|
||||
|
||||
### Build dengan cargo-chef (Rust)
|
||||
```dockerfile
|
||||
FROM lukemathwalker/cargo-chef:latest-rust-1.89.0 AS chef
|
||||
WORKDIR /app
|
||||
FROM chef AS planner
|
||||
COPY apps/scraper .
|
||||
RUN cargo chef prepare --recipe-path recipe.json
|
||||
FROM chef AS builder
|
||||
COPY --from=planner /app/recipe.json recipe.json
|
||||
RUN cargo chef cook --release --recipe-path recipe.json
|
||||
COPY apps/scraper .
|
||||
RUN cargo build --release
|
||||
```
|
||||
|
||||
### Runtime minimal untuk Rust binary
|
||||
```dockerfile
|
||||
FROM debian:bookworm-slim AS runtime
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl libssl3 && rm -rf /var/lib/apt/lists/*
|
||||
```
|
||||
|
||||
## Manual Deploy Steps
|
||||
```bash
|
||||
# 1. Login GHCR
|
||||
echo $GITHUB_TOKEN | docker login ghcr.io -u asepharyana --password-stdin
|
||||
|
||||
# 2. Full stack
|
||||
docker compose -f infra/compose/traefik.yml \
|
||||
-f infra/compose/shared.yml \
|
||||
-f infra/compose/nats.yml \
|
||||
-f infra/compose/dapr.yml \
|
||||
-f infra/compose/scraper.yml \
|
||||
--env-file .env up -d --remove-orphans
|
||||
|
||||
# 3. Selective (hanya satu service)
|
||||
docker compose -f infra/compose/scraper.yml --env-file .env up -d
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container reach Tailscale
|
||||
Pastikan route ke Tailscale di main table:
|
||||
```bash
|
||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
||||
systemctl restart tailscale-routes
|
||||
```
|
||||
|
||||
### Healthcheck gagal di scratch images
|
||||
NATS dan Dapr placement pake scratch — tidak bisa healthcheck. Cukup `service_started` di depends_on.
|
||||
|
||||
### Dapr sidecar crash
|
||||
```bash
|
||||
docker logs scraper-api-dapr | grep -iE "fatal|error"
|
||||
```
|
||||
Penyebab umum: komponen config salah, NATS/Dapr placement belum siap.
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
name: event-driven
|
||||
description: Event-driven architecture patterns with Dapr + NATS for Asepharyana Hub services
|
||||
---
|
||||
|
||||
# Event-Driven Architecture — Asepharyana Hub
|
||||
|
||||
## Stack
|
||||
- **Message Backbone**: NATS + JetStream (untuk streaming & job queue)
|
||||
- **Pub/Sub Runtime**: Dapr sidecar per service (pubsub via Redis built-in)
|
||||
- **State Store**: Dapr → Redis
|
||||
|
||||
## Event Topics Convention
|
||||
|
||||
```
|
||||
hub.<domain>.<action>
|
||||
|
||||
Contoh:
|
||||
hub.image.cached → Image selesai di-cache ke CDN
|
||||
hub.image.repaired → Image diperbaiki (CNAME change)
|
||||
hub.scrape.anime.done → Scrape anime selesai
|
||||
hub.system.alert → Error/alert dari service
|
||||
```
|
||||
|
||||
## CloudEvents Format
|
||||
|
||||
```json
|
||||
{
|
||||
"specversion": "1.0",
|
||||
"type": "hub.image.cached",
|
||||
"source": "scraper-api",
|
||||
"subject": "anime-poster",
|
||||
"id": "uuid-v4",
|
||||
"time": "2026-07-21T10:00:00Z",
|
||||
"datacontenttype": "application/json",
|
||||
"data": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Publish Event (Rust via HTTP API)
|
||||
|
||||
Gunakan `reqwest` langsung ke Dapr sidecar (SDK Rust masih experimental):
|
||||
|
||||
```rust
|
||||
let event = serde_json::json!({
|
||||
"specversion": "1.0",
|
||||
"type": "hub.image.cached",
|
||||
"source": "scraper-api",
|
||||
"id": Uuid::new_v4().to_string(),
|
||||
"time": chrono::Utc::now().to_rfc3339(),
|
||||
"datacontenttype": "application/json",
|
||||
"data": { "original_url": url, "cdn_url": cdn_url }
|
||||
});
|
||||
|
||||
reqwest::Client::new()
|
||||
.post("http://localhost:3500/v1.0/publish/pubsub/hub.image.cached")
|
||||
.json(&event)
|
||||
.send()
|
||||
.await?;
|
||||
```
|
||||
|
||||
## Service Invocation
|
||||
|
||||
```bash
|
||||
curl http://localhost:3500/v1.0/invoke/<app-id>/method/<path>
|
||||
```
|
||||
|
||||
## State Store
|
||||
|
||||
```bash
|
||||
# Set
|
||||
curl -X POST http://localhost:3500/v1.0/state/statestore \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '[{"key": "mykey", "value": "myvalue"}]'
|
||||
|
||||
# Get
|
||||
curl http://localhost:3500/v1.0/state/statestore/mykey
|
||||
|
||||
# Delete
|
||||
curl -X DELETE http://localhost:3500/v1.0/state/statestore/mykey
|
||||
```
|
||||
|
||||
## Scraper Event Integration
|
||||
|
||||
File yang perlu dimodifikasi untuk event-driven:
|
||||
|
||||
| File | Perubahan |
|
||||
|------|-----------|
|
||||
| `src/events/bus.rs` | Ganti backend dari tokio broadcast ke Dapr pub/sub |
|
||||
| `src/bootstrap/mod.rs` | Init DaprClient, inject ke AppState |
|
||||
| `src/presentation/state.rs` | Tambah `dapr_client` field |
|
||||
| `src/proxy/use_cases.rs` | Publish `ImageRepaired` & `ImageCached` events |
|
||||
| `src/infrastructure/services/images/cache.rs` | Emit event tiap cache selesai |
|
||||
| `Cargo.toml` | Tambah `reqwest`, `uuid`, `chrono` (jika belum ada) |
|
||||
|
||||
## Event Handlers (Subscribe)
|
||||
|
||||
Buat `src/subscribers/` untuk handler:
|
||||
|
||||
```rust
|
||||
// src/subscribers/image_handler.rs
|
||||
pub async fn handle_image_cached(event: CloudEvent) -> Result<()> {
|
||||
// Log, notifikasi, update status
|
||||
}
|
||||
```
|
||||
|
||||
Daftarkan subscribers di `bootstrap/mod.rs` dengan spawn task:
|
||||
```rust
|
||||
tokio::spawn(async move {
|
||||
let mut stream = dapr_client.subscribe("pubsub", "hub.image.cached");
|
||||
while let Some(event) = stream.next().await {
|
||||
handle_image_cached(event).await;
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Testing Event-Driven Code
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_publish_event() {
|
||||
let client = MockDaprClient::new();
|
||||
client.expect_publish()
|
||||
.with(...)
|
||||
.returning(|_| Ok(()));
|
||||
// ... test
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
name: hub-rules
|
||||
description: Repository structure, submodule strategy, infrastructure patterns, and architecture of Asepharyana Hub
|
||||
---
|
||||
|
||||
# Asepharyana Hub — Repository Rules
|
||||
|
||||
## Struktur Repository
|
||||
|
||||
```
|
||||
asepharyana-hub/
|
||||
├── apps/ # Git submodules — source code aplikasi
|
||||
├── docs/ # Dokumentasi, ADR, deployment guide
|
||||
├── infra/ # Infrastructure as code
|
||||
│ ├── compose/ # Satu compose file per service
|
||||
│ ├── dapr/ # Dapr component configs
|
||||
│ ├── docker/ # Dockerfiles per service
|
||||
│ └── traefik/ # Static & dynamic Traefik config
|
||||
├── scripts/ # Utility scripts (cleanup, update-deps)
|
||||
└── .github/workflows/ # CI/CD pipelines
|
||||
```
|
||||
|
||||
### Aturan Submodule
|
||||
- Setiap aplikasi di `apps/` adalah **submodule** ke repo terpisah.
|
||||
- Perubahan kode aplikasi dilakukan di **repo masing-masing**, bukan di sini.
|
||||
- Submodule pointer diupdate oleh CI/CD (bukan manual).
|
||||
|
||||
## Infrastructure Patterns
|
||||
|
||||
### Networking
|
||||
- Semua service join **`app-shared-net`** (external Docker bridge)
|
||||
- Service discovery via Docker DNS (container alias)
|
||||
- Traefik sebagai ingress untuk HTTP/S eksternal
|
||||
- Tailscale untuk cross-VPS (PostgreSQL, Redis)
|
||||
|
||||
### Compose File Pattern
|
||||
```yaml
|
||||
services:
|
||||
<service>:
|
||||
container_name: <service>
|
||||
image: ghcr.io/asepharyana/asepharyana-hub/<service>:sha-<sha>
|
||||
restart: always
|
||||
networks:
|
||||
app-shared-net:
|
||||
aliases:
|
||||
- <service>
|
||||
env_file:
|
||||
- ../../.env
|
||||
|
||||
networks:
|
||||
app-shared-net:
|
||||
name: app-shared-net
|
||||
external: true
|
||||
```
|
||||
|
||||
### Dapr Sidecar Pattern
|
||||
```yaml
|
||||
<service>-dapr:
|
||||
container_name: <service>-dapr
|
||||
image: daprio/daprd:latest
|
||||
restart: always
|
||||
depends_on:
|
||||
nats:
|
||||
condition: service_started
|
||||
dapr-placement:
|
||||
condition: service_started
|
||||
networks:
|
||||
- app-shared-net
|
||||
command:
|
||||
- './daprd'
|
||||
- '--app-id=<service>'
|
||||
- '--app-port=<port>'
|
||||
- '--dapr-http-port=3500'
|
||||
- '--dapr-grpc-port=50001'
|
||||
- '--placement-host-address=dapr-placement:50005'
|
||||
- '--resources-path=/components'
|
||||
volumes:
|
||||
- ../../infra/dapr/components:/components
|
||||
```
|
||||
|
||||
### Traefik Routing
|
||||
- Router + service definition di `infra/traefik/dynamic/apps.yaml`
|
||||
- Subdomain pattern: `<service>.asepharyana.my.id` + `<service>.asepharya.web.id`
|
||||
- TLS cert dari volume mount (bukan auto-acme)
|
||||
|
||||
### Image Tagging
|
||||
- `sha-<short-sha>` — immutable, untuk rollback
|
||||
- `latest` — mutable, untuk convenience
|
||||
- Registry: `ghcr.io/asepharyana/asepharyana-hub/<service>`
|
||||
|
||||
### CI/CD
|
||||
- `docker-build-push.yml` — build per service, push ke GHCR, update compose manifest
|
||||
- `deploy-docker.yml` — SSH ke orangevps, pull images, restart
|
||||
- Selective deploy: hanya compose file yg berubah
|
||||
|
||||
## Deployment Order
|
||||
1. `shared.yml` (Redis)
|
||||
2. `nats.yml` (NATS message bus)
|
||||
3. `dapr.yml` (Dapr placement)
|
||||
4. `traefik.yml` (Reverse proxy)
|
||||
5. Service compose files (apps + Dapr sidecar)
|
||||
Reference in New Issue
Block a user