chore: initial hub repo structure
This commit is contained in:
asepharyana
@@ -0,0 +1,686 @@
|
||||
# Moonrepo Migration Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Migrate the monorepo to moonrepo for centralized task orchestration, caching, and consistent dependency management across all 7 apps.
|
||||
|
||||
**Architecture:** moonrepo sits as a task orchestration layer above the existing Nix build system. `.moon/tasks/` holds shared task definitions (TypeScript and Rust). Each app gets a `moon.yml` with tags and inter-project dependencies. Nix, Docker Compose, git submodules, and infra remain untouched.
|
||||
|
||||
**Tech Stack:** moonrepo CLI, Node 22, Bun, TypeScript 5+, Cargo (Rust)
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Install moonrepo CLI and scaffold .moon/ directory
|
||||
|
||||
**Files:**
|
||||
- Create: `.moon/workspace.yml`
|
||||
- Create: `.moon/toolchain.yml`
|
||||
- Create: `.moon/tasks/` (directory)
|
||||
|
||||
- [ ] **Step 1: Install moonrepo CLI via curl**
|
||||
|
||||
```bash
|
||||
curl -fsSL https://moonrepo.dev/install/moon.sh | bash
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify installation**
|
||||
|
||||
Run: `moon --version`
|
||||
Expected: prints version number (e.g., `moon 1.x.x`)
|
||||
|
||||
- [ ] **Step 3: Create .moon/ scaffold directories**
|
||||
|
||||
```bash
|
||||
mkdir -p .moon/tasks
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .moon/
|
||||
git commit -m "chore: scaffold .moon/ directory for moonrepo"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Configure workspace.yml
|
||||
|
||||
**Files:**
|
||||
- Create: `.moon/workspace.yml`
|
||||
|
||||
- [ ] **Step 1: Write .moon/workspace.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/workspace
|
||||
$schema: "https://moonrepo.dev/schemas/workspace.json"
|
||||
|
||||
projects:
|
||||
- "apps/*"
|
||||
|
||||
vcs:
|
||||
manager: "git"
|
||||
defaultBranch: "main"
|
||||
|
||||
runner:
|
||||
implicitDeps:
|
||||
# TypeScript apps: lint depends on build (for typecheck path)
|
||||
- "typescript-build.build"
|
||||
|
||||
cacheTtl: 604800
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Validate config structure**
|
||||
|
||||
Run: `moon check`
|
||||
Expected: no errors (will warn about missing project configs — expected)
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add .moon/workspace.yml
|
||||
git commit -m "chore: configure moonrepo workspace with project glob"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Configure toolchain.yml
|
||||
|
||||
**Files:**
|
||||
- Create: `.moon/toolchain.yml`
|
||||
|
||||
- [ ] **Step 1: Write .moon/toolchain.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/toolchain
|
||||
$schema: "https://moonrepo.dev/schemas/toolchain.json"
|
||||
|
||||
node:
|
||||
version: "22.11.0"
|
||||
packageManager: "bun"
|
||||
bun:
|
||||
version: "1.3.11"
|
||||
|
||||
typescript:
|
||||
syncProjectReferences: true
|
||||
createMissingConfig: false
|
||||
routeOutDirToCache: false
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add .moon/toolchain.yml
|
||||
git commit -m "chore: configure moonrepo toolchain (Node 22, Bun 1.3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Create shared TypeScript task definitions
|
||||
|
||||
**Files:**
|
||||
- Create: `.moon/tasks/typescript-build.yml`
|
||||
- Create: `.moon/tasks/typescript-lint.yml`
|
||||
- Create: `.moon/tasks/typescript-test.yml`
|
||||
|
||||
- [ ] **Step 1: Write .moon/tasks/typescript-build.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
build:
|
||||
command: "bun run build"
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "tsconfig.json"
|
||||
- "package.json"
|
||||
outputs:
|
||||
- ".next"
|
||||
- "dist"
|
||||
- ".output"
|
||||
options:
|
||||
cache: true
|
||||
dev:
|
||||
command: "bun run dev"
|
||||
local: true
|
||||
options:
|
||||
persistent: true
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write .moon/tasks/typescript-lint.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
lint:
|
||||
command: "bun run lint"
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "eslint.config.mjs"
|
||||
- "tsconfig.json"
|
||||
options:
|
||||
cache: false
|
||||
typecheck:
|
||||
command: "bun run check-types"
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "tsconfig.json"
|
||||
options:
|
||||
cache: false
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write .moon/tasks/typescript-test.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
test:
|
||||
command: "bun test"
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "test/**/*"
|
||||
- "tests/**/*"
|
||||
- "vitest.config.ts"
|
||||
options:
|
||||
cache: false
|
||||
e2e:
|
||||
command: "noop"
|
||||
local: true
|
||||
options:
|
||||
cache: false
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .moon/tasks/typescript-build.yml .moon/tasks/typescript-lint.yml .moon/tasks/typescript-test.yml
|
||||
git commit -m "chore: add shared TypeScript task definitions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Create shared Rust task definitions
|
||||
|
||||
**Files:**
|
||||
- Create: `.moon/tasks/rust-build.yml`
|
||||
- Create: `.moon/tasks/rust-test.yml`
|
||||
- Create: `.moon/tasks/rust-lint.yml`
|
||||
|
||||
- [ ] **Step 1: Write .moon/tasks/rust-build.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
build:
|
||||
command: "cargo build --release"
|
||||
platform: system
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "Cargo.toml"
|
||||
- "Cargo.lock"
|
||||
- "build.rs"
|
||||
outputs:
|
||||
- "target/release/*"
|
||||
options:
|
||||
cache: true
|
||||
envFile: false
|
||||
dev:
|
||||
command: "cargo run"
|
||||
platform: system
|
||||
local: true
|
||||
options:
|
||||
persistent: true
|
||||
envFile: false
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write .moon/tasks/rust-test.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
test:
|
||||
command: "cargo test"
|
||||
platform: system
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "Cargo.toml"
|
||||
- "Cargo.lock"
|
||||
- "tests/**/*"
|
||||
options:
|
||||
cache: false
|
||||
envFile: false
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write .moon/tasks/rust-lint.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/tasks
|
||||
$schema: "https://moonrepo.dev/schemas/tasks.json"
|
||||
|
||||
tasks:
|
||||
lint:
|
||||
command: "cargo clippy -- -D warnings"
|
||||
platform: system
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
- "Cargo.toml"
|
||||
options:
|
||||
cache: false
|
||||
envFile: false
|
||||
fmt-check:
|
||||
command: "cargo fmt --check"
|
||||
platform: system
|
||||
inputs:
|
||||
- "src/**/*"
|
||||
options:
|
||||
cache: false
|
||||
envFile: false
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .moon/tasks/rust-build.yml .moon/tasks/rust-test.yml .moon/tasks/rust-lint.yml
|
||||
git commit -m "chore: add shared Rust task definitions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Create per-app moon.yml for TypeScript apps
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/nextjs/moon.yml`
|
||||
- Create: `apps/elysia/moon.yml`
|
||||
- Create: `apps/solidjs/moon.yml`
|
||||
|
||||
- [ ] **Step 1: Write apps/nextjs/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "typescript"
|
||||
platform: "node"
|
||||
tags:
|
||||
- "lang:typescript"
|
||||
- "type:frontend"
|
||||
|
||||
dependsOn:
|
||||
- id: "rust-auth"
|
||||
- id: "elysia"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write apps/elysia/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "typescript"
|
||||
platform: "bun"
|
||||
tags:
|
||||
- "lang:typescript"
|
||||
- "type:backend"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write apps/solidjs/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "typescript"
|
||||
platform: "bun"
|
||||
tags:
|
||||
- "lang:typescript"
|
||||
- "type:frontend"
|
||||
|
||||
dependsOn:
|
||||
- id: "elysia"
|
||||
- id: "rust-auth"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/nextjs/moon.yml apps/elysia/moon.yml apps/solidjs/moon.yml
|
||||
git commit -m "chore: add moon.yml for TypeScript apps (nextjs, elysia, solidjs)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Create per-app moon.yml for Rust apps
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/rust/moon.yml`
|
||||
- Create: `apps/rust-auth/moon.yml`
|
||||
- Create: `apps/leptos/moon.yml`
|
||||
|
||||
- [ ] **Step 1: Write apps/rust/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "rust"
|
||||
platform: "system"
|
||||
tags:
|
||||
- "lang:rust"
|
||||
- "type:backend"
|
||||
|
||||
fileGroups:
|
||||
sources:
|
||||
- "src/**/*.rs"
|
||||
- "Cargo.toml"
|
||||
- "Cargo.lock"
|
||||
- "build.rs"
|
||||
- "rustfmt.toml"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write apps/rust-auth/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "rust"
|
||||
platform: "system"
|
||||
tags:
|
||||
- "lang:rust"
|
||||
- "type:backend"
|
||||
|
||||
fileGroups:
|
||||
sources:
|
||||
- "src/**/*.rs"
|
||||
- "Cargo.toml"
|
||||
- "Cargo.lock"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Write apps/leptos/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "rust"
|
||||
platform: "system"
|
||||
tags:
|
||||
- "lang:rust"
|
||||
- "type:frontend"
|
||||
|
||||
dependsOn:
|
||||
- id: "rust"
|
||||
|
||||
fileGroups:
|
||||
sources:
|
||||
- "src/**/*.rs"
|
||||
- "Cargo.toml"
|
||||
- "Cargo.lock"
|
||||
- "Trunk.toml"
|
||||
- "rust-toolchain.toml"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/rust/moon.yml apps/rust-auth/moon.yml apps/leptos/moon.yml
|
||||
git commit -m "chore: add moon.yml for Rust apps (rust, rust-auth, leptos)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Create moon.yml for 9router
|
||||
|
||||
**Files:**
|
||||
- Create: `apps/9router/moon.yml`
|
||||
|
||||
- [ ] **Step 1: Write apps/9router/moon.yml**
|
||||
|
||||
```yaml
|
||||
# https://moonrepo.dev/docs/config/project
|
||||
$schema: "https://moonrepo.dev/schemas/project.json"
|
||||
|
||||
type: "application"
|
||||
language: "unknown"
|
||||
platform: "system"
|
||||
tags:
|
||||
- "type:router"
|
||||
|
||||
fileGroups:
|
||||
sources:
|
||||
- "src/**/*"
|
||||
- "next.config.mjs"
|
||||
- "package.json"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add apps/9router/moon.yml
|
||||
git commit -m "chore: add moon.yml for 9router"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Update flake.nix — add moon CLI to devShell
|
||||
|
||||
**Files:**
|
||||
- Modify: `flake.nix`
|
||||
|
||||
- [ ] **Step 1: Add moon to nativeBuildInputs in flake.nix**
|
||||
|
||||
Find the `devShells.default` block in `flake.nix`. Add `moon` to `nativeBuildInputs`:
|
||||
|
||||
```
|
||||
devShells.default = pkgs.mkShell {
|
||||
name = "ultimate-asepharyana-dev";
|
||||
nativeBuildInputs = with pkgs; [
|
||||
rustToolchain
|
||||
bun
|
||||
nodejs_22
|
||||
pkg-config
|
||||
openssl
|
||||
trunk
|
||||
wasm-bindgen-cli
|
||||
binaryen
|
||||
process-compose
|
||||
mysql84
|
||||
redis
|
||||
minio-client
|
||||
gh
|
||||
git
|
||||
moon # <-- add this line
|
||||
];
|
||||
# ... shellHook unchanged
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify Nix can still evaluate the flake**
|
||||
|
||||
Run: `nix flake check --no-build 2>&1 | head -20`
|
||||
Expected: no evaluation errors
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add flake.nix
|
||||
git commit -m "chore: add moon CLI to Nix devShell"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Update .gitignore
|
||||
|
||||
**Files:**
|
||||
- Modify: `.gitignore`
|
||||
|
||||
- [ ] **Step 1: Add moonrepo cache entries to .gitignore**
|
||||
|
||||
Append to `.gitignore`:
|
||||
|
||||
```gitignore
|
||||
# moonrepo
|
||||
.moon/cache
|
||||
.~moon
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add .gitignore
|
||||
git commit -m "chore: add moonrepo cache entries to .gitignore"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 11: Validate installation with moon check
|
||||
|
||||
**Files:** (none — validation only)
|
||||
|
||||
- [ ] **Step 1: Run moon check**
|
||||
|
||||
```bash
|
||||
moon check
|
||||
```
|
||||
|
||||
Expected: `OK` or zero errors. If warnings about unresolved project IDs appear, verify that `apps/*` glob in `workspace.yml` matches all project directories.
|
||||
|
||||
- [ ] **Step 2: Run moon query projects**
|
||||
|
||||
```bash
|
||||
moon query projects
|
||||
```
|
||||
|
||||
Expected: lists all 7 projects with their tags: nextjs, elysia, solidjs, rust, rust-auth, leptos, 9router
|
||||
|
||||
- [ ] **Step 3: Verify tag queries work**
|
||||
|
||||
```bash
|
||||
moon query projects --tag lang:typescript
|
||||
```
|
||||
|
||||
Expected: nextjs, elysia, solidjs
|
||||
|
||||
```bash
|
||||
moon query projects --tag lang:rust
|
||||
```
|
||||
|
||||
Expected: rust, rust-auth, leptos
|
||||
|
||||
- [ ] **Step 4: Commit (if any fixes were needed)**
|
||||
|
||||
No commit needed if check passes clean.
|
||||
|
||||
---
|
||||
|
||||
### Task 12: Test — moon run build on TypeScript apps
|
||||
|
||||
**Files:** (none — test only)
|
||||
|
||||
- [ ] **Step 1: Build nextjs**
|
||||
|
||||
```bash
|
||||
moon run nextjs:build
|
||||
```
|
||||
|
||||
Expected: `next build` runs successfully, outputs to `.next/`
|
||||
|
||||
- [ ] **Step 2: Build elysia**
|
||||
|
||||
```bash
|
||||
moon run elysia:build
|
||||
```
|
||||
|
||||
Expected: `bun build` runs successfully, outputs to `dist/`
|
||||
|
||||
- [ ] **Step 3: Build solidjs**
|
||||
|
||||
```bash
|
||||
moon run solidjs:build
|
||||
```
|
||||
|
||||
Expected: `vinxi build` runs successfully, outputs to `.output/`
|
||||
|
||||
- [ ] **Step 4: Verify caching on second build (nextjs)**
|
||||
|
||||
```bash
|
||||
moon run nextjs:build
|
||||
```
|
||||
|
||||
Expected: `Cached` — no rebuild, uses moonrepo cache
|
||||
|
||||
---
|
||||
|
||||
### Task 13: Test — moon run lint and test
|
||||
|
||||
**Files:** (none — test only)
|
||||
|
||||
- [ ] **Step 1: Run lint across TypeScript apps**
|
||||
|
||||
```bash
|
||||
moon run :lint
|
||||
```
|
||||
|
||||
Expected: all apps with a `lint` task run it. Note failures as they exist pre-migration (not caused by moonrepo).
|
||||
|
||||
- [ ] **Step 2: Run tests across TypeScript apps**
|
||||
|
||||
```bash
|
||||
moon run :test
|
||||
```
|
||||
|
||||
Expected: all apps with a `test` task run it.
|
||||
|
||||
- [ ] **Step 3: Run tag-scoped commands**
|
||||
|
||||
```bash
|
||||
moon run --tag lang:typescript :build
|
||||
```
|
||||
|
||||
Expected: builds nextjs, elysia, solidjs only (not Rust apps).
|
||||
|
||||
---
|
||||
|
||||
### Task 14: Final commit and documentation
|
||||
|
||||
**Files:**
|
||||
- Modify: `.gitignore` (if any final updates)
|
||||
|
||||
- [ ] **Step 1: Final moon check**
|
||||
|
||||
```bash
|
||||
moon check
|
||||
```
|
||||
|
||||
Expected: clean, no errors.
|
||||
|
||||
- [ ] **Step 2: Commit any remaining changes**
|
||||
|
||||
```bash
|
||||
git status
|
||||
```
|
||||
|
||||
If nothing outstanding, move on.
|
||||
|
||||
- [ ] **Step 3: Verify the full workspace is clean**
|
||||
|
||||
```bash
|
||||
git status
|
||||
```
|
||||
|
||||
Expected: working tree clean.
|
||||
@@ -0,0 +1,712 @@
|
||||
# PostgreSQL Migration Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Migrate elysia (Drizzle ORM + MySQL) and rust (SeaORM + MySQL) apps to PostgreSQL via direct cutover.
|
||||
|
||||
**Architecture:** Export MySQL data, convert schema to PostgreSQL format, import into target PostgreSQL instance, update app drivers and connection strings, deploy and validate.
|
||||
|
||||
**Tech Stack:** MySQL (source), PostgreSQL (target), Drizzle ORM (elysia), SeaORM (rust), pgloader or manual conversion for schema migration.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
### Elysia App Changes
|
||||
- `apps/elysia/src/db/lib/database.ts` — Replace mysql2 driver with postgres driver
|
||||
- `apps/elysia/src/db/lib/schema.ts` — Replace mysqlTable with pgTable, update column types
|
||||
- `apps/elysia/package.json` — Replace mysql2 with pg dependency
|
||||
- `.env` or config file — Update DATABASE_URL to PostgreSQL connection string
|
||||
|
||||
### Rust App Changes
|
||||
- `apps/rust/Cargo.toml` — Replace sqlx-mysql feature with sqlx-postgres
|
||||
- `apps/rust/src/infra/db_setup.rs` — Update DbBackend::MySql to DbBackend::Postgres, adjust SQL syntax
|
||||
- Config/environment — Update DATABASE_URL to PostgreSQL connection string
|
||||
|
||||
### Migration Artifacts
|
||||
- `mysql_backup.sql` — MySQL dump (created during migration, not committed)
|
||||
- `converted.sql` — PostgreSQL-compatible dump (created during migration, not committed)
|
||||
|
||||
---
|
||||
|
||||
## Task Breakdown
|
||||
|
||||
### Task 1: Backup MySQL and Export Schema
|
||||
|
||||
**Files:**
|
||||
- Create: `mysql_backup.sql` (temporary, not committed)
|
||||
|
||||
- [ ] **Step 1: Export MySQL database**
|
||||
|
||||
```bash
|
||||
cd /mnt/code/bp3/ultimate-asepharyana.tech
|
||||
mysqldump -u <mysql_user> -p <mysql_password> -h <mysql_host> <database_name> > mysql_backup.sql
|
||||
```
|
||||
|
||||
Expected: File created with full schema + data. Verify file size > 1MB (contains data).
|
||||
|
||||
-[]**Step 2: Verify backup integrity**
|
||||
|
||||
```bash
|
||||
# Check row counts in backup
|
||||
grep "INSERT INTO" mysql_backup.sql | wc -l
|
||||
```
|
||||
|
||||
Expected: Multiple INSERT statements present. Note row counts for later validation.
|
||||
|
||||
- [ ] **Step 3: Document backup location**
|
||||
|
||||
Store `mysql_backup.sql` in safe location (not in git). This is rollback insurance.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Convert MySQL Schema to PostgreSQL
|
||||
|
||||
**Files:**
|
||||
- Create: `converted.sql` (temporary, not committed)
|
||||
|
||||
- [ ] **Step 1: Install pgloader (if not present)**
|
||||
|
||||
```bash
|
||||
# macOS
|
||||
brew install pgloader
|
||||
|
||||
# Linux (Ubuntu/Debian)
|
||||
sudo apt-get install pgloader
|
||||
|
||||
# Or use Docker
|
||||
docker run --rm -v $(pwd):/data pgloader/pgloader pgloader /data/mysql_backup.sql postgresql://asephs:hunterz@100.108.1.124:5432/hub
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Convert MySQL dump to PostgreSQL**
|
||||
|
||||
```bash
|
||||
pgloader mysql_backup.sql postgresql://asephs:hunterz@100.108.1.124:5432/hub
|
||||
```
|
||||
|
||||
Or manually convert if pgloader unavailable:
|
||||
- Replace `AUTO_INCREMENT` with `SERIAL` or `BIGSERIAL`
|
||||
- Replace `DATETIME` with `TIMESTAMP`
|
||||
- Replace backticks with double quotes
|
||||
- Update index syntax for PostgreSQL
|
||||
|
||||
Expected: Conversion completes without errors. Check for warnings about type conversions.
|
||||
|
||||
- [ ] **Step 3: Verify conversion output**
|
||||
|
||||
```bash
|
||||
# If using pgloader, it creates converted.sql automatically
|
||||
# If manual, save converted schema to file
|
||||
cat converted.sql | head -50
|
||||
```
|
||||
|
||||
Expected: PostgreSQL-compatible SQL syntax (no backticks, SERIAL types, TIMESTAMP).
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Test PostgreSQL Import
|
||||
|
||||
**Files:**
|
||||
- Target: PostgreSQL instance at `postgresql://asephs:hunterz@100.108.1.124:5432/hub`
|
||||
|
||||
- [ ] **Step 1: Connect to target PostgreSQL**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub
|
||||
```
|
||||
|
||||
Expected: Connected to PostgreSQL. Prompt shows `hub=#`.
|
||||
|
||||
- [ ] **Step 2: Import converted schema**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub < converted.sql
|
||||
```
|
||||
|
||||
Expected: Import completes. Check for errors (should be none).
|
||||
|
||||
- [ ] **Step 3: Validate table creation**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "\dt"
|
||||
```
|
||||
|
||||
Expected: All tables listed (User, Account, Session, Role, Permission, UserRole, ImageCache, etc.).
|
||||
|
||||
- []**Step 4: Validate row counts**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) FROM \"User\";"
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) FROM \"Account\";"
|
||||
```
|
||||
|
||||
Expected: Row counts match MySQL backup (from Task 1, Step 2).
|
||||
|
||||
- [ ] **Step 5: Validate indexes**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "\di"
|
||||
```
|
||||
|
||||
Expected: All indexes present (email_idx, username_idx, userId_idx, sessionToken_idx, etc.).
|
||||
|
||||
- [ ] **Step 6: Validate foreign keys**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT constraint_name, table_name FROM information_schema.table_constraints WHERE constraint_type = 'FOREIGN KEY';"
|
||||
```
|
||||
|
||||
Expected: Foreign key constraints listed (User→Account, User→Session, etc.).
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Update Elysia Database Driver
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/elysia/src/db/lib/database.ts`
|
||||
- Modify: `apps/elysia/src/db/lib/schema.ts`
|
||||
- Modify: `apps/elysia/package.json`
|
||||
|
||||
- [ ] **Step 1: Update package.json dependencies**
|
||||
|
||||
Replace mysql2 with pg:
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": {
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"pg": "^8.11.0",
|
||||
"elysia": "^1.4.28"
|
||||
},
|
||||
"devDependencies": {
|
||||
"drizzle-kit": "^0.31.10"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run: `cd apps/elysia && bun install`
|
||||
|
||||
Expected: pg installed, mysql2 removed from node_modules.
|
||||
|
||||
- [ ] **Step 2: Update database.ts driver**
|
||||
|
||||
Replace entire file:
|
||||
|
||||
```typescript
|
||||
import type { PostgresJsDatabase } from 'drizzle-orm/postgres-js'
|
||||
import { drizzle } from 'drizzle-orm/postgres-js'
|
||||
import postgres from 'postgres'
|
||||
import * as schema from './schema'
|
||||
|
||||
export type Database = PostgresJsDatabase<typeof schema>
|
||||
|
||||
let dbInstance: Database | null = null
|
||||
let sqlInstance: ReturnType<typeof postgres> | null = null
|
||||
|
||||
export function initializeDb(databaseUrl: string): Database {
|
||||
if (dbInstance) {
|
||||
return dbInstance
|
||||
}
|
||||
|
||||
sqlInstance = postgres(databaseUrl)
|
||||
dbInstance = drizzle(sqlInstance, { schema, mode: 'default' })
|
||||
|
||||
return dbInstance
|
||||
}
|
||||
|
||||
export function getDb(): Database {
|
||||
if (!dbInstance) {
|
||||
throw new Error('Database not initialized. Call initializeDb first.')
|
||||
}
|
||||
return dbInstance
|
||||
}
|
||||
|
||||
export async function closeDb() {
|
||||
if (sqlInstance) {
|
||||
await sqlInstance.end()
|
||||
sqlInstance = null
|
||||
dbInstance = null
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Expected: File updated. No syntax errors.
|
||||
|
||||
- [ ] **Step 3: Update schema.ts imports**
|
||||
|
||||
Replace:
|
||||
```typescript
|
||||
import {
|
||||
index,
|
||||
int,
|
||||
mysqlTable,
|
||||
primaryKey,
|
||||
text,
|
||||
timestamp,
|
||||
varchar,
|
||||
} from 'drizzle-orm/mysql-core'
|
||||
```
|
||||
|
||||
With:
|
||||
```typescript
|
||||
import {
|
||||
index,
|
||||
integer,
|
||||
pgTable,
|
||||
primaryKey,
|
||||
text,
|
||||
timestamp,
|
||||
varchar,
|
||||
} from 'drizzle-orm/postgres-core'
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update schema.ts table definitions**
|
||||
|
||||
Replace all `mysqlTable` with `pgTable` and `int` with `integer`:
|
||||
|
||||
```typescript
|
||||
// Before
|
||||
export const users = mysqlTable(
|
||||
'User',
|
||||
{
|
||||
id: varchar('id', { length: 255 }).primaryKey(),
|
||||
name: varchar('name', { length: 255 }),
|
||||
// ...
|
||||
},
|
||||
// ...
|
||||
)
|
||||
|
||||
// After
|
||||
export const users = pgTable(
|
||||
'User',
|
||||
{
|
||||
id: varchar('id', { length: 255 }).primaryKey(),
|
||||
name: varchar('name', { length: 255 }),
|
||||
// ...
|
||||
},
|
||||
// ...
|
||||
)
|
||||
```
|
||||
|
||||
Do this for all tables: users, accounts, sessions, roles, permissions, userRoles, and any others.
|
||||
|
||||
Expected: All `mysqlTable` → `pgTable`, all `int` → `integer`.
|
||||
|
||||
- [ ] **Step 5: Update environment variable**
|
||||
|
||||
Set DATABASE_URL in `.env` or deployment config:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgresql://asephs:hunterz@100.108.1.124:5432/hub
|
||||
```
|
||||
|
||||
Expected: Environment variable set and accessible to elysia app.
|
||||
|
||||
- [ ] **Step 6: Test elysia connection**
|
||||
|
||||
```bash
|
||||
cd apps/elysia
|
||||
bun run src/index.ts
|
||||
```
|
||||
|
||||
Expected: App starts without connection errors. Check logs for "Database initialized" or similar.
|
||||
|
||||
- [ ] **Step 7: Commit elysia changes**
|
||||
|
||||
```bash
|
||||
cd /mnt/code/bp3/ultimate-asepharyana.tech
|
||||
git add apps/elysia/src/db/lib/database.ts apps/elysia/src/db/lib/schema.ts apps/elysia/package.json
|
||||
git commit -m "feat(elysia): migrate database driver from MySQL to PostgreSQL"
|
||||
```
|
||||
|
||||
Expected: Commit created with message.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Update Rust Database Driver
|
||||
|
||||
**Files:**
|
||||
- Modify: `apps/rust/Cargo.toml`
|
||||
- Modify: `apps/rust/src/infra/db_setup.rs`
|
||||
|
||||
- [ ] **Step 1: Update Cargo.toml features**
|
||||
|
||||
Replace:
|
||||
```toml
|
||||
sea-orm = { version = "1.1.19", features = ["sqlx-mysql", "runtime-tokio-rustls", "macros", "with-chrono", "with-uuid"] }
|
||||
```
|
||||
|
||||
With:
|
||||
```toml
|
||||
sea-orm = { version = "1.1.19", features = ["sqlx-postgres", "runtime-tokio-rustls", "macros", "with-chrono", "with-uuid"] }
|
||||
```
|
||||
|
||||
Expected: Cargo.toml updated. Feature changed from sqlx-mysql to sqlx-postgres.
|
||||
|
||||
- [ ] **Step 2: Update db_setup.rs backend check**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
match backend {
|
||||
DbBackend::MySql => {
|
||||
// MySQL-specific logic
|
||||
}
|
||||
_ => {
|
||||
info!("ℹ️ Skipping schema init for non-MySQL backend");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
With:
|
||||
```rust
|
||||
match backend {
|
||||
DbBackend::Postgres => {
|
||||
// PostgreSQL-specific logic
|
||||
let tables = vec![(
|
||||
"ImageCache",
|
||||
schema
|
||||
.create_table_from_entity(image_cache::Entity)
|
||||
.if_not_exists()
|
||||
.to_owned(),
|
||||
)];
|
||||
|
||||
for (name, stmt) in tables {
|
||||
match db.execute(backend.build(&stmt)).await {
|
||||
Ok(_) => info!(" ✓ Table '{}' checked/created", name),
|
||||
Err(e) => {
|
||||
error!(" [!] Failed to create table '{}': {}", name, e);
|
||||
return Err(e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// PostgreSQL index creation (different syntax)
|
||||
let index_sql = "CREATE INDEX IF NOT EXISTS idx_image_cache_cdn_url ON \"ImageCache\" (cdn_url)";
|
||||
match db.execute(Statement::from_string(backend, index_sql)).await {
|
||||
Ok(_) => info!(" ✓ Index 'idx_image_cache_cdn_url' ensured"),
|
||||
Err(e) => {
|
||||
let err_str = e.to_string();
|
||||
// PostgreSQL duplicate index error
|
||||
if err_str.contains("already exists") {
|
||||
info!(" ✓ Index 'idx_image_cache_cdn_url' already exists");
|
||||
} else {
|
||||
error!(" [!] Failed to create index on ImageCache: {}", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!("✅ Database schema initialization complete.");
|
||||
}
|
||||
_ => {
|
||||
info!("ℹ️ Skipping schema init for non-PostgreSQL backend");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Expected: db_setup.rs updated with PostgreSQL backend handling.
|
||||
|
||||
- [ ] **Step 3: Update environment variable**
|
||||
|
||||
Set DATABASE_URL in `.env` or deployment config:
|
||||
|
||||
```bash
|
||||
DATABASE_URL=postgresql://asephs:hunterz@100.108.1.124:5432/hub
|
||||
```
|
||||
|
||||
Expected: Environment variable set and accessible to rust app.
|
||||
|
||||
- [ ] **Step 4: Rebuild rust app**
|
||||
|
||||
```bash
|
||||
cd apps/rust
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
Expected: Build completes without errors. Compilation uses sqlx-postgres feature.
|
||||
|
||||
- [ ] **Step 5: Test rust connection**
|
||||
|
||||
```bash
|
||||
cd apps/rust
|
||||
cargo run
|
||||
```
|
||||
|
||||
Expected: App starts without connection errors. Check logs for "Database schema initialization complete" or similar.
|
||||
|
||||
- [ ] **Step 6: Commit rust changes**
|
||||
|
||||
```bash
|
||||
cd /mnt/code/bp3/ultimate-asepharyana.tech
|
||||
git add apps/rust/Cargo.toml apps/rust/src/infra/db_setup.rs
|
||||
git commit -m "feat(rust): migrate database driver from MySQL to PostgreSQL"
|
||||
```
|
||||
|
||||
Expected: Commit created with message.
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Smoke Tests - Elysia App
|
||||
|
||||
**Files:**
|
||||
- Test: Manual testing via HTTP requests or app UI
|
||||
|
||||
- [ ] **Step 1: Start elysia app**
|
||||
|
||||
```bash
|
||||
cd apps/elysia
|
||||
bun run src/index.ts
|
||||
```
|
||||
|
||||
Expected: App running on configured port (check logs for port).
|
||||
|
||||
- [ ] **Step 2: Test user login**
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/auth/login \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"email":"test@example.com","password":"password"}'
|
||||
```
|
||||
|
||||
Expected: Response 200 or 401 (auth error is OK, connection error is not).
|
||||
|
||||
-[]**Step 3: Test user creation (if endpoint exists)**
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:3000/users \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"Test User","email":"newuser@example.com"}'
|
||||
```
|
||||
|
||||
Expected: Response 200/201 or 400 (validation error is OK).
|
||||
|
||||
- [ ] **Step 4: Test session retrieval**
|
||||
|
||||
```bash
|
||||
curl -X GET http://localhost:3000/sessions \
|
||||
-H "Authorization: Bearer <token>"
|
||||
```
|
||||
|
||||
Expected: Response 200 with session data or 401 (auth error is OK).
|
||||
|
||||
- [ ] **Step 5: Check database logs**
|
||||
|
||||
```bash
|
||||
# In elysia app logs, verify queries are executing against PostgreSQL
|
||||
# Look for connection strings or query logs showing PostgreSQL
|
||||
```
|
||||
|
||||
Expected: Logs show PostgreSQL queries (not MySQL).
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Smoke Tests - Rust App
|
||||
|
||||
**Files:**
|
||||
- Test: Manual testing via HTTP requests or app UI
|
||||
|
||||
- [ ] **Step 1: Start rust app**
|
||||
|
||||
```bash
|
||||
cd apps/rust
|
||||
cargo run --release
|
||||
```
|
||||
|
||||
Expected: App running on configured port (check logs for port).
|
||||
|
||||
- [ ] **Step 2: Test image cache endpoint (if exists)**
|
||||
|
||||
```bash
|
||||
curl -X GET http://localhost:8000/api/cache/status
|
||||
```
|
||||
|
||||
Expected: Response 200 with cache status or 404 (endpoint may not exist).
|
||||
|
||||
- [ ] **Step 3: Test scraping/CDN endpoint**
|
||||
|
||||
```bash
|
||||
curl -X GET http://localhost:8000/api/health
|
||||
```
|
||||
|
||||
Expected: Response 200 with health status.
|
||||
|
||||
- [ ] **Step 4: Check database logs**
|
||||
|
||||
```bash
|
||||
# In rust app logs, verify queries are executing against PostgreSQL
|
||||
# Look for connection strings or query logs showing PostgreSQL
|
||||
```
|
||||
|
||||
Expected: Logs show PostgreSQL queries (not MySQL).
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Data Integrity Validation
|
||||
|
||||
**Files:**
|
||||
- Test: PostgreSQL queries
|
||||
|
||||
- [] **Step 1: Verify user count**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) as user_count FROM \"User\";"
|
||||
```
|
||||
|
||||
Expected: Count matches MySQL backup count (from Task 1, Step 2).
|
||||
|
||||
- [ ] **Step 2: Verify account count**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) as account_count FROM \"Account\";"
|
||||
```
|
||||
|
||||
Expected: Count matches MySQL backup.
|
||||
|
||||
- [ ] **Step 3: Verify session count**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) as session_count FROM \"Session\";"
|
||||
```
|
||||
|
||||
Expected: Count matches MySQL backup.
|
||||
|
||||
- [] **Step 4: Verify no orphaned foreign keys**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "
|
||||
SELECT a.id FROM \"Account\" a
|
||||
LEFT JOIN \"User\" u ON a.user_id = u.id
|
||||
WHERE u.id IS NULL;
|
||||
"
|
||||
```
|
||||
|
||||
Expected: No rows returned (no orphaned accounts).
|
||||
|
||||
- [ ] **Step 5: Verify role/permission relationships**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) as role_count FROM \"Role\";"
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "SELECT COUNT(*) as permission_count FROM \"Permission\";"
|
||||
```
|
||||
|
||||
Expected: Counts match MySQL backup.
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Performance Baseline
|
||||
|
||||
**Files:**
|
||||
- Test: Query performance comparison
|
||||
|
||||
- [ ] **Step 1: Benchmark user query on PostgreSQL**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "EXPLAIN ANALYZE SELECT * FROM \"User\" WHERE email = 'test@example.com';"
|
||||
```
|
||||
|
||||
Expected: Query plan shows index usage (Seq Scan or Index Scan). Note execution time.
|
||||
|
||||
- [ ] **Step 2: Benchmark account query on PostgreSQL**
|
||||
|
||||
```bash
|
||||
psql postgresql://asephs:hunterz@100.108.1.124:5432/hub -c "EXPLAIN ANALYZE SELECT * FROM \"Account\" WHERE user_id = 'user-123';"
|
||||
```
|
||||
|
||||
Expected: Query plan shows index usage. Note execution time.
|
||||
|
||||
- [ ] **Step 3: Compare with MySQL baseline (if available)**
|
||||
|
||||
If MySQL is still running, run same queries and compare execution times.
|
||||
|
||||
Expected: PostgreSQL performance similar or better than MySQL.
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Cleanup and Documentation
|
||||
|
||||
**Files:**
|
||||
- Create: `MIGRATION_LOG.md` (optional, for documentation)
|
||||
|
||||
- [ ] **Step 1: Remove temporary files**
|
||||
|
||||
```bash
|
||||
rm mysql_backup.sql converted.sql
|
||||
```
|
||||
|
||||
Expected: Temporary migration files deleted.
|
||||
|
||||
- [ ] **Step 2: Document migration completion**
|
||||
|
||||
Create `MIGRATION_LOG.md`:
|
||||
|
||||
```markdown
|
||||
# PostgreSQL Migration Log
|
||||
|
||||
**Date:** 2026-05-25
|
||||
**Status:** ✅ Complete
|
||||
|
||||
## Summary
|
||||
- Migrated elysia app from MySQL to PostgreSQL
|
||||
- Migrated rust app from MySQL to PostgreSQL
|
||||
- All data validated and integrity confirmed
|
||||
- Apps tested and operational
|
||||
|
||||
## Changes
|
||||
- elysia: Updated database driver (mysql2 → postgres), schema (mysqlTable → pgTable)
|
||||
- rust: Updated Cargo.toml feature (sqlx-mysql → sqlx-postgres), db_setup.rs backend handling
|
||||
|
||||
## Validation
|
||||
- Row counts match pre-migration
|
||||
- Foreign keys intact
|
||||
- Indexes present and performant
|
||||
- Auth flow functional
|
||||
- Session management working
|
||||
|
||||
## Rollback
|
||||
MySQL backup available at: [location if kept]
|
||||
To rollback: Restore MySQL from backup, revert connection strings, redeploy apps.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Final commit**
|
||||
|
||||
```bash
|
||||
git add MIGRATION_LOG.md
|
||||
git commit -m "docs: add PostgreSQL migration completion log"
|
||||
```
|
||||
|
||||
Expected: Commit created.
|
||||
|
||||
- [ ] **Step 4: Verify all apps running**
|
||||
|
||||
```bash
|
||||
# Check elysia
|
||||
curl http://localhost:3000/health
|
||||
|
||||
# Check rust
|
||||
curl http://localhost:8000/health
|
||||
```
|
||||
|
||||
Expected: Both apps respond with 200 status.
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec Coverage:**
|
||||
- ✅ Pre-migration (backup, convert, test) — Tasks 1-3
|
||||
- ✅ Elysia code updates (driver, schema, env) — Task 4
|
||||
- ✅ Rust code updates (Cargo.toml, db_setup.rs, env) — Task 5
|
||||
- ✅ Smoke tests (auth, endpoints, logs) — Tasks 6-7
|
||||
- ✅ Data validation (row counts, foreign keys) — Task 8
|
||||
- ✅ Performance baseline — Task 9
|
||||
- ✅ Cleanup and documentation — Task 10
|
||||
|
||||
**Placeholder Scan:**
|
||||
- ✅ No TBD/TODO
|
||||
- ✅ All code blocks complete
|
||||
- ✅ All commands exact with expected output
|
||||
- ✅ All file paths exact
|
||||
|
||||
**Type Consistency:**
|
||||
- ✅ Database type: `PostgresJsDatabase` (elysia), `DbBackend::Postgres` (rust)
|
||||
- ✅ Connection string format consistent: `postgresql://asephs:hunterz@100.108.1.124:5432/hub`
|
||||
- ✅ Table names consistent: "User", "Account", "Session", etc.
|
||||
@@ -0,0 +1,533 @@
|
||||
# Production GitHub Actions Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Rework GitHub Actions build/deploy workflows into a production baseline with reliable React submodule updates, least-privilege permissions, clear deploy behavior, and current official action versions.
|
||||
|
||||
**Architecture:** Keep two workflows: `docker-build-push.yml` for detect/build/manifest updates, and `deploy-docker.yml` for VPS deployment. Add dispatch submodule SHA readiness checks before parent pointer updates, and remove recursive submodule checkout from deploy runner.
|
||||
|
||||
**Tech Stack:** GitHub Actions YAML, GitHub-hosted Ubuntu runners, Docker Buildx, GHCR, git submodules, Docker Compose over SSH.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Modify `.github/workflows/docker-build-push.yml`: add default permissions, validate repository dispatch payloads, wait for submodule SHAs, keep selective matrix builds, harden manifest update.
|
||||
- Modify `.github/workflows/deploy-docker.yml`: add default permissions, remove recursive checkout, keep auto deploy from successful build, make deploy logs clearer.
|
||||
- Modify `.github/dependabot.yml`: add GitHub Actions update config so official actions stay current.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Harden build workflow permissions and dispatch validation
|
||||
|
||||
**Files:**
|
||||
- Modify: `.github/workflows/docker-build-push.yml`
|
||||
|
||||
- [ ] **Step 1: Add workflow-level read permissions**
|
||||
|
||||
At top level, after `concurrency`, add:
|
||||
|
||||
```yaml
|
||||
permissions:
|
||||
contents: read
|
||||
```
|
||||
|
||||
Expected shape:
|
||||
|
||||
```yaml
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
REGISTRY: ghcr.io
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace dispatch parser with payload validation**
|
||||
|
||||
In `.github/workflows/docker-build-push.yml`, replace `Parse repository_dispatch payload` step body with:
|
||||
|
||||
```yaml
|
||||
- name: Parse repository_dispatch payload
|
||||
id: dispatch
|
||||
if: github.event_name == 'repository_dispatch'
|
||||
env:
|
||||
SERVICE: ${{ github.event.client_payload.service }}
|
||||
SHA: ${{ github.event.client_payload.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
if [ -z "${SERVICE:-}" ]; then
|
||||
echo "::error::repository_dispatch payload missing service"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "${SHA:-}" ]; then
|
||||
echo "::error::repository_dispatch payload missing sha"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
case "$SERVICE" in
|
||||
rust-api|elysia-api|react-web|9router) ;;
|
||||
*)
|
||||
echo "::error::Unsupported service '$SERVICE'. Expected one of: rust-api, elysia-api, react-web, 9router"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$SHA" in
|
||||
*[!0-9a-fA-F]*|???????????????????????????????????????|?????????????????????????????????????????*)
|
||||
echo "::error::Invalid sha '$SHA'. Expected 40 hex characters"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
declare -a SERVICES=("rust-api" "elysia-api" "react-web" "9router")
|
||||
for svc in "${SERVICES[@]}"; do
|
||||
if [ "$SERVICE" = "$svc" ]; then
|
||||
echo "${svc}=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "${svc}=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
done
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run YAML syntax check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
for path in Path('.github/workflows').glob('*.yml'):
|
||||
yaml.safe_load(path.read_text())
|
||||
print(f'OK {path}')
|
||||
PY
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
OK .github/workflows/deploy-docker.yml
|
||||
OK .github/workflows/docker-build-push.yml
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/docker-build-push.yml
|
||||
git commit -m "ci: validate dispatch payloads"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add submodule SHA readiness wait before build/update
|
||||
|
||||
**Files:**
|
||||
- Modify: `.github/workflows/docker-build-push.yml`
|
||||
|
||||
- [ ] **Step 1: Add readiness job after changes job**
|
||||
|
||||
Insert this job between `changes` and `build`:
|
||||
|
||||
```yaml
|
||||
wait-submodule-ref:
|
||||
needs: [changes]
|
||||
if: github.event_name == 'repository_dispatch'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Wait for submodule ref
|
||||
env:
|
||||
SERVICE: ${{ github.event.client_payload.service }}
|
||||
SHA: ${{ github.event.client_payload.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
case "$SERVICE" in
|
||||
"rust-api") REPO="https://github.com/MythEclipse/ultimate-asepharyana-tech-rust.git" ;;
|
||||
"elysia-api") REPO="https://github.com/MythEclipse/ultimate-asepharyana-tech-elysia.git" ;;
|
||||
"react-web") REPO="https://github.com/MythEclipse/ultimate-asepharyana-tech-react.git" ;;
|
||||
"9router") REPO="https://github.com/MythEclipse/9router.git" ;;
|
||||
*)
|
||||
echo "::error::Unsupported service '$SERVICE'"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Waiting for $SERVICE ref $SHA in $REPO"
|
||||
for attempt in {1..30}; do
|
||||
if git ls-remote --exit-code "$REPO" "$SHA" >/dev/null 2>&1; then
|
||||
echo "Submodule ref $SHA is fetchable for $SERVICE"
|
||||
exit 0
|
||||
fi
|
||||
echo "Attempt $attempt/30: $SHA not visible yet; waiting 10s"
|
||||
sleep 10
|
||||
done
|
||||
|
||||
echo "::error::Submodule ref $SHA for $SERVICE was not fetchable after 300s"
|
||||
exit 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Make build wait for readiness job without blocking push/manual events**
|
||||
|
||||
Change build job header from:
|
||||
|
||||
```yaml
|
||||
build:
|
||||
needs: [changes]
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```yaml
|
||||
build:
|
||||
needs: [changes, wait-submodule-ref]
|
||||
if: |
|
||||
always() &&
|
||||
needs.changes.result == 'success' &&
|
||||
(needs.wait-submodule-ref.result == 'success' || needs.wait-submodule-ref.result == 'skipped') &&
|
||||
needs.changes.outputs.matrix != '[]'
|
||||
```
|
||||
|
||||
Remove existing build-level line:
|
||||
|
||||
```yaml
|
||||
if: needs.changes.outputs.matrix != '[]'
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Make update-manifest wait for readiness job**
|
||||
|
||||
Change update-manifest header from:
|
||||
|
||||
```yaml
|
||||
update-manifest:
|
||||
needs: [changes, build]
|
||||
if: |
|
||||
always() &&
|
||||
(needs.build.result == 'success' || needs.build.result == 'skipped')
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```yaml
|
||||
update-manifest:
|
||||
needs: [changes, wait-submodule-ref, build]
|
||||
if: |
|
||||
always() &&
|
||||
needs.changes.result == 'success' &&
|
||||
(needs.wait-submodule-ref.result == 'success' || needs.wait-submodule-ref.result == 'skipped') &&
|
||||
(needs.build.result == 'success' || needs.build.result == 'skipped')
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run YAML syntax check**
|
||||
|
||||
Run same command from Task 1 Step 3.
|
||||
|
||||
Expected both workflow files print `OK`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/docker-build-push.yml
|
||||
git commit -m "ci: wait for submodule refs before builds"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Harden manifest update and submodule checkout
|
||||
|
||||
**Files:**
|
||||
- Modify: `.github/workflows/docker-build-push.yml`
|
||||
|
||||
- [ ] **Step 1: Add job permissions to build and manifest jobs**
|
||||
|
||||
Ensure build job contains:
|
||||
|
||||
```yaml
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
```
|
||||
|
||||
Ensure update-manifest job contains:
|
||||
|
||||
```yaml
|
||||
permissions:
|
||||
contents: write
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace dispatch submodule checkout block**
|
||||
|
||||
Inside `Update tags and submodules`, replace the repository_dispatch submodule update block with:
|
||||
|
||||
```bash
|
||||
if [ "${{ github.event_name }}" == "repository_dispatch" ] && [ "${{ github.event.client_payload.service }}" == "$id" ]; then
|
||||
SHA_DISPATCH="${{ github.event.client_payload.sha }}"
|
||||
SUB_PATH="${PATHS[$id]}"
|
||||
if [ -n "$SHA_DISPATCH" ]; then
|
||||
echo "Updating submodule $SUB_PATH to $SHA_DISPATCH"
|
||||
git submodule update --init "$SUB_PATH"
|
||||
git -C "$SUB_PATH" fetch origin "$SHA_DISPATCH"
|
||||
git -C "$SUB_PATH" checkout "$SHA_DISPATCH"
|
||||
git add "$SUB_PATH"
|
||||
CHANGED=true
|
||||
fi
|
||||
fi
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add pull/rebase retry before push**
|
||||
|
||||
Replace:
|
||||
|
||||
```bash
|
||||
git commit -m "chore: update manifests and submodules [skip ci]"
|
||||
git pull --rebase origin main
|
||||
git push origin main
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```bash
|
||||
git commit -m "chore: update manifests and submodules [skip ci]"
|
||||
|
||||
for attempt in {1..3}; do
|
||||
if git pull --rebase origin main && git push origin main; then
|
||||
exit 0
|
||||
fi
|
||||
echo "Manifest push attempt $attempt/3 failed; retrying"
|
||||
git rebase --abort || true
|
||||
git pull --rebase origin main || true
|
||||
sleep 5
|
||||
done
|
||||
|
||||
echo "::error::Failed to push manifest update after 3 attempts"
|
||||
exit 1
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run YAML syntax check**
|
||||
|
||||
Run same command from Task 1 Step 3.
|
||||
|
||||
Expected both workflow files print `OK`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/docker-build-push.yml
|
||||
git commit -m "ci: harden manifest updates"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Make deploy checkout submodule-free and least privilege
|
||||
|
||||
**Files:**
|
||||
- Modify: `.github/workflows/deploy-docker.yml`
|
||||
|
||||
- [ ] **Step 1: Add workflow-level read permissions**
|
||||
|
||||
After `concurrency`, add:
|
||||
|
||||
```yaml
|
||||
permissions:
|
||||
contents: read
|
||||
```
|
||||
|
||||
Expected shape:
|
||||
|
||||
```yaml
|
||||
concurrency:
|
||||
group: deploy-vps
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
```
|
||||
|
||||
If current `cancel-in-progress` is `true`, change it to `false`.
|
||||
|
||||
- [ ] **Step 2: Make checkout non-recursive**
|
||||
|
||||
Replace checkout step:
|
||||
|
||||
```yaml
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```yaml
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
submodules: false
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add deploy context log**
|
||||
|
||||
At start of `Deploy with Docker Compose on VPS` run script, after `set -euo pipefail`, add:
|
||||
|
||||
```bash
|
||||
echo "Deploy event: ${{ github.event_name }}"
|
||||
echo "Deploy ref: ${{ github.ref }}"
|
||||
echo "Deploy sha: ${{ github.sha }}"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run YAML syntax check**
|
||||
|
||||
Run same command from Task 1 Step 3.
|
||||
|
||||
Expected both workflow files print `OK`.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/deploy-docker.yml
|
||||
git commit -m "ci: avoid submodule checkout during deploy"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Add GitHub Actions Dependabot updates
|
||||
|
||||
**Files:**
|
||||
- Modify: `.github/dependabot.yml`
|
||||
|
||||
- [ ] **Step 1: Add github-actions ecosystem**
|
||||
|
||||
Append this update entry under `updates:`:
|
||||
|
||||
```yaml
|
||||
- package-ecosystem: 'github-actions'
|
||||
directory: '/'
|
||||
schedule:
|
||||
interval: weekly
|
||||
groups:
|
||||
github-actions:
|
||||
patterns:
|
||||
- '*'
|
||||
```
|
||||
|
||||
Expected file shape:
|
||||
|
||||
```yaml
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: 'devcontainers'
|
||||
directory: '/'
|
||||
schedule:
|
||||
interval: weekly
|
||||
|
||||
- package-ecosystem: 'github-actions'
|
||||
directory: '/'
|
||||
schedule:
|
||||
interval: weekly
|
||||
groups:
|
||||
github-actions:
|
||||
patterns:
|
||||
- '*'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run YAML syntax check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
paths = [Path('.github/dependabot.yml'), *Path('.github/workflows').glob('*.yml')]
|
||||
for path in paths:
|
||||
yaml.safe_load(path.read_text())
|
||||
print(f'OK {path}')
|
||||
PY
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```text
|
||||
OK .github/dependabot.yml
|
||||
OK .github/workflows/deploy-docker.yml
|
||||
OK .github/workflows/docker-build-push.yml
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/dependabot.yml
|
||||
git commit -m "ci: enable github actions dependency updates"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Final validation
|
||||
|
||||
**Files:**
|
||||
- Validate: `.github/workflows/docker-build-push.yml`
|
||||
- Validate: `.github/workflows/deploy-docker.yml`
|
||||
- Validate: `.github/dependabot.yml`
|
||||
|
||||
- [ ] **Step 1: Run YAML syntax check**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
from pathlib import Path
|
||||
import yaml
|
||||
paths = [Path('.github/dependabot.yml'), *Path('.github/workflows').glob('*.yml')]
|
||||
for path in paths:
|
||||
yaml.safe_load(path.read_text())
|
||||
print(f'OK {path}')
|
||||
PY
|
||||
```
|
||||
|
||||
Expected all files print `OK`.
|
||||
|
||||
- [ ] **Step 2: Check workflows recognized by GitHub CLI**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
gh workflow list
|
||||
```
|
||||
|
||||
Expected output includes:
|
||||
|
||||
```text
|
||||
Build and Push Docker Images
|
||||
Deploy Docker to VPS
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Inspect final diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git diff -- .github/workflows .github/dependabot.yml
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `docker-build-push.yml` has dispatch validation, `wait-submodule-ref`, job permissions, and manifest push retry.
|
||||
- `deploy-docker.yml` has non-recursive checkout and read-only permissions.
|
||||
- `dependabot.yml` has `github-actions` updates.
|
||||
|
||||
- [ ] **Step 4: Commit any final validation fixes**
|
||||
|
||||
If Step 1 or Step 2 required fixes, commit them:
|
||||
|
||||
```bash
|
||||
git add .github/workflows .github/dependabot.yml
|
||||
git commit -m "ci: finalize production workflow hardening"
|
||||
```
|
||||
|
||||
If no fixes were needed, do not create an empty commit.
|
||||
@@ -0,0 +1,29 @@
|
||||
# React Direct Runtime and Images Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Serve the React SPA without nginx and remove frontend image-cache proxy calls so browser uses direct image/API URLs.
|
||||
|
||||
**Architecture:** The React Docker image still builds with Bun/Vite, but runtime uses `vite preview` from Bun instead of nginx. `CachedImage` normalizes image URLs and renders them directly, keeping fallback/retry UI but removing `/proxy/image-cache` auditing. Traefik keeps routing `react-web:80`, so compose and dynamic routing stay stable.
|
||||
|
||||
**Tech Stack:** Bun, Vite, React, TypeScript, Docker, Docker Compose, Traefik.
|
||||
|
||||
---
|
||||
|
||||
## Tasks
|
||||
|
||||
### Task 1: Switch React Runtime From nginx to Bun/Vite Preview
|
||||
|
||||
Modify `infra/docker/react.Dockerfile` so runtime stage uses `oven/bun:1-alpine`, installs production deps, copies `/app/dist`, exposes 80, and runs `bunx vite preview --host 0.0.0.0 --port 80`. Verify nginx runtime/copy lines are gone and Vite preview command exists.
|
||||
|
||||
### Task 2: Remove Image Cache Proxy From CachedImage
|
||||
|
||||
Modify `apps/react/src/components/ui/cached-image.tsx` to remove `API_BASE_URL` import, `/proxy/image-cache` POST, audit state/function, and auditing overlay. Keep direct normalized image URLs, retry behavior, and fallback image.
|
||||
|
||||
### Task 3: Verify Build and Config
|
||||
|
||||
Run static checks for both tasks, `bun --cwd apps/react run build`, `docker build -f infra/docker/react.Dockerfile -t react-web:test .`, start test container on `18080:80`, curl root, clean container, and run `git diff --check`.
|
||||
|
||||
### Task 4: Commit and Push
|
||||
|
||||
Commit changed files: `infra/docker/react.Dockerfile`, `apps/react/src/components/ui/cached-image.tsx`, `docs/superpowers/plans/2026-05-27-react-direct-runtime-and-images.md`. Push branch only after verification if user asks; do not push from worktree unless explicitly instructed.
|
||||
Reference in New Issue
Block a user