mcpedia/.hermes/skills/devops/nix-ci-deploy/references/bun-source-ssh-deploy.md

SSH deploy to bun-source services (not Nix): gotchas

This repo (asepharyana/mcpedia) deploys bun source via systemd — NOT via Nix like GMW. The pattern is: CI builds in GitHub Actions → upload artifact → deploy job downloads artifact → SCP tarball to VPS → SSH → git pull + unpack + index + systemctl restart. The VPS does NOT build.

Gotchas (learned during mcpedia deploy setup, 2026-08-20/21)

1. SSH host = public IP, NOT Tailscale IP

The VPS appears in ~/.ssh/config as Host orange → HostName 100.79.111.61. That IP is a Tailscale tailscale0 interface address in the CGNAT range (100.64.0.0/10). GitHub Actions runners are NOT on the Tailscale network, so they get Connection timed out (dropped at the network layer, not refused).

Fix: Use the VPS's real public IP (45.127.35.244, discovered via curl https://api.ipify.org from the VPS). Port 22 is open in iptables (ACCEPT tcp dpt:22 from 0.0.0.0/0).

2. SSH key MUST be stored directly from file (not shell variable)

Storing the deploy key via a shell variable corrupts it:

# ❌ WRONG — newlines get mangled by the shell → "ssh: no key found"
PRIV_KEY=$(cat keyfile)
gh secret set SSH_DEPLOY_KEY --body "$PRIV_KEY"

# ✅ CORRECT — pipe the file directly so GitHub preserves all bytes
cat keyfile | gh secret set SSH_DEPLOY_KEY --repo asepharyana/mcpedia

Symptom: appleboy/ssh-action fails with ssh.ParsePrivateKey: ssh: no key found.

3. Use appleboy/ssh-action@v1 (not @v1.1.0)

• @v1.1.0 (very old) has a key-parsing bug that rejects valid keys (ssh.ParsePrivateKey: ssh: no key found).
• @v1 (latest) handles OpenSSH ed25519 keys correctly.

4. appleboy/ssh-action needs explicit envs to pass through secret-derived vars

The envs parameter passes environment variables to the remote script. Include any secret you reference in the script:

envs: SSH_DEPLOY_HOST  # passes SSH_DEPLOY_HOST into the remote script

Without it, echo $SSH_DEPLOY_HOST on the VPS returns empty even though the action connected.

5. Always pass -o IdentitiesOnly=yes

Without it, SSH offers ALL loaded identities (deploy key + default keys) and the server may reject after "Too many authentication failures". appleboy/ssh-action handles this internally via the key input, but if you use raw ssh add -o IdentitiesOnly=yes.

6. GitHub workflow_run does NOT carry the push commit SHA

workflow_run events fire after CI completes, but actions/checkout checks out the default branch tip, not the specific commit. This is fine for deploy (the script does git pull origin main anyway), but verify the checkout ref matches what CI built if you rely on it.

7. DB migrations: db:push prompt is non-interactive-unfriendly in SSH deploy

When the schema includes a new column (e.g. adding extra_fields JSONB), drizzle-kit push in the SSH deploy script needs to be run. Two issues:

• --strict mode (config default): drizzle.config.ts has strict: true, which makes drizzle-kit push prompt No, abort / Yes, I want to execute all statements. In a non-interactive SSH script (no TTY), the prompt never receives input and the command times out → SIGTERM → exit code 124 → deploy fails. set -e then kills the whole deploy.
• bun run <script> ambiguity: bun run index resolves the index script from package.json. In a monorepo with workspace packages, ensure the script name is unique or use the explicit file path (e.g. bun run scripts/indexer.ts).

Fix: Use psql for schema changes instead of db:push:

psql "$DATABASE_URL" -c \
  "ALTER TABLE documents ADD COLUMN IF NOT EXISTS extra_fields jsonb DEFAULT '{}'::jsonb NOT NULL;"

After deploy, check the public URL (not just systemctl is-active):

curl -sk -o /dev/null -w "%{http_code}" https://wiki.asepharyana.my.id/
curl -s https://wiki.asepharyana.my.id/docs/websocket/contract | grep -c "frontmatter-pattern"

Deploy workflow template (bun-source, not Nix) — build in CI, deploy-only on VPS

User correction (2026-08-21): "alur ci nya juga benarkan lah masa build di vps,harusnya kan di vps hanya deploy,dan seharusnya pakai nix"

The VPS must not build. Build happens in CI; the VPS only deploys the pre-built artifact. Use a single workflow with a needs: chain (not workflow_run — see pitfall #8 below).

Pitfalls for bun-source deploy workflows

• bun --cwd X run Y is broken. bun run does NOT accept --cwd as a subcommand flag — it prints usage and exits 0 (silent failure). Use working-directory: X in the step, or cd X && bun run Y. (Root cause of CI appearing to pass while the build never actually ran. This also affects the smoke test step: bun --cwd apps/mcp run smoke was silently failing.)
• .next/ is a hidden directory. actions/upload-artifact@v4 excludes hidden files/dirs by default (include-hidden-files: false). Set include-hidden-files: true or the artifact will be empty.
• workflow_run cannot download artifacts from the CI run. This is a known GitHub Actions limitation — the deploy workflow doesn't have access to the CI run's artifacts. Fix: merge CI + deploy into a single workflow with deploy: needs: [build].
• actions/checkout@v4 → Node 20 deprecation. Bump to @v5.
• SCP'ing hundreds of individual files from .next/ can time out. Tar first, then SCP the single tarball: tar -czf app.tar.gz .next, SCP, then SSH to unpack. Also exclude .next/cache/ to reduce size (~73MB → ~49MB).
• download-artifact path resolution. When downloading with path: apps/web/.next, the artifact (which contains the .next directory contents from upload path: apps/web/.next) gets extracted correctly to apps/web/.next/. Do NOT use path: .next — that creates .next/.next/.

Final working workflow (verified 2026-08-21)

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  build:
    name: typecheck + build
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v5

      - name: Set up Bun
        uses: oven-sh/setup-bun@v2
        with:
          bun-version: "1.3.14"

      - name: Install dependencies
        run: bun install --frozen-lockfile

      - name: Typecheck
        run: bun run typecheck

      - name: Build web
        working-directory: apps/web
        run: bun run build

      # Upload .next/ — include-hidden-files is REQUIRED (hidden dir)
      - name: Upload web build artifact
        uses: actions/upload-artifact@v4
        with:
          name: mcpedia-web-build
          path: apps/web/.next
          include-hidden-files: true
          if-no-files-found: error

      # Smoke test: requires Postgres + Redis — skipped in CI without secrets
      - name: MCP smoke test
        if: env.DATABASE_URL != ''
        working-directory: apps/mcp
        env:
          DATABASE_URL: ${{ secrets.DATABASE_URL }}
          REDIS_URL: ${{ secrets.REDIS_URL }}
        run: bun run smoke

      - name: Test
        run: bun run test

  deploy:
    name: deploy to VPS
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v5

      - name: Download build artifact
        uses: actions/download-artifact@v4
        with:
          name: mcpedia-web-build
          path: apps/web/.next

      - name: Tar .next
        run: tar -C apps/web --exclude='.next/cache' -czf mcpedia-web-next.tar.gz .next

      - name: Copy tarball to VPS
        uses: appleboy/scp-action@v1
        with:
          host: ${{ secrets.SSH_DEPLOY_HOST }}
          port: ${{ secrets.SSH_DEPLOY_PORT }}
          username: ${{ secrets.SSH_DEPLOY_USER }}
          key: ${{ secrets.SSH_DEPLOY_KEY }}
          source: "mcpedia-web-next.tar.gz"
          target: "/tmp/"

      - name: Unpack and restart on VPS
        uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.SSH_DEPLOY_HOST }}
          port: ${{ secrets.SSH_DEPLOY_PORT }}
          username: ${{ secrets.SSH_DEPLOY_USER }}
          key: ${{ secrets.SSH_DEPLOY_KEY }}
          envs: SSH_DEPLOY_HOST
          script: |
            set -e
            cd /home/code/mcpedia
            git pull origin main
            /home/code/.bun/bin/bun install --frozen-lockfile
            /home/code/.bun/bin/bun run scripts/indexer.ts
            rm -rf apps/web/.next
            tar -xzf /tmp/mcpedia-web-next.tar.gz -C apps/web/
            rm -f /tmp/mcpedia-web-next.tar.gz
            sudo systemctl restart mcpedia-web mcpedia-api mcpedia-mcp mcpedia-worker
            sleep 3
            systemctl --no-pager status mcpedia-web mcpedia-api mcpedia-mcp mcpedia-worker --no-legend

Secrets to configure (per-repo GitHub secrets)

Secret	Value	How
SSH_DEPLOY_HOST	VPS public IP (e.g. 45.127.35.244)	gh secret set SSH_DEPLOY_HOST --body "45.127.35.244"
SSH_DEPLOY_PORT	22	gh secret set SSH_DEPLOY_PORT --body "22"
SSH_DEPLOY_USER	code (or whatever user owns the services)	gh secret set SSH_DEPLOY_USER --body "code"
SSH_DEPLOY_KEY	SSH deploy private key (ed25519)	cat keyfile | gh secret set SSH_DEPLOY_KEY

And on the VPS, add the public key to the deploy user's authorized_keys:

ssh code@<public-ip> "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys <<< '$(cat keyfile.pub)'"
chmod 600 ~/.ssh/authorized_keys

CI monitoring

gh run list --repo asepharyana/mcpedia --workflow "CI" --limit 3
gh run view <run_id> --log-failed   # only failing-step logs