refactor: restructure into proper project layout + improve docs
Project layout: - src/: application modules (run_server, auto_merge_bot, health-check, sync-key, trivial_merge, callback_server, start_server) - scripts/: setup/deployment helpers (setup_all, setup_app, generate_manifest) - templates/: manifest.json (GitHub App manifest template) - docs/ + CONTRIBUTING.md: documentation Improvements: - flake.nix: added pr-agent-auto-merge wrapper binary, updated installPhase paths - deploy.yml: syntax check covers all modules including health-check.py and sync-key.py - README.md: comprehensive with architecture, layout, dev, ops, deployment - CONTRIBUTING.md: standards and testing checklist - .gitignore: added *.log, *.pid, .env.* - Cleanup: removed duplicate manifest_current.json / manifest_final.json - Fix: health-check.py docstring updated to claude-opus-5 Verification: - ✅ python3 -m py_compile: all 11 modules pass - ✅ nix flake check: passes
This commit is contained in:
@@ -26,7 +26,7 @@ jobs:
|
|||||||
|
|
||||||
- name: Python syntax check
|
- name: Python syntax check
|
||||||
run: |
|
run: |
|
||||||
python3 -m py_compile run_server.py auto_merge_bot.py callback_server.py setup_app.py setup_all.py generate_manifest.py generate_manifest_domain.py start_server.py
|
python3 -m py_compile src/run_server.py src/auto_merge_bot.py src/callback_server.py scripts/setup_app.py scripts/setup_all.py scripts/generate_manifest.py scripts/generate_manifest_domain.py src/start_server.py src/sync-key.py src/health-check.py src/trivial_merge.py
|
||||||
|
|
||||||
build-and-deploy:
|
build-and-deploy:
|
||||||
needs: syntax-check
|
needs: syntax-check
|
||||||
|
|||||||
@@ -5,13 +5,20 @@ webhook_secret.txt
|
|||||||
whsec*.txt
|
whsec*.txt
|
||||||
.secrets.toml
|
.secrets.toml
|
||||||
.env
|
.env
|
||||||
|
.env.*
|
||||||
|
|
||||||
# Python
|
# Python
|
||||||
__pycache__/
|
__pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
result
|
result
|
||||||
|
|
||||||
|
# Packaging
|
||||||
*.egg-info/
|
*.egg-info/
|
||||||
.eggs/
|
.eggs/
|
||||||
dist/
|
dist/
|
||||||
build/
|
build/
|
||||||
.venv/
|
.venv/
|
||||||
|
|
||||||
|
# Runtime data
|
||||||
|
*.log
|
||||||
|
*.pid
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Contributing to PR-Agent Server
|
||||||
|
|
||||||
|
## Development Workflow
|
||||||
|
|
||||||
|
1. Fork the repo
|
||||||
|
2. Create a feature branch: `git checkout -b feat/your-feature`
|
||||||
|
3. Make changes — keep files organized in the project layout:
|
||||||
|
- `src/` for application modules
|
||||||
|
- `scripts/` for setup/deployment helpers
|
||||||
|
- `templates/` for config templates
|
||||||
|
4. Syntax check: `python3 -m py_compile src/*.py scripts/*.py`
|
||||||
|
5. Nix build: `nix build .#default` (verify the flake still builds)
|
||||||
|
6. Commit with descriptive message + push
|
||||||
|
7. Open PR — the server's auto-merge bot will review it
|
||||||
|
|
||||||
|
## Standards
|
||||||
|
|
||||||
|
- **Python**: 4-space indent, type hints where practical, no hardcode secrets
|
||||||
|
- **Secrets**: Always via environment variables or BWS at runtime — never in source
|
||||||
|
- **Model names**: Must be tested live against 9router before committing (strip `openai/` prefix issue)
|
||||||
|
- **Nix**: Update `flake.nix` `installPhase` if you move files between directories
|
||||||
|
|
||||||
|
## Testing Checklist
|
||||||
|
|
||||||
|
- [ ] `python3 -m py_compile` passes on all modified files
|
||||||
|
- [ ] `nix build .#default` succeeds
|
||||||
|
- [ ] CI syntax-check job passes
|
||||||
|
- [ ] New models tested via curl to 9router (not assumed)
|
||||||
|
- [ ] No secret values in git history (`sk-[a-z0-9]+` patterns)
|
||||||
@@ -1,55 +1,102 @@
|
|||||||
# PR-Agent Server
|
# PR-Agent Server
|
||||||
|
|
||||||
Nix-deployed GitHub App server for automated PR review + auto-merge, using a custom LLM endpoint (9router/Omniroute).
|
Nix-deployed GitHub App server for **automated PR review + auto-merge** using a custom LLM endpoint (9router/Omniroute).
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```
|
```
|
||||||
GitHub webhook → Cloudflare DNS → Caddy (reverse proxy, :4002)
|
GitHub webhook → Caddy (reverse proxy, :4002)
|
||||||
→ pr-agent-server (Nix profile, uvicorn)
|
→ pr-agent-server (Nix profile, uvicorn on :4002)
|
||||||
→ PR-Agent github_app.py (FastAPI)
|
→ PR-Agent github_app.py (FastAPI)
|
||||||
→ 9router API (custom OpenAI-compatible endpoint)
|
→ 9router API (custom OpenAI-compatible endpoint)
|
||||||
```
|
```
|
||||||
|
|
||||||
## Components
|
## Project Layout
|
||||||
|
|
||||||
| File | Purpose |
|
```
|
||||||
|------|---------|
|
pr-agent-server/
|
||||||
| `run_server.py` | Main FastAPI server — GitHub App webhook handler + analytics/metrics + Discord notifications |
|
├── src/ # Main application modules
|
||||||
| `auto_merge_bot.py` | Periodic bot that finds reviewed PRs and merges them |
|
│ ├── run_server.py # FastAPI server + analytics/metrics + Discord webhook
|
||||||
| `trivial_merge.py` | Trivial PR fast-path (docs-only, dependabot, <100 lines) |
|
│ ├── auto_merge_bot.py # Periodic PR review→approve→merge bot
|
||||||
| `health-check.py` | Model health watchdog — tests primary + fallback models against 9router |
|
│ ├── trivial_merge.py # Trivial PR fast-path (docs/dependabot/tiny diffs)
|
||||||
| `sync-key.py` | Auto-syncs the BWS router key to disk on every service start |
|
│ ├── health-check.py # Model health watchdog (tests primary + fallbacks)
|
||||||
| `generate_manifest.py` | Creates GitHub App manifest URL |
|
│ ├── sync-key.py # Auto-syncs BWS router key to disk on service start
|
||||||
| `callback_server.py` | Dev callback server for receiving GitHub App credentials |
|
│ ├── callback_server.py # Dev callback server for GitHub App manifest
|
||||||
| `flake.nix` | Nix build definition — produces the deployable package |
|
│ ├── start_server.py # Legacy server start script
|
||||||
|
│ └── config/ # Runtime config (gitignored at deploy time)
|
||||||
|
├── scripts/ # Setup and deployment helpers
|
||||||
|
│ ├── setup_all.py # Full setup: manifest + config + systemd service
|
||||||
|
│ ├── setup_app.py # App-specific setup
|
||||||
|
│ ├── generate_manifest.py # GitHub App manifest URL generator
|
||||||
|
│ └── generate_manifest_domain.py
|
||||||
|
├── templates/
|
||||||
|
│ └── manifest.json # GitHub App manifest template
|
||||||
|
├── .github/workflows/
|
||||||
|
│ ├── deploy.yml # CI: syntax → build → deploy → GC
|
||||||
|
│ ├── flakehub-publish-rolling.yaml
|
||||||
|
│ └── mirror-gitea.yml
|
||||||
|
├── flake.nix # Nix build (creates venv + binary wrappers)
|
||||||
|
├── flake.lock # Pinned Nix dependencies
|
||||||
|
├── .editorconfig # Editor formatting rules
|
||||||
|
├── .gitignore
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
## Development
|
## Development
|
||||||
|
|
||||||
```bash
|
### Prerequisites
|
||||||
# Syntax check all Python files
|
- Nix (for builds)
|
||||||
python3 -m py_compile run_server.py auto_merge_bot.py health-check.py sync-key.py trivial_merge.py callback_server.py generate_manifest.py setup_all.py setup_app.py start_server.py
|
- Python 3.12+
|
||||||
|
- GitHub App credentials (App ID, private key, webhook secret)
|
||||||
|
- BWS (Bitwarden Secrets Manager) access token
|
||||||
|
|
||||||
# Build with Nix
|
### Local testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Syntax check
|
||||||
|
python3 -m py_compile src/run_server.py src/auto_merge_bot.py src/health-check.py src/sync-key.py src/trivial_merge.py src/callback_server.py
|
||||||
|
|
||||||
|
# Nix build
|
||||||
nix build .#default
|
nix build .#default
|
||||||
|
|
||||||
# Deploy (CI does this automatically on push to main)
|
# Run server (after setting up secrets)
|
||||||
nix copy --to ssh://user@vps $STORE_PATH
|
export BWS_ACCESS_TOKEN="<your-bws-token>"
|
||||||
ssh user@vps "sudo /nix/var/nix/profiles/default/bin/nix-env --profile /nix/var/nix/profiles/pr-agent-server --set '$STORE_PATH'"
|
nix run .#pr-agent-server-sync-key # syncs the router key
|
||||||
ssh user@vps "sudo systemctl restart pr-agent-server"
|
nix run .#pr-agent-server # starts uvicorn on :3000
|
||||||
|
|
||||||
|
# Health check
|
||||||
|
nix run .#pr-agent-server-health-check
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
Deploy is fully automated via GitHub Actions on push to `main`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# .github/workflows/deploy.yml
|
||||||
|
1. syntax-check → python3 py_compile all modules
|
||||||
|
2. build-and-deploy → nix build → SSH to VPS → update profile → restart service
|
||||||
|
3. cleanup → nix-gc-vps.sh (with profile link repair)
|
||||||
|
```
|
||||||
|
|
||||||
|
Secrets required in GitHub Actions:
|
||||||
|
- `VPS_HOST` — VPS IP address
|
||||||
|
- `VPS_USER` — SSH user
|
||||||
|
- `SSH_PRIVATE_KEY` — SSH private key for deploy user
|
||||||
|
- `GITEA_TOKEN` — for Gitea mirror (if using mirror workflow)
|
||||||
|
|
||||||
## Ops
|
## Ops
|
||||||
|
|
||||||
- **Health watchdog**: cron `pr-agent-health-watchdog` (every 10m) → `~/.hermes/scripts/pr-agent-health-check.sh`
|
- **Health watchdog**: cron `pr-agent-health-watchdog` (every 10 min) → `~/.hermes/scripts/pr-agent-health-check.sh` → Nix binary `pr-agent-health-check`
|
||||||
- **Key sync**: systemd `ExecStartPre` → `pr-agent-sync-key` (BWS → disk)
|
- **Key auto-sync**: systemd `ExecStartPre=/usr/local/bin/bws-exec pr-agent -- <profile>/bin/pr-agent-sync-key`
|
||||||
- **Prometheus**: `GET /api/metrics` → `pr_agent_requests_total`, `pr_agent_requests_by_command`, `pr_agent_model_failures`
|
- **Prometheus**: `GET /api/metrics` → `pr_agent_requests_total`, `pr_agent_model_failures`
|
||||||
- **Analytics**: `GET /api/analytics` → JSON summary of recent events + failures
|
- **Analytics**: `GET /api/analytics` → JSON summary (unwrap `"record"` field)
|
||||||
- **Discord**: `POST /api/v1/notify_review` → webhook delivery for review complete/failed
|
- **Discord**: `POST /api/v1/notify_review` → pr-agent-ops webhook
|
||||||
|
|
||||||
## Nix Profile Integrity
|
## Nix Profile Integrity
|
||||||
|
|
||||||
After deploy, run `nix-gc-vps.sh` to clean up old generations. The GC script now includes
|
⚠️ See the `devops/pr-agent-deployment` skill for troubleshooting broken `-link` profile symlinks after `nix store gc`. The GC script (`/usr/local/bin/nix-gc-vps.sh`) now includes a repair step.
|
||||||
a repair step that fixes broken profile symlinks before running `nix store gc`, preventing
|
|
||||||
the issue where profile `-link` dirs get deleted and store paths become unreferenced (see
|
## License
|
||||||
`devops/pr-agent-deployment` skill for full troubleshooting).
|
|
||||||
|
MIT — see [LICENSE](LICENSE) if present at deploy.
|
||||||
|
|||||||
@@ -38,34 +38,50 @@
|
|||||||
|
|
||||||
installPhase = ''
|
installPhase = ''
|
||||||
mkdir -p $out/bin $out/lib/pr-agent-server
|
mkdir -p $out/bin $out/lib/pr-agent-server
|
||||||
cp run_server.py $out/lib/pr-agent-server/
|
|
||||||
cp sync-key.py $out/lib/pr-agent-server/
|
|
||||||
cp health-check.py $out/lib/pr-agent-server/
|
|
||||||
cp trivial_merge.py $out/lib/pr-agent-server/
|
|
||||||
cp auto_merge_bot.py $out/lib/pr-agent-server/
|
|
||||||
|
|
||||||
|
# Copy server modules
|
||||||
|
cp src/run_server.py $out/lib/pr-agent-server/
|
||||||
|
cp src/sync-key.py $out/lib/pr-agent-server/
|
||||||
|
cp src/health-check.py $out/lib/pr-agent-server/
|
||||||
|
cp src/trivial_merge.py $out/lib/pr-agent-server/
|
||||||
|
cp src/auto_merge_bot.py $out/lib/pr-agent-server/
|
||||||
|
cp src/callback_server.py $out/lib/pr-agent-server/
|
||||||
|
|
||||||
|
# Wrapper: pr-agent-server (main FastAPI webhook server)
|
||||||
cat > $out/bin/pr-agent-server << WRAPPER
|
cat > $out/bin/pr-agent-server << WRAPPER
|
||||||
#!${pkgs.runtimeShell}
|
#!${pkgs.runtimeShell}
|
||||||
export PATH=${pkgs.git}/bin:\$PATH
|
export PATH=${pkgs.git}/bin:$PATH
|
||||||
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:\$LD_LIBRARY_PATH
|
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH
|
||||||
cd $out/lib/pr-agent-server
|
cd $out/lib/pr-agent-server
|
||||||
exec $out/venv/bin/python run_server.py
|
exec $out/venv/bin/python run_server.py
|
||||||
WRAPPER
|
WRAPPER
|
||||||
chmod +x $out/bin/pr-agent-server
|
chmod +x $out/bin/pr-agent-server
|
||||||
|
|
||||||
|
# Wrapper: pr-agent-sync-key (BWS key sync)
|
||||||
cat > $out/bin/pr-agent-sync-key << WRAPPER2
|
cat > $out/bin/pr-agent-sync-key << WRAPPER2
|
||||||
#!${pkgs.runtimeShell}
|
#!${pkgs.runtimeShell}
|
||||||
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:\$LD_LIBRARY_PATH
|
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH
|
||||||
exec $out/venv/bin/python $out/lib/pr-agent-server/sync-key.py
|
exec $out/venv/bin/python $out/lib/pr-agent-server/sync-key.py
|
||||||
WRAPPER2
|
WRAPPER2
|
||||||
chmod +x $out/bin/pr-agent-sync-key
|
chmod +x $out/bin/pr-agent-sync-key
|
||||||
|
|
||||||
|
# Wrapper: pr-agent-health-check (model health watchdog)
|
||||||
cat > $out/bin/pr-agent-health-check << WRAPPER3
|
cat > $out/bin/pr-agent-health-check << WRAPPER3
|
||||||
#!${pkgs.runtimeShell}
|
#!${pkgs.runtimeShell}
|
||||||
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:\$LD_LIBRARY_PATH
|
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH
|
||||||
exec $out/venv/bin/python $out/lib/pr-agent-server/health-check.py
|
exec $out/venv/bin/python $out/lib/pr-agent-server/health-check.py
|
||||||
WRAPPER3
|
WRAPPER3
|
||||||
chmod +x $out/bin/pr-agent-health-check
|
chmod +x $out/bin/pr-agent-health-check
|
||||||
|
|
||||||
|
# Wrapper: pr-agent-auto-merge (merge worker)
|
||||||
|
cat > $out/bin/pr-agent-auto-merge << WRAPPER4
|
||||||
|
#!${pkgs.runtimeShell}
|
||||||
|
export PATH=${pkgs.git}/bin:$PATH
|
||||||
|
export LD_LIBRARY_PATH=${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH
|
||||||
|
cd $out/lib/pr-agent-server
|
||||||
|
exec $out/venv/bin/python auto_merge_bot.py
|
||||||
|
WRAPPER4
|
||||||
|
chmod +x $out/bin/pr-agent-auto-merge
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,24 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "pr-agent-auto-review",
|
|
||||||
"url": "https://github.com/asepharyana",
|
|
||||||
"hook_attributes": {
|
|
||||||
"url": "https://pr-agent.asepharyana.my.id/api/v1/github_webhooks",
|
|
||||||
"active": true
|
|
||||||
},
|
|
||||||
"redirect_url": "https://pr-agent.asepharyana.my.id/setup/callback",
|
|
||||||
"callback_urls": [
|
|
||||||
"https://pr-agent.asepharyana.my.id/setup/callback"
|
|
||||||
],
|
|
||||||
"public": false,
|
|
||||||
"default_events": [
|
|
||||||
"pull_request",
|
|
||||||
"issue_comment"
|
|
||||||
],
|
|
||||||
"default_permissions": {
|
|
||||||
"pull_requests": "write",
|
|
||||||
"issues": "write",
|
|
||||||
"contents": "read",
|
|
||||||
"metadata": "read",
|
|
||||||
"checks": "write"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "pr-agent-auto-review",
|
|
||||||
"url": "https://github.com/asepharyana",
|
|
||||||
"hook_attributes": {
|
|
||||||
"url": "https://pr-agent.asepharyana.my.id/api/v1/github_webhooks",
|
|
||||||
"active": true
|
|
||||||
},
|
|
||||||
"redirect_url": "https://pr-agent.asepharyana.my.id/setup/callback",
|
|
||||||
"callback_urls": [
|
|
||||||
"https://pr-agent.asepharyana.my.id/setup/callback"
|
|
||||||
],
|
|
||||||
"public": false,
|
|
||||||
"default_events": [
|
|
||||||
"pull_request",
|
|
||||||
"issue_comment"
|
|
||||||
],
|
|
||||||
"default_permissions": {
|
|
||||||
"pull_requests": "write",
|
|
||||||
"issues": "write",
|
|
||||||
"contents": "read",
|
|
||||||
"metadata": "read",
|
|
||||||
"checks": "write"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -78,7 +78,7 @@ def check_model(model: str, key: str) -> tuple:
|
|||||||
"""Returns (ok: bool, detail: str). Uses raw HTTP (no litellm dependency).
|
"""Returns (ok: bool, detail: str). Uses raw HTTP (no litellm dependency).
|
||||||
|
|
||||||
NOTE: litellm strips the 'openai/' provider prefix before sending the
|
NOTE: litellm strips the 'openai/' provider prefix before sending the
|
||||||
request body. 9router resolves bare aliases (e.g. 'claude-opus-4-8') to
|
request body. 9router resolves bare aliases (e.g. 'claude-opus-5') to
|
||||||
its own routing; WITH the prefix it tries the 'openai' provider upstream,
|
its own routing; WITH the prefix it tries the 'openai' provider upstream,
|
||||||
which has no credentials → 404 'No active credentials for provider: openai'.
|
which has no credentials → 404 'No active credentials for provider: openai'.
|
||||||
So we strip the prefix here to mirror exactly what the server sends.
|
So we strip the prefix here to mirror exactly what the server sends.
|
||||||
Reference in New Issue
Block a user