feat: add web_browse tool for navigating and interacting with web pages using Bun's WebView
ci / check (macos-latest) (push) Canceled after 0s
ci / check (ubuntu-latest) (push) Canceled after 0s
ci / check (windows-latest) (push) Canceled after 0s

This commit is contained in:
asepharyana
2026-09-04 22:37:38 +07:00
parent bb36af62e2
commit 922519aa01
6 changed files with 415 additions and 1 deletions
+67
View File
@@ -0,0 +1,67 @@
# WebView Tool — Spec
## TL;DR
A `web_browse` tool that uses Bun's built-in WebView to navigate, interact with, and
screenshot web pages — no Puppeteer, no Playwright, no separate browser download.
## Problem
The existing `web_fetch` tool only extracts static HTML. SPAs, pages requiring JS
execution, form submissions, and interactive testing are impossible. The agent needs
a real browser to handle modern web apps.
## Goal
- Navigate to any URL and extract text/HTML after JS execution
- Click buttons, fill forms, scroll pages
- Take screenshots (save to file, return path)
- Evaluate arbitrary JavaScript in the page context
- Generate PDFs from pages
## Architecture
### Tool: `web_browse`
Input schema:
```ts
{
url: string; // URL to navigate to
action?: 'text' | 'html' | 'screenshot' | 'evaluate' | 'click' | 'type' | 'scroll' | 'pdf';
selector?: string; // CSS selector for click/type/scroll
text?: string; // text to type (for 'type' action)
script?: string; // JS to evaluate (for 'evaluate' action)
wait?: number; // ms to wait after navigation (default: 1000)
}
```
Default action: `text` — navigate and return page text content.
### Implementation (src/tools-webview.ts)
- Uses `Bun.WebView` with Chrome backend (headless)
- One view per call (create → navigate → act → close)
- Screenshot saved to `os.tmpdir()/shiro-webview-{timestamp}.png`
- PDF saved to `os.tmpdir()/shiro-webview-{timestamp}.pdf`
- Timeout: 15s per navigation, 5s per action
- Error handling: browser not found → suggest install; page error → return error text
### Registration
- Add to `tools.ts` as a `net` tool (network access)
- Add to `TOOL_META` as 'net'
- Add to `TOOL_SETS.net` set
- NOT in default tool set (opt-in via toolSets config)
### Tests
- Unit: tool schema validation
- Integration: navigate to data: URL, verify text extraction
- Integration: screenshot saves file
- Integration: evaluate JS returns result
- Edge case: invalid URL returns error
- Edge case: browser not available returns helpful message
## Files touched
- **NEW**: `src/tools-webview.ts`
- **EDIT**: `src/tools.ts` — register tool, add to TOOL_META and TOOL_SETS
- **NEW**: `test/webview.test.ts`
- **NEW**: `docs/webview.md`
+83
View File
@@ -0,0 +1,83 @@
# WebView Tool (web_browse)
A built-in browser tool powered by Bun's native WebView. No Puppeteer, no Playwright,
no separate browser download — just the runtime's own headless browser.
## What it does
- **Navigate** to any URL and extract text/HTML after JS execution
- **Click** buttons, links, and interactive elements
- **Type** into input fields and forms
- **Scroll** to elements on the page
- **Screenshot** pages as PNG files
- **Evaluate** arbitrary JavaScript in the page context
## When to use it
Use `web_browse` instead of `web_fetch` when:
- The page is a SPA (Single Page Application) that needs JavaScript
- You need to interact with the page (click, type, scroll)
- You need a visual screenshot of the page
- `web_fetch` returns incomplete or empty content
- You need to fill forms or submit data
Use `web_fetch` when:
- The page is static HTML
- You just need the text content quickly
- No JS execution is needed
## Actions
| Action | Description | Requires |
|--------|-------------|----------|
| `text` | Page text content after JS execution | — |
| `html` | Raw HTML of the page | — |
| `screenshot` | Save page as PNG, return file path | — |
| `evaluate` | Run JavaScript, return result | `script` |
| `click` | Click an element | `selector` |
| `type` | Type text into an input | `selector` + `text` |
| `scroll` | Scroll to an element | `selector` |
## Examples
```json
// Get page text
{ "url": "https://example.com", "action": "text" }
// Take a screenshot
{ "url": "https://example.com", "action": "screenshot" }
// Click a button
{ "url": "https://example.com", "action": "click", "selector": "button.submit" }
// Fill a form
{ "url": "https://example.com", "action": "type", "selector": "#email", "text": "user@example.com" }
// Run custom JS
{ "url": "https://example.com", "action": "evaluate", "script": "document.querySelectorAll('a').length" }
```
## Requirements
- **macOS**: Uses system WKWebView (no install needed)
- **Linux/Windows**: Requires Chrome, Chromium, Edge, or Brave installed
- Bun 1.2+ for WebView support
## Tool set
`web_browse` is in the `net` tool set. Enable it in config:
```json
{ "toolSets": ["core", "git", "net"] }
```
Or enable all sets with `toolSets: ["*"]`.
## Limitations
- Each call creates a fresh browser instance (no persistent sessions by default)
- Maximum 15s navigation timeout, 5s action timeout
- Screenshots are saved to the system temp directory
- Not available in Bun versions before 1.2