Replace socket daemon with direct CDP WebSocket connection Drops the Unix socket subprocess in favor of talking CDP directly over WebSocket from the extension. Removes the protocol's send/health helpers, simplifies setup to just verifying Chrome's remote debugging port, and tightens types in dynamic-tools. Bumps Node engine to >=22 and adds the ws dependency.
amankumarsingh77 amankumarsingh7702@gmal.com
Wed, 29 Apr 2026 01:38:17 +0530
8 files changed,
629 insertions(+),
929 deletions(-)
M
README.md
→
README.md
@@ -1,513 +1,210 @@
# pi-browser-harness -> **Full browser control for pi agents.** Navigate, click, type, screenshot, and extract data from a real Chrome browser — all through CDP, all from your coding agent. - -<p align="center"> - <img alt="Version" src="https://img.shields.io/npm/v/@pi-browser-harness?style=flat-square&color=2563eb"> - <img alt="License" src="https://img.shields.io/badge/license-MIT-blue?style=flat-square"> - <img alt="Pi Package" src="https://img.shields.io/badge/pi-package-2563eb?style=flat-square"> - <img alt="Node" src="https://img.shields.io/badge/node-%3E%3D18-339933?style=flat-square&logo=node.js&logoColor=white"> -</p> - -**pi-browser-harness** gives pi agents full control of a real Chrome browser. No headless mode, no emulation — your agent sees what you see, clicks where you'd click, and reads pages exactly as they render. Browser tools coexist seamlessly with all standard pi tools (`read`, `bash`, `edit`, `write`). - ---- - -## Why pi-browser-harness? - -| | | -|---|---| -| 🖱️ **Real browser, real pixels** | Connects to your running Chrome via CDP. Compositor-level clicks work through iframes, shadow DOM, and cross-origin content. | -| 🧩 **Seamless pi integration** | `browser_*` tools appear alongside `read`, `bash`, `edit` — no context switching, no separate daemon management. | -| 🔧 **Self-extending harness** | The agent writes its own tools mid-task. Need a paginated scraper? Bulk CSV exporter? Register it at runtime, callable next turn. | -| ⚡ **Fast data extraction** | `browser_http_get` bypasses the browser entirely for APIs — 10-50x faster than DOM scraping. `browser_execute_js` for rendered data. | -| 🛡️ **Dialog-aware** | Auto-detects JS dialogs (alert, confirm, prompt, beforeunload) before they block your session. | -| 📸 **Visual feedback** | Screenshots before and after every action. The agent can see exactly what it's doing. | +Full browser control for pi agents. Navigate, click, type, screenshot, and extract data from your real Chrome — driven by your coding agent. --- ## Quick Start ```bash -# 1. Install the package -pi install npm:pi-browser-harness - -# 2. Start Chrome with remote debugging -# macOS: -/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222 & +# 1. Install (from this repo, until published to npm) +pi install git:github.com/amankumarsingh77/pi-browser-harness -# Linux: -google-chrome --remote-debugging-port=9222 & +# 2. Enable Chrome remote debugging +# Open chrome://inspect/#remote-debugging in Chrome, +# tick the "Discover network targets" checkbox, click Allow. +# No Chrome relaunch needed. -# 3. Run the guided setup inside pi +# 3. Connect pi to Chrome /browser-setup ``` -**That's it.** Your pi agent now has 20 browser control tools. Try: +That's it. Try: ``` > navigate to https://github.com and take a screenshot ``` +### Requirements + +- pi (latest) +- Node.js ≥ 22 +- Chrome / Chromium / Edge (any recent version) + +> If `chrome://inspect/#remote-debugging` isn't available, you can launch Chrome with `--remote-debugging-port=9222` instead. The harness will find it either way. + --- -## Features +## What you can do -### 🧭 Navigation & Tabs +### Navigation & tabs | Tool | What it does | -|------|-------------| -| `browser_navigate` | Navigate to a URL (safe: creates new tab on first call) | +|------|--------------| +| `browser_navigate` | Go to a URL (reuses the attached tab) | | `browser_new_tab` | Open a new tab, optionally navigate | -| `browser_open_urls` | Open multiple URLs in parallel tabs (great after web search) | +| `browser_open_urls` | Open many URLs in parallel tabs | | `browser_go_back` / `browser_go_forward` | History navigation | | `browser_reload` | Reload the current page | -| `browser_page_info` | Get URL, title, viewport, scroll position, or detect JS dialogs | -| `browser_list_tabs` / `browser_current_tab` | Tab discovery and identification | -| `browser_switch_tab` | Switch to a specific tab by targetId | +| `browser_list_tabs` / `browser_current_tab` | Discover tabs | +| `browser_switch_tab` | Switch the active tab | +| `browser_page_info` | URL, title, viewport, scroll, dialog state | -### 🖱️ Interaction +### Interaction | Tool | What it does | -|------|-------------| -| `browser_click` | Click at viewport CSS-pixel coordinates (compositor-level) | -| `browser_type` | Type text into the focused element | -| `browser_press_key` | Press any key with modifier support (Ctrl, Alt, Meta, Shift) | -| `browser_scroll` | Scroll the page by delta pixels | -| `browser_handle_dialog` | Accept or dismiss JS dialogs | +|------|--------------| +| `browser_click` | Click at viewport coordinates (works through iframes / shadow DOM) | +| `browser_type` | Type into the focused element | +| `browser_press_key` | Send a key with optional modifiers | +| `browser_scroll` | Scroll by delta pixels | +| `browser_handle_dialog` | Accept or dismiss `alert` / `confirm` / `prompt` | -### 📸 Visual & Data +### Visual & data | Tool | What it does | -|------|-------------| -| `browser_screenshot` | Capture a PNG screenshot of the current page | -| `browser_execute_js` | Execute JavaScript and return the result | -| `browser_http_get` | Direct HTTP GET (bypasses browser — 10-50x faster for APIs) | -| `browser_wait` | Wait N seconds | -| `browser_wait_for_load` | Wait for `document.readyState === 'complete'` | +|------|--------------| +| `browser_screenshot` | Capture the current page as PNG | +| `browser_execute_js` | Run JS in the page and return the result | +| `browser_http_get` | Direct HTTP GET — bypasses the browser, much faster for APIs | +| `browser_wait` / `browser_wait_for_load` | Sleep, or wait for `readyState === 'complete'` | -### 🔧 Self-Extending Harness +### Extending the harness at runtime | Tool | What it does | -|------|-------------| -| `list_dynamic_tools` | See what's already been registered | -| `register_tool` | Inject a new tool (available immediately, next turn) | -| `remove_tool` | Retire a tool that's no longer needed | - -> **Philosophy:** "The agent writes what's missing, mid-task." When the built-in tools aren't enough, the agent extends the harness itself — no reload, no restart. - ---- - -## Installation - -### Prerequisites - -| Requirement | Version | Notes | -|------------|---------|-------| -| **pi** | Latest | The pi coding agent CLI | -| **Node.js** | ≥ 18 | Runtime for the extension | -| **Chrome / Chromium / Edge** | Any recent | Must support CDP (Chrome DevTools Protocol) | -| **browser-harness daemon** | Latest | Installed automatically by `/browser-setup` | -| **uv** (optional) | Latest | Python package manager; used for daemon installation | -| **git** (optional) | Any | Fallback installation method | - -### Installing the package - -```bash -# From npm (recommended) -pi install npm:pi-browser-harness - -# From git -pi install git:github.com/amankumarsingh77/pi-browser-harness - -# Locally (for development) -pi install ./pi-browser-harness -``` - -### Installing the browser-harness daemon - -The daemon (`daemon.py`) is what connects pi-browser-harness to Chrome. Run the guided setup: - -``` -/browser-setup -``` - -This will: -1. Check for an existing browser-harness installation -2. Auto-install via `uv tool install browser-harness` or `git clone` -3. Verify Chrome is running with remote debugging -4. Start the daemon and test the connection - -**Manual installation:** - -```bash -# Option A: uv (recommended) -uv tool install browser-harness - -# Option B: git clone -git clone https://github.com/browser-use/browser-harness ~/Developer/browser-harness -cd ~/Developer/browser-harness && uv sync -``` - -### Enabling Chrome Remote Debugging - -Start Chrome with the `--remote-debugging-port` flag: - -```bash -# macOS -/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \ - --remote-debugging-port=9222 - -# Linux -google-chrome --remote-debugging-port=9222 - -# Windows -"C:\Program Files\Google\Chrome\Application\chrome.exe" --remote-debugging-port=9222 -``` - -> **Edge / Chromium**: Works identically — just replace the binary path. +|------|--------------| +| `list_dynamic_tools` | List tools registered this session | +| `register_tool` | Add a new tool from JS — callable next turn | +| `remove_tool` | Retire a registered tool | --- ## Usage -### Core Workflow +The core pattern is **screenshot → act → screenshot → verify.** Don't assume an action worked — look. -The fundamental pattern: **screenshot → interact → screenshot → verify.** +### Navigate and screenshot ``` -browser_screenshot() → find the target → browser_click(x, y) → browser_screenshot() -``` - -Every interaction is verified visually. Never assume an action worked. - -### Navigation - -```bash -# First navigation: creates a new tab (preserves user's active tab) browser_navigate("https://github.com/browser-use/browser-harness") browser_wait_for_load() browser_screenshot() ``` -### Form Filling +### Fill a form -```bash -browser_screenshot() # See the form -browser_click(400, 200) # Click the search input -browser_type("pi browser harness") # Type the query -browser_press_key("Enter") # Submit +``` +browser_screenshot() # see the form +browser_click(400, 200) # click the search input +browser_type("pi browser harness") # type +browser_press_key("Enter") # submit browser_wait_for_load() -browser_screenshot() # Verify results +browser_screenshot() # verify ``` -### Data Extraction +### Extract data -```bash -# Fast: Direct HTTP for APIs +```js +// Fast: hit the API directly, no browser browser_http_get("https://api.github.com/repos/browser-use/browser-harness") -# DOM scraping: Execute JS in the rendered page +// DOM scraping for rendered content browser_execute_js("document.querySelector('.price').innerText") -# Structured: Extract arrays of data +// Structured arrays browser_execute_js(` JSON.stringify( Array.from(document.querySelectorAll('.result')).map(el => ({ title: el.querySelector('h3')?.textContent, - link: el.querySelector('a')?.href, + link: el.querySelector('a')?.href, })) ) `) ``` -### Research Workflow (Search + Tabs) - -```bash -# Step 1: Search -browser_navigate("https://google.com/search?q=chrome+devtools+protocol+reference") +### Open multiple results in parallel -# Step 2: Open result links in parallel tabs +``` +browser_navigate("https://google.com/search?q=chrome+devtools+protocol") browser_open_urls({ urls: [ "https://chromedevtools.github.io/devtools-protocol/", "https://developer.chrome.com/docs/devtools", -] }) - -# Step 3: List all tabs to get target IDs +]}) browser_list_tabs() - -# Step 4: Switch to the tab you want -browser_switch_tab({ targetId: "ABC123..." }) - -# Step 5: Inspect -browser_wait_for_load() -browser_screenshot() -browser_execute_js("document.querySelector('.main-content').innerText") +browser_switch_tab({ targetId: "..." }) ``` -### SPAs (React, Vue, etc.) +### SPAs -`browser_wait_for_load()` only checks `document.readyState === 'complete'`. SPAs often render content after this. Check for specific elements: - -```bash -browser_navigate("https://my-spa.example.com") -browser_wait_for_load() +`browser_wait_for_load` only checks `readyState`. For React/Vue/etc., poll for the actual element you want: -# Poll for the element to appear +```js browser_execute_js("!!document.querySelector('.loaded-content')") ``` -### Keyboard Modifiers +### Keyboard modifiers -`browser_press_key` supports modifier bitfield combinations: - -| Combo | Bitfield | Example | -|-------|----------|---------| -| Ctrl+C | 2 | `browser_press_key("c", { modifiers: 2 })` | -| Cmd+V | 4 | `browser_press_key("v", { modifiers: 4 })` | -| Ctrl+Shift+T | 10 (2\|8) | `browser_press_key("T", { modifiers: 10 })` | -| Cmd+Shift+N | 12 (4\|8) | `browser_press_key("N", { modifiers: 12 })` | - ---- - -## Self-Extending Harness - -When the built-in tools aren't enough, the agent writes its own: +`browser_press_key` takes a modifier bitfield: `Alt=1`, `Ctrl=2`, `Meta/Cmd=4`, `Shift=8`. Combine with `|`. ``` - ● agent: needs to scrape 47 paginated search result pages - │ - ● calls list_dynamic_tools() → empty - │ - ● calls register_tool({ - │ name: "scrape_paginated", - │ implementation: "..." - │ }) - │ - ✓ next turn: scrape_paginated() is available and callable +browser_press_key("c", { modifiers: 2 }) // Ctrl+C +browser_press_key("v", { modifiers: 4 }) // Cmd+V +browser_press_key("T", { modifiers: 10 }) // Ctrl+Shift+T ``` -### Tool Lifecycle - -1. **`list_dynamic_tools`** — Check what's already registered -2. **`register_tool`** — Inject a new tool with custom JavaScript -3. **`remove_tool`** — Retire a tool you no longer need - -### Dynamic Tool Bindings - -Tools receive these bindings at runtime: - -| Binding | Description | -|---------|-------------| -| `params` | Tool arguments (as passed by the caller) | -| `daemon` | Browser daemon — `daemon.cdp()`, `daemon.evaluateJS()`, `daemon.getPageInfo()` | -| `require` | Node.js `require()` for builtins and installed packages | -| `signal` | `AbortSignal` for cancellation | -| `onUpdate` | Progress callback: `onUpdate({ content: [{ type: 'text', text: '...' }] })` | -| `ctx` | `ExtensionContext` with `cwd`, `sessionManager`, `ui`, `signal` | -| `console`, `fetch`, `JSON`, `Buffer`, `setTimeout`, `clearTimeout` | Standard globals | +### Writing your own tool mid-task -### Example: Structured Data Extractor +When the built-ins aren't enough, `register_tool` injects a new one — available on the next turn. -```javascript -// register_tool implementation: -const info = await daemon.getPageInfo(); -if ("dialog" in info) throw new Error("Dialog is blocking: " + info.dialog.message); - -const data = await daemon.evaluateJS(` - JSON.stringify( - Array.from(document.querySelectorAll('${params.rowSelector}')).map(el => ({ - title: el.querySelector('${params.titleSelector}')?.textContent?.trim(), - price: el.querySelector('${params.priceSelector}')?.textContent?.trim(), - link: el.querySelector('a')?.href, - })) - ) -`); - -const parsed = JSON.parse(data); -return { - content: [{ type: 'text', text: `Extracted ${parsed.length} items:\n\n${JSON.stringify(parsed, null, 2)}` }], - details: { items: parsed, count: parsed.length } -}; +```js +register_tool({ + name: "scrape_results", + implementation: ` + const data = await daemon.evaluateJS(\` + JSON.stringify( + Array.from(document.querySelectorAll('${params.selector}')) + .map(el => el.textContent.trim()) + ) + \`); + return { + content: [{ type: "text", text: data }], + details: { items: JSON.parse(data) }, + }; + `, +}); ``` ---- - -## Architecture - -``` -┌─────────────────────────────────────────────────┐ -│ pi agent │ -│ │ │ -│ ├─ browser_navigate() │ -│ ├─ browser_screenshot() │ -│ ├─ browser_click(x, y) │ -│ ├─ browser_execute_js(...) │ -│ ├─ list_dynamic_tools() │ -│ ├─ register_tool(...) │ -│ └─ remove_tool(...) │ -│ │ -│ pi-browser-harness (TypeScript extension) │ -└────────────────────┬────────────────────────────┘ - │ Unix socket - │ /tmp/bu-<namespace>.sock - ▼ - ┌────────────────┐ - │ daemon.py │ - │ (Python) │ - └───────┬────────┘ - │ CDP WebSocket - │ ws://localhost:9222 - ▼ - ┌────────────────┐ - │ Chrome / Edge │ - │ (user's real │ - │ browser) │ - └────────────────┘ -``` - -### Data Flow - -1. **pi agent** calls a `browser_*` tool (e.g., `browser_screenshot()`) -2. **pi-browser-harness extension** translates the call into a CDP command -3. **JSON-line request** sent over Unix socket to `daemon.py` -4. **daemon.py** relays the command over CDP WebSocket to Chrome -5. **Chrome** executes the command and returns the result -6. Result flows back through the chain to the agent - -### Session Persistence - -- Tab history, daemon namespace, and preferences persist across session reloads and branch navigation -- State is stored via `pi.appendEntry()` in the session's JSONL file -- On `session_start`, state is restored by scanning the branch for the last `browser-harness-state` entry - -### Key Design Decisions - -- **One socket connection per request** — stateless, matches the daemon's internal `_send` pattern -- **Session auto-recovery** — stale CDP sessions are automatically re-attached -- **Compositor-level clicks** — `Input.dispatchMouseEvent` works through iframes, shadow DOM, and cross-origin content without selectors -- **Daemon auto-restart** — if the daemon socket dies mid-request, it's restarted and the request is retried once +Tool implementations receive `params`, `daemon`, `require`, `signal`, `onUpdate`, `ctx`, plus the standard JS globals. --- -## Configuration - -### CLI Flags - -| Flag | Type | Default | Description | -|------|------|---------|-------------| -| `--browser-namespace` | `string` | auto-generated | Daemon namespace (`BU_NAME`). Use to run multiple instances. | -| `--browser-debug-clicks` | `boolean` | `false` | Enable debug click overlay (saves annotated screenshots to `/tmp`). | - -### Commands +## Commands | Command | Description | |---------|-------------| -| `/browser-setup` | Guided setup wizard — installs browser-harness, connects to Chrome | -| `/browser-status` | Show daemon health, current page URL/title/viewport | -| `/browser-reload-daemon` | Restart the browser daemon | +| `/browser-setup` | Connect pi to Chrome (run once) | +| `/browser-status` | Show daemon health and current page | +| `/browser-reload-daemon` | Restart the connection | --- ## Troubleshooting -| Symptom | Likely Cause | Fix | -|---------|-------------|-----| -| Daemon not starting | Chrome remote debugging not enabled | Start Chrome with `--remote-debugging-port=9222` | -| "DevToolsActivePort not found" | Chrome didn't accept the debugging connection | Open `chrome://inspect/#remote-debugging`, tick the checkbox, click Allow | -| Daemon starts but no tools work | Stale or dead CDP session | Run `/browser-reload-daemon` to restart | -| Dialog blocking page | JS `alert`/`confirm`/`prompt` is open | Run `browser_page_info()` to detect it, then `browser_handle_dialog()` | -| Page not fully loaded | SPA with async rendering | After `browser_wait_for_load()`, use `browser_execute_js()` to check for specific elements | -| `uv` not found | Python package manager not installed | Install via `curl -LsSf https://astral.sh/uv/install.sh \| sh` or use the git fallback | -| Socket permission denied | Stale socket file from previous run | `rm /tmp/bu-*.sock /tmp/bu-*.pid` and restart | -| Windows: `ps` not found | `setup.ts` uses Unix commands | Windows detection is best-effort; see manual installation steps above | +| Symptom | Fix | +|---------|-----| +| `DevToolsActivePort not found` | Open `chrome://inspect/#remote-debugging`, tick the checkbox, click Allow, then retry. | +| Connection fails after Chrome restart | Run `/browser-reload-daemon`. | +| Page seems loaded but content is missing | SPA — `browser_wait_for_load` isn't enough. Poll with `browser_execute_js` for the element you need. | +| A JS dialog is blocking actions | `browser_page_info` will report the dialog. Use `browser_handle_dialog` to accept or dismiss. | --- ## Security -> **Warning:** This extension gives your pi agent full control over your real Chrome browser. It can see your open tabs, read page content, submit forms, and interact with authenticated sessions. - -- **No sandboxing** — the agent has access to whatever Chrome has access to -- **Credentials** — if the agent hits an auth wall, it should stop and ask you. Never store credentials in agent prompts. -- **Dynamic tools** — `register_tool` evaluates user-supplied JavaScript in the pi process with full `require` access. Review any dynamic tool implementations you didn't write yourself. -- **Data handling** — Screenshots are saved to `/tmp`. HTTP responses and JS evaluation results may contain sensitive page data. -- **Session persistence** — Tab history and daemon state are stored in pi's session file (`.jsonl`). These contain page URLs and titles. - ---- - -## Contributing - -We welcome contributions! Here's how to get started: - -```bash -# Clone the repository -git clone https://github.com/amankumarsingh77/pi-browser-harness -cd pi-browser-harness - -# Install dependencies -npm install - -# Type-check -npm run typecheck - -# Test locally -pi install ./ # Install from local path -``` - -### Development Guidelines - -- **TypeScript strict mode** — all code must pass `tsc --noEmit` -- **Error handling** — tools must catch errors and return `isError: true` with a descriptive message -- **Tool names** — use `snake_case` for consistency with pi conventions -- **Parameters** — use `typebox` schemas for all tool parameters -- **Prompt guidelines** — every tool must include `promptSnippet` and `promptGuidelines` for LLM context - -### Project Structure - -``` -pi-browser-harness/ -├── src/ -│ ├── index.ts # Extension entry point, lifecycle, flags, commands -│ ├── daemon.ts # BrowserDaemon: spawns daemon.py, CDP transport -│ ├── tools.ts # All browser_* tool registrations -│ ├── dynamic-tools.ts # Self-extending harness (register_tool, etc.) -│ ├── protocol.ts # Unix socket protocol types and low-level transport -│ ├── prompt.ts # System prompt injection -│ ├── renderers.ts # Custom TUI renderers -│ ├── setup.ts # /browser-setup wizard -│ └── state.ts # Session persistence -├── skills/ -│ └── pi-browser-harness/ -│ └── SKILL.md # User-facing skill documentation -├── package.json -├── tsconfig.json -├── LICENSE -├── CHANGELOG.md -└── README.md -``` - ---- - -## Roadmap - -- [ ] Windows-native setup wizard (remove `ps`/`grep` dependency) -- [ ] iframe-target evaluation support in `browser_execute_js` -- [ ] Remote browser support (connect to Chrome running on another machine) -- [ ] Screenshot annotations (click markers, element highlights) -- [ ] Performance benchmarks and CI test suite -- [ ] Configurable screenshot output directory -- [ ] Recording/playback of interaction sequences -- [ ] Better audio/video support for media-heavy pages +This extension drives your real Chrome. The agent can see open tabs, read page content, submit forms, and act inside authenticated sessions. `register_tool` evaluates JavaScript in the pi process with full `require` access — review any dynamic tool you didn't write yourself. --- ## License -MIT — see [LICENSE](LICENSE) for details. - ---- - -<p align="center"> - <sub>Built with ❤️ for the pi ecosystem. Makes coding agents see the web.</sub> -</p> +MIT — see [LICENSE](LICENSE).
M
package.json
→
package.json
@@ -23,7 +23,7 @@ "url": "https://github.com/amankumarsingh77/pi-browser-harness/issues"
}, "homepage": "https://github.com/amankumarsingh77/pi-browser-harness#readme", "engines": { - "node": ">=18.0.0" + "node": ">=22.0.0" }, "files": [ "src/",@@ -50,6 +50,7 @@ },
"devDependencies": { "@mariozechner/pi-coding-agent": "*", "@mariozechner/pi-tui": "*", + "@types/ws": "^8.18.1", "typebox": "*", "typescript": "^5.0.0" },@@ -59,5 +60,8 @@ "prepublishOnly": "tsc --noEmit"
}, "publishConfig": { "access": "public" + }, + "dependencies": { + "ws": "^8.20.0" } }
M
src/daemon.ts
→
src/daemon.ts
@@ -1,49 +1,127 @@
/** - * BrowserDaemon — manages the browser-harness daemon process lifecycle. + * BrowserDaemon — manages a direct CDP WebSocket connection to Chrome. * * Architecture: - * Node.js → Unix socket (/tmp/bu-<namespace>.sock) → daemon.py → CDP WS → Chrome + * Node.js → CDP WebSocket (ws://localhost:9222/...) → Chrome * - * The daemon is spawned as a child process on start() and communicates via - * the protocol defined in protocol.ts. All CDP calls go through this class. + * No Python proxy, no Unix socket. Direct CDP over a single persistent + * WebSocket with message-ID multiplexing, auto-reconnect, and event buffering. */ -import { spawn, type ChildProcess } from "node:child_process"; -import { existsSync } from "node:fs"; -import { access, readFile, unlink, writeFile } from "node:fs/promises"; -import { homedir, tmpdir } from "node:os"; -import { basename, join } from "node:path"; +import { readFile, writeFile } from "node:fs/promises"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { connect as netConnect } from "node:net"; +import WebSocket from "ws"; import { - sendRequest, - daemonAlive, - healthProbe, - isInternalUrl, type CDPEvent, - type DaemonRequest, - type DaemonResponse, type DaemonStatus, type DialogInfo, type PageInfo, type PageInfoResult, type RemoteConfig, type TabInfo, + isInternalUrl, } from "./protocol"; // ── Constants ──────────────────────────────────────────────────────────────── -/** Common locations where browser-harness might be cloned */ -const BH_SEARCH_PATHS = [ - join(homedir(), "Developer", "browser-harness"), - join(homedir(), "src", "browser-harness"), - join(homedir(), "browser-harness"), - join(homedir(), "dev", "browser-harness"), - join(homedir(), "Projects", "browser-harness"), -]; +const CDP_CONNECT_TIMEOUT_MS = 10_000; +const CDP_REQUEST_TIMEOUT_MS = 15_000; +const CDP_RECONNECT_BASE_MS = 500; +const CDP_MAX_RECONNECT_ATTEMPTS = 5; +const PORT_PROBE_DEADLINE_MS = 30_000; +const PORT_PROBE_INTERVAL_MS = 1_000; + +/** + * Discover Chrome's CDP WebSocket URL by reading the per-profile + * `DevToolsActivePort` file written when remote debugging is enabled + * via chrome://inspect/#remote-debugging. Mirrors browser-harness/daemon.py. + */ +async function discoverWsUrl(): Promise<string> { + const dirs = chromeProfileDirs(); + const tried: string[] = []; + for (const base of dirs) { + const portFile = join(base, "DevToolsActivePort"); + let raw: string; + try { + raw = await readFile(portFile, "utf8"); + } catch { + continue; + } + tried.push(base); + const lines = raw.trim().split("\n"); + if (lines.length < 2) continue; + const port = lines[0].trim(); + const path = lines[1].trim(); + if (!port || !path) continue; -const DAEMON_START_TIMEOUT_MS = 30_000; -const DAEMON_POLL_INTERVAL_MS = 200; -const DAEMON_SHUTDOWN_TIMEOUT_MS = 5_000; -const SESSION_ATTACH_TIMEOUT_MS = 5_000; + await waitForPort(Number(port)); + return `ws://127.0.0.1:${port}${path}`; + } + throw new Error( + `DevToolsActivePort not found in ${dirs.join(", ")} — open chrome://inspect/#remote-debugging in your browser, tick the checkbox, click Allow, then retry. Or set BU_CDP_WS to a remote browser endpoint.`, + ); +} + +/** Probe TCP port until reachable or deadline expires. */ +async function waitForPort(port: number): Promise<void> { + const deadline = Date.now() + PORT_PROBE_DEADLINE_MS; + let lastErr: Error | null = null; + while (Date.now() < deadline) { + try { + await new Promise<void>((resolve, reject) => { + const sock = netConnect({ host: "127.0.0.1", port }); + const onError = (err: Error) => { + sock.destroy(); + reject(err); + }; + sock.setTimeout(1000, () => onError(new Error("probe timeout"))); + sock.once("error", onError); + sock.once("connect", () => { + sock.end(); + resolve(); + }); + }); + return; + } catch (err) { + lastErr = err instanceof Error ? err : new Error(String(err)); + await new Promise((r) => setTimeout(r, PORT_PROBE_INTERVAL_MS)); + } + } + throw new Error( + `Chrome's remote-debugging page is open, but DevTools is not live yet on 127.0.0.1:${port} — if Chrome opened a profile picker, choose your normal profile first, then tick the checkbox and click Allow if shown (last error: ${lastErr?.message ?? "unknown"})`, + ); +} + +// Chrome / Edge / Chromium profile directories that may contain DevToolsActivePort. +// Mirrors browser-harness/daemon.py PROFILES. +function chromeProfileDirs(): string[] { + const home = homedir(); + return [ + join(home, "Library/Application Support/Google/Chrome"), + join(home, "Library/Application Support/Microsoft Edge"), + join(home, "Library/Application Support/Microsoft Edge Beta"), + join(home, "Library/Application Support/Microsoft Edge Dev"), + join(home, "Library/Application Support/Microsoft Edge Canary"), + join(home, ".config/google-chrome"), + join(home, ".config/chromium"), + join(home, ".config/chromium-browser"), + join(home, ".config/microsoft-edge"), + join(home, ".config/microsoft-edge-beta"), + join(home, ".config/microsoft-edge-dev"), + join(home, ".var/app/org.chromium.Chromium/config/chromium"), + join(home, ".var/app/com.google.Chrome/config/google-chrome"), + join(home, ".var/app/com.brave.Browser/config/BraveSoftware/Brave-Browser"), + join(home, ".var/app/com.microsoft.Edge/config/microsoft-edge"), + join(home, "AppData/Local/Google/Chrome/User Data"), + join(home, "AppData/Local/Chromium/User Data"), + join(home, "AppData/Local/Microsoft/Edge/User Data"), + join(home, "AppData/Local/Microsoft/Edge Beta/User Data"), + join(home, "AppData/Local/Microsoft/Edge Dev/User Data"), + join(home, "AppData/Local/Microsoft/Edge SxS/User Data"), + ]; +} const VIRTUAL_KEY_CODES: Record<string, number> = { Enter: 13,@@ -66,20 +144,23 @@ // ── BrowserDaemon ────────────────────────────────────────────────────────────
export class BrowserDaemon { readonly namespace: string; - readonly socketPath: string; - readonly pidPath: string; - readonly logPath: string; - private proc: ChildProcess | null = null; + private ws: WebSocket | null = null; private _sessionId: string | null = null; - private _pid: number | null = null; + private _targetId: string | null = null; + private _currentDialog: DialogInfo | null = null; + private _eventBuffer: CDPEvent[] = []; + private _pending = new Map<number, PendingRequest>(); + private _nextId = 1; + private _connectResolve: ((v: void) => void) | null = null; + private _connectReject: ((err: Error) => void) | null = null; + private _connectTimer: ReturnType<typeof setTimeout> | null = null; + private _reconnectAttempts = 0; + private _stopping = false; private remote: RemoteConfig | null = null; constructor(namespace = "default", remote?: RemoteConfig) { this.namespace = namespace; - this.socketPath = `/tmp/bu-${namespace}.sock`; - this.pidPath = `/tmp/bu-${namespace}.pid`; - this.logPath = `/tmp/bu-${namespace}.log`; this.remote = remote ?? null; }@@ -87,19 +168,15 @@ // ── Public getters ───────────────────────────────────────────────────────
get sessionId(): string | null { return this._sessionId; - } - - get pid(): number | null { - return this._pid; } getStatus(): DaemonStatus { return { - alive: this._sessionId !== null, + alive: this._sessionId !== null && this.ws?.readyState === WebSocket.OPEN, sessionId: this._sessionId, - pid: this._pid, + pid: null, namespace: this.namespace, - socketPath: this.socketPath, + socketPath: "", remoteBrowserId: this.remote?.browserId, }; }@@ -107,153 +184,92 @@
// ── Lifecycle ──────────────────────────────────────────────────────────── /** - * Start the daemon process and wait for it to become ready. - * Idempotent — returns immediately if already alive and healthy. + * Connect to Chrome via CDP WebSocket. + * Idempotent — returns immediately if already connected and healthy. */ async start(): Promise<void> { - // Already healthy? - if (await healthProbe(this.socketPath)) { - await this.refreshSessionId(); - return; - } - - // Stale socket but daemon alive? Stop it first. - if (await daemonAlive(this.socketPath)) { - await this.stop(); - } - - const bhDir = await findBrowserHarnessDir(); - const env = this.buildEnv(); - - this.proc = spawn("uv", ["run", "daemon.py"], { - cwd: bhDir, - env: { ...process.env, ...env }, - stdio: ["ignore", "ignore", "ignore"], - detached: true, - }); - this.proc.unref(); - - if (this.proc.pid) { - this._pid = this.proc.pid; + if (this._sessionId && this.ws?.readyState === WebSocket.OPEN) { + return; // Already connected } - // Poll until socket accepts connections - const deadline = Date.now() + DAEMON_START_TIMEOUT_MS; - let lastError = ""; + this._stopping = false; + this._reconnectAttempts = 0; - while (Date.now() < deadline) { - if (this.proc.exitCode !== null) { - lastError = await this.readLogTail(); + try { + await this.connect(); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + if (this.needsChromeSetup(msg)) { throw new Error( - `Browser daemon exited (code ${this.proc.exitCode}): ${lastError || "unknown error"}`, + `Chrome remote debugging not enabled. Open chrome://inspect/#remote-debugging and tick the Allow checkbox, then retry.\n\nDetails: ${msg}`, ); } - - if (await daemonAlive(this.socketPath)) { - // Verify CDP is healthy (not just socket) - if (await healthProbe(this.socketPath)) { - await this.refreshSessionId(); - return; - } - // Socket alive but CDP not ready — keep polling - } - - await sleep(DAEMON_POLL_INTERVAL_MS); + throw err; } - - lastError = await this.readLogTail(); - // Check if Chrome needs remote debugging setup - if (this.needsChromeSetup(lastError)) { - throw new Error( - `Chrome remote debugging not enabled. Start Chrome with --remote-debugging-port=9222, then retry.\n\nDaemon log: ${lastError}`, - ); - } - - throw new Error( - `Daemon "${this.namespace}" failed to start after ${DAEMON_START_TIMEOUT_MS / 1000}s. ` + - `Log: ${lastError || "no log output"}`, - ); } /** - * Gracefully stop the daemon: send shutdown, wait for exit, force-kill if needed. + * Gracefully disconnect from Chrome. */ async stop(): Promise<void> { - // Send shutdown via socket - try { - await sendRequest(this.socketPath, { meta: "shutdown" }, 3000); - } catch { - // Socket may already be dead — that's fine + this._stopping = true; + + // Reject any pending connect + if (this._connectReject) { + this._connectReject(new Error("Daemon stopped")); + this._connectReject = null; + } + if (this._connectTimer) { + clearTimeout(this._connectTimer); + this._connectTimer = null; } - // Wait for process to exit - if (this.proc && this.proc.exitCode === null) { - const deadline = Date.now() + DAEMON_SHUTDOWN_TIMEOUT_MS; - while (Date.now() < deadline) { - if (this.proc.exitCode !== null) break; - await sleep(100); - } - - // Force kill if still alive - if (this.proc.exitCode === null) { - try { - this.proc.kill("SIGTERM"); - } catch { - // already dead - } - } + // Reject all pending requests + for (const [id, req] of this._pending) { + req.reject(new Error("Daemon stopped")); + this._pending.delete(id); } - // Clean up files - for (const f of [this.socketPath, this.pidPath]) { + // Close the WebSocket + if (this.ws) { try { - await unlink(f); + this.ws.close(1000, "Shutdown"); } catch { - // file doesn't exist + // best-effort } + this.ws = null; } - this.proc = null; this._sessionId = null; - this._pid = null; + this._targetId = null; + this._currentDialog = null; + this._eventBuffer = []; } /** - * Ensure daemon is alive and healthy. Restarts if stale. + * Ensure daemon is alive and healthy. Reconnects if stale. * Call before every tool execution. */ async ensureAlive(): Promise<void> { - if (!(await healthProbe(this.socketPath))) { - await this.stop(); - await this.start(); + const isOpen = this.ws?.readyState === WebSocket.OPEN; + if (isOpen && this._sessionId) { + // Quick health check with a lightweight CDP call + try { + await this.cdp("Target.getTargets", {}); + return; + } catch { + // Connection is stale — reconnect + } } + + await this.stop(); + await this.start(); } // ── CDP Communication ──────────────────────────────────────────────────── /** - * Send a raw request to the daemon and get the response. - */ - async send(request: DaemonRequest): Promise<DaemonResponse> { - await this.ensureAlive(); - - try { - const response = await sendRequest(this.socketPath, request); - return response; - } catch (err) { - // Socket died mid-request — restart and retry once - const message = err instanceof Error ? err.message : String(err); - if (message.includes("ENOENT") || message.includes("ECONNREFUSED") || message.includes("EPIPE")) { - await this.stop(); - await this.start(); - return sendRequest(this.socketPath, request); - } - throw err; - } - } - - /** - * Convenience: send a CDP method and return the result. + * Send a CDP method and return the result. * Handles session management — Target.* methods skip session, * others use explicit sessionId or the daemon's default session. */@@ -262,50 +278,33 @@ method: string,
params?: Record<string, unknown>, sessionId?: string | null, ): Promise<unknown> { - // Browser-level Target.* calls must NOT use a session - const sid = method.startsWith("Target.") ? null : (sessionId ?? this._sessionId); + // Browser-level Target.* calls must NOT use a session. + // Everything else uses the explicit sessionId (any session attached + // with flatten:true on this connection is reachable via the top-level + // sessionId field — that's what flat-protocol mode means). + const effectiveSid = method.startsWith("Target.") ? null : (sessionId ?? this._sessionId); - const response = await this.send({ - method, - params: params ?? {}, - session_id: sid, - }); - - if (response.error) { - // Auto-recover from stale session - if ( - response.error.includes("Session with given id not found") && - sid === this._sessionId - ) { + try { + return await this.sendRawCdp(method, params ?? {}, effectiveSid); + } catch (err) { + // Auto-recover from stale default session — retry once + if (err instanceof SessionNotFoundError && effectiveSid === this._sessionId) { await this.attachFirstPage(); - const retry = await this.send({ - method, - params: params ?? {}, - session_id: this._sessionId, - }); - if (retry.error) { - throw new Error(`CDP error after session recovery (${method}): ${retry.error}`); - } - return retry.result; + return this.sendRawCdp(method, params ?? {}, this._sessionId); } - throw new Error(`CDP error (${method}): ${response.error}`); + throw err; } - - return response.result; } // ── High-level Browser Helpers ──────────────────────────────────────────── /** Get page state or dialog info */ async getPageInfo(): Promise<PageInfoResult> { - // Check for JS dialog first - try { - const dialogResp = await sendRequest(this.socketPath, { meta: "pending_dialog" }); - if (dialogResp.dialog) { - return { dialog: dialogResp.dialog }; - } - } catch { - // ignore — dialog check is best-effort + // Check for buffered dialog first + if (this._currentDialog) { + const dialog = this._currentDialog; + this._currentDialog = null; // consume it + return { dialog }; } const js = `JSON.stringify({url:location.href,title:document.title,w:innerWidth,h:innerHeight,sx:scrollX,sy:scrollY,pw:document.documentElement.scrollWidth,ph:document.documentElement.scrollHeight})`;@@ -340,9 +339,14 @@ }
return tabs; } - /** Get current tab info */ + /** Get current tab info — uses the targetId we attached to. */ async currentTab(): Promise<TabInfo> { - const result = (await this.cdp("Target.getTargetInfo")) as { + if (!this._targetId) { + throw new Error("No tab attached. Call newTab/switchTab first."); + } + const result = (await this.cdp("Target.getTargetInfo", { + targetId: this._targetId, + })) as { targetInfo: { targetId: string; url: string; title: string }; }; const t = result.targetInfo;@@ -370,12 +374,17 @@ targetId,
flatten: true, })) as { sessionId: string }; - // Tell the daemon to use this as default session this._sessionId = result.sessionId; - await sendRequest(this.socketPath, { - meta: "set_session", - session_id: result.sessionId, - }); + this._targetId = targetId; + + // Enable core domains for the new session + for (const domain of ["Page", "DOM", "Runtime", "Network"]) { + try { + await this.cdp(`${domain}.enable`); + } catch { + // best-effort + } + } // Mark the tab with a green circle for user visibility try {@@ -394,7 +403,7 @@ if (expression.includes("return ") && !expression.trim().startsWith("(")) {
expression = `(function(){${expression}})()`; } - const sid = targetId || null; // explicit target or daemon default + const sid = targetId || null; const result = (await this.cdp( "Runtime.evaluate", {@@ -411,10 +420,11 @@ }
return result.result?.value; } - /** Drain buffered CDP events from the daemon */ + /** Drain buffered CDP events from the internal buffer */ async drainEvents(): Promise<CDPEvent[]> { - const resp = await sendRequest(this.socketPath, { meta: "drain_events" }); - return resp.events ?? []; + const events = this._eventBuffer; + this._eventBuffer = []; + return events; } /** Get virtual key code for a key name or character */@@ -422,16 +432,206 @@ getVirtualKeyCode(key: string): number {
return VIRTUAL_KEY_CODES[key] ?? (key.length === 1 ? key.charCodeAt(0) : 0); } - /** Get the key code string for CDP (uses the key name or the key itself) */ + /** Get the key code string for CDP */ getKeyCode(key: string): string { return key.length === 1 && !VIRTUAL_KEY_CODES[key] ? key : key; } - // ── Internal Helpers ────────────────────────────────────────────────────── + // ── Internal: WebSocket Connection ──────────────────────────────────────── + + private async connect(): Promise<void> { + const timeoutMs = CDP_CONNECT_TIMEOUT_MS; + const wsUrl = this.remote?.cdpUrl + ? this.remote.cdpUrl + : process.env.BU_CDP_WS + ? process.env.BU_CDP_WS + : await discoverWsUrl(); + + if (!this.remote) { + this.remote = { + cdpUrl: wsUrl, + browserId: wsUrl.split("/").pop() || "unknown", + }; + } + + await this.openWebSocket(wsUrl, timeoutMs); + await this.attachFirstPage(); + } + + private openWebSocket(url: string, timeoutMs: number): Promise<void> { + return new Promise((resolve, reject) => { + this._connectResolve = resolve; + this._connectReject = reject; + + this._connectTimer = setTimeout(() => { + this._connectTimer = null; + this._connectReject = null; + if (this.ws) { + try { this.ws.close(); } catch { /* ignore */ } + this.ws = null; + } + reject(new Error(`CDP WebSocket connection timed out after ${timeoutMs}ms`)); + }, timeoutMs); + + try { + this.ws = new WebSocket(url); + } catch (err) { + clearTimeout(this._connectTimer!); + this._connectTimer = null; + reject(err); + return; + } + + this.ws.onopen = () => { + if (this._connectTimer) { + clearTimeout(this._connectTimer); + this._connectTimer = null; + } + this._connectResolve = null; + this._connectReject = null; + this._reconnectAttempts = 0; + resolve(); + }; + + this.ws.onmessage = (event: WebSocket.MessageEvent) => { + const data = event.data; + this.handleMessage(typeof data === "string" ? data : data.toString()); + }; + + this.ws.onerror = (_event: WebSocket.ErrorEvent) => { + // Only reject during the connect phase; otherwise it means a disconnect + if (this._connectReject) { + const err = new Error("CDP WebSocket error during connection"); + this._connectReject(err); + this._connectReject = null; + if (this._connectTimer) { + clearTimeout(this._connectTimer); + this._connectTimer = null; + } + } + }; + + this.ws.onclose = (_event: WebSocket.CloseEvent) => { + // If we're shutting down intentionally, don't reconnect + if (this._stopping) return; + + // Auto-reconnect with backoff + if (this._reconnectAttempts < CDP_MAX_RECONNECT_ATTEMPTS) { + this._sessionId = null; + const delay = Math.min( + CDP_RECONNECT_BASE_MS * 2 ** this._reconnectAttempts, + 15_000, + ); + this._reconnectAttempts++; + setTimeout(() => { + if (!this._stopping) { + this.start().catch(() => { + // Reconnection ultimately failed — tools will report errors + }); + } + }, delay); + } + }; + }); + } + + // ── Internal: Message Handling ──────────────────────────────────────────── + + private handleMessage(data: string): void { + let msg: CDPMessage; + try { + msg = JSON.parse(data); + } catch { + return; // Corrupt frame, ignore + } + + // CDP events have no `id` — they have a `method` field + if (msg.id === undefined) { + // Dialog detection + if (msg.method === "Page.javascriptDialogOpening") { + this._currentDialog = { + type: (msg.params?.type as DialogInfo["type"]) || "alert", + message: (msg.params?.message as string) || "", + defaultPrompt: msg.params?.defaultPrompt as string | undefined, + }; + return; + } + + // Dialog closed — clear the stored dialog + if (msg.method === "Page.javascriptDialogClosed") { + this._currentDialog = null; + return; + } + + // Buffer other events for drainEvents() + this._eventBuffer.push({ + method: msg.method!, + params: msg.params as unknown, + session_id: msg.sessionId as string | undefined, + }); + return; + } + + // It's a response to a pending request + const pending = this._pending.get(msg.id); + if (!pending) return; // Stale response, ignore + + this._pending.delete(msg.id); + clearTimeout(pending.timer); + + if (msg.error) { + // Handle stale session recovery + const errMsg = msg.error.message || String(msg.error); + if ( + errMsg.includes("Session with given id not found") && + pending.sessionId === this._sessionId + ) { + // Schedule recovery and reject — caller can retry + pending.reject(new SessionNotFoundError(errMsg)); + return; + } + pending.reject(new Error(`CDP error (${pending.method}): ${errMsg}`)); + return; + } + + pending.resolve(msg.result); + } + + /** Send a CDP command directly to the browser session */ + private sendRawCdp( + method: string, + params: Record<string, unknown>, + sessionId: string | null, + ): Promise<unknown> { + if (!this.ws || this.ws.readyState !== WebSocket.OPEN) { + return Promise.reject(new Error("Browser not connected. Is Chrome running?")); + } + + const id = this._nextId++; + const payload: Record<string, unknown> = { id, method, params }; + if (sessionId) payload.sessionId = sessionId; + const msg = JSON.stringify(payload); + + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + this._pending.delete(id); + reject(new Error(`CDP timeout after ${CDP_REQUEST_TIMEOUT_MS}ms: ${method}`)); + }, CDP_REQUEST_TIMEOUT_MS); + + this._pending.set(id, { resolve, reject, timer, method, sessionId }); + try { + this.ws!.send(msg); + } catch (err) { + clearTimeout(timer); + this._pending.delete(id); + reject(err); + } + }); + } /** Attach to the first real page target. Creates about:blank if none exist. */ private async attachFirstPage(): Promise<string> { - const result = (await this.cdp("Target.getTargets")) as { + const result = (await this.sendRawCdp("Target.getTargets", {}, null)) as { targetInfos: Array<{ targetId: string; type: string; url: string }>; };@@ -440,112 +640,91 @@ (t) => t.type === "page" && !isInternalUrl(t.url),
); if (pages.length === 0) { - const created = (await this.cdp("Target.createTarget", { - url: "about:blank", - })) as { targetId: string }; + const created = (await this.sendRawCdp( + "Target.createTarget", + { url: "about:blank" }, + null, + )) as { targetId: string }; pages.push({ targetId: created.targetId, type: "page", url: "about:blank" }); } - const attachResult = (await this.cdp("Target.attachToTarget", { - targetId: pages[0].targetId, - flatten: true, - })) as { sessionId: string }; + const attachResult = (await this.sendRawCdp( + "Target.attachToTarget", + { targetId: pages[0].targetId, flatten: true }, + null, + )) as { sessionId: string }; this._sessionId = attachResult.sessionId; + this._targetId = pages[0].targetId; // Enable core domains for (const domain of ["Page", "DOM", "Runtime", "Network"]) { try { - await this.cdp(`${domain}.enable`); + await this.sendRawCdp(`${domain}.enable`, {}, this._sessionId); } catch { // best-effort } } - return attachResult.sessionId; - } - - /** Refresh session ID from the daemon's current session */ - private async refreshSessionId(): Promise<void> { + // Mark tab try { - const resp = await sendRequest(this.socketPath, { meta: "session" }); - if (resp.session_id) { - this._sessionId = resp.session_id; - } + await this.sendRawCdp( + "Runtime.evaluate", + { + expression: `if(!document.title.startsWith('🟢'))document.title='🟢 '+document.title`, + }, + this._sessionId, + ); } catch { - // will be set on first attach + // best-effort } + + return attachResult.sessionId; } - /** Build environment variables for the daemon process */ - private buildEnv(): Record<string, string> { - const env: Record<string, string> = { - BU_NAME: this.namespace, - }; - - if (this.remote) { - env.BU_CDP_WS = this.remote.cdpUrl; - env.BU_BROWSER_ID = this.remote.browserId; - } - - return env; + /** Auto-recover from a stale session error by re-attaching */ + async recoverSession(): Promise<void> { + await this.attachFirstPage(); } - /** Read the last line of the daemon log */ - private async readLogTail(): Promise<string> { - try { - const content = await readFile(this.logPath, "utf-8"); - const lines = content.trim().split("\n"); - return lines[lines.length - 1] || ""; - } catch { - return ""; - } - } + // ── Error diagnostics ──────────────────────────────────────────────────── - /** Check if the error message suggests Chrome needs remote debugging setup */ private needsChromeSetup(errorMsg: string): boolean { const lower = errorMsg.toLowerCase(); return ( - lower.includes("devtoolsactiveport not found") || - lower.includes("enable chrome://inspect") || - lower.includes("not live yet") || - (lower.includes("ws handshake failed") && + lower.includes("devtoolsactiveport") || + lower.includes("cannot reach chrome") || + lower.includes("ecode") && lower.includes("econnrefused") || + (lower.includes("ws handshake") && (lower.includes("403") || lower.includes("timed out"))) ); } } -// ── Utility ────────────────────────────────────────────────────────────────── +// ── Internal Types ─────────────────────────────────────────────────────────── -function sleep(ms: number): Promise<void> { - return new Promise((resolve) => setTimeout(resolve, ms)); +interface PendingRequest { + resolve: (value: unknown) => void; + reject: (err: Error) => void; + timer: ReturnType<typeof setTimeout>; + method: string; + sessionId: string | null; } -/** - * Find the browser-harness repository directory. - * Searches common locations and falls back to cwd. - */ -async function findBrowserHarnessDir(): Promise<string> { - // Check common locations - for (const dir of BH_SEARCH_PATHS) { - if ( - existsSync(join(dir, "daemon.py")) && - existsSync(join(dir, "helpers.py")) - ) { - return dir; - } - } +interface CDPMessage { + id?: number; + method?: string; + params?: Record<string, unknown>; + result?: unknown; + error?: { message: string; code?: number }; + sessionId?: string; +} + +// ── Error Classes ──────────────────────────────────────────────────────────── - // Check cwd - const cwd = process.cwd(); - if (existsSync(join(cwd, "daemon.py")) && existsSync(join(cwd, "helpers.py"))) { - return cwd; +class SessionNotFoundError extends Error { + constructor(msg: string) { + super(msg); + this.name = "SessionNotFoundError"; } - - throw new Error( - "browser-harness not found. Clone it first:\n" + - " git clone https://github.com/browser-use/browser-harness ~/Developer/browser-harness\n" + - " cd ~/Developer/browser-harness && uv sync\n\n" + - `Searched: ${BH_SEARCH_PATHS.join(", ")}`, - ); }
M
src/dynamic-tools.ts
→
src/dynamic-tools.ts
@@ -26,7 +26,7 @@ interface DynamicToolDef {
name: string; label: string; description: string; - execute: (params: Record<string, unknown>, signal: AbortSignal | undefined, onUpdate: ((update: unknown) => void) | undefined, ctx: unknown) => Promise<{ content: Array<{ type: "text"; text: string }>; details?: Record<string, unknown> }>; + execute: (params: Record<string, unknown>, signal: AbortSignal | undefined, onUpdate: ((update: unknown) => void) | undefined, ctx: unknown) => Promise<{ content: Array<{ type: "text"; text: string }>; details: Record<string, unknown> | undefined }>; } // ── AsyncFunction constructor ────────────────────────────────────────────────@@ -247,7 +247,7 @@ Buffer,
setTimeout, clearTimeout, ); - return result as { content: Array<{ type: "text"; text: string }>; details?: Record<string, unknown> }; + return result as { content: Array<{ type: "text"; text: string }>; details: Record<string, unknown> | undefined }; }, };@@ -267,7 +267,7 @@ }
return tool.execute( (execParams ?? {}) as Record<string, unknown>, execSignal, - execOnUpdate, + execOnUpdate as ((update: unknown) => void) | undefined, execCtx, ); },
M
src/index.ts
→
src/index.ts
@@ -57,7 +57,7 @@ // ═══════════════════════════════════════════════════════════════════════════
// Commands // ═══════════════════════════════════════════════════════════════════════════ pi.registerCommand("browser-status", { - description: "Show browser daemon status and current page", + description: "Show browser connection status and current page", handler: async (_args, ctx) => { if (!daemon) { ctx.ui.notify("Browser daemon not started. Run /browser-setup first.", "warning");@@ -66,15 +66,12 @@ }
const status = daemon.getStatus(); const lines = [ - `Browser Daemon: ${status.alive ? "🟢 Connected" : "🔴 Disconnected"}`, - `Namespace: ${status.namespace}`, - `Socket: ${status.socketPath}`, + `Browser: ${status.alive ? "🟢 Connected" : "🔴 Disconnected"}`, `Session: ${status.sessionId || "none"}`, - `PID: ${status.pid || "none"}`, ]; if (status.remoteBrowserId) { - lines.push(`Remote Browser: ${status.remoteBrowserId}`); + lines.push(`Browser ID: ${status.remoteBrowserId}`); } // Try to get current page info
M
src/protocol.ts
→
src/protocol.ts
@@ -1,35 +1,9 @@
/** - * Protocol types and low-level socket communication for the browser-harness daemon. + * Protocol types for pi-browser-harness. * - * The daemon exposes a Unix socket at /tmp/bu-<NAME>.sock. - * Protocol: one JSON-line request → one JSON-line response per connection. + * Defines the shape of CDP communication and browser state. + * No socket transport — the daemon talks CDP directly over WebSocket. */ - -import { createConnection } from "node:net"; - -// ── Request ────────────────────────────────────────────────────────────────── - -export interface DaemonRequest { - /** CDP method (e.g. "Page.navigate") — absent for meta requests */ - method?: string; - /** CDP parameters */ - params?: Record<string, unknown>; - /** CDP session ID — null/absent for browser-level (Target.*) calls */ - session_id?: string | null; - /** Meta command: "drain_events" | "set_session" | "pending_dialog" | "shutdown" */ - meta?: string; -} - -// ── Response ───────────────────────────────────────────────────────────────── - -export interface DaemonResponse { - result?: unknown; - error?: string; - events?: CDPEvent[]; - session_id?: string; - dialog?: DialogInfo; - ok?: boolean; -} // ── CDP Event ────────────────────────────────────────────────────────────────@@ -75,9 +49,9 @@
export interface DaemonStatus { alive: boolean; sessionId: string | null; - pid: number | null; + pid: number | null; // deprecated — always null (no subprocess) namespace: string; - socketPath: string; + socketPath: string; // deprecated — always "" (no socket) remoteBrowserId?: string; }@@ -101,101 +75,3 @@
export function isInternalUrl(url: string): boolean { return INTERNAL_PREFIXES.some((p) => url.startsWith(p)); } - -// ── Low-level socket send ──────────────────────────────────────────────────── - -/** - * Send a single JSON-line request to the daemon socket and read one JSON-line response. - * Opens a fresh connection per request (stateless, matches helpers.py _send pattern). - */ -export function sendRequest( - socketPath: string, - request: DaemonRequest, - timeoutMs = 15_000, -): Promise<DaemonResponse> { - return new Promise((resolve, reject) => { - const socket = createConnection(socketPath); - - const timer = setTimeout(() => { - socket.destroy(); - reject(new Error(`Daemon request timed out after ${timeoutMs}ms: ${JSON.stringify(request.method ?? request.meta)}`)); - }, timeoutMs); - - let buffer = ""; - - socket.on("connect", () => { - socket.write(JSON.stringify(request) + "\n"); - }); - - socket.on("data", (chunk: Buffer) => { - buffer += chunk.toString(); - // Response is one JSON line ending with \n - if (buffer.includes("\n")) { - clearTimeout(timer); - const line = buffer.split("\n")[0]; - try { - const response: DaemonResponse = JSON.parse(line); - socket.end(); - resolve(response); - } catch (e) { - socket.end(); - reject(new Error(`Failed to parse daemon response: ${line.slice(0, 200)}`)); - } - } - }); - - socket.on("error", (err: NodeJS.ErrnoException) => { - clearTimeout(timer); - socket.destroy(); - reject(new Error(`Daemon socket error (${socketPath}): ${err.message}`)); - }); - - socket.on("close", () => { - clearTimeout(timer); - if (buffer.length === 0) { - reject(new Error(`Daemon socket closed without response (${socketPath})`)); - } - }); - }); -} - -/** - * Quick check if the daemon socket accepts connections. - */ -export function daemonAlive(socketPath: string, timeoutMs = 1000): Promise<boolean> { - return new Promise((resolve) => { - const socket = createConnection(socketPath); - const timer = setTimeout(() => { - socket.destroy(); - resolve(false); - }, timeoutMs); - - socket.on("connect", () => { - clearTimeout(timer); - socket.end(); - resolve(true); - }); - - socket.on("error", () => { - clearTimeout(timer); - resolve(false); - }); - }); -} - -/** - * True health check: send Target.getTargets and verify "result" key in response. - * A stale daemon can accept socket connections but have a dead CDP websocket. - */ -export async function healthProbe(socketPath: string): Promise<boolean> { - try { - const response = await sendRequest( - socketPath, - { method: "Target.getTargets", params: {} }, - 3000, - ); - return "result" in response && response.result !== undefined; - } catch { - return false; - } -}
M
src/setup.ts
→
src/setup.ts
@@ -1,18 +1,15 @@
/** - * Setup command — guides the user through installing and connecting - * browser-harness to their Chrome browser. + * Setup command — verifies Chrome is running with remote debugging + * and connects the pi agent to it. */ import type { ExtensionAPI, ExtensionContext } from "@mariozechner/pi-coding-agent"; import { execSync } from "node:child_process"; -import { existsSync } from "node:fs"; -import { homedir } from "node:os"; -import { join } from "node:path"; import type { BrowserDaemon } from "./daemon"; export function registerSetupCommand(pi: ExtensionAPI, daemon: BrowserDaemon): void { pi.registerCommand("browser-setup", { - description: "Install and connect browser-harness to Chrome", + description: "Connect pi to your Chrome browser", handler: async (_args, ctx) => { await runSetup(ctx, daemon); },@@ -20,30 +17,9 @@ });
} async function runSetup(ctx: ExtensionContext, daemon: BrowserDaemon): Promise<void> { - ctx.ui.notify("browser-harness setup: checking installation...", "info"); + ctx.ui.notify("browser setup: checking Chrome...", "info"); - // Step 1: Check for browser-harness installation - const bhInstalled = await checkBrowserHarnessInstalled(); - if (!bhInstalled) { - ctx.ui.notify( - "browser-harness not found. Installing...", - "warning", - ); - const installed = await installBrowserHarness(ctx); - if (!installed) { - ctx.ui.notify( - "Installation failed. Please install manually:\n" + - " git clone https://github.com/browser-use/browser-harness ~/Developer/browser-harness\n" + - " cd ~/Developer/browser-harness && uv sync", - "error", - ); - return; - } - } - - ctx.ui.notify("browser-harness found ✓", "info"); - - // Step 2: Check Chrome is running + // Step 1: Check Chrome is running const chromeRunning = checkChromeRunning(); if (!chromeRunning) { ctx.ui.notify(@@ -54,23 +30,26 @@ return;
} ctx.ui.notify("Chrome is running ✓", "info"); - // Step 3: Try to start the daemon - ctx.ui.notify("Connecting to Chrome...", "info"); + // Step 2: Try to connect to Chrome DevTools + ctx.ui.notify("Connecting to Chrome DevTools...", "info"); try { await daemon.start(); - ctx.ui.notify("Daemon connected ✓", "info"); + ctx.ui.notify("Connected to Chrome ✓", "info"); } catch (err) { const msg = err instanceof Error ? err.message : String(err); const lower = msg.toLowerCase(); if ( lower.includes("devtoolsactiveport") || - lower.includes("remote debugging") + lower.includes("remote debugging") || + lower.includes("econnrefused") || + lower.includes("cannot reach chrome devtools") ) { ctx.ui.notify( "Chrome remote debugging needs to be enabled.\n\n" + - "Start Chrome with: --remote-debugging-port=9222\n" + - "Then run /browser-setup again.", + "Open chrome://inspect/#remote-debugging in your browser, tick the\n" + + "\"Discover network targets\" / Allow checkbox, then run /browser-setup again.\n\n" + + "Or set BU_CDP_WS to a remote browser WebSocket URL.", "warning", ); return;@@ -80,10 +59,10 @@ ctx.ui.notify(`Connection failed: ${msg}`, "error");
return; } - // Step 4: Verify with test navigation + // Step 3: Verify with test navigation ctx.ui.notify("Testing browser control...", "info"); try { - await daemon.newTab("https://github.com/browser-use/browser-harness"); + await daemon.newTab("https://github.com"); const info = await daemon.getPageInfo(); if ("dialog" in info) { await daemon.cdp("Page.handleJavaScriptDialog", { accept: true });@@ -97,66 +76,6 @@ ctx.ui.notify(
`Browser connected but test navigation failed: ${err instanceof Error ? err.message : String(err)}`, "warning", ); - } -} - -// ── Helpers ────────────────────────────────────────────────────────────────── - -async function checkBrowserHarnessInstalled(): Promise<boolean> { - const searchPaths = [ - join(homedir(), "Developer", "browser-harness"), - join(homedir(), "src", "browser-harness"), - join(homedir(), "browser-harness"), - join(homedir(), "dev", "browser-harness"), - join(homedir(), "Projects", "browser-harness"), - process.cwd(), - ]; - - for (const dir of searchPaths) { - if ( - existsSync(join(dir, "daemon.py")) && - existsSync(join(dir, "helpers.py")) - ) { - return true; - } - } - - // Check if uv tool is installed globally - try { - execSync("uv tool list 2>/dev/null | grep browser-harness", { timeout: 5000 }); - return true; - } catch { - // not found via uv - } - - return false; -} - -async function installBrowserHarness(ctx: ExtensionContext): Promise<boolean> { - // Try uv tool install - try { - ctx.ui.notify("Installing browser-harness via uv...", "info"); - execSync("uv tool install browser-harness", { - timeout: 60_000, - stdio: "pipe", - }); - return true; - } catch { - // uv install failed - } - - // Try git clone - try { - const targetDir = join(homedir(), "Developer", "browser-harness"); - ctx.ui.notify(`Cloning browser-harness to ${targetDir}...`, "info"); - execSync( - `git clone https://github.com/browser-use/browser-harness "${targetDir}"`, - { timeout: 30_000, stdio: "pipe" }, - ); - execSync("uv sync", { cwd: targetDir, timeout: 60_000, stdio: "pipe" }); - return true; - } catch { - return false; } }
M
src/tools.ts
→
src/tools.ts
@@ -140,6 +140,7 @@ const info = await daemon.getPageInfo();
if ("dialog" in info) { return { content: [{ type: "text" as const, text: `Navigated to ${params.url}\nTarget: ${targetId}\n⚠️ A dialog is open: ${info.dialog.type} — "${info.dialog.message}"\nUse browser_handle_dialog to dismiss or accept it.` }], + details: undefined, }; }@@ -149,11 +150,13 @@ content: [{
type: "text" as const, text: `${prefix}Navigated to: ${info.url}\nTitle: ${info.title}\nViewport: ${info.width}x${info.height}\nTarget: ${targetId}`, }], + details: undefined, }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Navigation failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -187,6 +190,7 @@ content: [{
type: "text" as const, text: `⚠️ DIALOG OPEN: ${info.dialog.type}\nMessage: ${info.dialog.message}${info.dialog.defaultPrompt ? `\nDefault: ${info.dialog.defaultPrompt}` : ""}\n\nUse browser_handle_dialog to accept or dismiss.`, }], + details: undefined, }; }@@ -195,11 +199,13 @@ content: [{
type: "text" as const, text: `URL: ${info.url}\nTitle: ${info.title}\nViewport: ${info.width}x${info.height}\nScroll: (${info.scrollX}, ${info.scrollY})\nPage size: ${info.pageWidth}x${info.pageHeight}`, }], + details: undefined, }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -225,15 +231,16 @@ entries: Array<{ id: number; url: string; title: string }>;
currentIndex: number; }; if (history.currentIndex <= 0) { - return { content: [{ type: "text" as const, text: "Already at the beginning of history." }] }; + return { content: [{ type: "text" as const, text: "Already at the beginning of history." }], details: undefined }; } const targetEntry = history.entries[history.currentIndex - 1]; await daemon.cdp("Page.navigateToHistoryEntry", { entryId: targetEntry.id }); - return { content: [{ type: "text" as const, text: `Navigated back to: ${targetEntry.url}` }] }; + return { content: [{ type: "text" as const, text: `Navigated back to: ${targetEntry.url}` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -259,15 +266,16 @@ entries: Array<{ id: number; url: string; title: string }>;
currentIndex: number; }; if (history.currentIndex >= history.entries.length - 1) { - return { content: [{ type: "text" as const, text: "Already at the end of history." }] }; + return { content: [{ type: "text" as const, text: "Already at the end of history." }], details: undefined }; } const targetEntry = history.entries[history.currentIndex + 1]; await daemon.cdp("Page.navigateToHistoryEntry", { entryId: targetEntry.id }); - return { content: [{ type: "text" as const, text: `Navigated forward to: ${targetEntry.url}` }] }; + return { content: [{ type: "text" as const, text: `Navigated forward to: ${targetEntry.url}` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -291,13 +299,14 @@ await daemon.ensureAlive();
await daemon.cdp("Page.reload"); const info = await daemon.getPageInfo(); if ("dialog" in info) { - return { content: [{ type: "text" as const, text: `Reloaded. ⚠️ Dialog open: ${info.dialog.type}` }] }; + return { content: [{ type: "text" as const, text: `Reloaded. ⚠️ Dialog open: ${info.dialog.type}` }], details: undefined }; } - return { content: [{ type: "text" as const, text: `Reloaded: ${info.title} (${info.url})` }] }; + return { content: [{ type: "text" as const, text: `Reloaded: ${info.title} (${info.url})` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -341,11 +350,12 @@ y: params.y,
button: params.button || "left", clickCount: params.clicks || 1, }); - return { content: [{ type: "text" as const, text: `Clicked at (${params.x}, ${params.y})` }] }; + return { content: [{ type: "text" as const, text: `Clicked at (${params.x}, ${params.y})` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Click failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -374,11 +384,12 @@ async execute(_id, params) {
try { await daemon.ensureAlive(); await daemon.cdp("Input.insertText", { text: params.text }); - return { content: [{ type: "text" as const, text: `Typed: "${params.text}"` }] }; + return { content: [{ type: "text" as const, text: `Typed: "${params.text}"` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Type failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -448,11 +459,12 @@ windowsVirtualKeyCode: vk,
nativeVirtualKeyCode: vk, }); - return { content: [{ type: "text" as const, text: `Pressed: "${k}"${mod ? ` (modifiers: ${mod})` : ""}` }] }; + return { content: [{ type: "text" as const, text: `Pressed: "${k}"${mod ? ` (modifiers: ${mod})` : ""}` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Key press failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -508,11 +520,12 @@ deltaX: dx,
deltaY: dy, }); - return { content: [{ type: "text" as const, text: `Scrolled by (${dx}, ${dy}) at (${cx}, ${cy})` }] }; + return { content: [{ type: "text" as const, text: `Scrolled by (${dx}, ${dy}) at (${cx}, ${cy})` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Scroll failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -549,6 +562,7 @@ } catch (err) {
return { isError: true, content: [{ type: "text" as const, text: `Screenshot failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -592,6 +606,7 @@ } catch (err) {
return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -637,11 +652,13 @@ await daemon.ensureAlive();
const tab = await daemon.currentTab(); return { content: [{ type: "text" as const, text: `Current tab: ${tab.title || "(no title)"}\nURL: ${tab.url}\nID: ${tab.targetId}` }], + details: undefined, }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -668,11 +685,13 @@ await daemon.switchTab(params.targetId);
const tab = await daemon.currentTab(); return { content: [{ type: "text" as const, text: `Switched to: ${tab.title || "(no title)"}\nURL: ${tab.url}` }], + details: undefined, }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Switch failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -700,11 +719,12 @@ const targetId = await daemon.newTab(params.url);
const msg = params.url ? `New tab created and navigated to: ${params.url}\nTarget: ${targetId}` : `New tab created: about:blank\nTarget: ${targetId}`; - return { content: [{ type: "text" as const, text: msg }] }; + return { content: [{ type: "text" as const, text: msg }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -742,6 +762,7 @@ } catch (err) {
return { isError: true, content: [{ type: "text" as const, text: `JS execution failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -840,6 +861,7 @@ } catch (err) {
return { isError: true, content: [{ type: "text" as const, text: `HTTP request failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -896,7 +918,7 @@ }),
async execute(_id, params) { const sec = params.seconds || 1; await new Promise((r) => setTimeout(r, sec * 1000)); - return { content: [{ type: "text" as const, text: `Waited ${sec}s.` }] }; + return { content: [{ type: "text" as const, text: `Waited ${sec}s.` }], details: undefined }; }, });@@ -926,7 +948,7 @@
while (Date.now() < deadline) { const state = await daemon.evaluateJS("document.readyState"); if (state === "complete") { - return { content: [{ type: "text" as const, text: "Page loaded (readyState: complete)." }] }; + return { content: [{ type: "text" as const, text: "Page loaded (readyState: complete)." }], details: undefined }; } await new Promise((r) => setTimeout(r, 300)); }@@ -934,11 +956,13 @@
return { isError: true, content: [{ type: "text" as const, text: `Page load timed out after ${timeout}s. Current state unknown.` }], + details: undefined, }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Wait failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -975,11 +999,12 @@ } else {
await daemon.cdp("Page.handleJavaScriptDialog", { accept: false }); } - return { content: [{ type: "text" as const, text: `Dialog ${params.action}ed.` }] }; + return { content: [{ type: "text" as const, text: `Dialog ${params.action}ed.` }], details: undefined }; } catch (err) { return { isError: true, content: [{ type: "text" as const, text: `Dialog handling failed: ${err instanceof Error ? err.message : String(err)}` }], + details: undefined, }; } },@@ -1032,6 +1057,7 @@ url,
targetId: "", ok: false as const, error: err instanceof Error ? err.message : String(err), + details: undefined, }; } });@@ -1067,6 +1093,7 @@ return {
...tab, ok: false as const, error: err instanceof Error ? err.message : String(err), + details: undefined, }; } finally { completed++;@@ -1079,6 +1106,7 @@ type: "text" as const,
text: `Opening URLs… ${completed}/${okCount} navigated`, }, ], + details: undefined, }); } });