all repos — sz-chrome-remote-extension @ 0c3613891604d10e332d31762c9c3345c357a3bb

pi-chrome, patched to drive a signed-in browser on another host through a broker, with an MCP server

0.15.26: update docs for benchmark gates
Tianren Dong tianrendong.dtr@gmail.com
Sat, 16 May 2026 14:12:13 -0400
commit

0c3613891604d10e332d31762c9c3345c357a3bb

parent

b792f22f2c2ef4dfb7bd300e7381a186e34e0c18

M CHANGELOG.md → CHANGELOG.md

@@ -2,6 +2,11 @@ # Changelog

All notable user-facing changes to `pi-chrome`. +## 0.15.26 — 2026-05-16 + +- **Documentation accuracy.** README, FAQ, examples, comparison, and test-suite docs now describe the 41-challenge suite, gate buckets, strict-CSP fallback, and current human-vs-extension limitations. +- **Published benchmark assets.** The npm package now includes `test-suite/` so the documented benchmark pages are available from installed packages, not only from the repository checkout. + ## 0.15.25 — 2026-05-16 - **Reload after older installs.** `/reload` now recovers from stale singleton state left by pi-chrome 0.15.19 and earlier instead of skipping the freshly loaded extension.
M README.md → README.md

@@ -149,6 +149,10 @@ | **Observe** | `chrome_list_console_messages`, `chrome_list_network_requests`, `chrome_get_network_request` (with response body) |

Each tool is documented inline in Pi — agents see the parameters and gotchas (Chrome input, CSP limits, file upload behavior) without trial-and-error. +### Known limits vs. human Chrome use + +pi-chrome is strongest on web-page workflows exposed through DOM, screenshots, tabs, and Chrome input. It is not a full human/OS substitute. Current limitations include native Chrome/OS surfaces (print/save dialogs, permission bubbles, password-manager prompts), cross-origin iframe DOM access, rich multitouch/pinch/stylus gestures, visual CAPTCHA/bot challenges, hardware-backed auth (passkeys/security keys/biometrics), and arbitrary OS app interaction. For strict-CSP pages, use screenshot + coordinate input when `chrome_snapshot`/`chrome_evaluate` are blocked. + --- ## Click & input behavior

@@ -213,9 +217,13 @@ ---

## Built-in benchmark suite -[`test-suite/`](./test-suite) is a benchmark for **any** browser-control agent (not just pi-chrome). It includes **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks**. +[`test-suite/`](./test-suite) is a benchmark for **any** browser-control agent (not just pi-chrome). It includes **41 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks**. -Scoring tracks expected outcomes per challenge rather than raw PASS count, so tools are judged against their declared browser-control capability. +Scoring tracks expected outcomes per challenge rather than raw PASS count, so tools are judged against their declared browser-control capability. Unit challenges are split into gate buckets: + +- `core` — expected release blockers for normal trusted-mode browser control. +- `conditional` — capability/environment gated (clipboard, touch, dialogs, native UI, etc.). +- `quality` — adversarial humanization/fingerprint signals; report trends, don't block general release by default. Each challenge exposes `window.__verdict` / `window.__reason` / `window.__events` and a manifest entry with expected results per mode.

@@ -224,7 +232,7 @@ cd test-suite && python3 -m http.server 8765

# open http://127.0.0.1:8765/ in the Chrome window pi-chrome controls ``` -Categories: `real-input`, `pointer-humanization`, `keyboard`, `activation-gates`, `scroll`, `drag-drop`, `clipboard`, `native-controls`, `frameworks`, `editing`, `dom-complexity`, `frames`, `files`, `observability`, `fingerprint`, `agent-safety`. +Categories include: `trusted-input`, `pointer-humanization`, `keyboard`, `focus-keyboard`, `activation-gates`, `scroll`, `drag-drop`, `clipboard`, `native-controls`, `frameworks`, `editing`, `dom-complexity`, `frames`, `files`, `observability`, `csp`, `lazy-loading`, `dialogs`, `popups`, `spa-routing`, `fingerprint`, and `agent-safety`. If you build a competing tool, please open a PR with your scores. We benchmark in public.

@@ -247,7 +255,8 @@

`pi-chrome` is actively shipped. Things on the near roadmap: - More observability tools (DOM mutation streams, performance traces) -- First-class iframe + Shadow-DOM uid stability across snapshots +- First-class cross-origin iframe + Shadow-DOM uid stability across snapshots +- Native-browser surface coverage where extension APIs allow it (downloads, permissions, context menus) - Web Push & service worker introspection - Recorder mode that emits agent prompts from your own clicks
M docs/COMPARISON.md → docs/COMPARISON.md

@@ -134,7 +134,7 @@ ---

## Public benchmarks worth knowing (for axis 2 / axis 3 comparison) -Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **38 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering real input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, file uploads, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count. That's **driver-level** grading. +Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **41 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering trusted input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, iframes, file uploads, strict-CSP screenshot fallback, dynamic waits, tab lifecycle, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count, with `core`, `conditional`, and `quality` gate buckets. That's **driver-level** grading. For **agent-level** comparison (axis 2), the public benchmarks worth citing:

@@ -156,7 +156,7 @@ ---

## Reproducing pi-chrome's driver-level claims -Run [`../test-suite/`](../test-suite) against any browser-control tool. Each challenge exposes `window.__verdict` / `window.__reason` / `window.__events`, so any tool (Playwright, Puppeteer, Selenium, Stagehand, pi-chrome) can grade itself deterministically. +Run [`../test-suite/`](../test-suite) against any browser-control tool. Each challenge exposes `window.__verdict` / `window.__reason` / `window.__events`, so any tool (Playwright, Puppeteer, Selenium, Stagehand, pi-chrome) can grade itself deterministically. Headline release scoring should use the `core` gate; `conditional` depends on declared environment capabilities, and `quality` tracks adversarial/humanization regressions. ```bash cd test-suite && python3 -m http.server 8765
M docs/EXAMPLES.md → docs/EXAMPLES.md

@@ -159,6 +159,8 @@ - sign-in flows

- guarded buttons - audio/video controls - fullscreen and other user-activation checks -- pages with strict CSP or user-activation checks +- pages where DOM injection/evaluate is limited, if the agent can use screenshots + coordinates + +Strict CSP note: `chrome_snapshot`/`chrome_evaluate` may be blocked on pages that disallow `unsafe-eval`; `chrome_screenshot`, tab/navigation tools, and real input still work. Chrome may show its debugger banner while pi-chrome is attached.
M docs/FAQ.md → docs/FAQ.md

@@ -14,9 +14,9 @@ By default no — extensions need explicit "Allow in incognito" permission. Toggle it on `chrome://extensions` if you want pi-chrome to see incognito tabs. We don't recommend it for sensitive work.

## Will sites detect that I'm automating? -Interactive controls use Chrome's real input layer via CDP: pointer paths are humanized, key cadence has variance, and normal user-activation gates are satisfied. Some detectors check for the `chrome.debugger` API attached and Chrome will show the "Chrome is being debugged" banner. +Interactive controls use Chrome's real input layer via CDP, so normal user-activation gates are satisfied and input is closer to real browser use than DOM-dispatched events. pi-chrome also shapes pointer/keyboard/scroll behavior, but this is not a guarantee of undetectability. Some detectors check for the `chrome.debugger` API attached, and Chrome will show the "Chrome is being debugged" banner. -The [`test-suite/`](../test-suite) grades browser-control behavior against common detection signals. +The [`test-suite/`](../test-suite) grades browser-control behavior against common detection signals. Its `quality` bucket is adversarial signal, not a blanket promise that every site will treat automation as human. ## Why do I see a banner saying "Pi Chrome Connector started debugging this browser"?

@@ -48,6 +48,10 @@

## Can I script it without Pi? The Pi-facing tools are thin wrappers around an HTTP bridge at `127.0.0.1:17318`. You could call it directly from any process, but the API is internal and may change. If you need a stable scripting interface, file an issue and we'll consider stabilizing. + +## What can humans do that pi-chrome cannot? + +pi-chrome controls web pages through Chrome extension APIs, page inspection, screenshots, and browser input. It is not full OS-level human control. Known gaps include native Chrome/OS dialogs (print/save-as, some permission bubbles, password-manager prompts), arbitrary OS app interaction, visual CAPTCHA challenges, hardware-backed auth (passkeys/security keys/biometrics), rich multi-touch/pinch/stylus gestures, and DOM inspection inside cross-origin iframes. Some of these can still be handled with screenshot + coordinate input or user assistance, but they are not first-class deterministic workflows. ## Does `chrome_evaluate` work on strict-CSP pages?
M extensions/chrome-profile-bridge/browser-extension/manifest.json → extensions/chrome-profile-bridge/browser-extension/manifest.json

@@ -1,7 +1,7 @@

{ "manifest_version": 3, "name": "Pi Chrome Connector", - "version": "0.15.25", + "version": "0.15.26", "description": "Lets Pi control tabs in Chrome via a local connector at 127.0.0.1.", "permissions": [ "tabs",
M package-lock.json → package-lock.json

@@ -1,12 +1,12 @@

{ "name": "pi-chrome", - "version": "0.15.25", + "version": "0.15.26", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "pi-chrome", - "version": "0.15.25", + "version": "0.15.26", "license": "MIT", "peerDependencies": { "@earendil-works/pi-coding-agent": "*",
M package.json → package.json

@@ -1,6 +1,6 @@

{ "name": "pi-chrome", - "version": "0.15.25", + "version": "0.15.26", "scripts": { "version": "node scripts/sync-manifest-version.js", "prepublishOnly": "node scripts/sync-manifest-version.js"

@@ -52,6 +52,7 @@ "type": "commonjs",

"files": [ "extensions", "docs", + "test-suite", "README.md", "CHANGELOG.md", "CONTRIBUTING.md",