docs/ARCHITECTURE.md (view raw)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
# pi-chrome architecture
`pi-chrome` connects Pi to your existing Chrome profile through a local-only bridge and an unpacked Chrome extension.
```text
+----------------------+ +--------------------------+
| Pi agent (terminal) | -- 127.0.0.1:17318 ->| Chrome extension |
| chrome_* tools | | (your real profile) |
+-----------+----------+ +-------------+------------+
| same machine |
v v
Other Pi sessions Tabs you already have open
share same bridge (GitHub, Linear, Stripe, etc.)
```
## Components
- **Pi extension** — exposes `chrome_*` tools and `/chrome` commands inside Pi.
- **Loopback bridge** — listens on `127.0.0.1:17318`; no external network bind by default.
- **Chrome companion extension** — loaded unpacked into your real Chrome profile.
- **Chrome debugger / CDP** — drives input, screenshots, network/console observation, and evaluation.
## Session model
Multiple Pi sessions can use same Chrome companion extension. First session opens local bridge; later sessions detect it and pipe commands through.
Each Pi session owns its own automation target:
- First chrome action without explicit target opens dedicated automation window.
- If separate window cannot be created, pi-chrome falls back to dedicated tab.
- Target survives `/reload` and Chrome service-worker restarts.
- Ownership is tracked by id and mirrored to `chrome.storage.session`.
- Cleanup closes calling session's automation target and every tab it created through `tab.new`.
- Existing user tabs adopted into a session group are preserved, and ungrouped only if still in that group.
- Cleanup removes individual owned tabs, never whole windows. User/other-session tabs moved into a Pi window remain open; Chrome closes a window automatically when its final tab is removed.
- Shutdown waits at most two seconds for cleanup before stopping the bridge. `/reload` preserves resources; revoke starts cleanup without blocking. Hard process termination or unavailable Chrome can still leave owned tabs open.
To point pi-chrome at an existing tab, pass `targetId`, `urlIncludes`, or `titleIncludes`.
## Tab management guards
`chrome_tab` management actions are guarded:
- `activate`, `close`, `group`, and `ungroup` without explicit target act on session automation tab if it exists.
- If no automation tab exists, operation errors instead of touching your active tab.
## Background mode
The existing background setting is a hard session policy, enabled by default. No separate lock/unlock command is needed.
```text
/chrome background on # enforce no explicit window focus/tab activation
/chrome background off # allow foreground/watch mode
```
- With background on, per-call `background:false` and legacy `foreground:true` cannot override the policy. `chrome_tab activate` errors with instructions to ask the user to turn background off.
- With background off, calls may focus Chrome; per-call `background:true` still avoids explicit focus/tab activation.
- Policy is applied in `authorizedBridgeSend` for every tool, including `chrome_launch(url)`, tab creation, and tools without a background parameter. Each session sends its own effective `background`/`foreground` flags through the shared bridge.
- All worker window-focus/tab-activation writes go through a guarded helper. Background tab creation uses `active:false`; implicit automation windows remain `focused:false`.
- Screenshots use CDP `Page.captureScreenshot` with `fromSurface:true` and `captureBeyondViewport:false`. They target a tab, not whichever tab happens to be visible. There is no `captureVisibleTab`/activation fallback on debugger or capture failure. PNG/JPEG output is unchanged; full-page capture retains tiles plus a JSON manifest and restores scroll position best-effort, including on failure.
- Background tab creation and screenshots use internal `tab.new.background` / `page.screenshot.background` wire actions. Old companions reject them before changing tabs; Pi reports a reload instruction instead of retrying an unsafe legacy action. No capability-probe race or extra round trip is needed.
- Real CDP input and existing explicit DOM-fallback controls are unchanged. Background mode never silently substitutes synthetic input to avoid focus.
### Scope and risks
This is a policy against **explicit pi-chrome focus/activation**, not an OS focus sandbox. Page scripts (`window.open`, `window.focus`), trusted input, native dialogs, debugger banners, Chrome window/Spaces behavior, and closing an active tab can still change focus or selection. Other sessions and human actions remain independent. Requests already dispatched before a mode change keep their earlier policy.
Inactive/minimized tabs can throttle timers or rendering, and clipboard/fullscreen/other focus-gated workflows may fail. Screenshots now require debugger attachment, which can conflict with DevTools or other extensions; hidden-tab rendering can differ or be unavailable. No automatic foreground retry is allowed. Full-page capture temporarily scrolls the target page. Reload both Pi and the Chrome companion after upgrading, and live-test tab selection, OS focus, and screenshot fidelity on supported Chrome/OS versions.
## Authorization
Bridge connection alone is not enough. Chrome control stays locked until current Pi session runs:
```text
/chrome authorize
```
Authorization expires after configured duration, on `/chrome revoke`, or when Pi exits.
## Unpacked extension choice
`pi-chrome` ships browser extension source as an unpacked folder on purpose:
- easy to inspect before loading
- no Web Store release delay
- MIT-licensed source in repo
- `/chrome doctor` can compare loaded extension version against installed package
Loaded extension has broad tab/scripting permissions inside profile where it is installed. Install only from trusted package source.
|