pi-browser-harness

Browser control for pi agents, in the Chrome you're already using.

Your profile, your logins, your cookies. The agent reads pages as an accessibility tree with stable element handles, fills forms that React won't revert, captures network traffic and console errors, searches the web, and drops to raw CDP when the tools run out. 40 tools over a single CDP connection — no MCP hop, no second browser, no extra API keys.

---

How it compares

	pi-browser-harness	Playwright MCP	Puppeteer MCP	Stagehand
Browser	Your running Chrome — real profile, live sessions	Launches a clean instance	Launches a clean instance	Launches a clean instance
Element targeting	AX-tree refs ([eN]) that re-resolve at action time; coordinate and selector fallbacks	Selectors / AX locators	CSS selectors	LLM-inferred selectors
Input dispatch	Compositor-level CDP input — works through iframes, shadow DOM, cross-origin frames	DOM-level	DOM-level	DOM-level
Framework-safe writes	browser_type types with real input events, reads the value back, and falls back to the native setter when typing doesn't take	Native fill	Manual	Native fill
Transport	Native pi extension, in-process	MCP / JSON-RPC round trip	MCP / JSON-RPC round trip	In-process + its own LLM calls
Extra credentials	None	None	None	Its own model API key
Concurrency	Reads run in parallel; mutations auto-serialize through a shared mutex	Sequential	Sequential	Sequential
Isolation	Dedicated Chrome window per session; refuses tabs it didn't open	N/A (own browser)	N/A	N/A
Terminal output	Inline screenshots and Ctrl+O-expandable tool results	Plain text	Plain text	Plain text
Escape hatch	browser_run_script — raw CDP + full Node in one call	—	—	—

Reflects each project's default configuration. Corrections welcome via PR.

The short version: if you live in pi and want an agent driving the same Chrome you're already signed into, this is the one that fits.

---

Install

pi install npm:pi-browser-harness

Then enable remote debugging in your browser — either open chrome://inspect/#remote-debugging (or brave://inspect, edge://inspect), tick Discover network targets, and click Allow — or relaunch the browser with --remote-debugging-port=9222.

Finally, run /browser-setup in pi to connect. It's a one-time step — the connection survives across pi sessions and reconnects on its own when Chrome restarts.

The first time you connect, you'll be asked which browser profile the agent should work in:

Select the browser profile the agent should use
    Work ([email protected])
  ✓ Personal ([email protected])
    Testing ([email protected])
    — Clear selection (use whichever window is focused) —

That choice is saved to ~/.pi/agent/browser-harness.json and reused by every project until you change it with /browser-profile. It decides which logins, cookies, and extensions the agent works with, so pinning it keeps runs reproducible. Skip the prompt and the harness behaves as it always did: tabs open in whichever profile currently has focus.

Requires: pi (latest) · Node.js ≥ 22 · Chrome, Chromium, Brave, or Edge

---

The mental model

Interaction is ref-first. browser_snapshot returns the accessibility tree, and every interactive element carries a stable ref ([eN]) plus click coordinates @(x,y):

[e7]  textbox  "Email"          @(412,288)
[e9]  combobox "Country"        @(412,344)
[e12] button   "Create account" @(456,412)

Pass the ref — not the coordinates — to browser_click, browser_type, browser_select_option, browser_focus, browser_upload_file, or browser_dispatch_key. Refs are keyed to element identity, so they re-resolve position at action time and survive the re-renders that SPA forms trigger on every keystroke. Coordinates go stale the moment the layout reflows. browser_click by ref scrolls the element into view first (inside scrolling lists too) and warns when something covers it; an x/y outside the viewport is refused instead of clicking nothing.

Three things follow from that:

• Never screenshot to find a click target. The snapshot already has refs and coordinates.
• Mutating calls append a "Page changes" diff. Read it to confirm the step landed — a panel closed, a new [eN] appeared — instead of taking a screenshot to check.
• "ref is stale" means the page changed. Re-snapshot for fresh refs; don't retry blindly.

Which tool answers your question

You need to know…	Reach for
What's on the page, what's clickable	browser_snapshot — the default
One element's text, value, or geometry	browser_execute_js — one round trip
An article's readable content	browser_read_page
Why nothing happened after an action	browser_console
What the page requested over the network	browser_network_requests
Whether something rendered correctly	browser_screenshot — last resort, pixels only

---

Tools

Reading the page

Tool	Purpose
browser_snapshot	Default. Accessibility tree with [eN] refs and @(x,y) per interactive element. Optional includeScreenshot.
browser_execute_js	Surgical DOM reads — text, attributes, getBoundingClientRect(). Cheapest precise read.
browser_page_info	URL, title, viewport, scroll position — or the pending JS dialog, if one is blocking.
browser_console	JS errors and console output. Pass sinceSeq from the previous nextCursor to see only what an action caused.
browser_screenshot	PNG/JPEG capture. For verifying visual rendering only.

Web research

Tool	Purpose
browser_web_search	Query → ranked {title, url, snippet, rank}, by scraping a real search results page in your Chrome. No API key. Links only. Walks the fallback ladder Google → Bing → DuckDuckGo → Brave until one answers, or searches only the engine you name.
browser_read_page	Reader mode. A url (opened and closed in its own tab) or an owned targetId → clean article text, boilerplate stripped.

Both run in isolated tabs and never disturb your current page.

Interacting

Tool	Purpose
browser_click	Click by ref (preferred) or viewport x/y. Compositor-level, so iframes and shadow DOM work.
browser_select_option	Choose in a native <select> by value, label, or index. Returns available options on no match.
browser_focus	Deterministic .focus() — no coordinate accuracy needed.
browser_type	Enter text into a field by ref/selector (replaces its value; clear:false appends) or into the focused element. Real input events, so autocomplete opens; returns the value read back.
browser_press_key	Press a key with an optional modifier bitfield.
browser_dispatch_key	Synthetic DOM KeyboardEvent on one element, for pages that ignore raw CDP keys.
browser_scroll	Scroll by delta. W3C wheel convention: positive deltaY = down.
browser_drag_and_drop	Drag between two coordinate pairs.

Navigation and tabs

Tool	Purpose
browser_navigate	Go to a URL in the current tab.
browser_new_tab / browser_open_urls	Open one tab, or many URLs in parallel.
browser_go_back / browser_go_forward / browser_reload	History.
browser_list_tabs / browser_current_tab	List tabs (scope:"owned" by default) and inspect the attached one.
browser_switch_tab / browser_close_tab	Switch or close — owned tabs only.

Waiting and dialogs

Tool	Purpose
browser_wait_for	Wait until a selector appears (or disappears with gone:true), or text shows up. Prefer this over sleeping.
browser_wait_for_load	Wait for document.readyState === "complete".
browser_wait	Fixed delay, in seconds.
browser_handle_dialog	Accept or dismiss alert / confirm / prompt.

Network

Tool	Purpose
browser_network_requests	Requests captured on the current tab since attach. Filter by URL, method, status, resource type, or recency; optional response bodies.
browser_http_get	Plain GET outside the browser. Much faster than navigate-then-read for JSON APIs — but no JS rendering.
browser_set_headers	Set extra HTTP request headers on the current owned tab. Values are not echoed in output.
browser_clear_headers	Clear extra HTTP request headers from the current owned tab.

Extra headers persist for subsequent document, subresource, fetch, and XHR requests until cleared or the tab closes. Create or select an owned tab before setting them, and clear them before switching tabs or navigating the tab to another origin.

Files and viewport

Tool	Purpose
browser_upload_file	Set files on a file input, with a DataTransfer fallback for stubborn pages.
browser_download	Set the download directory and suppress the save-as prompt, or restore the default. Applies to the user's own downloads until restored.
browser_print_to_pdf	Print the page to PDF via Page.printToPDF.
browser_viewport_resize	Override viewport size and device pixel ratio for responsive checks.

Escape hatches

Tool	Purpose
browser_run_script	Run a temporary script with raw CDP and full Node access. See Extending.
browser_save_recipe	Save the browser task the agent just finished as a replayable recipe. See Saved recipes.
browser_run_recipe	Replay a saved recipe with no model calls, or list recipes when called with no name.
browser_setup	Agent-callable setup. Idempotent — how the agent recovers from not initialized on its own.

browser_snapshot, browser_network_requests, browser_console, browser_execute_js, browser_web_search, and browser_read_page ship custom TUI rendering: compact by default, full output on Ctrl+O (app.tools.expand). The model always receives the complete result regardless of render state.

---

Recipes

Fill a form on an SPA

Wait, snapshot once, then act by ref. No screenshots anywhere in this loop.

browser_wait_for({ selector: "#email" })
browser_snapshot()                                  // → [e7] textbox, [e9] combobox, [e12] button
browser_type({ ref: "e7", text: "[email protected]" })
browser_select_option({ ref: "e9", label: "India" })
browser_click({ ref: "e12" })                       // re-resolves position even after reflow

Each mutating call returns a "Page changes" diff — that's your confirmation the step landed.

Extract data

// One value
browser_execute_js({ expression: "document.querySelector('.price').innerText" })

// Structured
browser_execute_js({ expression: `JSON.stringify(
  Array.from(document.querySelectorAll('.result')).map(el => ({
    title: el.querySelector('h3')?.textContent,
    link:  el.querySelector('a')?.href,
  }))
)` })

// Skip the browser entirely for a JSON endpoint
browser_http_get({ url: "https://api.github.com/repos/amankumarsingh77/pi-browser-harness" })

Debug a failing page

browser_click({ ref: "e12" })
browser_console({ levels: ["error", "warn"] })      // did JS throw?
browser_network_requests({                          // did a request 4xx/5xx?
  urlPattern: "/api/",
  statusFilter: { min: 400 },
  includeResponseBodies: true
})

Search, then read

browser_web_search({ query: "chrome devtools protocol overview", limit: 5 })
browser_read_page({ url: "<the best result's url>" })

// Search one engine only, with no fallback:
browser_web_search({ query: "chrome devtools protocol overview", engine: "bing" })

details.engine names the engine that answered, and details.fallbacks lists any engines that failed first (CAPTCHA, no results, markup change, load failure).

Both tools open their own isolated tab, so neither disturbs the tab you are working in.

Keyboard modifiers

Modifiers are a bitfield: 1 Alt · 2 Ctrl · 4 Meta/Cmd · 8 Shift.

browser_press_key({ key: "c", modifiers: 2 })    // Ctrl+C
browser_press_key({ key: "T", modifiers: 10 })   // Ctrl+Shift+T

---

Saved recipes

Every page-changing browser call is journaled to ~/.pi/agent/browser-harness/journal/<session>.jsonl (user-only permissions, pruned after 30 days). Password values are never written.

After a multi-step task the agent offers to save it. browser_save_recipe turns that task into a YAML file under ~/.pi/agent/browser-recipes/<site>/<name>.yaml (or .pi/browser-recipes/ in the project with project: true), replacing the values you name as params with {{param}}. You confirm the recipe before it is written. Saving over a name bumps version and keeps the old file under .history/.

Run one yourself with /browser-recipe vendor-invoices-export month=2026-10 (no model involved), or ask the agent ("run the invoices recipe for October"), which calls browser_run_recipe. /browser-recipe with no name lists recipes; names tab-complete. Each step finds its element by role and name, waits up to 5s for it, and checks that the page changed the way the recorded run did. A status or alert message the recorded run saw counts when its first words match, so "Exported 43 rows" satisfies "Exported 42 rows".

When a step fails, the replay stops and hands the agent the step, what it expected and a fresh snapshot. The agent finishes your task from that step first, then calls browser_save_recipe with repair_step, which replaces that step onward with what it just did, bumps version and keeps the old file. A failed /browser-recipe goes to the agent the same way. If the page asks you to sign in, the replay stops and the agent asks you to sign in; it never types credentials.

A recipe in the project's .pi/browser-recipes/ comes from whoever wrote the repo, so it needs your approval before its first run: you see its file, site, the URLs it opens, the files it uploads and its downloads. The approval holds for those exact file bytes (kept in ~/.pi/agent/browser-harness/trusted-recipes.json); if the file changes, you are asked again. Without a UI an unapproved project recipe does not run.

Reads are kept too: a browser_http_get, or a browser_read_page of a URL, becomes a read step, and a replay returns what each one fetched (browser_run_recipe in its result, /browser-recipe as a message in the session that starts no model turn). A read made with request headers or a tab id is left out, since headers can carry credentials and a tab id means nothing next run. A browser_execute_js that changed nothing is left out as well, so a replay does not return what it read; read data with browser_http_get or browser_read_page when the recipe should hand it back.

If the site shows Cloudflare's "Just a moment..." check, a replay waits up to 30s for it to clear; if it does not, the replay stops and the agent asks you to complete the check in the browser.

A task that typed into a password field, or used a browser_execute_js that changed the page or browser_run_script, cannot be saved.

---

Concurrency and isolation

Parallel reads, serialized writes. Observation tools run concurrently with each other and with mutations. Mutations (click, fill, type, scroll, navigate, switch_tab, …) pass through a shared async mutex and FIFO-queue automatically, so emitting several in one turn is safe.

browser_snapshot() + browser_network_requests({ sinceMs: 5000 }) + browser_http_get({ url: "..." })

Tab ownership. On first attach the harness opens a dedicated Chrome window and only operates inside it. browser_list_tabs defaults to scope:"owned" (pass scope:"all" for a read-only view of your other tabs); browser_switch_tab and browser_close_tab refuse anything this session didn't open. Owned tabs are closed at session shutdown, so nothing stale accumulates in your browser.

---

Extending

When the built-in tools can't express a workflow, write a script and run it. It executes inside the harness process with a live CDP handle — far faster than chaining tool calls.

write("<tmpdir>/scrape.js", `
  const results = [];
  for (const url of params.urls) {
    await daemon.session().call("Page.navigate", { url });
    await new Promise(r => setTimeout(r, 2000));
    results.push({ url, title: await daemon.evaluateJs("document.title") });
  }
  return { content: [{ type: "text", text: JSON.stringify(results, null, 2) }] };
`)

browser_run_script({ path: "<tmpdir>/scrape.js", params: { urls: [...] } })

Bindings: params, daemon, require, signal, onUpdate, ctx, console, fetch, JSON, Buffer, setTimeout, clearTimeout.

daemon exposes evaluateJs, pageInfo, listTabs, switchTab, newTab, closeTab, current, status, and session() for raw CDP (call · callOnTarget · callBrowser · takeDialog).

Scripts must live under the OS temp dir, the cwd, or BH_SCRIPT_DIR, and every call carries a mandatory timeout (60s default, 600s max). They land on disk first, so they're reviewable and re-runnable.

Contributing tools from another pi extension

Another extension can add its own browser_* tools to this harness. Contributed tools share the same client, mutation mutex, tab ownership, and lane prompt as the built-ins, so a serialized tool you add queues behind browser_click exactly as browser_type does.

Define the tool with defineBrowserTool and hand it to registerBrowserTools at your extension's load time, before pi emits session_start. The harness registers it with pi on the next session_start alongside the built-ins.

// my-extension/src/index.ts
import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
import { Type } from "typebox";
import { defineBrowserTool, registerBrowserTools, ok } from "pi-browser-harness/src/index.ts";
import { evalJs } from "pi-browser-harness/src/domains/cdp-call.ts";

const pageHeightTool = defineBrowserTool({
  name: "browser_page_height",
  label: "Browser Page Height",
  description: "Report the document's full scroll height in CSS pixels.",
  promptSnippet: "Measure the page's scroll height",
  promptGuidelines: [],
  parameters: Type.Object({}),
  concurrency: "parallel",
  handler: async (_args, { client }) => {
    const height = await evalJs(client, "document.documentElement.scrollHeight");
    if (!height.success) return height;
    return ok({ text: `Scroll height: ${String(height.data)}px` });
  },
});

export default function myExtension(_pi: ExtensionAPI): void {
  const added = registerBrowserTools([pageHeightTool]);
  if (!added.success) console.warn(added.error.message);
}

registerBrowserTools returns a Result. The whole batch is rejected, and nothing is added, when any name is already taken by a built-in, an earlier contribution, or another tool in the same batch. Pick concurrency: "serialized" for anything that changes page or tab state; "parallel" for reads. browserTools() lists what the harness will register, built-ins plus contributions so far.

A tool that wants a custom result view declares a details schema next to parameters. The handler then returns details of that shape, and renderResult receives them typed. pi stores details with the session and replays them later; the harness validates replayed details against the schema first, so a renderer never sees a stale shape from an older session (it falls back to the stored text instead). Details must be plain JSON: no Maps, and omit a key rather than setting it to undefined.

import { Text } from "@mariozechner/pi-tui";

const PageHeightDetails = Type.Object({ height: Type.Number(), unit: Type.Literal("px") });

const pageHeightTool = defineBrowserTool({
  // ...name, label, description, promptSnippet, promptGuidelines, parameters, concurrency as above
  details: PageHeightDetails,
  handler: async (_args, { client }) => {
    const height = await evalJs(client, "document.documentElement.scrollHeight");
    if (!height.success) return height;
    const px = typeof height.data === "number" ? height.data : 0;
    return ok({ text: `Scroll height: ${px}px`, details: { height: px, unit: "px" } });
  },
  renderResult: (result, _expanded, theme) =>
    new Text(theme.fg("muted", `${result.details.height}${result.details.unit}`), 0, 0),
});

Tools without a details schema need nothing extra: they may still attach any JSON object for logs, and their renderResult (if any) sees it untyped.

Known limits

Two extension points are documented but not implemented; the escape hatches that exist today are listed with each.

• No per-tab extension slot. TabSession in src/cdp/session.ts carries only the harness's own per-tab state (session id, dialog, page-info cache). A contributed tool cannot stash its own state on a tab. Keep per-tab state in your extension keyed by targetId from browser_current_tab or browser_list_tabs, or compute it on demand with browser_run_script through daemon.session(targetId).
• Hardcoded browser list. src/profile/paths.ts knows the user-data directories of a fixed set of Chromium browsers, so /browser-profile cannot discover one that is not on that list. Point the harness at any Chromium profile directory with CHROME_USER_DATA_DIR, or skip discovery entirely by attaching to a running browser's DevTools endpoint with BU_CDP_WS.

---

Commands, flags, and environment

Command	Purpose
/browser-setup	Connect pi to your browser. Run once; idempotent.
/browser-profile	Pick which browser profile the agent works in. Remembered across sessions.
/browser-status	Connection state, selected profile, and current page.
/browser-reload-daemon	Force a client restart. Rarely needed — the transport reconnects on its own.
/browser-recipe	Replay a saved recipe: /browser-recipe NAME key=value. No name lists recipes.

Flag / variable	Effect
--browser-namespace <name>	Override the session namespace (default: auto-generated).
--browser-debug-clicks	Save annotated click screenshots for debugging. Equivalent to BH_DEBUG_CLICKS=1.
BU_CDP_WS	Attach to a remote browser by WebSocket URL instead of discovering a local one. The daemon reads this when it starts, so stop a running daemon first.
BH_DEBUG_CLICKS	Same as --browser-debug-clicks.
BU_CDP_PORTS	Extra ports to probe during CDP discovery, beyond 9222.
BH_SCRIPT_DIR	Additional directory browser_run_script will accept paths from.
PI_BROWSER_NODE	Node executable used to run the daemon. Defaults to the one running pi, or node from PATH when pi is a Bun-compiled binary.

---

Anti-patterns

• Don't launch your own browser. You're attached to the user's real Chrome.
• Don't type credentials. Hit an auth wall — stop and ask.
• Don't screenshot to understand a page, find coordinates, or read a value. browser_snapshot and browser_execute_js are cheaper and exact.
• Don't guess CSS selectors when a snapshot ref exists.
• Don't sleep to wait. browser_wait_for is deterministic; browser_wait is a last resort.
• Don't ignore dialogs. They freeze the page's JS thread — browser_page_info reports them, browser_handle_dialog clears them.

---

Architecture

pi agent
   │  browser_* tools (in-process extension)
   ▼
pi-browser-harness  ──►  daemon (Unix socket / named pipe)  ──►  Chrome
                              CDP WebSocket

A long-lived daemon owns the CDP connection, so it outlives individual pi sessions — which is why you only see Chrome's "Allow Remote Debugging" prompt once. browser_run_script executes in the harness process with direct daemon and Node access.

Prompt guidance

Tool descriptions and parameter schemas own per-tool usage. Tool guidelines add only decisions and caveats not already covered there. The browser prompt carries cross-tool rules: tab isolation, concurrency, checking mutations, and stale-ref recovery. These rules are available before and after connection. Longer recipes remain in this README and the bundled skill rather than being copied into the browser prompt.

---

Troubleshooting

Symptom	Fix
DevToolsActivePort not found	Open chrome://inspect/#remote-debugging, tick Discover network targets, click Allow. Or relaunch with --remote-debugging-port=9222.
No browser instance running	Chrome/Brave/Edge isn't open, or only helper processes remain. Open the browser, then /browser-setup.
Browser harness not initialized	Run /browser-setup — or let the agent call browser_setup itself.
ref is stale	The page re-rendered. Re-run browser_snapshot for fresh refs.
Page looks loaded but content is missing	SPA still rendering. Use browser_wait_for({ selector }), not a fixed sleep.
Click did nothing	Check browser_console for a thrown error, then browser_network_requests for a failed call.
Actions all hang	A JS dialog is blocking. browser_page_info reports it; browser_handle_dialog clears it.
No [eN] ref for your target	The element wasn't exposed as interactive. Fall back to a CSS selector, or browser_execute_js with getBoundingClientRect().
Connected but the wrong browser responds	A stale DevToolsActivePort file. Quit every Chromium-family browser, reopen one, then /browser-setup.
Agent acts as the wrong account	No profile is pinned, so tabs open in whichever profile has focus. Run /browser-profile and pick one.
couldn't open a window in "…" automatically	The profile window couldn't be opened for you. Open that profile from the browser's profile menu, then retry — the harness never falls back to a different profile silently.

---

Security

This extension drives your real browser. The agent can read any page you're signed into, submit forms, and act inside authenticated sessions. browser_run_script is full code execution in the pi process with unrestricted require — review any script before running it. Scripts are path-restricted and timeout-bounded, but that is a blast-radius limit, not a sandbox.

---

Contributing

See CONTRIBUTING.md for setup, conventions, and the PR process. npm run check runs the typecheck, the boundary scan, the type-aware ESLint pass and the test suite in that order; it is the one command a PR must pass. Release history lives in CHANGELOG.md.

License

MIT — see LICENSE.