← Blog

BLOG

Browser AI Agents: Runtime Architecture for Production

Browser AI agents need a real Chromium runtime, not a local Chrome tab. Learn the architecture, trade-offs, and how to run them on hosted sessions.

9 min readRemote Browser
Ivory computer towers connected across terracotta platforms in a quiet desert landscape
The infrastructure behind your browserREMOTE BROWSER / ILLUSTRATION

Browser AI agents are programs that use a real browser as their execution environment: they read the DOM, click, type, navigate, and observe results the same way a person would. The hard part is rarely the model. It is the runtime underneath it — the browser session the agent drives, how it is isolated, how it persists state, and how you observe it when something goes wrong. This article covers the architecture of browser AI agents, the trade-offs between local and hosted runtimes, and the production criteria that decide whether an agent works reliably at scale.

If you already know you need a hosted runtime, the remote browser for AI agents guide covers setup directly. This piece is about the design decisions that come before that.

What a browser AI agent actually is

A browser AI agent is a loop: observe the page, decide on an action, execute it, observe again. Each step depends on a browser that can be driven programmatically and inspected reliably.

That loop has four components:

  • A control layer. The agent framework or LLM decides what to do next. It might be browser-use, Stagehand, a custom planner, or a Claude/Codex-style agent with browser tools.
  • A transport. The control layer talks to the browser over a protocol. In practice this is the Chrome DevTools Protocol (CDP), usually through Playwright, Puppeteer, or Selenium.
  • A browser instance. A real Chromium process with a rendering engine, network stack, cookie jar, and storage.
  • A session boundary. The isolation, profile, proxy, and lifecycle controls that keep one agent's work separate from another's.

Most tutorials collapse these into "run Playwright locally." That works for a demo. It breaks down when you need concurrency, persistence, or debugging across many runs. The runtime is where browser AI agents succeed or fail.

Why the runtime matters more than the model

Model quality gets the attention, but in production the runtime is the more common failure source. A stronger model cannot recover from a browser that crashed mid-task, a session that lost its cookies, or a CAPTCHA that blocked the flow with no way to intervene.

Concretely, the runtime determines:

  • Determinism of observation. If the agent reads a stale DOM or a half-rendered page, its decision is wrong regardless of model quality. Stable session state and predictable page-load behavior matter.
  • State continuity. Logged-in flows require persistent profiles. A fresh browser per step loses auth and forces re-login, which most sites rate-limit.
  • Isolation. Two agents sharing a browser instance corrupt each other's cookies, tabs, and storage. Session isolation is a correctness requirement, not a nicety.
  • Observability. When an agent fails, you need to see what it saw. A live viewer and session recordings turn a black-box failure into a debuggable one.
  • Cost control. Browser time is metered. Idle sessions, leaked processes, and runaway loops are the main cost drivers.

Treat the runtime as infrastructure with an SLA, not as a library you happen to call.

Local Chrome vs hosted Chromium for browser AI agents

The first architectural decision is where the browser runs. Both options are legitimate; they fail in different ways.

CriterionLocal Chrome / self-managedHosted Chromium runtime
SetupInstall browsers, manage versions per machineConnect over CDP to a ready session
ConcurrencyBounded by the host's CPU and memoryScales with the provider's capacity, subject to your plan
IsolationManual — separate profiles or containersPer-session isolation by default
PersistenceYou manage profile directories and backupsPersistent profiles as a managed feature
DebuggingLocal DevTools, hard to shareLive viewer and session recordings
Network egressYour IP; proxy setup is manualConfigurable proxy and browser settings
Failure recoveryYou restart and rebuild stateSession lifecycle handled by the runtime
Cost modelFixed infra plus engineering timeMetered browser time — see /pricing

Local Chrome is the right choice when you are prototyping, when the task needs a browser on a specific machine, or when data cannot leave your network. Hosted Chromium is the right choice when you need concurrency, isolation, and observability without building a browser fleet yourself.

The trade-off is control versus operational burden. Self-managed gives you full control and full responsibility. Hosted moves the fleet management off your plate but adds a dependency and a metered cost. Neither is universally better; pick based on how many concurrent agents you run and how much browser infrastructure you want to own.

The control path: CDP, Playwright, and agent frameworks

Almost every browser AI agent ends up speaking CDP, directly or through a library. Understanding the path clarifies what a runtime must expose.

  • CDP is the low-level protocol. It gives you DOM access, network interception, input events, and page lifecycle events.
  • Playwright and Puppeteer wrap CDP with a higher-level API. Playwright's connectOverCDP lets you attach to an existing browser instead of launching one.
  • Agent frameworks (browser-use, Stagehand, custom planners) sit on top and translate model decisions into Playwright or CDP calls.

A hosted runtime's job is to hand you a CDP endpoint that any of these layers can drive. That is what makes a runtime framework-agnostic: if it speaks CDP, it works with Playwright, Puppeteer, Selenium, and most agent frameworks without special integration.

Connecting a Playwright agent to a hosted session

The pattern is the same whether you use a raw CDP endpoint or a provider SDK. You obtain a WebSocket endpoint for a session, then attach:

import { chromium, Browser, Page } from "playwright";

interface SessionInfo {
  cdpUrl: string; // wss://... provided by the runtime
}

async function runAgentTask(session: SessionInfo): Promise<void> {
  let browser: Browser | undefined;
  try {
    // Attach to the already-running hosted Chromium instance.
    browser = await chromium.connectOverCDP(session.cdpUrl);

    // Reuse the existing context so persistent profile state is preserved.
    const context = browser.contexts()[0] ?? (await browser.newContext());
    const page: Page = context.pages()[0] ?? (await context.newPage());

    await page.goto("https://example.com/login", { waitUntil: "domcontentloaded" });

    // Agent-driven steps: fill, click, wait for a stable signal.
    await page.getByLabel("Email").fill("agent@example.com");
    await page.getByLabel("Password").fill(process.env.AGENT_PASSWORD!);
    await page.getByRole("button", { name: "Sign in" }).click();

    // Wait on a deterministic condition, not a fixed timeout.
    await page.waitForURL("**/dashboard", { timeout: 30_000 });

    const heading = await page.getByRole("heading", { level: 1 }).innerText();
    console.log("Landed on:", heading);
  } finally {
    // Close the connection, not the remote browser, unless the runtime
    // ties session teardown to disconnect.
    await browser?.close();
  }
}

Two details matter in production. First, connectOverCDP attaches to an existing browser — you are not launching one, so browser version and flags are the runtime's responsibility. Second, wait on deterministic signals (waitForURL, role-based locators) rather than fixed sleeps; agents that poll on timeouts are both slow and flaky.

The Chrome DevTools Protocol documentation is the authoritative reference for the underlying domains if you need to go below Playwright.

Production criteria for browser AI agents

When you evaluate a runtime — hosted or self-managed — these are the criteria that predict whether agents hold up under load.

Session isolation

Each agent run should get its own browser context or instance. Shared state causes cross-contamination: cookies from one task leak into another, tabs collide, and failures become non-reproducible. Isolation is the baseline for correctness.

Persistent profiles

Many real tasks require staying logged in across runs. Persistent profiles let an agent resume authenticated sessions without re-entering credentials every time. The runtime should let you name, reuse, and reset profiles deliberately — not silently share one global profile.

Configurable browser settings

Network egress, user-agent, locale, and timezone affect how sites respond. A runtime should expose these as configurable browser settings. Be precise about what you need: "configurable proxy and browser settings" is a real requirement; claims of perfect anonymity are not something any runtime can honestly guarantee.

Live viewer and recordings

The single biggest debugging accelerator is watching the session live. A viewer lets a human step in when an agent stalls — solving a CAPTCHA, approving a step, or diagnosing a selector that broke. Recordings let you replay a failed run after the fact.

Lifecycle and usage controls

Sessions must be created, reused, and torn down predictably. Leaked sessions are the most common hidden cost. Look for explicit session lifecycle APIs and usage metering you can monitor. Current rates and limits are on /pricing.

Framework compatibility

If the runtime speaks CDP, it works with Playwright, Puppeteer, Selenium, and agent frameworks without bespoke glue. Avoid runtimes that only work with one library — you will want to change frameworks eventually.

Common failure modes and how the runtime addresses them

Most production incidents with browser AI agents fall into a small set of patterns:

  • Stale observation. The agent acts on a page state that has already changed. Fix: wait on deterministic signals and re-read the DOM before acting.
  • Lost auth. A fresh browser per step forces re-login and triggers rate limits. Fix: persistent profiles.
  • Cross-task contamination. Shared cookies and storage corrupt results. Fix: per-session isolation.
  • Silent stalls. The agent waits on an element that never appears. Fix: live viewer plus bounded timeouts with explicit failure.
  • Runaway cost. Idle or leaked sessions burn browser time. Fix: lifecycle controls and usage monitoring.
  • Selector drift. A site changes its markup and the agent breaks. Fix: role-based and text-based locators, plus recordings to diagnose quickly.

None of these are model problems. They are runtime and harness problems, which is why the runtime deserves the same engineering attention as the agent logic.

When to move from local to hosted

A useful heuristic: move to a hosted runtime when any of these become true.

  • You need more concurrent agents than one machine can host.
  • You are spending engineering time on browser versions, crashes, and cleanup instead of agent logic.
  • You need persistent profiles and isolation that are painful to manage locally.
  • You need to watch or replay sessions for debugging or compliance.
  • You need configurable network egress without building a proxy layer.

If none of these apply, local Chrome is fine and cheaper. The decision is about operational load, not about which approach is "more advanced."

Getting started

The practical path for most teams:

  1. Prototype locally with Playwright and a real Chromium to validate the agent loop.
  2. Move the browser to a hosted session and connect over CDP — the agent code barely changes.
  3. Add persistent profiles for authenticated flows and isolation for concurrency.
  4. Wire up the live viewer and recordings before you scale, not after the first incident.
  5. Monitor usage and set lifecycle policies so idle sessions do not accumulate.

For the setup specifics, see the remote browser for AI agents guide, the remote browser online overview, and the remote control browser guide. The /documentation covers CDP endpoints, session APIs, and profile management in detail.

Browser AI agents are only as reliable as the browser they run in. Get the runtime right — isolation, persistence, observability, and lifecycle — and the model has a stable environment to reason about. Get it wrong, and no amount of model quality will save a session that lost its cookies halfway through a checkout flow.