← Blog

BLOG

Browser AI Agent: What It Is and the Runtime It Needs

A browser AI agent drives real Chromium to complete web tasks. Learn how it works, where it breaks in production, and how to run one on hosted sessions.

9 min readRemote Browser
Ivory computer towers connected across terracotta platforms in a quiet desert landscape
The infrastructure behind your browserREMOTE BROWSER / ILLUSTRATION

A browser AI agent is a model-driven loop that observes a web page, decides on an action, and executes it inside a real browser — clicking, typing, scrolling, and navigating until a task is done. The model supplies judgment; the browser supplies the environment. Most production failures come from the second half. This guide explains how a browser AI agent actually works, where local Chrome setups fall apart at scale, and what to evaluate in the runtime underneath it.

If you already know you need a hosted runtime, Remote Browser for AI agents covers the architecture directly. If you are still deciding, read on — the runtime decision is the one that determines whether your browser AI agent is a demo or a product.

How a browser AI agent works

Every browser AI agent, regardless of framework, runs the same four-stage loop:

  1. Observe. The agent captures the current state of the page — a DOM snapshot, an accessibility tree, a screenshot, or some combination. This is the model's only view of reality.
  2. Reason. The model receives the observation plus the task and produces a decision: click this element, type this text, navigate to this URL, or declare the task complete.
  3. Act. The runtime translates that decision into a browser action — a CDP command, a Playwright call, or a synthetic input event.
  4. Verify. The agent re-observes to confirm the action had the intended effect, then loops.

The quality of the loop depends on all four stages, but stages 1 and 3 are where the runtime earns its keep. A model that reasons perfectly against a stale or partial observation will still fail. An action layer that drops events, races the page load, or loses the session mid-task will fail regardless of model quality.

What "real browser" means here

A browser AI agent needs a genuine Chromium instance, not an HTTP client with a DOM parser. Modern sites render client-side, gate content behind JavaScript, and behave differently under automation. The agent must execute the same JavaScript the site ships, hold cookies and storage across navigations, and expose the page through the Chrome DevTools Protocol so the agent framework can inspect and control it.

That requirement is why the runtime layer matters. You are not running a script; you are running a stateful, long-lived browser process that a model drives turn by turn.

Where local browser setups break down

Running a browser AI agent against a local Chrome install works fine for development. It stops working when you move to production, for predictable reasons:

  • Session state is fragile. A crash, a laptop sleep, or a container restart kills the browser and the agent's entire context with it. Long tasks that span minutes or hours cannot survive that.
  • Concurrency is bounded by the machine. Ten parallel agents need ten browser processes, each consuming a large share of memory. Scaling means provisioning machines, not just spawning tasks.
  • Observability is missing. When an agent fails at step 14, you need the screenshot, the DOM, the network log, and the console output from that exact moment. Local runs rarely capture them.
  • Environment drift. Chrome versions, fonts, locales, and proxy settings differ across machines, so an agent that works on your laptop behaves differently in CI or on a worker fleet.
  • Credential handling is ad hoc. Agents that log in need persistent profiles, and storing those profiles on ephemeral workers is a security and reliability problem.

None of these are model problems. They are runtime problems, and they are the reason hosted browser infrastructure exists as a category.

What a production runtime provides

A hosted runtime moves the browser off your machine and behind an API. The agent framework connects to a remote Chromium session over CDP and drives it exactly as it would a local one. The differences show up in the operational layer:

CapabilityLocal ChromeHosted runtime
Session lifetimeTied to process/machineIndependent of your worker; survives restarts
ConcurrencyBounded by host resourcesProvisioned per session via API
Live debuggingAttach a debugger locallyLive viewer stream of the session
Persistent stateManual profile directoriesManaged persistent profiles
Network controlsLocal proxy configConfigurable proxy and browser settings per session
IsolationShared host environmentIsolated session per agent
ObservabilityWhatever you instrumentSession-level access to page state and logs
Version consistencyDrifts per machineConsistent Chromium build per session

The practical effect: your agent code stays the same, but the failure modes change from "the machine died" to "the task logic was wrong" — which is the failure mode you can actually fix.

CDP is the contract

The Chrome DevTools Protocol is the interface between your browser AI agent and the browser. Playwright, Puppeteer, Selenium, and most agent frameworks speak it, directly or through a wrapper. A hosted runtime that exposes a CDP endpoint is compatible with all of them without a proprietary SDK.

That compatibility is the main reason to prefer CDP-native runtimes over vendor-specific APIs. You keep the ability to swap frameworks, run the same agent locally for debugging, and avoid rewriting your action layer when you change providers. The Chrome DevTools Protocol documentation is the authoritative reference for the domains your agent will use — Page, Runtime, DOM, Input, and Network cover most agent workloads.

Connecting a browser AI agent over CDP

The connection pattern is the same across frameworks: create a session through the runtime's API, get a CDP WebSocket endpoint, and connect. Here is the Playwright version in TypeScript:

import { chromium, Browser, Page } from "playwright";

interface SessionInfo {
  cdpUrl: string; // wss://... from the runtime's session API
  sessionId: string;
}

async function createSession(): Promise<SessionInfo> {
  // Replace with your runtime's session-creation call.
  // Remote Browser returns a CDP WebSocket URL per session.
  const res = await fetch(`${process.env.BROWSER_API_URL}/sessions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${process.env.BROWSER_API_KEY}`,
    },
    body: JSON.stringify({
      // Configurable browser settings: proxy, locale, viewport, profile.
      profile: "agent-checkout",
      viewport: { width: 1280, height: 800 },
    }),
  });
  if (!res.ok) throw new Error(`session create failed: ${res.status}`);
  return (await res.json()) as SessionInfo;
}

async function runAgentTask(task: string) {
  const session = await createSession();
  let browser: Browser | undefined;
  try {
    browser = await chromium.connectOverCDP(session.cdpUrl);
    const context = browser.contexts()[0] ?? (await browser.newContext());
    const page: Page = context.pages()[0] ?? (await context.newPage());

    await page.goto("https://example.com", { waitUntil: "domcontentloaded" });

    // Agent loop: observe -> decide -> act -> verify.
    // The model call is your own; the browser side is standard Playwright.
    for (let step = 0; step < 20; step++) {
      const snapshot = await page.accessibility.snapshot();
      const action = await decideNextAction(task, snapshot); // your LLM call
      if (action.type === "done") break;

      if (action.type === "click") {
        await page.locator(action.selector).click({ timeout: 10_000 });
      } else if (action.type === "type") {
        await page.locator(action.selector).fill(action.text);
      } else if (action.type === "goto") {
        await page.goto(action.url, { waitUntil: "domcontentloaded" });
      }
      await page.waitForLoadState("networkidle").catch(() => {});
    }
  } finally {
    // Disconnect without killing the remote session if you want to resume later.
    await browser?.close();
  }
}

Two details matter more than they look. First, connectOverCDP attaches to an existing browser rather than launching one, so the session lifecycle belongs to the runtime, not your process. Second, closing the Playwright connection does not necessarily end the remote session — that separation is what lets a long task survive a worker restart and resume from a persistent profile.

If you are wiring this up for the first time, the documentation covers session creation, CDP endpoints, and profile management in detail.

Production criteria for evaluating a runtime

When you compare runtimes for a browser AI agent, evaluate against the failure modes you will actually hit:

Session durability. Can a session outlive the process that created it? Can you reconnect to a running session from a different worker? Long agent tasks need both.

Profile persistence. Agents that log in need cookies and storage to survive across sessions. Check how profiles are created, named, and reused, and whether they are isolated between tenants.

Network controls. Many tasks require a specific egress region or proxy. Look for configurable proxy settings per session rather than a single global configuration.

Observability. A live viewer that streams the session in real time is the fastest debugging tool you have. Pair it with access to console logs and network activity at the session level.

Framework compatibility. CDP-native access means Playwright, Puppeteer, Selenium, and agent frameworks all work. A proprietary action API means lock-in.

Usage controls. You need visibility into how many sessions run concurrently and how long they live. Metering and limits should be explicit — see pricing for how Remote Browser meters browser-hours.

Isolation. Each agent session should be isolated from others: separate browser process, separate storage, separate network identity where configured.

The stealth question

Agents that hit anti-bot systems need more than a clean IP. Runtimes differ in what they expose: configurable browser settings, proxy pools, and header controls vary widely, and claims about detection evasion are hard to verify. Treat stealth as a configuration surface to test against your actual targets, not a checkbox. Run a small benchmark against the sites you care about before committing.

Common mistakes when building a browser AI agent

  • Treating the browser as stateless. Every navigation is a state transition. Agents that assume a page is loaded when the DOM event fires will fail on client-rendered sites. Wait on the condition you need, not a fixed timeout.
  • Ignoring the observation budget. Full DOM dumps blow through context windows and cost. Accessibility trees and targeted element extraction are usually better inputs than raw HTML.
  • No step limit. Every agent loop needs a hard cap on iterations and a wall-clock timeout. Without them, a confused agent burns browser-hours doing nothing.
  • Skipping verification. An agent that clicks and moves on without confirming the result will compound errors silently. Re-observe after every action.
  • Coupling to one framework. If your action layer is written against a vendor SDK, migrating is a rewrite. CDP keeps the door open.

When a hosted runtime is the right call

A hosted runtime is the right choice when any of these are true: your agents run longer than a few minutes, you need more than a handful in parallel, you need persistent logins, or you need to debug failures after the fact. It is overkill for a single script that runs once a day on your laptop.

The middle ground — running Chromium in a container you manage — works until you need session durability and a viewer, at which point you are rebuilding hosted infrastructure. For most teams shipping a browser AI agent, the runtime is not the interesting problem; the agent logic is. Buy the runtime, own the logic.

To see what a hosted session looks like in practice, start with running a remote browser online, or go straight to the documentation and create a session.