Browser Web Agent: Runtime Choices for Production

# Browser Web Agent: Runtime Choices for Production

A browser web agent is a program that drives a real browser — navigating pages, clicking, typing, reading the DOM — to complete a task on the open web. The agent is the decision layer; the browser is the execution layer. Most teams get the first part working in a day and then spend months on the second. This guide covers what a browser web agent actually needs from its runtime, how the main options compare, and how to wire one up over CDP without owning the Chromium fleet.

If you are choosing a runtime today, the short version: local Chrome is fine for development, self-hosted Chromium is fine if you have platform engineers to spare, and a hosted runtime is the default for anything that has to run unattended. The rest of this post explains why and shows the code.

What a browser web agent actually needs

Strip away the model and the prompt and a browser web agent is a loop: observe the page, decide on an action, execute it, observe again. Each of those steps has a runtime requirement that is easy to underestimate.

• A real browser, not a headless shim. Modern sites lean on JavaScript, service workers, and layout-dependent behavior. Agents that parse raw HTML miss most of what a user would see.
• A stable control channel. The agent process and the browser process should not share a machine if you want to scale or survive restarts. CDP over a WebSocket endpoint is the standard answer.
• Session isolation. Two agents running concurrently must not share cookies, storage, or tabs. Isolation has to be enforced by the runtime, not by convention.
• Persistent state where it matters. Logins, consent banners, and multi-step flows often span more than one session. You need profiles that survive a session ending.
• Observability. When an agent fails at step 14, you need to see what the page looked like. Screenshots and a live viewer are not nice-to-haves; they are the debugging interface.
• Cost control. Browser time is metered. A runaway agent that leaves sessions open is a billing incident waiting to happen.

None of these are exotic. The problem is that assembling all six yourself is a platform project, and most agent teams would rather spend that time on the agent.

The three runtime options, compared

There are really only three ways to give a browser web agent a browser. They differ in who owns the operational surface.

The middle column is where most teams get stuck. Self-hosting Chromium is not hard to start — a Docker image and a CDP port gets you going. It is hard to keep running: zombie processes, memory leaks across long sessions, version drift between the browser and your Playwright client, and the concurrency scheduler you end up writing from scratch. If you want the longer argument, remote browsers for AI agents covers the operational failure modes in detail.

Hosted runtimes are not automatically better. They are better when the alternative is you building and maintaining a browser fleet. If your workload is ten sessions a day on a schedule, self-hosting is defensible. If it is many concurrent agents with retries, it is not.

Where the popular options sit

The market has split into three rough categories, and the names you will see in search results map onto them.

Agent frameworks like browser-use give you the decision loop — the LLM-driven planner that decides what to click. They are not runtimes. They need a browser to drive, and they will drive whatever CDP endpoint you hand them.

Hosted browser providers like Browserbase, Hyperbrowser, Steel, Kernel, and Anchor sell sessions. They differ on pricing model, stealth characteristics, session startup latency, and how much of the agent layer they bundle. Some are closer to raw infrastructure; others ship opinionated agent tooling on top.

Self-hostable open-source options like Browserless and Steel (in its self-hosted form) let you run the same kind of session API on your own infrastructure. This is the right answer if data residency or existing cloud commitments force it, and the wrong answer if you are trying to avoid building a platform team.

Remote Browser sits in the hosted category with a deliberate emphasis on the runtime primitives rather than the agent: hosted Chromium sessions, CDP access, Playwright/Puppeteer/Selenium compatibility, a live viewer, persistent profiles, configurable browser and proxy settings, and per-session isolation. It does not try to be your planner. You bring the agent; it provides the browser and the operational surface around it.

A practical way to evaluate any of these: ask what happens when a session dies mid-task, whether you can attach to a running session to debug it, and how concurrency is billed. The answers separate real runtimes from thin wrappers. Current plan details and metering are on the pricing page.

Wiring a browser web agent over CDP

The integration is the same regardless of which hosted runtime you pick, because they all speak CDP. Here is a TypeScript example using Playwright. It assumes your runtime exposes a WebSocket debugger URL per session.

import { chromium, Browser, Page } from "playwright";

interface SessionInfo {
  cdpUrl: string;   // wss://... from your runtime's session API
  sessionId: string;
}

async function createSession(): Promise<SessionInfo> {
  // Replace with your runtime's session-creation endpoint.
  const res = await fetch(`${process.env.BROWSER_API_URL}/sessions`, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.BROWSER_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      // Configurable browser settings: region, proxy, profile, etc.
      profile: "checkout-agent",
      timeoutSeconds: 900,
    }),
  });
  if (!res.ok) throw new Error(`session create failed: ${res.status}`);
  const data = await res.json();
  return { cdpUrl: data.cdpUrl, sessionId: data.id };
}

async function runAgentTask(session: SessionInfo, task: string) {
  const browser: Browser = await chromium.connectOverCDP(session.cdpUrl);
  const context = browser.contexts()[0] ?? await browser.newContext();
  const page: Page = context.pages()[0] ?? await context.newPage();

  try {
    await page.goto("https://example.com", { waitUntil: "domcontentloaded" });

    // Your agent loop goes here. Each iteration: observe, decide, act.
    // Keep actions small and re-read state after every mutation.
    const title = await page.title();
    console.log(`[${session.sessionId}] on page: ${title} — task: ${task}`);

    // Example single action; a real agent would loop until done.
    await page.getByRole("link", { name: /more information/i }).click();
    await page.waitForLoadState("networkidle");
  } finally {
    // Disconnect the client; the runtime decides when the session ends.
    await browser.close();
  }
}

(async () => {
  const session = await createSession();
  await runAgentTask(session, "find the contact page");
})();

A few things worth noting in that snippet:

• connectOverCDP attaches to an existing browser rather than launching one. That is the whole point — the browser lives somewhere else.
• browser.close() disconnects the client. Whether the underlying session terminates is a runtime policy, so check your provider's semantics before relying on it for cleanup.
• The profile field is where persistent state lives. Reusing a profile name across sessions is how you keep logins and cookies without re-authenticating every run.

The Playwright side of this is documented in the BrowserType.connectOverCDP reference, and the underlying protocol is specified in the Chrome DevTools Protocol docs. If you are using Puppeteer instead, the equivalent is puppeteer.connect({ browserWSEndpoint }).

Production criteria that actually matter

Once the agent works locally, the questions change. These are the ones that decide whether a runtime holds up.

Session lifecycle and cleanup

Every session should have a hard timeout and a defined teardown path. Agents fail in ways that leave browsers open — a thrown exception before close(), a hung waitForNavigation, a model that loops. The runtime should reap orphaned sessions without you writing a cron job. Ask what the maximum session duration is and what happens at the boundary.

Isolation guarantees

Concurrent sessions must not leak state. Verify this by running two sessions against the same site with different profiles and confirming cookies do not cross. Isolation is usually enforced at the container or context level; the implementation detail matters less than the guarantee.

Debuggability

When an agent fails, you want the page as it was. A live viewer that lets you watch a session in real time, plus screenshots captured at each step, turns a two-hour investigation into a two-minute one. This is the single biggest quality-of-life difference between a hosted runtime and a hand-rolled one. See remote control browser for how live session access works in practice.

Network and browser configuration

Proxies, geolocation, locale, and user agent are all things agents need to set per task. The runtime should expose these as configuration, not as code you patch into the browser launch. Be precise about what you need: "configurable browser settings" covers proxy and locale. Any claims about advanced anti-detection features should be verified against the provider's documentation rather than assumed.

Cost visibility

Browser time is the unit of cost. You want per-session attribution so you can tell which agent or customer is consuming hours. If you cannot break down spend by workload, you cannot optimize it. Remote Browser's metering model is described on pricing; the same discipline applies to any provider.

Client compatibility

If your team already writes Playwright, you should not have to rewrite to a proprietary SDK. CDP compatibility means Playwright, Puppeteer, and Selenium all work against the same session. That is a portability guarantee as much as a convenience one — it means switching runtimes later is a config change, not a migration.

Common failure modes

A few patterns show up repeatedly when teams move a browser web agent from demo to production.

Treating the agent loop as the hard part. The planner is usually the easy bit. The failures are environmental: a session that died, a page that never finished loading, a selector that matched the wrong element because the DOM shifted. Budget engineering time for the runtime layer accordingly.

No retry semantics. Agents fail transiently. A retry needs a fresh session or a clean page state, not a re-run on top of a half-completed flow. Decide this before you ship.

Ignoring page load semantics. networkidle is not a reliable signal on sites with long-polling or analytics beacons. Prefer explicit waits on the element the agent needs next.

Leaving sessions open on error paths. Wrap the agent loop in try/finally and make sure the runtime enforces its own timeout as a backstop.

Skipping the viewer during development. Watching the agent run is the fastest way to find out that it is clicking the wrong button. Use it before you need it in an incident.

Choosing a runtime

The decision comes down to three questions. Do you have platform engineers who can own a Chromium fleet? Does your data policy require self-hosting? And how much concurrency do you need to sustain?

If the answer to the first is no and the second is no, a hosted runtime is the pragmatic choice, and the remaining work is picking one whose primitives match your agent. Look for CDP access, persistent profiles, a live viewer, and clear per-session metering. If you want to see what a session looks like end to end before committing, remote browser online walks through the full flow, and the documentation covers the API surface in detail.

The browser web agent pattern is not going away — it is how a growing share of web work gets done. The teams that ship reliably are the ones that treat the browser as infrastructure and stop treating it as a local dependency.