Browser Use Models: Choosing a Runtime for AI Agents

# Browser Use Models: Choosing a Runtime for AI Agents

"Browser use models" is a phrase that gets used two ways, and the confusion costs teams real time. Sometimes it means the language models that drive browser-use agents — the reasoning layer that reads a page and decides what to click. Sometimes it means the browser runtime those models actually run inside: the Chromium instance, the CDP endpoint, the session that survives a retry. Both matter, but only one of them is the thing you can control when a task fails at 3 a.m.

This post is about the second meaning. If you are wiring up a browser use model — whether that is a hosted model, a local model, or an agent framework like Hermes — the runtime underneath determines whether your agent finishes tasks or burns tokens on timeouts. We will cover what a browser use model actually needs from its runtime, how hosted Chromium compares to local Chrome, and how to connect a Hermes agent-browser setup over CDP without the usual breakage.

What "browser use models" actually refers to

The term shows up in three overlapping contexts:

• The model layer. An LLM that takes a DOM snapshot, a screenshot, or an accessibility tree and outputs an action. This is the "browser use" part of browser-use, and it is what most benchmark discussions focus on.
• The agent framework. The orchestration code that loops: observe → decide → act → verify. Hermes, browser-use, and similar frameworks live here.
• The browser runtime. The actual Chromium process, its CDP endpoint, its profile, its network egress. This is where reliability is won or lost.

Most teams spend their effort on layer one and discover too late that layer three is the bottleneck. A model that scores well on a static benchmark can still fail in production because the browser it drives is slow to start, gets fingerprinted, or loses its session between steps.

If you want the deeper architectural argument, our post on remote browsers for AI agents covers why the runtime layer became a distinct product category.

Why the runtime, not the model, decides task success

Consider what happens during a single browser-use step:

1. The agent requests a page state.
2. The runtime serializes the DOM or captures a screenshot.
3. The model reasons over that state and emits an action.
4. The runtime executes the action via CDP.
5. The agent verifies the result and loops.

Steps 1, 2, 4, and 5 are all runtime work. If the runtime is a local Chrome instance on a developer laptop, you inherit every problem that comes with it: cold-start latency, profile corruption, memory leaks across long sessions, and no isolation between concurrent agents.

Hosted Chromium changes the failure profile. Sessions start from a known-good image, run in isolated containers, and expose a CDP endpoint you connect to over the network. The model layer stays the same; the runtime stops being a variable you debug by hand.

The three failure modes that trace back to the runtime

• Session death mid-task. A local browser crashes or the tab is closed by another process. Hosted sessions are isolated, so one agent cannot kill another's browser.
• Detection and blocking. Datacenter IPs and default Chromium fingerprints get flagged. Configurable browser settings and proxy options give you levers here, though no runtime can guarantee a site will not block you.
• Non-reproducible state. Without persistent profiles, every run starts from zero and you cannot debug what the agent saw. Persistent profiles and a live viewer make failures inspectable.

Hosted Chromium vs local Chrome for browser use models

The comparison below is the one that matters when you move from a notebook to a service.

The trade-off is real: hosted runtimes add network latency between your agent and the browser. For most browser-use workloads, that latency is smaller than the variance you get from local cold starts and crashes. For latency-critical, single-step tasks, local may still win.

Connecting a Hermes agent-browser setup over CDP

The most common integration question we see is some variant of "hermes remote browser not working." Almost always the cause is one of four things: the CDP URL is wrong, the WebSocket is being blocked, the agent is launching its own browser instead of connecting, or the session expired.

The fix is to treat the remote browser as an endpoint, not a launch target. Here is a TypeScript example using Playwright's connectOverCDP, which is the same path Puppeteer and raw CDP clients use:

import { chromium } from 'playwright';

// The CDP endpoint comes from your Remote Browser session.
// Treat it like any other secret — do not hardcode it.
const cdpUrl = process.env.REMOTE_BROWSER_CDP_URL;

if (!cdpUrl) {
  throw new Error('REMOTE_BROWSER_CDP_URL is not set');
}

async function runAgentTask() {
  // connectOverCDP attaches to an existing browser instead of
  // launching a new one. This is the key difference from chromium.launch().
  const browser = await chromium.connectOverCDP(cdpUrl, {
    timeout: 30_000,
  });

  const context = browser.contexts()[0] ?? (await browser.newContext());
  const page = context.pages()[0] ?? (await context.newPage());

  try {
    await page.goto('https://example.com', { waitUntil: 'domcontentloaded' });

    // Your browser use model observes state here and emits actions.
    const title = await page.title();
    console.log('Page title:', title);

    // Persist anything the agent needs before the session ends.
    await context.storageState({ path: 'state.json' });
  } finally {
    // Close the connection, not the remote browser, unless you own its lifecycle.
    await browser.close();
  }
}

runAgentTask().catch((err) => {
  console.error('Agent task failed:', err);
  process.exit(1);
});

Three details in that snippet matter more than they look:

• `connectOverCDP`, not `launch`. If your Hermes agent-browser code calls chromium.launch(), it will start a local browser and ignore your remote endpoint entirely. This is the single most common cause of "hermes browser connect" failures.
• `browser.close()` semantics. Closing the connection is not the same as terminating the remote session. Check your runtime's session lifecycle so you do not leak browser-hours.
• Timeouts. Remote connections need explicit timeouts. The default is often too short for a cold session and too long for a dead one.

For the protocol-level view, the Chrome DevTools Protocol documentation is the authoritative reference for what your client can actually send over that WebSocket.

Debugging "hermes remote browser not working"

Work through these in order:

1. Verify the endpoint. Open the CDP URL in a WebSocket client. If it does not connect, the problem is upstream of your agent.
2. Check for a local launch call. Grep your Hermes browser skill for launch, puppeteer.launch, or chromium.launch. Replace with a connect call.
3. Confirm the session is alive. Sessions have lifetimes. If your agent sleeps between steps, the session may have been reclaimed.
4. Inspect via the live viewer. Watching the session in real time tells you whether the agent is acting on a stale page or a blank one.
5. Check egress. If the page loads locally but not remotely, the issue is network policy, not CDP.

What to look for in a browser runtime for agents

When you evaluate a hosted runtime — Remote Browser or otherwise — these are the criteria that separate a demo from infrastructure:

• CDP compatibility. Does it expose a standard CDP endpoint, or a proprietary protocol? Standard CDP means Playwright, Puppeteer, and Selenium all work without adapters.
• Session isolation. One agent's crash should not affect another's session.
• Persistent profiles. Can you carry cookies and storage across runs, and can you reset them deliberately?
• Live observability. A viewer and session logs turn "it failed" into "here is what it saw."
• Configurable browser settings. Proxy selection, user-agent, locale, and viewport should be settable per session. Be skeptical of anyone promising guaranteed evasion.
• Usage controls. You need to know what a session costs before you run ten thousand of them. See /pricing for current rates and limits.
• Documentation depth. The /documentation page should answer connection, lifecycle, and profile questions without a sales call.

If you are still deciding between running Chromium yourself and using a hosted runtime, remote browser online walks through the operational differences.

Where browser use models and runtimes meet

The interesting engineering question is not "which model is best" but "what does the model need from the runtime to succeed." A few concrete implications:

• Screenshot-based models need fast, consistent rendering. If your runtime throttles or renders inconsistently, the model sees a different page than a human would.
• DOM-based models need stable selectors. Dynamic class names break agents. A runtime that lets you inject a stable accessibility tree or a cleaned DOM helps more than a better model.
• Multi-step agents need session continuity. Every session reset is a chance to lose context. Persistent profiles reduce that risk.
• Long-horizon agents need cost visibility. A model that retries five times costs five times the browser-hours. Metering matters.

This is why "browser use models" as a search term is slightly misleading. The model is one input. The runtime is the system.

Practical migration path

If you are moving a browser use model from local Chrome to a hosted runtime, the sequence that causes the least pain:

1. Abstract the connection. Put your CDP URL behind an environment variable today, even if you are still local.
2. Switch to `connectOverCDP`. Verify your agent connects to an existing browser before you change anything else.
3. Add session lifecycle handling. Know when a session starts, when it expires, and how to request a new one.
4. Move profiles to the runtime. Stop shipping state.json files around; use persistent profiles.
5. Instrument cost. Track browser-hours per task before you scale.
6. Add the live viewer to your debugging loop. It replaces most of the guesswork.

None of these steps require changing your model. That is the point — the runtime is a separable layer, and treating it as one is what makes browser use models production-viable.

Summary

Browser use models are only as good as the browser they run in. Hosted Chromium gives you isolated sessions, standard CDP access, persistent profiles, and observability that local Chrome cannot match. The trade-off is added network latency and a dependency on your runtime provider's reliability. For most agent workloads, that trade is worth it.

If you are debugging a Hermes agent-browser integration, start with the connection: use connectOverCDP, verify the endpoint, and watch the session in the live viewer. Nine times out of ten, "hermes remote browser not working" is a launch call that should have been a connect call.

Ready to test it? Start with the /documentation, then check /pricing to understand what a session costs before you scale.