Hermes Browser Use: Run AI Agents on Hosted Chromium

# Hermes Browser Use: Run AI Agents on Hosted Chromium

Hermes browser use refers to running a Hermes-style AI agent — a model-driven loop that plans, calls tools, and observes results — against a real browser instead of a static HTTP client. The agent decides what to click, type, or read; the browser executes it and returns the resulting DOM, screenshot, or accessibility tree. The hard part is not the reasoning loop. It is giving that loop a browser that is fast, isolated, observable, and cheap to run at scale. This guide covers how Hermes browser use actually works, where local Chrome breaks down, and how to wire an agent to hosted Chromium over CDP with Playwright.

If you are already running agents locally, the pattern will look familiar. What changes is the runtime underneath: instead of chromium.launch() on your laptop, you connect to a remote session that already exists, with its own profile, proxy settings, and lifecycle. That shift is what makes agent workloads viable in production.

What "Hermes Browser Use" Actually Means

The term gets used loosely, so it helps to separate the layers:

• The agent loop. A model receives a task, chooses an action (navigate, click, extract), and repeats until done. This is the "Hermes" part — the reasoning and tool-calling layer.
• The browser tool. The interface the agent calls to act on the web. Usually a thin wrapper over Playwright, Puppeteer, or raw CDP.
• The browser runtime. The actual Chromium process: where it runs, how long it lives, what profile and network identity it uses, and how you observe it.

Most tutorials focus on the first two layers and treat the third as an afterthought. In practice, the runtime is where agent projects succeed or stall. A reasoning loop that works in a demo will fail in production if sessions leak, profiles reset, or you cannot see what the agent saw when it made a bad decision.

Hermes browser use, then, is really a question of runtime design: how do you give an autonomous loop a browser it can trust?

Why Local Chrome Stops Working for Agents

Running playwright install and launching Chromium locally is fine for development. It breaks down in predictable ways once agents run continuously:

• Session state collides. Two agents sharing one Chrome profile overwrite each other's cookies, localStorage, and login state.
• Resource contention. Headless Chromium is memory-hungry. Ten concurrent agents on one machine will thrash, and the failure mode is a timeout, not a clean error.
• No isolation. A crashed agent can leave zombie processes and locked profile directories that block the next run.
• No observability. When an agent fails at step 14, you have logs but no replay. You cannot see the page it saw.
• Network identity is fixed. Your laptop's IP is your agent's IP. That is a problem for any workload that needs stable or varied egress.

None of these are reasoning problems. They are infrastructure problems, and they compound as you scale from one agent to many.

The Runtime Criteria That Matter

Before comparing options, define what a production agent runtime needs. These are the criteria that separate a demo from something you can leave running:

The right-hand column is not automatically better — it is better *for agent workloads specifically*, because agents are long-running, concurrent, and unpredictable. A test suite that runs for 90 seconds in CI does not need most of this. An agent that runs for 20 minutes across 40 steps does.

Connecting a Hermes-Style Agent to Hosted Chromium

The connection mechanism is the Chrome DevTools Protocol. Playwright exposes it through connectOverCDP, which attaches to an already-running browser rather than launching one. This is the same primitive you would use to attach to a local Chrome instance, just pointed at a remote endpoint.

Here is a minimal TypeScript example. It assumes you have a session endpoint and a CDP WebSocket URL from your runtime provider:

import { chromium, Browser, Page } from 'playwright';

interface AgentSession {
  cdpUrl: string;
}

async function connectAgentSession(session: AgentSession): Promise<{
  browser: Browser;
  page: Page;
}> {
  // Attach to the already-running remote Chromium over CDP.
  // No local Chrome is launched; the browser lives in the hosted runtime.
  const browser = await chromium.connectOverCDP(session.cdpUrl, {
    timeout: 30_000,
  });

  // Reuse the existing context so cookies and storage persist
  // across agent steps and across reconnects.
  const contexts = browser.contexts();
  const context = contexts.length > 0
    ? contexts[0]
    : await browser.newContext();

  const page = context.pages()[0] ?? (await context.newPage());

  // Give the agent a stable viewport and a hard navigation timeout.
  await page.setViewportSize({ width: 1280, height: 800 });
  page.setDefaultTimeout(15_000);

  return { browser, page };
}

async function runAgentStep(page: Page, action: AgentAction) {
  switch (action.type) {
    case 'navigate':
      await page.goto(action.url, { waitUntil: 'domcontentloaded' });
      break;
    case 'click':
      await page.locator(action.selector).click();
      break;
    case 'extract':
      return page.locator(action.selector).innerText();
  }
}

Two details matter here. First, connectOverCDP does not create a browser — it attaches to one. That means the session's profile, proxy, and lifecycle are managed by the runtime, not your script. Second, reusing browser.contexts()[0] instead of creating a fresh context is what preserves login state between agent steps. If you create a new context every time, you throw away the session's identity.

The Playwright documentation on connectOverCDP is the authoritative reference for the options and limitations of this call — notably that it is Chromium-only, which is exactly what hosted Chromium runtimes provide.

Where Hermes Browser Use Fits in a Real Stack

A production agent stack usually has four moving parts, and it helps to be explicit about which one you are building:

1. Orchestration. Task queue, retries, and state machine. Often your own code or a workflow engine.
2. Reasoning. The model and its tool schema. This is the Hermes layer.
3. Browser tool. The adapter that translates model actions into Playwright calls.
4. Browser runtime. The hosted Chromium session.

Most teams over-invest in layer 2 and under-invest in layer 4. The reasoning model is the visible part; the runtime is the part that determines whether your success rate holds up at 3 a.m. with 50 concurrent agents.

If you want a deeper treatment of how these layers interact, the remote browser for AI agents guide covers the runtime layer in more detail, and remote web browser covers the practical setup path.

Practical Setup Checklist

Before you point a Hermes-style agent at a hosted runtime, work through this list. Each item maps to a failure mode that shows up later if skipped.

• Pin the browser version. Agent behavior changes when Chromium updates. Know which version your sessions run.
• Set explicit timeouts. Navigation, action, and overall task timeouts should all be defined. Agents will otherwise wait indefinitely on a hung page.
• Decide profile strategy up front. Ephemeral sessions for scraping, persistent profiles for anything requiring login.
• Log the observation, not just the action. Store the DOM snapshot or screenshot the agent saw at each step. This is what makes debugging possible.
• Handle reconnects. CDP connections drop. Your agent loop should be able to reattach to the same session rather than starting over.
• Cap concurrency deliberately. Know your session limit and queue work rather than spawning unbounded browsers.

The reconnect point deserves emphasis. A naive agent loop treats a dropped CDP connection as a fatal error. A production loop reconnects to the same session, re-reads the current page state, and continues. This is only possible if the session outlives the connection — which is a property of the runtime, not your script.

Observability: The Difference Between Debugging and Guessing

When an agent fails, you need to answer one question: what did it see? Without a live viewer or session recording, you are reduced to reading logs and guessing.

A hosted runtime typically provides a live viewer — a real-time view of the session that a human can watch or take over. This is not a nice-to-have for agent work. It is the difference between a 10-minute fix and a two-day investigation. It also enables a useful pattern: when the agent gets stuck, a human takes over, completes the step, and hands control back. The session state is preserved because the browser never restarted.

For teams running remote control browser workflows, this handoff model is often the fastest path to reliability — not because the agent is bad, but because some steps genuinely need a human, and the cost of a handoff is far lower than the cost of a failed run.

Cost and Lifecycle Considerations

Agent workloads have a different cost profile than test suites. A test run is short and predictable. An agent run is long and variable — it might finish in 30 seconds or time out after 20 minutes.

This makes metered, per-session pricing a better fit than fixed infrastructure for most agent projects, because you pay for actual browser time rather than idle capacity. But it also means lifecycle discipline matters: an agent that forgets to close its session keeps billing. Build teardown into your orchestration layer, not your agent's happy path.

Current rates and session limits are listed on the pricing page. Check there rather than assuming a number, since these change.

Common Mistakes in Hermes Browser Use

A short list of patterns that cause most production failures:

• Launching a browser per action. Slow, expensive, and destroys session state. Connect once, act many times.
• Sharing one context across agents. Guarantees cookie and storage collisions.
• Ignoring the accessibility tree. Feeding raw HTML to a model wastes tokens and hurts accuracy. The accessibility snapshot is usually a better observation channel.
• No retry budget. Transient network failures are normal. An agent with no retries will fail on noise.
• Treating the runtime as an afterthought. The reasoning loop is the easy part to iterate on. The runtime is what you have to get right once.

Getting Started

The shortest path is: pick a runtime, get a CDP endpoint, and connect with connectOverCDP. Everything else — profiles, proxies, viewers, concurrency — is configuration on top of that primitive.

Start with the documentation for the connection details and session API, then read remote browser online if you want to run a session without any local setup at all. The goal is to spend your engineering time on the agent's reasoning and tool design, not on keeping Chromium processes alive.