← Blog

BLOG

Hermes Agent Web Interface: Connect It to a Remote Browser

Wire the Hermes agent web interface to hosted Chromium over CDP. Session setup, Playwright code, and production checks for reliable browser tasks.

9 min readRemote Browser
Ivory computer towers connected across terracotta platforms in a quiet desert landscape
The infrastructure behind your browserREMOTE BROWSER / ILLUSTRATION

The Hermes agent web interface is the browser-based control surface where you configure an agent, start runs, and watch what it does. If your Hermes agent needs to touch real websites — log in, fill forms, scrape a dashboard, click through a checkout — the web interface alone is not enough. It needs a browser it can drive, and that browser should not be a Chrome window on your laptop.

This guide covers the integration: how the Hermes agent web interface hands work to a browser, why a hosted Chromium session is the right target for anything beyond a demo, and the concrete CDP wiring that makes it work. The short version: point Hermes at a remote browser endpoint, connect over the Chrome DevTools Protocol, and keep the session lifecycle under your control.

What the Hermes agent web interface actually is

Hermes is an agent framework with a web UI layer. The Hermes agent web interface is where a human sets up the agent — model choice, tools, task prompt, credentials — and where run output surfaces. It is not a browser. It is a control plane.

That distinction matters because most "my agent can't browse" problems are not interface problems. They are runtime problems. The web interface can hold your task definition and display a live view of a session, but the session itself has to run somewhere with a real Chromium binary, a network path to the target site, and a way for the agent to issue commands.

Three layers are involved:

  • Control plane — the Hermes agent web interface. Configuration, run history, human oversight.
  • Agent runtime — the process that reasons over page state and decides the next action.
  • Browser runtime — the Chromium instance the agent drives, plus the CDP endpoint that exposes it.

When people search for a "hermes agent web interface," they usually want one of two things: a way to see the agent working in a browser, or a way to make the agent's browser work reliably. This article addresses the second, because the first is only useful if the second holds up.

Why the browser runtime decides whether Hermes works

An agent that drives a local Chrome install inherits every problem that install has. Version drift between the agent's Playwright or Puppeteer client and the local Chromium build. A single browser profile shared across concurrent runs, so two tasks fight over cookies. No isolation between a test run and a production run. No way to inspect a session after it fails, because the window closed.

A hosted browser runtime moves those concerns off the machine running the Hermes agent web interface. You get:

  • A fresh, isolated session per run — no cookie bleed between tasks.
  • A stable CDP endpoint — the agent connects to a WebSocket URL instead of spawning a process.
  • A live viewer — a human can watch the session in the web interface while it runs.
  • Persistent profiles when you need them — saved logins survive across sessions without sharing state across unrelated tasks.
  • Configurable browser settings — proxy routing, user-agent, and other session-level options set at creation time.

For a deeper treatment of the runtime layer, see Remote browsers for AI agents.

Connecting the Hermes agent web interface to hosted Chromium

The integration path is CDP. Hermes' browser tooling speaks the same protocol Playwright and Puppeteer use, so the wiring is a connection string plus a session lifecycle.

Step 1: Create a session and capture the endpoint

When your Hermes run starts, request a browser session from the Remote Browser API. The response includes a CDP WebSocket endpoint. Treat that endpoint as a secret — anyone holding it can drive the session.

Keep session creation inside your backend, not in the browser tab hosting the Hermes agent web interface. The web UI should call your service; your service calls the browser API; your service returns only what the UI needs to display.

Step 2: Connect over CDP

Here is a minimal Playwright connection that mirrors what a Hermes browser tool does under the hood:

import { chromium, type Browser, type Page } from "playwright";

interface SessionHandle {
  cdpUrl: string;
  sessionId: string;
}

async function connectHermesBrowser(session: SessionHandle): Promise<{
  browser: Browser;
  page: Page;
}> {
  const browser = await chromium.connectOverCDP(session.cdpUrl, {
    timeout: 30_000,
  });

  // A hosted session usually starts with one context and one page.
  const contexts = browser.contexts();
  const context = contexts.length > 0
    ? contexts[0]
    : await browser.newContext();

  const pages = context.pages();
  const page = pages.length > 0 ? pages[0] : await context.newPage();

  page.setDefaultTimeout(20_000);
  return { browser, page };
}

async function runHermesTask(session: SessionHandle, task: (p: Page) => Promise<void>) {
  const { browser, page } = await connectHermesBrowser(session);
  try {
    await task(page);
  } finally {
    // Disconnect the client; the hosted session is stopped separately.
    await browser.close();
  }
}

Two details are easy to get wrong. First, connectOverCDP attaches to an existing browser — it does not launch one, so browser.close() disconnects your client rather than killing the remote Chromium. Second, the session may already have a page open; reuse it instead of creating a second one, or your agent will act on a blank tab while the real page sits idle.

The Playwright documentation on connecting to an existing browser over CDP is the authoritative reference for the connection semantics.

Step 3: Stop the session explicitly

When the task finishes — success or failure — call the session-stop endpoint. Do not rely on the client disconnect to clean up. A session left running continues to consume browser-hours, and if your pricing is metered, that is real cost. Check pricing for how sessions are billed.

Step 4: Surface the live view in the web interface

If the Hermes agent web interface supports embedding a live session view, pass the viewer URL for the session alongside the run status. This is the feature that makes the web interface genuinely useful: a human can watch the agent navigate, spot a stuck loop, and intervene before the run burns its budget.

Runtime comparison: local Chrome vs hosted Chromium for Hermes

CriterionLocal Chrome on the Hermes hostHosted Chromium session
Session isolationShared profile unless you build isolationOne isolated session per request
ConcurrencyBounded by host CPU and memoryScales with your session quota
CDP endpointLocal port, not reachable remotelyWebSocket URL, reachable from any worker
Live viewingScreen share or VNC workaroundBuilt-in viewer URL
Persistent loginsManual profile managementProfiles you attach per session
Proxy configurationOS-level or launch flagsSet per session at creation
Failure debuggingWindow closes, state lostSession artifacts and viewer retained
Cost modelHost cost, effectively fixedMetered per browser-hour

The local option is fine for a single developer iterating on a task. It breaks the moment you run two Hermes agents in parallel, move the agent to a container without a display server, or need to debug a failure after the fact.

Production criteria for the Hermes browser integration

Before you ship a Hermes agent that browses, check these:

  • Session lifecycle is owned by your backend. The web interface triggers runs; it does not hold CDP credentials. Rotate or scope any token the UI can see.
  • Every run has a hard timeout. An agent that loops on a selector will run until something stops it. Set a task-level deadline and a session-level cap.
  • Reconnects are handled. CDP connections drop. Wrap the connection in retry logic that re-attaches to the same session rather than creating a new one, or you will orphan sessions.
  • Page state is asserted, not assumed. After every navigation or click, wait for a concrete condition — a selector, a network idle state, a URL change. Agents that act on stale DOM produce flaky runs.
  • Artifacts are captured on failure. Screenshot and page HTML at the point of failure turn a mystery into a fix.
  • Concurrency is planned. Know how many simultaneous sessions your plan allows and queue Hermes runs accordingly. Do not assume the plan permits unbounded parallelism.

If you are still deciding between running Chromium yourself and using a hosted runtime, Remote browser online covers the trade-offs in more depth.

Where the web interface and the browser runtime meet

The clean architecture separates three responsibilities:

  1. The Hermes agent web interface owns configuration, run initiation, and human oversight.
  2. Your orchestration service owns session creation, credential handling, and cleanup.
  3. The hosted browser owns Chromium, the CDP endpoint, and the live view.

When these blur — for example, when the web interface holds the CDP URL directly — you get credential leakage, orphaned sessions, and no audit trail. Keep the boundary sharp.

A practical pattern: the web interface posts a task to your service, the service creates a session, runs the Hermes agent against it, streams status back to the UI, and stops the session in a finally block. The UI never sees a CDP URL. It sees a run ID and a viewer link.

For teams that want the agent to be controllable from outside the web interface as well — a CLI, a chat integration, a scheduled job — the same service boundary works. See Remote control browser for how that generalizes.

Common failure modes and fixes

The agent connects but acts on a blank page. The session opened a default page and your agent created a second one. Reuse context.pages()[0] when it exists.

Runs work locally but fail in the web interface. The local run used your machine's network and cookies. The hosted session starts clean. Pass credentials through a persistent profile or an authenticated setup step.

Sessions pile up and cost climbs. Cleanup is missing or not in a finally. Add a session TTL on the provider side as a backstop.

The agent times out on a page that loads fine in your browser. The hosted session may be routed through a different proxy or geo. Check the session's network configuration and whether the target site blocks the exit IP.

CDP disconnects mid-run. Long-running tasks over a single WebSocket are fragile. Reconnect to the same session ID rather than starting over, and keep individual agent steps short.

Getting started

The fastest path is to create one session, connect with the Playwright snippet above, and drive a single page load end to end before adding agent logic. Once that works, add the Hermes task loop, then the live viewer, then the cleanup guarantees.

Full endpoint details, session options, and profile management are in the documentation. Current session limits and metering are on the pricing page.

The Hermes agent web interface is a good place to define and watch an agent. It is not a browser, and treating it as one is where most reliability problems start. Give the agent a real, isolated, hosted Chromium session over CDP, keep the lifecycle in your own service, and the web interface becomes what it should be: the window into a system that actually holds up under load.