BLOG
Hermes Agent Web Interface: Connect It to a Remote Browser
Wire the Hermes agent web interface to hosted Chromium over CDP. Session setup, Playwright code, and production checks for reliable browser tasks.

The Hermes agent web interface is the browser-based control surface where you configure an agent, start runs, and watch what it does. If your Hermes agent needs to touch real websites — log in, fill forms, scrape a dashboard, click through a checkout — the web interface alone is not enough. It needs a browser it can drive, and that browser should not be a Chrome window on your laptop.
This guide covers the integration: how the Hermes agent web interface hands work to a browser, why a hosted Chromium session is the right target for anything beyond a demo, and the concrete CDP wiring that makes it work. The short version: point Hermes at a remote browser endpoint, connect over the Chrome DevTools Protocol, and keep the session lifecycle under your control.
What the Hermes agent web interface actually is
Hermes is an agent framework with a web UI layer. The Hermes agent web interface is where a human sets up the agent — model choice, tools, task prompt, credentials — and where run output surfaces. It is not a browser. It is a control plane.
That distinction matters because most "my agent can't browse" problems are not interface problems. They are runtime problems. The web interface can hold your task definition and display a live view of a session, but the session itself has to run somewhere with a real Chromium binary, a network path to the target site, and a way for the agent to issue commands.
Three layers are involved:
- Control plane — the Hermes agent web interface. Configuration, run history, human oversight.
- Agent runtime — the process that reasons over page state and decides the next action.
- Browser runtime — the Chromium instance the agent drives, plus the CDP endpoint that exposes it.
When people search for a "hermes agent web interface," they usually want one of two things: a way to see the agent working in a browser, or a way to make the agent's browser work reliably. This article addresses the second, because the first is only useful if the second holds up.
Why the browser runtime decides whether Hermes works
An agent that drives a local Chrome install inherits every problem that install has. Version drift between the agent's Playwright or Puppeteer client and the local Chromium build. A single browser profile shared across concurrent runs, so two tasks fight over cookies. No isolation between a test run and a production run. No way to inspect a session after it fails, because the window closed.
A hosted browser runtime moves those concerns off the machine running the Hermes agent web interface. You get:
- A fresh, isolated session per run — no cookie bleed between tasks.
- A stable CDP endpoint — the agent connects to a WebSocket URL instead of spawning a process.
- A live viewer — a human can watch the session in the web interface while it runs.
- Persistent profiles when you need them — saved logins survive across sessions without sharing state across unrelated tasks.
- Configurable browser settings — proxy routing, user-agent, and other session-level options set at creation time.
For a deeper treatment of the runtime layer, see Remote browsers for AI agents.
Connecting the Hermes agent web interface to hosted Chromium
The integration path is CDP. Hermes' browser tooling speaks the same protocol Playwright and Puppeteer use, so the wiring is a connection string plus a session lifecycle.
Step 1: Create a session and capture the endpoint
When your Hermes run starts, request a browser session from the Remote Browser API. The response includes a CDP WebSocket endpoint. Treat that endpoint as a secret — anyone holding it can drive the session.
Keep session creation inside your backend, not in the browser tab hosting the Hermes agent web interface. The web UI should call your service; your service calls the browser API; your service returns only what the UI needs to display.
Step 2: Connect over CDP
Here is a minimal Playwright connection that mirrors what a Hermes browser tool does under the hood:
import { chromium, type Browser, type Page } from "playwright";
interface SessionHandle {
cdpUrl: string;
sessionId: string;
}
async function connectHermesBrowser(session: SessionHandle): Promise<{
browser: Browser;
page: Page;
}> {
const browser = await chromium.connectOverCDP(session.cdpUrl, {
timeout: 30_000,
});
// A hosted session usually starts with one context and one page.
const contexts = browser.contexts();
const context = contexts.length > 0
? contexts[0]
: await browser.newContext();
const pages = context.pages();
const page = pages.length > 0 ? pages[0] : await context.newPage();
page.setDefaultTimeout(20_000);
return { browser, page };
}
async function runHermesTask(session: SessionHandle, task: (p: Page) => Promise<void>) {
const { browser, page } = await connectHermesBrowser(session);
try {
await task(page);
} finally {
// Disconnect the client; the hosted session is stopped separately.
await browser.close();
}
}Two details are easy to get wrong. First, connectOverCDP attaches to an existing browser — it does not launch one, so browser.close() disconnects your client rather than killing the remote Chromium. Second, the session may already have a page open; reuse it instead of creating a second one, or your agent will act on a blank tab while the real page sits idle.
The Playwright documentation on connecting to an existing browser over CDP is the authoritative reference for the connection semantics.
Step 3: Stop the session explicitly
When the task finishes — success or failure — call the session-stop endpoint. Do not rely on the client disconnect to clean up. A session left running continues to consume browser-hours, and if your pricing is metered, that is real cost. Check pricing for how sessions are billed.
Step 4: Surface the live view in the web interface
If the Hermes agent web interface supports embedding a live session view, pass the viewer URL for the session alongside the run status. This is the feature that makes the web interface genuinely useful: a human can watch the agent navigate, spot a stuck loop, and intervene before the run burns its budget.
Runtime comparison: local Chrome vs hosted Chromium for Hermes
| Criterion | Local Chrome on the Hermes host | Hosted Chromium session |
|---|---|---|
| Session isolation | Shared profile unless you build isolation | One isolated session per request |
| Concurrency | Bounded by host CPU and memory | Scales with your session quota |
| CDP endpoint | Local port, not reachable remotely | WebSocket URL, reachable from any worker |
| Live viewing | Screen share or VNC workaround | Built-in viewer URL |
| Persistent logins | Manual profile management | Profiles you attach per session |
| Proxy configuration | OS-level or launch flags | Set per session at creation |
| Failure debugging | Window closes, state lost | Session artifacts and viewer retained |
| Cost model | Host cost, effectively fixed | Metered per browser-hour |
The local option is fine for a single developer iterating on a task. It breaks the moment you run two Hermes agents in parallel, move the agent to a container without a display server, or need to debug a failure after the fact.
Production criteria for the Hermes browser integration
Before you ship a Hermes agent that browses, check these:
- Session lifecycle is owned by your backend. The web interface triggers runs; it does not hold CDP credentials. Rotate or scope any token the UI can see.
- Every run has a hard timeout. An agent that loops on a selector will run until something stops it. Set a task-level deadline and a session-level cap.
- Reconnects are handled. CDP connections drop. Wrap the connection in retry logic that re-attaches to the same session rather than creating a new one, or you will orphan sessions.
- Page state is asserted, not assumed. After every navigation or click, wait for a concrete condition — a selector, a network idle state, a URL change. Agents that act on stale DOM produce flaky runs.
- Artifacts are captured on failure. Screenshot and page HTML at the point of failure turn a mystery into a fix.
- Concurrency is planned. Know how many simultaneous sessions your plan allows and queue Hermes runs accordingly. Do not assume the plan permits unbounded parallelism.
If you are still deciding between running Chromium yourself and using a hosted runtime, Remote browser online covers the trade-offs in more depth.
Where the web interface and the browser runtime meet
The clean architecture separates three responsibilities:
- The Hermes agent web interface owns configuration, run initiation, and human oversight.
- Your orchestration service owns session creation, credential handling, and cleanup.
- The hosted browser owns Chromium, the CDP endpoint, and the live view.
When these blur — for example, when the web interface holds the CDP URL directly — you get credential leakage, orphaned sessions, and no audit trail. Keep the boundary sharp.
A practical pattern: the web interface posts a task to your service, the service creates a session, runs the Hermes agent against it, streams status back to the UI, and stops the session in a finally block. The UI never sees a CDP URL. It sees a run ID and a viewer link.
For teams that want the agent to be controllable from outside the web interface as well — a CLI, a chat integration, a scheduled job — the same service boundary works. See Remote control browser for how that generalizes.
Common failure modes and fixes
The agent connects but acts on a blank page. The session opened a default page and your agent created a second one. Reuse context.pages()[0] when it exists.
Runs work locally but fail in the web interface. The local run used your machine's network and cookies. The hosted session starts clean. Pass credentials through a persistent profile or an authenticated setup step.
Sessions pile up and cost climbs. Cleanup is missing or not in a finally. Add a session TTL on the provider side as a backstop.
The agent times out on a page that loads fine in your browser. The hosted session may be routed through a different proxy or geo. Check the session's network configuration and whether the target site blocks the exit IP.
CDP disconnects mid-run. Long-running tasks over a single WebSocket are fragile. Reconnect to the same session ID rather than starting over, and keep individual agent steps short.
Getting started
The fastest path is to create one session, connect with the Playwright snippet above, and drive a single page load end to end before adding agent logic. Once that works, add the Hermes task loop, then the live viewer, then the cleanup guarantees.
Full endpoint details, session options, and profile management are in the documentation. Current session limits and metering are on the pricing page.
The Hermes agent web interface is a good place to define and watch an agent. It is not a browser, and treating it as one is where most reliability problems start. Give the agent a real, isolated, hosted Chromium session over CDP, keep the lifecycle in your own service, and the web interface becomes what it should be: the window into a system that actually holds up under load.