← Blog

BLOG

Computer Use MCP: Connect Agents to a Remote Browser

Learn how computer use MCP connects AI agents to browser tools, where CDP fits, and how to secure and operate remote Chromium sessions.

10 min readRemote Browser
Ivory computer towers connected across terracotta platforms in a quiet desert landscape
The infrastructure behind your browserREMOTE BROWSER / ILLUSTRATION

Computer use MCP is an approach to giving an AI agent tools for operating a graphical environment through the Model Context Protocol (MCP). An MCP server exposes actions such as taking a screenshot, clicking, typing, or navigating; an MCP client makes those tools available to the agent. The server then translates tool calls into actions in a browser or desktop environment.

MCP is the tool interface, not the computer itself. For web tasks, a typical stack uses an MCP server backed by Playwright and a hosted Chromium session. Remote Browser can provide that browser runtime and CDP access; you still need an MCP server that supports the connection and an agent or application that acts as an MCP client.

What computer use MCP does

A computer-use agent observes an interface, decides what to do, and calls a tool to act. The tool returns an observation, and the agent repeats the cycle until it finishes or needs help.

For a browser task, that loop might look like this:

  1. The agent calls a navigation tool with a URL.
  2. The tool returns the resulting page URL and, depending on the implementation, a screenshot or structured page information.
  3. The agent identifies a search field and calls a click or fill tool.
  4. The tool performs the action and returns an updated observation.
  5. The agent checks the result before reporting it or taking another action.

The exact tool names, arguments, and output formats depend on the MCP server. “Computer use MCP” does not identify one standard server or guarantee a particular set of tools. Inspect a server’s advertised tools and documentation before configuring an agent around it.

A computer-use server may control a full desktop, while a browser-focused server controls pages and browser sessions. That distinction matters: hosted Chromium is suitable for web applications, but it does not by itself provide control of arbitrary native desktop applications.

How MCP, Playwright, CDP, and the browser fit together

These components solve different problems:

ComponentResponsibilityDoes not provide by itself
MCP client and agentDiscover tools, select actions, and process tool resultsA running browser or desktop
MCP serverExpose tools and translate requests into environment operationsA model that decides what to do
PlaywrightControl browser pages through a programming APIAn MCP interface unless wrapped by a server
Chrome DevTools Protocol (CDP)Provide a browser debugging and control protocolAgent reasoning or a complete MCP server
Hosted Chromium runtimeRun browser sessions and expose a connection methodA universal desktop or guaranteed task success

A common web-only architecture is:

Agent / MCP client
        |
        | MCP tool calls
        v
Computer-use or browser MCP server
        |
        | Playwright connection over CDP
        v
Hosted Chromium session
        |
        v
Target website

The MCP server may run beside the agent or as a separate service. It must be able to reach the browser endpoint and use a control mechanism compatible with the runtime. Some MCP servers launch their own local browser and do not support attaching to an existing CDP endpoint; check that capability before choosing one.

For more on the browser layer, see remote browsers for AI agents and the remote browser documentation.

Choose the right computer-use tool surface

The most useful computer use MCP interface depends on what the agent needs to observe and manipulate. You do not have to choose between visual and structured interaction for every task; a server can offer both.

Screenshot and coordinate tools

A visual tool surface typically returns a screenshot and accepts actions such as clicking at (x, y), scrolling, pressing keys, or typing text. It can work with interfaces where structured element information is incomplete.

The trade-off is sensitivity to viewport size, scrolling, zoom, overlays, and rendering changes. A coordinate from an earlier screenshot may point to a different element after the page moves. Capture a fresh observation after navigation or a major layout change, and verify the result of consequential clicks.

Structured browser tools

A browser-specific server may expose page text, accessibility information, element references, or semantic actions such as filling a labeled field. These can reduce ambiguity when the page exposes useful structure.

Structured tools are not immune to failure: references can become stale, labels can be missing, and a page can change between observation and action. Treat returned element references as short-lived unless the server documents otherwise.

Full desktop tools

A desktop computer-use server can be appropriate when a workflow leaves the browser—for example, to interact with a native application. It also has a wider security and operational footprint: files, clipboard contents, other applications, and operating-system dialogs may become accessible.

Use a browser-only tool surface when the task is confined to the web. It narrows the environment you need to secure and is usually simpler to deploy.

Connect a browser runtime before adding MCP

Before debugging an agent’s decisions, confirm that the process hosting your MCP server can connect to the remote browser. The following TypeScript example checks a CDP connection with Playwright. It is not an MCP server; it validates the browser-control layer that a compatible MCP server could use.

Prerequisites

  • Node.js and a TypeScript runner such as tsx.
  • The playwright package.
  • A running Chromium session with a CDP WebSocket endpoint reachable from your process.
  • Permission to access the target site. The example uses example.com.

Install the dependencies:

npm install playwright
npm install --save-dev typescript tsx @types/node

Set CDP_WS_URL to the endpoint supplied for your session. Treat it as a secret: a CDP endpoint can grant substantial control over the browser. Do not commit it or include it in logs.

export CDP_WS_URL='wss://YOUR_SESSION_ENDPOINT'
npx tsx check-browser.ts

Save this as check-browser.ts:

import { chromium, type Browser } from "playwright";

async function main(): Promise<void> {
  const endpoint = process.env.CDP_WS_URL;
  if (!endpoint) {
    throw new Error("Set CDP_WS_URL to your browser session's CDP endpoint.");
  }

  let browser: Browser | undefined;
  let page: Awaited<ReturnType<
    Awaited<ReturnType<typeof chromium.connectOverCDP>>["newPage"]
  >> | undefined;

  try {
    browser = await chromium.connectOverCDP(endpoint);

    // Use the existing context supplied by the remote session.
    const context = browser.contexts()[0];
    if (!context) {
      throw new Error("No browser context is available in this session.");
    }

    page = await context.newPage();
    await page.goto("https://example.com", {
      waitUntil: "domcontentloaded",
      timeout: 30_000,
    });

    console.log("Page title:", await page.title());
    console.log("Page URL:", page.url());
    await page.screenshot({ path: "example.png" });
    console.log("Screenshot saved to example.png");
  } finally {
    // Close only the page created by this check.
    await page?.close().catch(() => {});
    // Disconnect from the browser; session lifecycle is provider-dependent.
    await browser?.close().catch(() => {});
  }
}

main().catch((error: unknown) => {
  // Avoid printing connection details or credentials in shared logs.
  console.error("Browser connection check failed.");
  process.exitCode = 1;
});

A successful run prints the title and URL and saves a screenshot. The process needs write access to its working directory for the screenshot. Depending on the provider and connection mode, closing the Playwright browser connection may also affect the remote session; use a disposable session for this test and check the runtime’s lifecycle documentation.

Playwright documents the connection method and its Chromium-specific limitations in the BrowserType `connectOverCDP` reference. A CDP connection is not the same as Playwright’s native protocol connection, and compatibility can vary across browser and Playwright versions.

Wire the MCP server to the session

Once the direct connection works, add the MCP layer. Configuration syntax varies by server and client, so use their documentation rather than assuming that every computer use MCP implementation accepts the same flags.

  1. Choose a server with the right control surface. Confirm whether it exposes visual actions, structured browser actions, or both. For hosted Chromium, verify that it supports attaching to an existing CDP endpoint.
  2. Start a browser session. Obtain its connection endpoint through your runtime’s documented process. Keep the endpoint in a secret store or environment variable accessible only to the server.
  3. Configure the MCP server. Supply the endpoint using that server’s supported configuration. Do not assume that a generic CDP_WS_URL variable is recognized just because the preceding test uses it.
  4. Register the server with your MCP client. Configure the client using the transport and launch or connection settings supported by both sides.
  5. Inspect the available tools. Check their input schemas, returned observations, timeout behavior, and whether they operate on the intended session.
  6. Run a low-risk task. Navigate to a permitted test page, read its title or take a screenshot, then verify the result outside the agent.
  7. Test cleanup and failure paths. Disconnect the client, trigger a timeout, and confirm who is responsible for stopping the browser session.

If the connection check succeeds but the MCP tool fails, inspect the server’s configuration and network access. A server running in another container or host may not be able to reach the same endpoint as your local shell.

Design tool calls for reliable outcomes

The quality of a computer use MCP integration depends on the contract between the agent and its tools. A robust implementation should make it easy for the agent to tell what happened without exposing unnecessary browser state.

For each tool, define:

  • Inputs: Accepted actions, coordinate units, URL rules, text limits, and any session identifier.
  • Outputs: Current URL, relevant observation, action status, and a clear error when the operation fails.
  • Timing: Navigation timeout, maximum tool duration, and whether the tool waits for a specific page condition.
  • Scope: Which browser session, context, page, or tab the call affects.
  • Side effects: Whether the action can submit data, download a file, or change account state.

For example, a click tool should distinguish “the click was dispatched” from “the intended transaction completed.” The latter requires a separate verification step, such as checking for a confirmation page or an updated record.

Avoid blindly retrying actions with side effects. A timeout after clicking Submit does not prove that submission failed. Check the resulting state before retrying, and use application-level idempotency mechanisms where available.

Secure the browser and the agent separately

A hosted browser reduces the need to operate Chromium on the agent host, but it does not remove the risks of granting an agent access to websites and credentials.

Restrict navigation and network access. Validate permitted destinations and enforce network-level egress rules. URL validation alone is insufficient: redirects, DNS changes, and subresources can reach unexpected addresses. Block access to internal services and cloud metadata endpoints unless explicitly required.

Treat page content as untrusted. A web page can contain instructions that try to redirect the agent or extract secrets. Keep task instructions and authorization decisions outside page content. Do not let a page grant itself new tools or permissions.

Separate sessions and identities. Use isolated sessions for unrelated users or jobs. Persistent profiles can preserve useful login state, but they also contain sensitive authentication material. Limit access, define retention, and revoke credentials when a profile is no longer needed.

Gate consequential actions. Require human approval or another policy check before payments, account changes, message sending, or destructive operations. Show the destination and proposed action to the approver, then verify that the page has not changed before proceeding.

Protect observations and logs. Screenshots, page text, downloads, and tool traces can contain personal data or secrets. Redact where practical, restrict access, and set retention limits. Never log CDP endpoints or authentication tokens.

Operate computer use MCP in production

A working demo is not enough to establish that a workflow is reliable. Measure outcomes at the task and tool levels, and keep enough evidence to diagnose failures without retaining more data than necessary.

Track at least:

  • Task outcomes: completed, failed, timed out, or escalated for human review.
  • Tool behavior: action errors, navigation timeouts, stale references, and retries.
  • Session lifecycle: creation, connection, duration, disconnection, and cleanup.
  • Verification: whether the expected final state was observed, not merely whether a click succeeded.
  • Resource use: browser session time and concurrency against your configured limits.

A live viewer can help an operator inspect a stuck session when access is authorized. Persistent profiles may reduce repeated logins, but they introduce credential management and isolation requirements. Confirm which settings and controls are available for your deployment in the documentation, and check pricing for current usage details and limits.

When to use Remote Browser

Remote Browser provides hosted Chromium sessions and CDP access for web-focused computer use MCP architectures. It also supports Playwright, Puppeteer, and Selenium workflows, with capabilities including a live viewer, persistent profiles, session isolation, configurable browser settings, and usage controls.

Use it as the browser runtime behind a compatible MCP server—not as a substitute for the MCP server, agent, approval policy, or a full desktop environment. Start by validating a disposable session over CDP, then connect your chosen server and test its tool behavior against a low-risk workflow. For broader runtime considerations, see remote browser control.