Web Automation API: How to Connect Code to Hosted Browsers

# Web Automation API: How to Connect Code to Hosted Browsers

A web automation API is the interface between your code and a browser that runs somewhere else. Instead of launching a local Chromium instance, your script sends commands over HTTP or WebSocket to a hosted browser session. This is the pattern behind modern AI agents, scraping pipelines, and test harnesses that need reliable, persistent browser access without managing infrastructure.

Remote Browser provides exactly this: a hosted Chromium runtime with a web automation API that supports Playwright, Puppeteer, Selenium, and raw Chrome DevTools Protocol (CDP) connections. This guide explains how the API works, what to look for when evaluating one, and how to connect your existing automation stack.

Why a Web Automation API Matters

Local browser automation breaks down in production. Your machine goes to sleep, the network changes, the browser crashes, and you have no way to inspect what happened. A hosted browser API solves these problems by moving the browser to infrastructure you control remotely.

The core value is not "browser in the cloud." It's the API contract that lets you treat a browser like a service. You request a session, connect to it, run commands, and tear it down—all through a well-defined interface.

For AI agents, this is especially important. An agent that needs to log into a portal, navigate a multi-step workflow, or extract data from a JavaScript-heavy site cannot afford to lose its session state mid-task. A web automation API with persistent profiles and live debugging gives you that reliability.

What to Look For in a Web Automation API

Not all browser APIs are equal. Here are the production criteria that matter:

The table above is a checklist, not a marketing pitch. If you're evaluating a web automation API, ask these questions before you commit.

How Remote Browser's Web Automation API Works

Remote Browser exposes a REST API for session management and a WebSocket endpoint for CDP traffic. The flow is straightforward:

1. Create a session via POST /v1/sessions with your API key.
2. Connect to the session using the returned WebSocket URL.
3. Run commands through Playwright, Puppeteer, or raw CDP.
4. Inspect the session live via the viewer.
5. Terminate the session when done.

Here's a TypeScript example using Playwright's connectOverCDP method:

import { chromium } from 'playwright';

async function connectToRemoteBrowser() {
  // 1. Create a session via the Remote Browser API
  const response = await fetch('https://api.remote-browser.dev/v1/sessions', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      profileId: 'persistent-profile-1',
      timeoutMinutes: 30
    })
  });

  const session = await response.json();
  console.log(`Session created: ${session.id}`);

  // 2. Connect to the session using CDP
  const browser = await chromium.connectOverCDP(session.cdpUrl);
  const context = browser.contexts()[0];
  const page = context.pages()[0] || await context.newPage();

  // 3. Run your automation
  await page.goto('https://example.com');
  await page.fill('#search', 'web automation api');
  await page.click('button[type="submit"]');
  await page.waitForLoadState('networkidle');

  const results = await page.title();
  console.log(`Page title: ${results}`);

  // 4. Keep the session alive for debugging or reuse
  // The session will persist until you terminate it or the timeout hits

  // 5. Terminate when done
  await fetch(`https://api.remote-browser.dev/v1/sessions/${session.id}`, {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer YOUR_API_KEY'
    }
  });

  await browser.close();
}

connectToRemoteBrowser().catch(console.error);

The key detail is connectOverCDP. This method lets Playwright attach to an existing browser instance rather than launching a new one. That's what makes the hosted browser feel like a local one—your code doesn't care where the browser runs.

Keeping Browser Sessions Alive Across Workers

One of the most common questions we hear is: how do you keep browser sessions alive across multiple cloud workers?

The answer is session persistence. When you create a session with a profile ID, Remote Browser maintains that browser context on the server. Your workers can connect and disconnect from the session without losing state.

Here's the pattern:

1. Worker A creates a session with profileId: 'user-123' and logs into a portal.
2. Worker A disconnects. The session stays alive on the server.
3. Worker B connects to the same session using the same profile ID.
4. Worker B sees the logged-in state and continues the task.

This is critical for AI agents that need to hand off tasks between workers or resume after a timeout. Without session persistence, you'd have to re-authenticate on every task, which breaks most real-world workflows.

The trade-off is cost. Persistent sessions consume resources even when idle. Remote Browser addresses this with configurable timeouts and usage controls—you set the maximum session duration, and the API enforces it. See our pricing page for current session costs.

Playwright Remote Browser: Connecting to a Hosted Instance

If you're already using Playwright, connecting to a remote browser is a small change to your existing code. Instead of chromium.launch(), you use chromium.connectOverCDP() with the WebSocket URL from your session.

The main differences from local Playwright:

• No browser download: You don't need to install Chromium locally. The hosted browser handles that.
• No launch options: You can't pass --headless or --proxy-server flags. Instead, you configure these via the Remote Browser API when creating the session.
• Network latency: Commands travel over the network. This adds a few milliseconds per command, which is negligible for most automation but matters for high-frequency operations.

For a deeper dive, see our guide on Playwright remote browser connections.

CDP: The Universal Web Automation API

Chrome DevTools Protocol is the foundation of all modern browser automation. Playwright and Puppeteer both use CDP under the hood. When you connect to a Remote Browser session, you're speaking CDP directly.

This matters because CDP gives you access to everything: DOM manipulation, network interception, performance metrics, and browser-level events. If you need to do something that Playwright doesn't expose, you can drop down to raw CDP commands.

Here's a minimal CDP example using WebSocket:

const WebSocket = require('ws');

const ws = new WebSocket('wss://remote-browser.dev/cdp/session-id');

ws.on('open', () => {
  // Enable the Page domain
  ws.send(JSON.stringify({
    id: 1,
    method: 'Page.enable'
  }));

  // Navigate to a URL
  ws.send(JSON.stringify({
    id: 2,
    method: 'Page.navigate',
    params: { url: 'https://example.com' }
  }));
});

ws.on('message', (data) => {
  const msg = JSON.parse(data);
  if (msg.method === 'Page.loadEventFired') {
    console.log('Page loaded');
    // Take a screenshot or extract data
  }
});

The Chrome DevTools Protocol documentation is the authoritative reference for what you can do over this interface. See the official CDP docs for the full method list.

Browser in the Cloud vs. Local Automation

The debate between hosted and local browsers comes down to trade-offs. Here's an honest comparison:

Local browsers:

• Zero latency for commands
• No API costs
• Full control over browser flags
• But: you manage the infrastructure, sessions die with your machine, and scaling means provisioning more machines

Hosted browsers (via a web automation API):

• Persistent sessions that survive worker restarts
• Centralized debugging and monitoring
• Scale without provisioning
• But: network latency, API costs, and you depend on the provider's uptime

For development and small-scale testing, local is fine. For production AI agents, scraping pipelines, or anything that needs to run 24/7, a hosted browser API is the practical choice. This is the argument we make in our remote browsers for AI agents post.

What Is an Agent Browser?

An agent browser is a browser runtime designed for AI agents. It's not just a browser—it's a browser with an API, session management, and observability built in.

The distinction matters because AI agents have different requirements than traditional test scripts:

• Long-running sessions: Agents may work on a task for hours. The browser must stay alive.
• State persistence: Agents need to maintain login state, cookies, and local storage across steps.
• Human oversight: Agents make mistakes. You need to see what they're doing and intervene.
• Safety controls: Agents can go off the rails. You need usage limits and session timeouts.

Remote Browser was built with these requirements in mind. The web automation API is the interface, but the runtime is what makes it work for agents. For more detail, see our agent browser runtime guide.

Browser Benchmark: Measuring Web Automation API Performance

When evaluating a web automation API, you need benchmarks. But most benchmarks are misleading because they measure the wrong things.

Here's what to measure:

1. Session creation time: How long from API call to ready-to-use browser? Sub-second is good.
2. Command latency: How long does a CDP command take round-trip? This includes network overhead.
3. Stability: How often do sessions crash or disconnect? This is more important than raw speed.
4. Concurrency: How many sessions can you run in parallel? This determines your throughput.

We publish our own benchmark methodology in our browser benchmark guide. The short version: don't trust synthetic benchmarks. Run your actual workload against the API and measure end-to-end task completion time.

Browser Remote Control: The Human-in-the-Loop Pattern

A web automation API isn't just for fully autonomous agents. It's also for human-in-the-loop workflows where a person needs to monitor or intervene.

Remote Browser's live viewer gives you real-time visibility into what the browser is doing. You can watch the agent navigate, see console errors, and even take over control if needed.

This is the remote control browser pattern. It's useful for:

• Customer support: An agent handles routine tasks, but a human steps in for edge cases.
• QA debugging: A test fails, and you need to see exactly what happened.
• Compliance: You need to audit what an agent did, step by step.

The API supports this by keeping sessions alive and providing a live view. You don't have to choose between full automation and full manual control—you get both.

Production Considerations for a Web Automation API

Before you put a web automation API into production, consider these factors:

Authentication and secrets. Your API key is the keys to the kingdom. Store it in a secret manager, not in your codebase. Rotate it regularly.

Session lifecycle. Define clear rules for when sessions are created and destroyed. Idle sessions cost money. Use timeouts aggressively.

Error handling. Network connections drop. The browser crashes. Your code should retry with exponential backoff and have a fallback strategy.

Observability. Log every API call, session ID, and error. You'll need this for debugging and cost analysis.

Compliance. If you're handling user data, make sure your browser sessions are isolated. Remote Browser provides session isolation by default, but you should verify this meets your compliance requirements.

For a full checklist, see our production readiness guide.

Getting Started with Remote Browser's Web Automation API

The fastest way to evaluate Remote Browser is to create a session and connect to it with Playwright. The API is designed to be familiar if you've used any browser automation tool before.

Here's the minimal flow:

1. Sign up for an API key.
2. Create a session via POST /v1/sessions.
3. Connect with chromium.connectOverCDP(session.cdpUrl).
4. Run your automation code.
5. Terminate the session.

The documentation has complete API references, code samples, and troubleshooting guides. If you're migrating from a local setup, start with our remote browser online guide, which covers the basics of connecting to hosted Chromium.

The Bottom Line

A web automation API is the missing layer between your code and a reliable browser runtime. It gives you persistent sessions, live debugging, and infrastructure that scales—without the operational burden of managing browsers yourself.

Remote Browser provides this API with support for Playwright, Puppeteer, Selenium, and raw CDP. Whether you're building an AI agent, a scraping pipeline, or a test harness, the connection pattern is the same: create a session, connect over CDP, and run your automation.

The trade-offs are real—network latency and API costs—but for production workloads, the benefits of persistence and observability far outweigh the costs. Start with a simple session, measure your actual task completion time, and decide if the hosted model fits your needs.