BLOG
Use the Web with a Remote Browser Runtime
Use the web from AI agents and backend workflows with hosted Chromium, CDP, Playwright, isolated sessions, and production controls.

To use the web from an AI agent or backend service, you need more than an HTTP client. Modern websites often require JavaScript execution, cookies, login state, navigation, clicking, file downloads, and interaction with pages that change after load.
The practical way to use the web programmatically is to run a real browser in an isolated session and control it through a browser automation protocol. Remote Browser provides hosted Chromium sessions that your application can access over the Chrome DevTools Protocol (CDP), with compatibility for Playwright, Puppeteer, Selenium, and browser-use workflows.
This approach moves browser operations out of your application server while preserving the capabilities that agents need:
- Open pages and follow links
- Inspect rendered content and accessibility information
- Fill forms and submit actions
- Reuse authenticated profiles
- Download or upload files
- Observe a live session while it runs
- Keep sessions isolated from one another
- Apply configurable browser, proxy, and usage settings
What it means to use the web programmatically
“Use the web” can describe several different technical tasks. Choosing the right execution method depends on what the workflow must do.
A direct HTTP request is often sufficient when you only need a public document, an API response, or static HTML. It is usually faster and easier to test than a browser. However, it cannot reproduce many user actions reliably, especially when a site renders data in the browser, requires a login, or changes its content after client-side JavaScript executes.
A browser runtime is appropriate when the workflow must behave like a user or inspect what a user would see. An agent may need to:
- Navigate to a page.
- Wait for client-side rendering to complete.
- Identify a button, form field, link, or table.
- Perform an interaction.
- Read the resulting page state.
- Decide what to do next.
That loop requires a browser process, not just a web request. A remote browser supplies that process without requiring your worker, notebook, or development machine to install and maintain Chromium.
For AI agents, this distinction matters. The model can plan actions, but the browser runtime executes them against a real page. The runtime also provides the session state, page events, screenshots, DOM access, and error signals required to make the next decision.
How a remote browser lets agents use the web
A typical architecture has four layers:
- Agent or application: Chooses the next action based on the task and current page state.
- Automation client: Playwright, Puppeteer, Selenium, or another compatible library translates actions into browser commands.
- Remote Browser session: Runs hosted Chromium and maintains pages, contexts, cookies, storage, and downloads.
- Target website: Receives ordinary browser traffic and returns rendered pages, responses, and events.
Your service creates or obtains a browser session, receives a CDP endpoint, and connects to that endpoint. The browser then runs independently of the process making the connection. Your worker does not need to launch a local Chrome binary or package browser dependencies into every deployment.
CDP is useful because it is a standard control path for Chromium-based browsers. Playwright can connect to an existing browser over CDP, while other libraries can use their own remote connection mechanisms. The control plane and browser execution remain separate from your agent logic.
This separation also makes it easier to run different workloads:
- A short-lived research task can use a fresh isolated session.
- A support workflow can use a persistent profile with saved login state.
- A test worker can create a clean session for each test.
- A long-running agent can reconnect to a session when its application process restarts.
- An operator can inspect a live viewer when an agent reaches an unexpected page.
Persistent profiles should be treated as sensitive state. They may contain cookies, local storage, and other authentication artifacts. Use separate profiles for separate customers or security boundaries, and avoid sharing one profile between unrelated concurrent tasks.
Use the web from TypeScript with Playwright and CDP
The connection flow is intentionally small. Your application receives a CDP endpoint from the Remote Browser session and passes it to Playwright.
The following example opens an existing remote Chromium session, navigates to a public page, extracts rendered text, and closes the connection:
import { chromium } from "playwright";
const endpoint = process.env.REMOTE_BROWSER_CDP_URL;
if (!endpoint) {
throw new Error("REMOTE_BROWSER_CDP_URL is required");
}
const browser = await chromium.connectOverCDP(endpoint);
const context = browser.contexts()[0];
if (!context) {
throw new Error("The remote browser did not expose a browser context");
}
const page = context.pages()[0] ?? await context.newPage();
try {
await page.goto("https://example.com", {
waitUntil: "domcontentloaded",
timeout: 30_000,
});
const title = await page.title();
const text = await page.locator("body").innerText();
console.log({ title, text: text.slice(0, 2_000) });
} finally {
await page.close();
await browser.close();
}The session-creation step is managed through the Remote Browser control plane. The exact endpoint and session options depend on the integration you use, so keep those details in configuration rather than embedding them in application code.
Playwright’s connectOverCDP documentation explains the client-side connection method and its limitations. CDP connections are most useful when you need to attach to an already-running Chromium instance. If you are starting a local browser directly, normal Playwright launch methods may be simpler.
For a production agent, replace the example’s fixed URL with an explicit task state machine. Record the action, target page, result, and failure reason at every important step. Do not rely on a single long timeout to determine whether the task succeeded.
A reliable workflow for using the web
An agent that can use the web still needs an execution policy. A practical workflow separates planning from browser control.
1. Define the task boundary
State what the agent is allowed to read or change. “Find three public product pages” is materially different from “change account billing information.” High-impact actions should require confirmation or a separate approval step.
2. Create the appropriate session
Choose a fresh session for public research or a persistent profile for an authenticated workflow. Configure the browser for the required environment, such as a region-specific proxy or other available browser settings. Avoid carrying an authenticated profile into tasks that do not need it.
3. Observe before acting
After navigation, inspect the page title, URL, visible text, roles, labels, and relevant page structure. An agent should not assume that a selector from a previous run still exists.
4. Act with stable targets
Prefer accessible roles, labels, and meaningful text over fragile CSS paths. Use explicit waits for a known state, such as a button becoming enabled or a result list appearing. Do not use arbitrary sleep calls as the primary synchronization method.
5. Verify the result
After a click or form submission, verify the URL, visible confirmation, response state, or resulting data. A successful click does not prove that the intended operation completed.
6. Capture the evidence
Store task-level logs, selected screenshots, page URLs, and structured outcomes. Redact credentials, tokens, personal data, and sensitive page content before sending diagnostic information to a model or logging system.
7. End or suspend the session deliberately
Close temporary sessions when the task completes. Keep persistent sessions only when their saved state has a clear owner and retention policy. Session cleanup is part of correctness, not just infrastructure housekeeping.
Choosing an approach
The best way to use the web depends on the page and the level of state your workflow needs.
| Approach | Best for | Strengths | Trade-offs |
|---|---|---|---|
| Direct HTTP client | APIs, static pages, public feeds | Fast, inexpensive to operate, easy to test | No JavaScript execution, browser storage, or user interaction |
| Local Playwright or Puppeteer | Development, small controlled jobs | Full browser control and simple debugging | Browser installation, scaling, isolation, and worker maintenance remain yours |
| Remote Browser over CDP | Agents, distributed workers, authenticated workflows | Hosted Chromium, remote execution, persistent profiles, session isolation, and live visibility | Requires session lifecycle management and network-aware error handling |
| Hosted task-specific API | Narrow, predefined operations | Small integration surface for one use case | Less flexible when page structure, state, or actions vary |
A remote browser is not automatically the right answer for every request. If a public API returns the required data, use the API. If you only need a deterministic local end-to-end test, local Playwright may be sufficient. Use a hosted browser when the workflow needs real browser behavior and you want that runtime separated from your application infrastructure.
Production criteria that matter
Session isolation
Concurrent tasks should not accidentally share cookies, local storage, downloads, or open pages. Use one session or profile boundary per logical security context. Treat session identifiers and CDP endpoints as secrets because possession of an active endpoint may provide control over the browser.
Authentication and profiles
Persistent profiles reduce repeated login work, but they increase the impact of a compromise. Store profile ownership in your application database, restrict which worker can access it, and define how long it may remain active. For one-off tasks, a clean session is usually easier to reason about.
Do not ask an agent to expose passwords or copy authentication tokens into its output. Prefer an explicit login handoff, a controlled credential mechanism, or a pre-authenticated profile with limited permissions.
Timeouts and retries
Use different timeouts for navigation, element readiness, downloads, and the overall task. Retry only operations that are safe to repeat. A retry after a payment submission or account change can create a duplicate action if the first request actually succeeded.
Record whether a failure happened before navigation, during page rendering, during an interaction, or while verifying the result. That classification is more useful than a generic “browser failed” message.
Observability
A live viewer is useful during development and for operational debugging, particularly when an agent reaches a consent page, unexpected redirect, or changed layout. For automated runs, combine live inspection with structured telemetry:
- Session creation and termination timestamps
- Current URL and page title
- Action type and target description
- Navigation and interaction duration
- Final task status
- Error category and retry count
Avoid recording every page indiscriminately. Capture enough context to diagnose failures while respecting user and website data.
Usage controls
Browser workloads consume more resources than ordinary HTTP requests. Set limits in your application for concurrent sessions, maximum task duration, navigation count, downloads, and model retries. Review current Remote Browser rates, included usage, and service limits on the pricing page before choosing a capacity model.
Security and operational boundaries
When your service can use the web, it can also reach untrusted pages. Treat page content as data, not instructions. A page may contain text intended to manipulate an agent into revealing secrets, ignoring task constraints, or taking an unrelated action.
Useful safeguards include:
- Keep CDP endpoints and API credentials on the server side.
- Separate browser permissions from model permissions.
- Require confirmation for irreversible actions.
- Block or review sensitive destinations where appropriate.
- Validate downloads before opening or processing them.
- Limit access to internal services from browser workflows.
- Redact sensitive content from logs and model context.
- Destroy temporary sessions after completion.
- Use separate profiles for separate tenants or accounts.
For a broader discussion of hosted execution and agent workloads, see the guide to remote browsers for AI agents. The Remote Browser documentation covers the available connection and session concepts.
When to use a remote browser
Use a remote browser when your application must use the web as a user-facing browser would: rendering JavaScript applications, maintaining login state, navigating multi-step flows, interacting with controls, or executing tasks across distributed workers.
Start with direct HTTP when the target provides a stable API. Move to browser automation when the API is unavailable or cannot represent the required user journey. Move from local automation to Remote Browser when browser installation, worker isolation, persistent profiles, remote debugging, or session scaling becomes an operational problem.
Remote execution does not remove the need for good selectors, task boundaries, retries, or security controls. It provides the browser runtime so your team can focus on those application-level decisions. If you need to use the web reliably from agents or backend services, a hosted Chromium session connected over CDP is a practical foundation.