Browser Use Start: A Practical Setup Guide for AI Agents

# Browser Use Start: A Practical Setup Guide for AI Agents

To browser use start correctly, you need two things: an agent that decides what to do, and a browser session it can actually drive. Most tutorials cover the first and hand-wave the second. That gap is where projects stall. This guide covers what browser use does, how to connect an agent to a real Chromium session over CDP, and what to check before you move from a laptop demo to something that runs unattended.

If you already know you want a hosted runtime, Remote Browser provides Chromium sessions with CDP access, persistent profiles, and a live viewer. If you are still deciding, read on — the decision criteria matter more than the vendor.

What Browser Use Actually Does

The term gets used two ways, and conflating them causes confusion.

Browser use as a library pattern. An LLM receives a task ("find the invoice for March and download the PDF"), inspects the current page state, decides on an action (click, type, scroll, extract), executes it, and repeats until the task is done or it gives up. The loop is the product. The browser is the substrate.

Browser use as a runtime concern. The session has to live somewhere. It needs a real rendering engine, a network path that isn't blocked, a profile that survives between runs, and a way for you to watch what happened when it fails.

Most "browser use" content focuses on the loop. The loop is the easy part — it's a while statement around a model call. The runtime is where production teams spend their time, because that's where the failure modes live: sessions that die mid-task, logins that expire, pages that render differently on a headless build, IPs that get flagged.

A useful mental model: the agent is a driver, the browser is the car, and the runtime is the road. You can write a brilliant driver and still not finish the trip if the road is closed.

The Three Ways to Give an Agent a Browser

There are three practical architectures, and they trade off differently.

Local Chrome. You launch a browser on your dev machine and point the agent at it. This is the right choice for the first hour of a project. It stops being right the moment you need two agents running at once, because they fight over the same profile, the same ports, and the same screen.

Self-hosted headless fleet. You run Chromium in containers, manage a scheduler, handle zombie processes, configure outbound network paths, and build your own session viewer. This is a real engineering project. Teams do it, and some do it well, but it is rarely the thing they set out to build.

Hosted runtime. You get a connection URL, hand it to Playwright or Puppeteer, and the provider handles lifecycle, isolation, and cleanup. The trade-off is a per-browser-hour cost and a dependency on someone else's uptime. For most agent products, that trade is worth making early and revisiting later.

Connecting an Agent Over CDP

The Chrome DevTools Protocol is the common language here. Playwright, Puppeteer, and Selenium can all attach to a remote Chromium instance over CDP, which means your agent code doesn't need to know whether the browser is local or hosted — only the connection string changes.

Here is a minimal TypeScript example using Playwright's connectOverCDP:

import { chromium, Browser, Page } from 'playwright';

interface SessionConfig {
  cdpUrl: string;      // e.g. wss://... from your runtime provider
  taskPrompt: string;
}

async function runAgentTask(config: SessionConfig): Promise<string> {
  let browser: Browser | null = null;

  try {
    // Attach to the hosted Chromium session over CDP.
    browser = await chromium.connectOverCDP(config.cdpUrl);

    // Reuse the default context so persistent profile state applies.
    const context = browser.contexts()[0] ?? await browser.newContext();
    const page: Page = context.pages()[0] ?? await context.newPage();

    await page.goto('https://example.com/login', {
      waitUntil: 'domcontentloaded',
      timeout: 30_000,
    });

    // Your agent loop goes here. A real implementation would
    // inspect the DOM, call a model, and dispatch actions.
    await page.fill('#username', process.env.APP_USER!);
    await page.fill('#password', process.env.APP_PASS!);
    await page.click('button[type="submit"]');
    await page.waitForLoadState('networkidle');

    const result = await page.title();
    return result;
  } finally {
    // Close the CDP connection, not the remote browser itself,
    // unless your provider expects an explicit session teardown.
    await browser?.close();
  }
}

Three details in that snippet matter more than they look.

`connectOverCDP` versus `launch`. launch starts a browser process on your machine. connectOverCDP attaches to one that already exists. The second is what you want for hosted sessions. The Playwright docs cover the CDP connection semantics in detail, including the caveat that connectOverCDP is Chromium-only.

Context reuse. If you call newContext() on every run, you throw away cookies, localStorage, and any logged-in state. For agents that operate behind a login, reusing the default context — or explicitly loading a persistent profile — is the difference between a working agent and one that re-authenticates every five minutes.

Teardown semantics. Closing the CDP connection is not always the same as terminating the remote session. Check your provider's model. Some bill until you explicitly stop the session; others tear down on disconnect. This is a real cost difference at scale.

What to Check Before You Commit

Once the connection works, the questions shift from "does it run" to "does it hold up." Here is the checklist that matters.

Session isolation

Two agents running concurrently must not share cookies, storage, or network identity. If they do, you get cross-contamination that is maddening to debug — agent A logs in as user 1, agent B inherits the session and starts acting as user 1. Confirm that each session gets its own browser context at minimum, and ideally its own process.

Persistent profiles

Some tasks need a warm profile: logged-in state, saved preferences, a session token that survives a restart. Others need a cold one every time. Your runtime should let you choose. A profile that persists across sessions is also a liability if it's shared across tenants, so check how profiles are scoped.

Proxy and network configuration

Agents that hit protected sites need a network path that looks ordinary. This usually means residential or mobile proxies, and it means the proxy is configured at the browser level, not bolted on afterward. Look for configurable browser settings — proxy endpoints, locale, timezone, user agent — rather than a single "stealth" toggle that hides what it's actually doing. Stealth is a spectrum, and you want to know where on it you're standing.

Observability

When an agent fails at step 14 of 20, you need to see what the page looked like. A live viewer plus session recording turns a two-hour debugging session into a five-minute one. This is the feature teams underrate most before they need it and value most after.

Usage controls

Browser time is metered. You want per-session limits, a way to kill runaway agents, and a dashboard that shows spend before the invoice arrives. Remote Browser meters by browser-hour; current rates and plan details are on the pricing page.

Local Versus Hosted: When to Switch

The honest answer is that local is fine until it isn't. The signals that you've outgrown it:

• You need more than two or three concurrent sessions.
• Your CI pipeline needs to run agent tests, and there's no display.
• You're spending more time managing outbound network configuration than on agent logic.
• Debugging requires reproducing a user's exact session, and you can't.
• Your laptop fans spin up every time a test suite runs.

None of these are emergencies on day one. All of them are emergencies on the day you demo to a customer.

The switch itself is small if you've kept the connection layer abstracted. If your agent code calls chromium.launch() directly in twenty places, the refactor is annoying. If it calls a getBrowser() helper, it's a one-line change.

A Note on "Browser Use" Tooling

There's a specific open-source project called Browser Use, and it's worth being precise about what it is and isn't. It's an agent framework — it handles the reasoning loop, the action space, and the page-state representation. It is not a browser runtime. It expects a browser to be provided.

That distinction matters because teams sometimes adopt the framework and assume the runtime comes with it, then hit a wall when they need concurrency, isolation, or a proxy. The framework and the runtime are separate purchases, and you should evaluate them separately. For a deeper comparison of how the framework and a hosted runtime fit together, see Remote Browser for AI agents.

Practical Starting Sequence

If you're starting today, here's an order that avoids rework:

1. Get one task working locally. Launch Chromium with Playwright, write the agent loop, confirm it completes a real task end to end. Don't optimize anything yet.
2. Abstract the connection. Replace direct launch() calls with a single function that returns a browser. This is the seam you'll swap later.
3. Add a second concurrent run. Watch it break. Note exactly what breaks — shared profile, port conflict, memory. That list is your requirements document.
4. Move to a hosted session. Point your connection function at a CDP URL. Confirm the same task passes.
5. Add observability. Turn on the live viewer. Run the task again and watch it. You will spot something you didn't know was happening.
6. Set usage limits. Cap session duration and concurrency before you run anything unattended.

Steps 1 and 2 take an afternoon. Steps 4 through 6 take another. The teams that struggle are the ones that skip step 2 and try to do step 4 across a codebase full of hardcoded launches.

Common Failure Modes

A short list of things that go wrong, and why:

• The agent works locally and fails hosted. Usually a headless rendering difference, a missing font, or a viewport size the site responds to differently. Set an explicit viewport and user agent.
• Login works once, then fails. The profile isn't persisting, or the site invalidated the session. Check whether your context is being recreated per run.
• Requests get blocked after a few hundred runs. The network identity is too consistent. This is where proxy configuration and configurable browser settings earn their keep.
• Sessions leak. Agents that crash without cleanup leave sessions running and billing. Always wrap the agent loop in a finally that tears down, and set a server-side max duration as a backstop.
• Debugging is impossible. No viewer, no recording, no logs. Add observability before you need it, not after.

Where Remote Browser Fits

Remote Browser is a hosted Chromium runtime for agents and browser-use workflows. It gives you CDP endpoints that Playwright, Puppeteer, and Selenium can attach to, per-session isolation, persistent profiles, configurable browser and proxy settings, a live viewer, and browser-hour metering with usage controls.

It is not an agent framework. You bring the reasoning loop — whether that's Browser Use, a custom planner, or a scripted sequence — and Remote Browser provides the session it drives. That separation is deliberate: model choices change faster than browser infrastructure, and you don't want to rewrite your runtime every time you swap a model.

If you want to see a session running before you write any code, the online remote browser walkthrough shows what the viewer looks like. If you're evaluating whether to drive sessions from code or from an interactive console, remote control browser covers that split.

The Short Version

Browser use start is two problems, not one: the agent loop and the browser runtime. Solve them separately. Get the loop working locally, abstract the connection, then move the browser to a hosted session when concurrency, isolation, or debugging forces your hand.

The runtime decision is not permanent, but it is load-bearing. Pick one that speaks CDP, isolates sessions, persists profiles when you need it, and lets you watch what your agent is doing. Everything else is a detail you can fix later.