Browser Use 81: What the Score Means for Production

# Browser Use 81: What the Score Means for Production

Browser Use 81 is a benchmark score, and like every benchmark score it answers a narrower question than the one you actually care about. If you are evaluating browser-use for a production workload, the number tells you something about task success on a fixed evaluation set. It does not tell you what happens when your agent hits a login wall at 3 a.m., when a session dies mid-checkout, or when you need forty concurrent browsers and only have budget for four.

This post breaks down what a score like 81 represents, where it stops being predictive, and what the runtime layer underneath browser-use has to do before that number survives contact with real traffic. If you want the runtime overview first, start with Remote Browser for AI agents.

What "Browser Use 81" Actually Refers To

Browser-use is an open-source framework that lets an LLM drive a browser: it observes the DOM, decides on an action, executes it through Playwright or CDP, and repeats until the task completes or it gives up. Benchmarks for this class of agent typically report a percentage of tasks completed correctly across a curated set of web tasks — form fills, multi-step navigation, data extraction, and increasingly, tasks on sites that actively resist automation.

A score of 81 sits in the range where the framework is clearly viable for real work but not yet a solved problem. Roughly one in five tasks fails on the evaluation set. On your workload, that ratio will be different — usually worse, because benchmark tasks are chosen to be representative and your tasks are chosen by your business.

Three things determine whether a benchmark number transfers to your stack:

• Task distribution. Benchmarks skew toward public, well-structured sites. Your agents probably touch authenticated dashboards, internal tools, and sites with aggressive bot detection.
• Runtime stability. A benchmark run that crashes halfway is usually retried or discarded. In production, a crashed session is a failed customer action.
• Cost per solved task. A 81% success rate at one price is a different product than 81% at ten times the price. This is the metric that actually shows up on your invoice.

That last point matters more than most teams expect. A framework that solves 81% of tasks but burns three browser-hours per attempt is more expensive than one that solves 75% and finishes in one.

Where the Score Stops Being Predictive

Benchmarks are run under conditions that production does not reproduce. Understanding the gap is the difference between a demo and a deployment.

Local Chrome is not the same as a hosted session

Most browser-use benchmark runs happen against a local Chromium instance on a developer machine or a CI runner. That environment has a residential IP, a real display, no proxy latency, and no concurrent sessions competing for CPU. Move the same agent to a cloud VM with a datacenter IP and the failure modes change entirely: CAPTCHAs appear, geo-restrictions trigger, and sites that loaded instantly now take four seconds.

Retries hide the real failure rate

If your harness retries a failed task twice, your effective success rate goes up and your cost goes up faster. A benchmark that reports 81% after retries is reporting a different number than one that reports first-attempt success. When you read a score, find out which one it is.

The long tail is where agents die

Benchmark task sets are finite. Production task sets are not. The 19% that fails in a benchmark is a sample; the failures you hit in production are drawn from a much larger distribution that includes sites you have never seen, layouts that changed last Tuesday, and anti-bot systems that update weekly.

This is why the runtime layer matters more than the framework score. The framework decides *what* to do. The runtime decides whether the browser is still alive, still authenticated, and still reachable when the agent tries to do it.

What a Production Runtime Has to Provide

If you are running browser-use agents against real sites, the runtime underneath needs to handle a specific set of problems. These are not optional features; they are the difference between a working agent and a support ticket.

The pattern here is that every row is a runtime concern, not an agent concern. Browser-use does not care whether the browser is local or remote — it cares that the CDP connection is stable and that the page state is what it expects. That separation is what makes hosted Chromium a drop-in for local Chrome in most browser-use setups.

For a deeper look at how the runtime layer is structured, see Remote Browser online.

Connecting Browser-Use to Hosted Chromium

The mechanics are straightforward. Browser-use accepts a CDP URL, so you point it at a hosted session instead of launching a local browser. Here is the Playwright-level version of that connection, which is what browser-use does under the hood:

import { chromium, Browser, BrowserContext, Page } from 'playwright';

interface SessionConfig {
  cdpUrl: string;        // wss://... from your runtime provider
  profileId?: string;    // reuse login state across runs
  proxyRegion?: string;  // route traffic through a specific region
}

async function connectToHostedBrowser(
  config: SessionConfig
): Promise<{ browser: Browser; context: BrowserContext; page: Page }> {
  const browser = await chromium.connectOverCDP(config.cdpUrl, {
    timeout: 30_000,
  });

  // Reuse the default context so persistent profile state is preserved.
  const context = browser.contexts()[0] ?? await browser.newContext({
    viewport: { width: 1280, height: 800 },
    locale: 'en-US',
  });

  const page = await context.newPage();

  // Fail fast on navigation stalls instead of hanging the agent loop.
  page.setDefaultNavigationTimeout(45_000);
  page.setDefaultTimeout(15_000);

  return { browser, context, page };
}

// Usage inside a browser-use style agent loop:
const { browser, page } = await connectToHostedBrowser({
  cdpUrl: process.env.REMOTE_BROWSER_CDP_URL!,
  profileId: 'checkout-agent-01',
});

await page.goto('https://example.com/dashboard', {
  waitUntil: 'domcontentloaded',
});

// ... agent observes, decides, acts ...

await browser.close(); // releases the hosted session

Two details matter here. First, connectOverCDP is the same call you would use against a local Chrome with --remote-debugging-port; the only change is the URL. Second, browser.close() is what releases the hosted session and stops the meter — forgetting it is the most common cause of runaway browser-hour costs.

The Playwright documentation for `browserType.connectOverCDP` covers the connection semantics, including the fact that CDP connections are Chromium-only. If your agent needs Firefox or WebKit, you are looking at a different connection path.

Cost, Concurrency, and the Numbers Benchmarks Don't Show

A benchmark score is a quality signal. It says nothing about the two constraints that actually gate production: cost per task and concurrency ceiling.

Cost per solved task

The useful metric is not cost per browser-hour. It is cost per *solved* task, which is:

cost_per_solved_task = (browser_hours_per_attempt × cost_per_hour) / success_rate

At an 81% success rate, you are paying for the 19% that fails too. If a task takes 90 seconds and you run it on a metered hosted browser, the arithmetic is easy to do — but only if you know your real success rate on *your* tasks, not the benchmark's. Measure it before you commit to a volume plan.

Current rates and metering details are on the pricing page. We deliberately do not quote numbers here because they change and because your effective cost depends on your task mix.

Concurrency and session isolation

Benchmarks run one task at a time. Production runs many. The questions to ask:

• Does each task get its own browser context, or do tasks share state?
• What happens when two agents hit the same site with the same profile?
• Can you cap concurrent sessions to control spend?
• Are sessions recycled after completion, or do they linger?

Session isolation is the one that bites hardest. If agent A logs into a shared account and agent B inherits those cookies, you get cross-contamination that is very hard to debug from logs alone. Hosted runtimes that give each task a clean context solve this by default; local setups usually do not.

Stealth, Proxies, and the Sites That Reject Agents

The related keyword "browser use stealth" points at a real problem: a meaningful fraction of production tasks run against sites that do not want automated traffic. Benchmarks rarely include these sites, which is part of why benchmark scores overstate real-world success.

What actually helps, in rough order of impact:

1. Residential or mobile proxies. Datacenter IP ranges are widely blocked. Routing through residential IPs in the right region removes a large class of failures.
2. Consistent browser settings. A browser that reports a Linux user agent, a Windows viewport, and a headless rendering path is trivially fingerprinted. Configurable browser settings that stay internally consistent matter more than any single spoofed value.
3. Realistic session behavior. Instant navigation, no mouse movement, and sub-100ms form fills are detectable regardless of IP.
4. Persistent profiles. A profile with history and cookies looks more like a returning user than a fresh context every time.

We are careful about what we claim here. Remote Browser exposes configurable browser settings and proxy configuration; it does not promise that any specific site will accept your traffic. Stealth is a spectrum, and the honest position is that you should test against your actual targets rather than trust a vendor's evasion claim.

When to Move Off Local Chrome

The decision point is usually not technical capability — it is operational pain. Move to a hosted runtime when any of these become true:

• Your agents run on servers where installing Chromium is awkward or impossible.
• You need more concurrent sessions than a single machine can host.
• You are spending more time debugging browser crashes than agent logic.
• You need to see what the agent saw after the fact, not just read a stack trace.
• Your tasks require persistent login state across runs.
• You want per-session cost visibility instead of a cloud bill you cannot attribute.

None of these require rewriting your agent. Browser-use talks CDP; hosted Chromium speaks CDP. The migration is usually a connection-string change plus a session lifecycle you now have to manage explicitly.

For the broader picture of how remote browser control fits into agent architectures, see remote control browser.

A Practical Evaluation Checklist

If you are comparing runtimes for a browser-use workload, score each candidate against these:

• CDP compatibility. Can you connect with connectOverCDP without a custom shim?
• Session lifecycle. Is there a clear create/connect/close path, and does close actually release resources?
• Profile persistence. Can you reuse login state, and can you scope it per account?
• Isolation guarantees. Does each task get a clean context?
• Proxy control. Can you select region and proxy type per session?
• Observability. Is there a live viewer, and can you retrieve session recordings or logs?
• Metering. Is cost per session visible, and can you set limits?
• Framework support. Playwright, Puppeteer, Selenium — all over CDP, all without local Chrome.

A benchmark score of 81 tells you the framework is worth evaluating. This checklist tells you whether the runtime will hold up when the evaluation is over and real traffic starts.

The Short Version

Browser Use 81 is a useful signal that the framework clears a meaningful bar on curated tasks. It is not a production guarantee, and treating it as one is how teams end up surprised by their first month of real traffic. The score measures the agent; the runtime measures everything else — session stability, isolation, proxy quality, cost control, and whether you can see what went wrong at 3 a.m.

If you are ready to test browser-use against a hosted runtime, the documentation covers the CDP connection path, session lifecycle, and profile handling. Start with one task you already know how to solve manually, measure first-attempt success and cost per solved task, and let that number — not the benchmark — drive the decision.