BAP vs Playwright

Feature matrix and benchmark comparison between BAP and Playwright MCP/CLI for AI agent browser automation.

Overview

BAP and Playwright both use the Playwright engine for browser automation. They differ in architecture, API design, and optimization for AI agent workflows.

Dimension	BAP	Playwright
Publisher	browseragentprotocol	Microsoft
License	Apache-2.0	Apache-2.0
Architecture	Two-process (WebSocket bridge)	Single-process
MCP Package	@browseragentprotocol/mcp	@playwright/mcp
CLI Package	@browseragentprotocol/cli	@playwright/cli

MCP Server Comparison

BAP MCP

• 23 MCP tools - Composite act batches N steps in 1 call - observe returns structured elements with refs - extract with JSON Schema - 6 fusion operations - Response tiers (full/interactive/minimal) - WebMCP discovery - Session persistence across restarts

Playwright MCP

• 31 tools (17 core + 6 vision + 5 test + 3 other) - No built-in batching - browser_snapshot returns raw accessibility tree - browser_evaluate for custom JS extraction - Lower per-call latency (single-process) - Simpler setup (npx @playwright/mcp) - 27.5k GitHub stars, Microsoft-backed

Feature Matrix

Feature	BAP	Playwright MCP
Composite actions (batch N steps)	Yes	No
Structured observation with refs	Yes	Accessibility tree
JSON Schema extraction	Yes	Custom JS
Navigate + observe fusion	Yes	No
Act + pre/post observe fusion	Yes	No
Incremental observation	Yes	No
Response compression tiers	Yes	No
WebMCP tool discovery	Yes	No
Session persistence	Yes	No
Multi-client (CLI + MCP)	Yes	No
Batched form filling	act (N fills + click = 1 call)	browser_fill_form (fills batched, click separate)
Per-call latency	+50-200ms (WebSocket)	Lower (in-process)
Element disambiguation	Semantic selectors (can be ambiguous)	Positional refs (always unique)
SKILL.md support	13 platforms	Claude Code, GitHub Copilot

CLI Comparison

Dimension	BAP CLI	Playwright CLI
Commands	26	~70+ (granular)
Composite actions	bap act fill:...=val click:...	Individual commands
Semantic selectors	role:button:"Submit", label:"Email"	Accessibility tree refs
Observation	bap observe --tier=interactive	playwright-cli snapshot
Extraction	bap extract --fields="title,price"	playwright-cli eval (custom JS)

Benchmark Results

All data from the reproducible benchmark suite. See Benchmarks for full methodology.

Scenario	BAP Standard	BAP Fused	Playwright	Std vs PW	Fused vs PW
baseline	2	2	2	Tie	Tie
observe	2	1	2	Tie	-50%
extract	2	2	2	Tie	Tie
form	4	3	5	-20%	-40%
ecommerce	8	5	11	-27%	-55%
workflow	5	4	5	Tie	-20%
Total	23	17	27	~15%	~37%

The fair comparison is BAP Standard vs Playwright. BAP Standard follows the same observe-then-act pattern as Playwright. BAP Fused is an optimization layer that is not apples-to-apples.

Where BAP Wins

• Composite act: Batching multiple steps into one call is the primary advantage. Most impactful in multi-step flows like ecommerce (8 vs 11 calls).
• Fused operations: navigate(observe:true) and act(postObserve:true) eliminate redundant roundtrips.
• Structured extract: JSON Schema-based extraction vs writing custom JS.
• Session persistence: Browser state survives across client restarts.
• Multi-client: CLI and MCP share the same browser simultaneously.

Where Playwright Wins

• Per-call latency: Single-process architecture. No WebSocket overhead.
• Element disambiguation: Positional snapshot refs uniquely identify elements. BAP’s observe can return ambiguous selectors for identical elements (e.g., 6 “Add to cart” buttons).
• Setup simplicity: npx @playwright/mcp — single process, no daemon management.
• Ecosystem: 27.5k GitHub stars, Microsoft-backed, extensive testing integration.
• Tool count: 31 tools covering more granular operations.

When to Use Which

Choose BAP

• Multi-step form automation - AI agent workflows (observe-decide-act loop) - Need CLI + MCP access to same browser - Session persistence matters - Token efficiency is critical

Choose Playwright

• Per-call latency is the priority - Already in the Playwright testing ecosystem - Need the simplest possible setup - Element disambiguation is critical - Vision-based automation (6 vision tools)