Form Automation

Automate login flows, multi-step forms, and complex inputs using bap act composite actions.

Login Flow with bap act

The most common form automation pattern is a login flow. With bap act, you batch fill + click into a single command:

bap act fill:label:"Email"="[e-mail]" fill:label:"Password"="secret123" click:role:button:"Sign In"

This executes three steps in one shell invocation — no intermediate observe needed.

Step Syntax

Each step in bap act follows the pattern action:selector=value:

Part	Format	Example
Action	fill, click, type, press, select, check, uncheck	fill
Selector	Any BAP selector format	label:"Email", role:button:"Submit", e5
Value	Right of = (optional for click)	"[e-mail]"

Positional Refs

After running bap observe, use the returned element refs directly:

# Observe the page first
bap observe

# Use refs from observe output
bap act fill:e1="[e-mail]" fill:e3="secret123" click:e5

TypeScript SDK

import { BAPClient, label, role } from "@browseragentprotocol/client";

const result = await client.act({
  steps: [
    BAPClient.step("action/fill", {
      selector: label("Email"),
      value: "[e-mail]",
    }),
    BAPClient.step("action/fill", {
      selector: label("Password"),
      value: "secret123",
    }),
    BAPClient.step("action/click", {
      selector: role("button", "Sign In"),
    }),
  ],
  postObserve: { includeInteractiveElements: true },
});

if (result.success) {
  console.log("Login successful");
  console.log("Next page elements:", result.postObservation?.interactiveElements);
}

Multi-Step Forms

For forms that span multiple pages or sections, chain bap act calls:

# Step 1: Personal info
bap act fill:label:"First Name"="Jane" fill:label:"Last Name"="Doe" click:text:"Next"

# Step 2: Address (after page transition)
bap act fill:label:"Street"="123 Main St" fill:label:"City"="San Francisco" select:label:"State"="CA" click:text:"Next"

# Step 3: Payment
bap act fill:label:"Card Number"="4242424242424242" fill:label:"Expiry"="12/28" fill:label:"CVV"="123" click:text:"Submit"

Conditional Steps

In the TypeScript SDK, add conditions to steps that should only run when a prerequisite is met:

await client.act({
  steps: [
    BAPClient.step("action/fill", {
      selector: label("Email"),
      value: "[e-mail]",
    }),
    {
      action: "action/click",
      params: { selector: role("button", "Next") },
      condition: {
        selector: role("button", "Next"),
        state: "enabled",
        timeout: 5000,
      },
    },
  ],
});

Error Recovery

Steps support three error handling strategies:

await client.act({
  steps: [
    {
      action: "action/click",
      params: { selector: text("Accept Cookies") },
      onError: "skip", // Cookie banner might not appear
    },
    {
      action: "action/fill",
      params: { selector: label("Email"), value: "[e-mail]" },
      onError: "retry",
      maxRetries: 3,
      retryDelay: 500,
    },
    {
      action: "action/click",
      params: { selector: role("button", "Submit") },
      onError: "stop", // Default: halt on failure
    },
  ],
});

Fused Observe + Act

Eliminate extra roundtrips by fusing observation with action:

# CLI: --observe flag adds post-observation
bap act fill:e1="test" click:e3 --observe

// SDK: preObserve captures state before, postObserve after
const result = await client.act({
  preObserve: { includeInteractiveElements: true },
  steps: [
    /* ... */
  ],
  postObserve: { includeInteractiveElements: true, responseTier: "interactive" },
});

// result.preObservation -- page state before form submission
// result.postObservation -- page state after (e.g., dashboard elements)

Recipes

For repeatable form flows, use bap recipe:

# Save a recipe
bap recipe save login -- act fill:label:"Email"="[e-mail]" fill:label:"Password"="pass" click:role:button:"Sign In"

# Run it later
bap recipe run login