Step Tracking & Debugging

Utils

Track automation progress and debug failures

Access step tracking through agent.utils.outOfSteps. Used for tracking automation progress and debugging when automations run out of steps or timeout.

OutOfStepsUtils Interface

TypeScript

interface OutOfStepsUtils {
  storeScreen(
    screen: AndroidNode,
    stage: string,
    screenState: string,
    remainingSteps: number,
    screenshotRecord: ScreenshotRecord
  ): Promise<void>;

  submit(
    type: "outOfSteps" | "timeout" | "debug",
    tag?: string
  ): Promise<{ success: false; error: string } | { success: true; id: string }>;
}

`storeScreen()`

TypeScript

storeScreen(screen: AndroidNode, stage: string, screenState: string, remainingSteps: number, screenshotRecord: ScreenshotRecord): Promise<void>

Stores the current screen state for debugging purposes. Call this periodically during automation to track progress and help diagnose issues when automations fail.

Parameters

Name	Type	Description
`screen`	`AndroidNode`	The current screen content from screenContent()
`stage`	`string`	Current stage/phase of the automation (e.g., 'login', 'checkout')
`screenState`	`string`	Description of the current screen state
`remainingSteps`	`number`	Number of steps remaining in the automation
`screenshotRecord`	`ScreenshotRecord`	Screenshot quality setting

Examples

TypeScript

const screen = await agent.actions.screenContent();

// Store screen state for debugging
await agent.utils.outOfSteps.storeScreen(
  screen,
  "login",
  "waiting_for_credentials",
  50,
  ScreenshotRecord.LOW_QUALITY
);

`submit()`

TypeScript

submit(type: "outOfSteps" | "timeout" | "debug", tag?: string): Promise<{ success: false; error: string } | { success: true; id: string }>

Submits the collected screen states for analysis. Call this when the automation ends unexpectedly or for debugging. The server keeps the 200 most recent records per type per automation — except "debug" records, which are kept per tag: each tag retains its own 200 most recent records.

Parameters

Name	Type	Description
`type`	`"outOfSteps" \| "timeout" \| "debug"`	Reason for submission
`tag`	`string`	Optional tag for "debug" submissions (defaults to "default", max 64 characters). Use distinct tags to keep separate debug histories — the server retains 200 debug records per tag. Ignored for non-debug submissions.

Returns

{ success: true; id: string } | { success: false; error: string }Result with submission ID on success or error message on failure

Examples

Submit on timeout

TypeScript

const result = await agent.utils.outOfSteps.submit("timeout");
if (result.success) {
  console.log("Debug data submitted with ID:", result.id);
} else {
  console.log("Failed to submit:", result.error);
}

Submit for debugging

TypeScript

// Collect screens during automation
await agent.utils.outOfSteps.storeScreen(screen1, "step1", "initial", 100, ScreenshotRecord.HIGH_QUALITY);
await agent.utils.outOfSteps.storeScreen(screen2, "step2", "processing", 80, ScreenshotRecord.LOW_QUALITY);

// Submit for analysis (stored under the "default" tag)
await agent.utils.outOfSteps.submit("debug");

Submit debug data with a tag

TypeScript

// Each tag keeps its own history of 200 records,
// so tagged submissions don't push out other debug data
await agent.utils.outOfSteps.submit("debug", "login-flow");

Supporting Types

ScreenshotRecord

Enum for screenshot quality settings.

TypeScript

enum ScreenshotRecord {
  HIGH_QUALITY,  // Full quality screenshot
  LOW_QUALITY,   // Compressed screenshot (faster, smaller)
  NONE           // No screenshot
}

Usage Pattern

Complete Debugging Flow

TypeScript

async function runAutomation() {
  let remainingSteps = 100;

  while (remainingSteps > 0) {
    const screen = await agent.actions.screenContent();

    // Store screen state periodically
    await agent.utils.outOfSteps.storeScreen(
      screen,
      getCurrentStage(),
      describeScreen(screen),
      remainingSteps,
      ScreenshotRecord.LOW_QUALITY
    );

    // Perform automation step
    const result = await performStep(screen);
    if (!result.success) {
      // Submit debug data on failure
      await agent.utils.outOfSteps.submit("outOfSteps");
      throw new Error("Automation failed");
    }

    remainingSteps--;
  }

  // Submit on timeout if loop exits without success
  await agent.utils.outOfSteps.submit("timeout");
}