AndroidNode

Interface

Accessibility tree nodes for UI element interaction

AndroidNode represents a node in the Android accessibility tree. Each node corresponds to a UI element on screen and contains properties like text, bounds, and state. Use methods like screenContent() to get the root node and traverse the tree.

Getting Screen Content

TypeScript

// Get the accessibility tree
const screen = await agent.actions.screenContent();

// Find elements
const button = screen.findTextOne("Submit");
const allInputs = screen.filterAdvanced(f => f.isEditText());

// Interact with elements
if (button) {
  const { left, top, right, bottom } = button.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}

AndroidNodeFilter

Learn about the builder pattern for complex node queries

Properties

Identification

Property	Type	Description
`falseId`	`string`	UUID unique per request (changes each time screen content is fetched)
`viewId?`	`string`	Android view ID (e.g., 'com.example:id/button')
`className`	`string \| null`	Android class name (e.g., 'android.widget.Button')
`packageName`	`string`	Package name of the app owning this node

Layout

Property	Type	Description
`boundsInScreen`	`{left, top, right, bottom}`	Screen coordinates of the node's bounding box
`index`	`number`	Index within parent's children

Text Content

Property	Type	Description
`text?`	`string`	Text content of the node
`description?`	`string`	Content description (accessibility label)
`hintText`	`string \| null`	Hint text for input fields
`maxTextLength`	`number`	Maximum text length for input fields
`inputType`	`string`	Input type for text fields
`isShowingHintText`	`boolean`	Whether hint text is currently shown
`tooltipText?`	`string \| null`	Tooltip textAndroid 9+
`paneTitle?`	`string \| null`	Pane title for accessibilityAndroid 9+
`containerTitle?`	`string \| null`	Container titleAndroid 14+

State

Property	Type	Description
`isEnabled`	`boolean`	Whether the node is enabled
`clickable`	`boolean`	Whether the node is clickable
`isLongClickable`	`boolean`	Whether the node responds to long click
`isContextClickable`	`boolean`	Whether the node responds to context click
`isFocusable`	`boolean`	Whether the node can receive focus
`isFocused`	`boolean \| null`	Whether the node currently has focus
`isScrollable`	`boolean`	Whether the node is scrollable
`isSelected`	`boolean`	Whether the node is selected
`isChecked`	`boolean \| null`	Checked state (for checkboxes, switches)
`isEditable`	`boolean`	Whether text can be edited
`isDismissable`	`boolean`	Whether the node can be dismissed
`isPassword`	`boolean`	Whether this is a password field
`isMultiLine`	`boolean`	Whether this is a multi-line text field
`isVisibleToUser`	`boolean`	Whether the node is visible on screen
`isImportantForAccessibility`	`boolean`	Whether important for accessibility
`isHeading?`	`boolean`	Whether this node is a headingAndroid 9+
`isTextSelectable?`	`boolean`	Whether text can be selectedAndroid 13+
`isTextEntryKey?`	`boolean`	Whether this is a keyboard keyAndroid 10+

Collection Info

Property	Type	Description
`collectionInfo?`	`object`	Info when node is a collection (ListView, RecyclerView). Contains rowCount, columnCount, itemCount, hierarchical, selectionMode
`collectionItemInfo?`	`object`	Info when node is a collection item. Contains rowIndex, columnIndex, rowSpan, columnSpan, isSelected

Hierarchy

Property	Type	Description
`children`	`AndroidNode[]`	Child nodes
`parent`	`AndroidNode \| null`	Parent node (null for root)

Actions

Property	Type	Description
`actions`	`number[]`	Array of supported action IDs (use with agent.constants.ACTION_*)
`actionLabels`	`{id: number, label?: string}[]`	Action labels with optional custom text

Methods

`allNodes()`

TypeScript

allNodes(): AndroidNode[]

Returns all nodes in the subtree as a flat array, including this node and all descendants.

Returns

AndroidNode[]Flat array of all nodes

Examples

TypeScript

const screen = await agent.actions.screenContent();
const allNodes = screen.allNodes();
console.log("Total nodes:", allNodes.length);

// Find all visible text
const textNodes = allNodes.filter(n => n.text && n.isVisibleToUser);

`findById()`

TypeScript

findById(id: string): AndroidNode[]

Finds all nodes with the matching viewId.

Parameters

Name	Type	Description
`id`	`string`	View ID to search for (e.g., 'com.example:id/button')

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

const buttons = screen.findById("com.example:id/submit_button");

`findByIdOne()`

TypeScript

findByIdOne(id: string): AndroidNode | null

Finds the first node with the matching viewId.

Parameters

Name	Type	Description
`id`	`string`	View ID to search for

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

const button = screen.findByIdOne("com.example:id/submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

`findText()`

TypeScript

findText(textOrRegex: string | RegExp): AndroidNode[]

Finds all nodes containing the text in description, text, or hintText (case-insensitive). Supports strings and regular expressions.

Parameters

Name	Type	Description
`textOrRegex`	`string \| RegExp`	Text to search for or regex pattern

Returns

AndroidNode[]Array of matching nodes

Examples

Find by string

TypeScript

const nodes = screen.findText("Submit");

Find by regex

TypeScript

const priceNodes = screen.findText(/\$\d+\.\d{2}/);

`findTextOne()`

TypeScript

findTextOne(textOrRegex: string | RegExp): AndroidNode | null

Finds the first node containing the text (case-insensitive).

Parameters

Name	Type	Description
`textOrRegex`	`string \| RegExp`	Text to search for or regex pattern

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

const submit = screen.findTextOne("Submit");
if (submit) {
  const { left, top, right, bottom } = submit.boundsInScreen;
  await agent.actions.tap((left + right) / 2, (top + bottom) / 2);
}

`find()`

TypeScript

find(predicate: (node: AndroidNode) => boolean): AndroidNode | null

Finds the first node matching a custom predicate function.

Parameters

Name	Type	Description
`predicate`	`(node: AndroidNode) => boolean`	Function to test each node

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

// Find first clickable button with specific text
const btn = screen.find(node =>
  node.className === "android.widget.Button" &&
  node.clickable &&
  node.text === "Submit"
);

// Find first visible input field
const input = screen.find(node =>
  node.isEditable && node.isVisibleToUser
);

`filter()`

TypeScript

filter(predicate: (node: AndroidNode) => boolean): AndroidNode[]

Filters all nodes matching a custom predicate function.

Parameters

Name	Type	Description
`predicate`	`(node: AndroidNode) => boolean`	Function to test each node

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

// Find all buttons
const buttons = screen.filter(node =>
  node.className === "android.widget.Button"
);

// Find all clickable, visible nodes
const clickables = screen.filter(node =>
  node.clickable && node.isVisibleToUser
);

// Find nodes with text containing dollar amounts
const prices = screen.filter(node =>
  node.text?.includes("$")
);

`findAdvanced()`

TypeScript

findAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode | null

Finds the first node using the AndroidNodeFilter builder pattern.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

AndroidNode | nullFirst matching node or null

Examples

TypeScript

// Find the submit button
const submit = screen.findAdvanced(f => f.isButton().hasText("Submit"));

// Find first editable text field
const input = screen.findAdvanced(f => f.isEditText().isEditable());

// Find by view ID
const header = screen.findAdvanced(f => f.hasId("com.example:id/header"));

`filterAdvanced()`

TypeScript

filterAdvanced(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): AndroidNode[]

Filters all nodes using the AndroidNodeFilter builder pattern.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

AndroidNode[]Array of matching nodes

Examples

TypeScript

// Find all clickable buttons
const buttons = screen.filterAdvanced(f => f.isButton().isClickable());

// Find text nodes with specific text
const labels = screen.filterAdvanced(f => f.isText().hasText("Welcome"));

// Find nodes in a specific list
const listItems = screen.filterAdvanced(f =>
  f.anyParent(p => p.hasId("com.example:id/list"))
);

`matches()`

TypeScript

matches(filterBuilder: (f: AndroidNodeFilter) => AndroidNodeFilter): boolean

Checks if this node matches the given filter.

Parameters

Name	Type	Description
`filterBuilder`	`(f: AndroidNodeFilter) => AndroidNodeFilter`	Function to build the filter

Returns

booleantrue if node matches the filter

Examples

TypeScript

// Check if a node is a clickable button
if (node.matches(f => f.isButton().isClickable())) {
  await node.performAction(agent.constants.ACTION_CLICK);
}

// Filter children manually
const clickableChildren = node.children.filter(child =>
  child.matches(f => f.isClickable())
);

`toJSON()`

TypeScript

toJSON(): object

Converts the node to a plain JSON object. Useful for debugging or serialization.

Returns

objectPlain object representation

Examples

TypeScript

console.log(JSON.stringify(node.toJSON(), null, 2));

Action Methods

These methods allow you to perform actions directly on nodes. Available since app version 2.119.

`performAction()`

TypeScript

performAction(actionInt: number, data?: object, fieldsToIgnore?: string[]): Promise<{ actionPerformed: boolean }>

Performs an accessibility action on this node. This is the preferred way to interact with UI elements as it uses the accessibility system.

Parameters

Name	Type	Description
`actionInt`	`number`	Action constant (use agent.constants.ACTION_*)
`data?`	`object`	Additional action data (optional, used for ACTION_SET_TEXT etc.)
`fieldsToIgnore?`	`string[]`	Fields to ignore when matching node (optional)

Returns

Promise<{ actionPerformed: boolean }>Whether the action was performed successfully

Examples

Click a button

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

Set text on an input field

TypeScript

const input = screen.findAdvanced(f => f.isEditText());
if (input) {
  await input.performAction(agent.constants.ACTION_SET_TEXT, {
    [agent.constants.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE]: "Hello World"
  });
}

Scroll a list

TypeScript

const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);
}

`randomClick()`

TypeScript

randomClick(): void

Performs a tap at a random position within this node's bounds. Uses boundsInScreen to determine the click area. Useful for more natural-looking automation.

Examples

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}

Click multiple items

TypeScript

const items = screen.filterAdvanced(f => f.isClickable().hasText(/Item \d+/));
for (const item of items) {
  item.randomClick();
  await sleep(500);
}

`randomSwipe()`

TypeScript

randomSwipe(direction: "up" | "down" | "left" | "right"): void

Performs a swipe gesture within this node's bounds. The swipe starts and ends at random positions within the node for more natural-looking automation.

Parameters

Name	Type	Description
`direction`	`"up" \| "down" \| "left" \| "right"`	Direction to swipe

Examples

Scroll a list down

TypeScript

const listView = screen.findAdvanced(f => f.isScrollable());
if (listView) {
  listView.randomSwipe("up"); // Swipe up to scroll down
}

Swipe through a carousel

TypeScript

const carousel = screen.findAdvanced(f => f.hasId("carousel"));
if (carousel) {
  carousel.randomSwipe("left"); // Swipe left to see next item
}

Common Patterns

Click a button (random position)

TypeScript

const button = screen.findTextOne("Submit");
if (button) {
  button.randomClick();
}

Click using accessibility action

TypeScript

const button = screen.findAdvanced(f => f.isButton().hasText("OK"));
if (button) {
  await button.performAction(agent.constants.ACTION_CLICK);
}

Type in an input field

TypeScript

const input = screen.findAdvanced(f => f.isEditText().isEditable());
if (input) {
  await input.performAction(agent.constants.ACTION_CLICK);
  await agent.actions.writeText("Hello World");
}

Scroll a list

TypeScript

const scrollView = screen.findAdvanced(f => f.isScrollable());
if (scrollView) {
  // Using accessibility action
  await scrollView.performAction(agent.constants.ACTION_SCROLL_FORWARD);

  // Or using random swipe for more natural scrolling
  scrollView.randomSwipe("up");
}