Writing Scripts
TypeScript basics, Agent API, and common patterns
Automation scripts are written in TypeScript and have access to the agent object which provides all the APIs for interacting with the device.
The Agent Object
The agent object is globally available and provides:
agent = { actions: { ... }, // Device interactions (tap, swipe, type, etc.) utils: { ... }, // Utilities (random helpers, job tasks, files) info: { ... }, // Device information control: { ... }, // Automation control (pause, delay) display: { ... }, // Display settings email: { ... }, // Email utilities notifications: { ... }, // Notification callbacks constants: { ... }, // Action constants arguments: { ... }, // Parameters and job variables}Common Actions
Touch Gestures
// Simple tap at coordinatesawait agent.actions.tap(500, 1000);
// Swipe from point A to B over 500msawait agent.actions.swipe(500, 1500, 500, 500, 500);
// Long press for 2 secondsawait agent.actions.hold(500, 1000, 2000);
// Double tap with 100ms intervalawait agent.actions.doubleTap(500, 1000, 100);
// Human-like random tap within node boundsconst button = screen.findTextOne("Submit");if (button) { button.randomClick();}Text Input
// Type text (requires focused input field)await agent.actions.writeText("Hello, World!");
// Copy text to clipboardawait agent.actions.copyText("Text to copy");
// Paste from clipboardawait agent.actions.paste();
// Hide keyboardawait agent.actions.hideKeyboard();Navigation
// System navigationawait agent.actions.goBack();await agent.actions.goHome();await agent.actions.recents();
// D-pad navigation (Android 13+ only)await agent.actions.dpad("down");await agent.actions.dpad("center");App Management
// Launch app by package nameawait agent.actions.launchApp("com.example.myapp");
// Launch fresh (tries closing the existing app first)await agent.actions.launchApp("com.example.myapp", true);
// Open URL on in-app browserawait agent.actions.browse("https://example.com");
// List all installed appsconst apps = await agent.actions.listApps();console.log(apps["com.android.chrome"]); // "Chrome"Screen Content
Get the current screen content as an accessibility tree:
// Get current screen contentconst screen = await agent.actions.screenContent();
// Get all nodes recursivelyconst allNodes = getAllNodes(screen);
// Find nodes by textconst buttons = allNodes.filter(node => node.text?.toLowerCase().includes("submit"));
// Find nodes by IDconst loginBtn = allNodes.find(node => node.viewId === "com.example:id/login_button");
// Find nodes by classconst editTexts = allNodes.filter(node => node.className === "android.widget.EditText");Helper Functions
These helper functions are globally available:
// Get all descendant nodes recursivelyconst allNodes = getAllNodes(screen);
// Find nodes by viewId (resourceId)const nodes = findNodesById(screen, "com.example:id/button");
// Find nodes by exact textconst textNodes = findNodesByText(screen, "Submit");
// Check if node has specific textconst hasText = nodeHasText(node, "Continue");Interacting with Nodes
// Find a button and tap its centerconst button = allNodes.find(n => n.text === "Submit" && n.clickable);if (button) { const { left, top, right, bottom } = button.boundsInScreen; const centerX = (left + right) / 2; const centerY = (top + bottom) / 2; await agent.actions.tap(centerX, centerY);}
// Or use accessibility action for more reliable clicksawait button.performAction(agent.constants.ACTION_CLICK);
// Scroll a nodeawait scrollableNode.performAction(agent.constants.ACTION_SCROLL_FORWARD);
// Focus an input fieldawait inputField.performAction(agent.constants.ACTION_FOCUS);Screenshots
// Take a screenshot (maxWidth, maxHeight, quality)const screenshot = await agent.actions.screenshot(1080, 1920, 80);
// Result contains base64 image dataconst { base64, width, height } = screenshot;
// Use for debugging or storingconsole.log(`Screenshot: ${width}x${height}`);Common Patterns
Sleep/Wait Function
// Simple sleep functionfunction sleep(ms: number): Promise<void> { return new Promise(resolve => setTimeout(resolve, ms));}
// Sleep with random range (more human-like)function sleepRandom(min: number, max: number): Promise<void> { const ms = Math.floor(Math.random() * (max - min) + min); return new Promise(resolve => setTimeout(resolve, ms));}
// Usageawait sleep(2000); // Wait 2 secondsawait sleepRandom(1000, 3000); // Wait 1-3 seconds randomlyWait for Screen State
// Wait for a specific element to appearasync function waitForElement( condition: (nodes: AndroidNode[]) => boolean, timeout: number = 10000): Promise<boolean> { const startTime = Date.now();
while (Date.now() - startTime < timeout) { const screen = await agent.actions.screenContent(); const allNodes = getAllNodes(screen);
if (condition(allNodes)) { return true; }
await sleep(500); }
return false;}
// Usage: Wait for "Home" text to appearconst found = await waitForElement(nodes => nodes.some(n => n.text === "Home"));Retry with Backoff
async function withRetry<T>( fn: () => Promise<T>, maxAttempts: number = 3, baseDelay: number = 1000): Promise<T> { let lastError: Error | undefined;
for (let attempt = 1; attempt <= maxAttempts; attempt++) { try { return await fn(); } catch (error) { lastError = error as Error; console.log(`Attempt ${attempt} failed: ${lastError.message}`);
if (attempt < maxAttempts) { const delay = baseDelay * Math.pow(2, attempt - 1); await sleep(delay); } } }
throw lastError;}Logging
// Standard console methods are availableconsole.log("Info message");console.warn("Warning message");console.error("Error message");
// Log objectsconsole.log("Screen content:", { nodeCount: allNodes.length });
// Debug with contextconsole.log(`[Stage: ${currentStage}] Processing screen...`);File Operations
// Check if file existsconst exists = agent.utils.files.exists("/sdcard/Download/data.json");
// Read file contentconst content = agent.utils.files.readFullFile("/sdcard/Download/data.json");const data = JSON.parse(content);
// List directoryconst files = agent.utils.files.list("/sdcard/Download");for (const file of files) { console.log(file.name, file.isDirectory);}
// Save file to deviceawait agent.actions.saveFile("result.json", JSON.stringify(data));Network Monitoring
// Track network connectivitylet isOnline = true;
agent.utils.setNetworkCallback((networkAvailable) => { isOnline = networkAvailable; if (!networkAvailable) { console.warn("Network connection lost!"); }});
// Check before network operationsif (!isOnline) { console.log("Waiting for network..."); await sleep(5000);}
// Refresh mobile IP (airplane mode toggle)await agent.actions.airplane();Best Practices
Use Random Delays
Add random delays between actions to appear more human-like. Use sleepRandom(1000, 2000) instead of fixed delays.
Use randomClick for Taps
Instead of tapping exact center of elements, use node.randomClick() to tap at random positions within the element bounds.
Prefer performAction over tap
Use node.performAction() with accessibility actions for more reliable interactions, especially for buttons and inputs.
Handle Unknown Screens
Always have fallback logic for screens you don't recognize. Log unknown states and retry after a short delay.