# Natural Language Actions Source: https://docs.uplink.build/api-reference/ai/act Perform browser actions using natural language instructions with page.act() The `page.act()` method allows you to control the browser using natural language instructions. The AI interprets your instruction, determines what actions to take, and executes them automatically. ## Signature ```typescript theme={null} page.act(instruction: string): Promise ``` **Parameters:** * `instruction` - Natural language description of what you want to do **Returns:** `Promise` ```typescript theme={null} { success: boolean actions: Array<{ type: string // 'click', 'input', 'goto', etc. params: unknown[] result?: unknown error?: string }> reasoning?: string } ``` ## How It Works When you call `page.act()`: 1. AI captures the current page state (DOM, visible elements, form fields, etc.) 2. Your instruction is sent to the AI provider with page context 3. AI determines which actions to perform (click, input, scroll, etc.) 4. Actions are executed sequentially on the page 5. Result is returned with details about what was done ## Basic Examples ### Simple Click Actions ```typescript theme={null} await page.goto('https://example.com') // Click a button await page.act('Click the sign up button') // Click a link await page.act('Click the privacy policy link') // Click by description await page.act('Click the blue submit button at the bottom') ``` ### Form Interactions ```typescript theme={null} // Fill a single field await page.act('Fill in the email field with john@example.com') // Fill multiple fields await page.act('Enter username "johndoe" and password "secret123"') // Select from dropdown await page.act('Select "United States" from the country dropdown') // Check a checkbox await page.act('Check the "I agree to terms" checkbox') ``` ## Intermediate Examples ### Multi-Step Workflows ```typescript theme={null} await page.goto('https://example.com/login') // AI performs multiple actions in sequence await page.act('Fill in the login form with username "john" and password "pass123", then click login') ``` ### Conditional Actions ```typescript theme={null} await page.goto('https://example.com/products') // AI finds and interacts with dynamic elements await page.act('If there is a cookie consent banner, click accept') await page.act('Find the product with the lowest price and click on it') ``` ### Navigation and Scrolling ```typescript theme={null} // Scroll to find elements await page.act('Scroll down to the footer and click the contact link') // Navigate through pages await page.act('Click the next page button') ``` ## Advanced Examples ### Complex Interactions ```typescript theme={null} await page.goto('https://example.com/checkout') // Multi-step form completion const result = await page.act(` Fill out the shipping form with: - Name: John Doe - Address: 123 Main St - City: San Francisco - ZIP: 94102 Then click continue `) if (result.success) { console.log('Form completed successfully') console.log('Actions taken:', result.actions) } ``` ### Dynamic Content Handling ```typescript theme={null} // Wait and interact with loaded content await page.act('Wait for the search results to load, then click the first result') // Handle popups await page.act('If a signup popup appears, close it') // Interact with carousels await page.act('Click the right arrow on the image carousel 3 times') ``` ### Search and Filter ```typescript theme={null} await page.goto('https://example.com/products') await page.act('Enter "laptop" in the search box and press enter') await page.act('Filter results by price: $500-$1000') await page.act('Sort by highest rating') ``` ## Best Practices ### Write Clear Instructions Be specific and descriptive. Instead of "click the button", say "click the blue submit button at the bottom of the form" **Good:** ```typescript theme={null} await page.act('Click the red "Add to Cart" button next to the product image') await page.act('Fill in the email field in the login form with test@example.com') ``` **Bad:** ```typescript theme={null} await page.act('Click it') // Too vague await page.act('Do the thing') // Not specific ``` ### Break Down Complex Tasks For very complex workflows, break them into steps: ```typescript theme={null} // Instead of one giant instruction await page.act('Fill in the form') await page.act('Select shipping method') await page.act('Click continue to payment') ``` ### Handle Errors Check the result and handle failures: ```typescript theme={null} const result = await page.act('Click the checkout button') if (!result.success) { console.error('Action failed:', result.actions) // Try alternative approach or manual automation } ``` ### Combine with Manual Automation Mix AI actions with traditional automation for best results: ```typescript theme={null} // Use AI for dynamic interactions await page.act('Accept cookie consent if it appears') // Use manual automation for precise control await page.click('#precise-selector') await page.input('#email', 'test@example.com') // Use AI for complex extraction await page.act('Find and click the product with best rating') ``` ## Common Use Cases ### Authentication Flows ```typescript theme={null} await page.goto('https://example.com/login') await page.act('Click the "Sign in with Google" button') // Handle OAuth popup... await page.act('Click the authorize button') ``` ### Form Filling ```typescript theme={null} await page.act(` Complete the contact form with: Name: John Doe Email: john@example.com Message: I would like more information Then submit the form `) ``` ### Shopping and E-commerce ```typescript theme={null} await page.goto('https://shop.example.com') await page.act('Search for "wireless headphones"') await page.act('Filter by price under $100') await page.act('Click on the product with the highest rating') await page.act('Select black color and add to cart') ``` ### Content Navigation ```typescript theme={null} await page.goto('https://news.example.com') await page.act('Find and click the article about technology') await page.act('Scroll down and click "Load more comments"') ``` ## Error Handling The `ActResult` includes details about what happened: ```typescript theme={null} const result = await page.act('Click the submit button') if (result.success) { console.log('✓ Action completed successfully') // See what actions were performed result.actions.forEach(action => { console.log(`${action.type}(${action.params.join(', ')})`) }) // View AI reasoning (if available) if (result.reasoning) { console.log('AI reasoning:', result.reasoning) } } else { console.error('✗ Action failed') // Check which step failed const failedAction = result.actions.find(a => a.error) if (failedAction) { console.error('Failed at:', failedAction.type, failedAction.error) } } ``` ## Limitations AI actions are powerful but have limitations: * Require API calls (adds latency and cost) * May not work on heavily obfuscated UIs * Success rate varies by page complexity * Not suitable for precise pixel-level interactions When AI actions fail, fall back to traditional selector-based automation. ## Performance Tips 1. **Use specific instructions** - Reduces AI thinking time 2. **Combine actions** - One instruction with multiple steps is faster than multiple calls 3. **Cache page state** - If performing multiple actions on same page 4. **Use Haiku model** - Faster for simple actions ## Related Extract structured data with page.extract() Configure AI agents and providers Learn effective AI automation patterns Manual interaction methods (click, input) # AI Automation Best Practices Source: https://docs.uplink.build/api-reference/ai/best-practices Guidelines and patterns for effective AI-powered browser automation Learn how to use AI automation effectively by following these best practices and patterns. ## When to Use AI vs Manual Automation ### Use AI When: * **Dynamic UIs** - Elements don't have stable selectors or IDs * **Complex interactions** - Multi-step workflows that are hard to script * **Natural workflows** - Actions easier to describe than code * **Rapid prototyping** - Quick proof-of-concept without precise selectors * **Handling variations** - Page structure varies based on state or user * **Data extraction** - Unstructured content needs to be parsed ### Use Manual Automation When: * **Performance critical** - AI adds 1-3 seconds per call * **High volume** - Thousands of operations (cost adds up) * **Stable selectors** - Elements have reliable IDs or classes * **Exact precision** - Need pixel-perfect control * **Offline scenarios** - No internet or API access * **Simple operations** - Basic click/input on known elements ### Decision Tree ``` Is the UI stable with reliable selectors? ├─ Yes → Use manual automation (click, input, etc.) └─ No → Is performance critical? ├─ Yes → Use manual automation with resilient selectors └─ No → Is cost a concern? ├─ Yes → Use manual for bulk, AI for complex parts └─ No → Use AI automation ``` ## Hybrid Approach (Recommended) The most effective pattern combines both AI and manual automation: ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) const browser = await client.launch() const page = await browser.newPage() // Manual: Fast, reliable navigation await page.goto('https://example.com/products') // AI: Handle dynamic cookie banner await page.act('Accept cookies if a banner appears') // Manual: Precise form filling await page.input('#search', 'wireless headphones') await page.click('#search-button') // AI: Complex filtering await page.act('Filter results to show only items under $100 with free shipping') // AI: Smart extraction const products = await page.extract( 'Extract the first 5 products with name, price, and rating', productsSchema ) // Manual: Precise navigation if (products.success && products.data) { await page.goto(products.data[0].url) } ``` ## Prompt Engineering ### Write Clear, Specific Instructions **Good prompts:** ```typescript theme={null} // Specific location and description await page.act('Click the blue "Sign In" button in the top-right corner') // Include context await page.act('In the shipping form, select "Express Delivery" from the shipping method dropdown') // Describe visual characteristics await page.act('Click the red "Delete" button next to the item named "Widget Pro"') ``` **Bad prompts:** ```typescript theme={null} // Too vague await page.act('Click the button') // Ambiguous await page.act('Do the login thing') // Missing context await page.act('Select the option') ``` ### Break Down Complex Tasks Instead of one massive instruction: ```typescript theme={null} // Bad: Too complex for one instruction await page.act('Login with username admin and password secret, then navigate to settings, update the email to newemail@example.com, and save') // Good: Break into logical steps await page.act('Fill in username "admin" and password "secret", then click login') await page.act('Click the settings link in the navigation') await page.act('Change the email field to newemail@example.com') await page.act('Click the save button') ``` ### Provide Context Help the AI understand the page structure: ```typescript theme={null} // Without context await page.extract('Get the price', priceSchema) // With context (better results) await page.extract( 'Extract the current price shown in the product details section', priceSchema ) ``` ## Error Handling Patterns ### Graceful Fallbacks Always have a backup plan: ```typescript theme={null} // Try AI first let result = await page.act('Click the submit button') if (!result.success) { // Fallback to manual console.log('AI failed, trying manual automation') try { await page.click('#submit-button') } catch (error) { console.error('Both AI and manual failed') throw error } } ``` ### Retry with Different Instructions If extraction fails, try rephrasing: ```typescript theme={null} let result = await page.extract('Get the price', priceSchema) if (!result.success) { // Try a different instruction result = await page.extract( 'Extract the product price shown in USD', priceSchema ) } if (!result.success) { // Fall back to manual extraction const priceText = await page.evaluate(() => document.querySelector('.price')?.textContent ) } ``` ### Validation and Verification Verify AI actions worked as expected: ```typescript theme={null} // Perform action await page.act('Add item to cart') // Verify result const cartCount = await page.evaluate(() => document.querySelector('.cart-count')?.textContent ) if (cartCount === '0') { console.error('Item was not added to cart') // Retry or use manual method } ``` ## Cost Management ### Optimize API Usage AI calls cost money - use them wisely: ```typescript theme={null} // Bad: Multiple separate calls await page.act('Click button A') await page.act('Click button B') await page.act('Click button C') // Good: Combine when possible await page.act('Click buttons A, B, and C in sequence') ``` ### Cache Extracted Data Don't extract the same data repeatedly: ```typescript theme={null} // Extract once const productData = await page.extract( 'Extract all product details', productSchema ) // Reuse the data if (productData.success && productData.data) { console.log(productData.data.name) console.log(productData.data.price) // Don't call extract again for the same data } ``` ### Use Cheaper Models for Simple Tasks ```typescript theme={null} // For simple actions, use Haiku (faster and cheaper) const cheapAgent = ai.createAgent({ provider: 'anthropic', model: 'claude-haiku-4-20241024', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) // Use for simple tasks page.setAgent(cheapAgent) await page.act('Click the agree button') // Switch to Sonnet for complex tasks const smartAgent = ai.createAgent({ provider: 'anthropic', model: 'claude-sonnet-4-5-20250929', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) page.setAgent(smartAgent) await page.extract('Extract complex nested data', complexSchema) ``` ### Batch Operations Process multiple items with fewer AI calls: ```typescript theme={null} // Bad: Extract each product separately (5 AI calls) for (let i = 0; i < 5; i++) { await page.goto(`/product/${i}`) await page.extract('Get product details', productSchema) } // Good: Extract all at once (1 AI call) await page.goto('/products') const result = await page.extract( 'Extract details for the first 5 products', z.object({ products: z.array(productSchema) }) ) ``` ## Testing and Reliability ### Test Against Real Pages Always test with actual websites: ```typescript theme={null} // Test extraction against real page const testUrl = 'https://example.com/product/test-item' const result = await page.extract( 'Extract product details', productSchema ) if (!result.success) { console.error('Extraction failed on test page:', result.error) // Adjust schema or instruction } ``` ### Handle Page Variations Pages may look different based on state: ```typescript theme={null} // Check for variations const isLoggedIn = await page.evaluate(() => !!document.querySelector('.user-profile') ) if (!isLoggedIn) { await page.act('Click the login button') // Handle login... } // Now proceed with main task await page.act('Navigate to dashboard') ``` ### Set Timeouts AI operations can take time: ```typescript theme={null} // Set reasonable timeout for complex extractions try { const result = await Promise.race([ page.extract('Extract data', schema), new Promise((_, reject) => setTimeout(() => reject(new Error('Timeout')), 30000) ) ]) } catch (error) { console.error('Extraction timed out, trying simpler approach') // Fallback strategy } ``` ### Monitor Success Rates Track how often AI automation succeeds: ```typescript theme={null} let successCount = 0 let totalAttempts = 0 async function automateWithTracking() { totalAttempts++ const result = await page.act('Perform action') if (result.success) { successCount++ } console.log(`Success rate: ${(successCount / totalAttempts * 100).toFixed(1)}%`) if (successCount / totalAttempts < 0.8) { console.warn('Success rate below 80%, consider switching to manual automation') } } ``` ## Schema Design ### Start Simple, Then Expand Begin with basic schemas and add complexity as needed: ```typescript theme={null} // Start simple const simpleSchema = z.object({ title: z.string(), price: z.number() }) // Test it let result = await page.extract('Extract product info', simpleSchema) // If successful, expand const detailedSchema = z.object({ title: z.string(), price: z.number(), rating: z.number(), reviews: z.array(z.object({ author: z.string(), text: z.string(), rating: z.number() })) }) ``` ### Use Appropriate Types Match types to the actual data format: ```typescript theme={null} // If price includes currency symbol const schema = z.object({ price: z.string() // "$29.99" }) // If you need numeric price, transform it const schema = z.object({ price: z.string().transform(str => parseFloat(str.replace(/[$,]/g, '')) ) }) // If price is already numeric in HTML const schema = z.object({ price: z.number() // 29.99 }) ``` ### Make Fields Optional When Appropriate Not all data is always present: ```typescript theme={null} const schema = z.object({ // Always present name: z.string(), price: z.number(), // Sometimes present discount: z.number().optional(), originalPrice: z.number().optional(), badge: z.string().optional(), // "Best Seller", "New", etc. // May be null reviewCount: z.number().nullable(), }) ``` ## Performance Optimization ### Minimize Page Context The AI analyzes page content - simpler pages = faster processing: ```typescript theme={null} // Remove unnecessary elements before extraction await page.evaluate(() => { // Hide ads, footers, etc. to reduce context document.querySelectorAll('.ad, footer, .sidebar').forEach(el => { el.style.display = 'none' }) }) // Now extract (faster with less content) const result = await page.extract('Extract article', schema) ``` ### Parallel Execution Run independent operations in parallel: ```typescript theme={null} // Sequential (slow) await page.act('Accept cookies') await page.act('Close popup') // Parallel (faster) await Promise.all([ page.act('Accept cookies if present'), page.act('Close popup if present') ]) ``` ### Pre-navigate and Cache Navigate ahead of time: ```typescript theme={null} // Pre-load pages const pages = await Promise.all([ browser.newPage(), browser.newPage(), browser.newPage() ]) await Promise.all([ pages[0].goto('https://example.com/page1'), pages[1].goto('https://example.com/page2'), pages[2].goto('https://example.com/page3') ]) // Now extract from all pages in parallel const results = await Promise.all([ pages[0].extract('Extract data', schema), pages[1].extract('Extract data', schema), pages[2].extract('Extract data', schema) ]) ``` ## Security Considerations ### Protect API Keys Never expose API keys in code: ```typescript theme={null} // Bad: Hardcoded key const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: 'sk-ant-api03-...' // DON'T DO THIS } }) // Good: Environment variables const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) ``` ### Sanitize Instructions If instructions come from user input, sanitize them: ```typescript theme={null} // If instruction includes user input const userQuery = getUserInput() // Validate and sanitize const sanitized = userQuery .trim() .substring(0, 500) // Limit length .replace(/[<>]/g, '') // Remove potentially harmful chars await page.act(`Search for ${sanitized}`) ``` ### Review AI Actions in Development Log actions during development to verify behavior: ```typescript theme={null} const result = await page.act('Perform action') // Review what AI did console.log('Actions taken:') result.actions.forEach(action => { console.log(`- ${action.type}(${JSON.stringify(action.params)})`) }) ``` ## Complete Example Here's a complete example following all best practices: ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' import { z } from 'zod' // Validate environment if (!process.env.ANTHROPIC_API_KEY) { throw new Error('ANTHROPIC_API_KEY required') } // Create agent const agent = ai.createAgent({ provider: 'anthropic', model: 'claude-sonnet-4-5-20250929', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) // Define schema const productSchema = z.object({ name: z.string(), price: z.number(), rating: z.number().optional(), inStock: z.boolean() }) const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) try { const browser = await client.launch() const page = await browser.newPage() // Manual navigation (fast) await page.goto('https://shop.example.com') // AI for dynamic content (flexible) await page.act('Accept cookies if a banner appears') // Manual search (precise) await page.input('#search', 'wireless headphones') await page.click('#search-submit') // Wait for results await page.waitForSelector('.search-results') // AI extraction (smart) const result = await page.extract( 'Extract the first 5 products with name, price, rating, and stock status', z.object({ products: z.array(productSchema) }) ) if (result.success && result.data) { // Process results result.data.products.forEach(product => { console.log(`${product.name}: $${product.price}`) }) } else { // Fallback to manual console.error('AI extraction failed, using manual method') const products = await page.evaluate(() => { return Array.from(document.querySelectorAll('.product')).map(el => ({ name: el.querySelector('.name')?.textContent, price: parseFloat(el.querySelector('.price')?.textContent || '0') })) }) } await page.close() await browser.close() } finally { await client.close() } ``` ## Related Learn about page.act() Learn about page.extract() Configure AI agents Manual automation methods # Type-Safe Data Extraction Source: https://docs.uplink.build/api-reference/ai/extract Extract structured data from web pages using AI with Zod schema validation The `page.extract()` method uses AI to extract structured data from web pages. You can optionally provide a Zod schema for type-safe extraction with automatic validation. ## Signature ```typescript theme={null} // With schema (type-safe) page.extract( instruction: string, schema: z.ZodType ): Promise> // Without schema (string extraction) page.extract( instruction: string ): Promise> ``` **Parameters:** * `instruction` - Natural language description of what data to extract * `schema` (optional) - Zod schema for validation and type safety **Returns:** `Promise>` ```typescript theme={null} { success: boolean data?: T // Typed based on your schema error?: string reasoning?: string // AI's explanation } ``` ## Installation You'll need to install Zod for schema-based extraction: ```bash theme={null} npm install zod ``` ## Basic Examples ### Simple Text Extraction Extract text without a schema: ```typescript theme={null} await page.goto('https://example.com/article') const result = await page.extract('Extract the article title') if (result.success) { console.log(result.data.extraction) // "How to Build Better Software" } ``` ### Single Object Extraction Extract a structured object with type safety: ```typescript theme={null} import { z } from 'zod' const productSchema = z.object({ name: z.string(), price: z.number(), inStock: z.boolean() }) await page.goto('https://shop.example.com/product/123') const result = await page.extract( 'Extract the product information', productSchema ) if (result.success && result.data) { console.log(result.data.name) // string console.log(result.data.price) // number console.log(result.data.inStock) // boolean // TypeScript knows the exact types! } ``` ## Intermediate Examples ### Multiple Fields Extract complex objects with many fields: ```typescript theme={null} const userSchema = z.object({ name: z.string(), email: z.string().email(), joinDate: z.string(), verified: z.boolean(), profileImage: z.string().url() }) await page.goto('https://example.com/profile/johndoe') const result = await page.extract( 'Extract user profile information', userSchema ) if (result.success && result.data) { const user = result.data console.log(`${user.name} (${user.email})`) console.log(`Joined: ${user.joinDate}`) console.log(`Verified: ${user.verified}`) } ``` ### Optional Fields Handle optional data with Zod: ```typescript theme={null} const articleSchema = z.object({ title: z.string(), author: z.string(), publishDate: z.string(), updatedDate: z.string().optional(), // May not be present tags: z.array(z.string()).optional(), readTime: z.number().optional() }) const result = await page.extract( 'Extract article metadata', articleSchema ) ``` ## Advanced Examples ### Array Extraction Extract lists of items: ```typescript theme={null} const searchResultsSchema = z.object({ query: z.string(), results: z.array(z.object({ title: z.string(), url: z.string().url(), description: z.string(), price: z.number().optional() })), totalResults: z.number() }) await page.goto('https://example.com/search?q=laptop') const result = await page.extract( 'Extract all search results with their details', searchResultsSchema ) if (result.success && result.data) { console.log(`Found ${result.data.totalResults} results for "${result.data.query}"`) result.data.results.forEach(item => { console.log(`${item.title} - $${item.price}`) console.log(item.url) }) } ``` ### Nested Objects Extract complex nested data structures: ```typescript theme={null} const restaurantSchema = z.object({ name: z.string(), rating: z.number(), priceLevel: z.string(), address: z.object({ street: z.string(), city: z.string(), state: z.string(), zip: z.string() }), hours: z.object({ monday: z.string(), tuesday: z.string(), wednesday: z.string(), thursday: z.string(), friday: z.string(), saturday: z.string(), sunday: z.string() }), reviews: z.array(z.object({ author: z.string(), rating: z.number(), text: z.string(), date: z.string() })) }) const result = await page.extract( 'Extract complete restaurant information including address, hours, and recent reviews', restaurantSchema ) ``` ### Table Extraction Extract data from HTML tables: ```typescript theme={null} const tableSchema = z.object({ headers: z.array(z.string()), rows: z.array(z.array(z.string())) }) await page.goto('https://example.com/data') const result = await page.extract( 'Extract the pricing table data', tableSchema ) if (result.success && result.data) { // Print as CSV console.log(result.data.headers.join(',')) result.data.rows.forEach(row => { console.log(row.join(',')) }) } ``` ## Zod Schema Primer ### Basic Types ```typescript theme={null} import { z } from 'zod' z.string() // string z.number() // number z.boolean() // boolean z.date() // Date object z.string().url() // URL string z.string().email() // Email string z.string().uuid() // UUID string z.literal('specific') // Exact value z.enum(['a', 'b', 'c']) // One of several values ``` ### Optional and Nullable ```typescript theme={null} z.string().optional() // string | undefined z.string().nullable() // string | null z.string().nullish() // string | null | undefined z.string().default('hello') // string with default value ``` ### Arrays and Objects ```typescript theme={null} z.array(z.string()) // string[] z.object({ // { name: string, age: number } name: z.string(), age: z.number() }) ``` ### Validation ```typescript theme={null} z.string().min(3) // At least 3 characters z.string().max(100) // At most 100 characters z.number().positive() // Must be positive z.number().int() // Must be integer z.number().min(0).max(100) // Between 0 and 100 ``` ## Error Handling Handle validation errors gracefully: ```typescript theme={null} const schema = z.object({ price: z.number().positive(), email: z.string().email() }) const result = await page.extract( 'Extract product price and contact email', schema ) if (result.success && result.data) { // Data is valid and typed console.log('Price:', result.data.price) console.log('Email:', result.data.email) } else { // Extraction failed or validation failed console.error('Extraction error:', result.error) // See AI reasoning if (result.reasoning) { console.log('AI reasoning:', result.reasoning) } } ``` ## Best Practices ### Be Specific in Instructions Clear instructions lead to better extraction results **Good:** ```typescript theme={null} await page.extract( 'Extract the product name, price in USD, and availability status from the product details section', productSchema ) ``` **Bad:** ```typescript theme={null} await page.extract('Get the info', productSchema) // Too vague ``` ### Design Schemas Carefully Match your schema to the actual data structure: ```typescript theme={null} // If prices include currency symbols const priceSchema = z.string() // "$29.99" // Not: z.number() (would fail validation) // Or extract and parse const priceSchema = z.string().transform(str => parseFloat(str.replace(/[$,]/g, '')) ) ``` ### Handle Missing Data Use optional fields for data that might not be present: ```typescript theme={null} const schema = z.object({ title: z.string(), // These might not always be present subtitle: z.string().optional(), author: z.string().optional(), rating: z.number().optional() }) ``` ### Test with Real Pages Always test extraction with actual pages: ```typescript theme={null} // Test extraction const result = await page.extract(instruction, schema) if (!result.success) { console.error('Extraction failed:', result.error) // Adjust instruction or schema } ``` ## Common Use Cases ### E-commerce Product Data ```typescript theme={null} const productSchema = z.object({ name: z.string(), brand: z.string(), price: z.number(), originalPrice: z.number().optional(), discount: z.number().optional(), rating: z.number(), reviewCount: z.number(), inStock: z.boolean(), images: z.array(z.string().url()), description: z.string() }) const result = await page.extract( 'Extract all product details', productSchema ) ``` ### Article Metadata ```typescript theme={null} const articleSchema = z.object({ headline: z.string(), subheadline: z.string().optional(), author: z.string(), publishDate: z.string(), readingTime: z.number(), tags: z.array(z.string()), summary: z.string() }) const result = await page.extract( 'Extract article metadata and summary', articleSchema ) ``` ### Contact Information ```typescript theme={null} const contactSchema = z.object({ name: z.string(), email: z.string().email().optional(), phone: z.string().optional(), address: z.string().optional(), website: z.string().url().optional(), socialMedia: z.object({ twitter: z.string().optional(), linkedin: z.string().optional(), facebook: z.string().optional() }).optional() }) const result = await page.extract( 'Extract contact information from the page', contactSchema ) ``` ### Reviews and Ratings ```typescript theme={null} const reviewsSchema = z.object({ overallRating: z.number(), totalReviews: z.number(), reviews: z.array(z.object({ author: z.string(), rating: z.number(), title: z.string(), text: z.string(), date: z.string(), helpful: z.number().optional() })) }) const result = await page.extract( 'Extract product reviews and ratings', reviewsSchema ) ``` ## Performance Tips 1. **Be specific** - Clear instructions reduce processing time 2. **Use appropriate schemas** - Don't over-complicate schemas 3. **Extract once** - Cache results instead of re-extracting 4. **Batch extraction** - Extract multiple fields at once rather than separate calls ## Limitations AI extraction has some limitations: * Requires API calls (adds latency and cost) * May not work on obfuscated or heavily JavaScript-rendered content * Accuracy depends on page structure and instruction clarity * Rate limits apply based on your AI provider ## Related Perform actions with page.act() Configure AI agents and providers Effective AI automation patterns Manual data extraction with evaluate() # AI Automation Overview Source: https://docs.uplink.build/api-reference/ai/overview Use AI to automate mobile browsers with natural language instructions The Uplink AI package enables you to control browsers using natural language instructions powered by large language models (LLMs). Instead of writing selectors and detailed automation scripts, you can describe what you want to do in plain English. ## Installation The AI package is optional and must be installed separately: ```bash theme={null} npm install @uplink-code/ai ``` You'll also need an API key from a supported AI provider (Anthropic or OpenAI). ## When to Use AI Automation AI automation is ideal when: * **Dynamic UIs**: The page structure changes frequently or unpredictably * **Complex interactions**: Multi-step workflows that are hard to script manually * **Data extraction**: Extracting structured information from unstructured content * **Natural workflows**: Actions that are easier to describe than to code Traditional selector-based automation works better when: * **Performance is critical**: AI calls add latency and cost * **Exact control needed**: You need pixel-perfect precision * **Stable selectors**: The page structure is predictable and stable * **Offline automation**: No internet connection or API access ## Quick Example Here's how AI automation works: ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' // Create an AI agent const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) // Connect with the agent const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) const browser = await client.launch() const page = await browser.newPage() await page.goto('https://example.com/products') // Use natural language to perform actions await page.act('Find the cheapest laptop and add it to cart') // Extract structured data with type safety import { z } from 'zod' const productSchema = z.object({ name: z.string(), price: z.number(), inStock: z.boolean() }) const result = await page.extract( 'Get the product details from this page', productSchema ) console.log(result.data) // { name: "ThinkPad X1", price: 1299.99, inStock: true } ``` ## Key Features ### Natural Language Actions The [`page.act()`](./act) method interprets your instruction and performs the necessary browser actions automatically: ```typescript theme={null} await page.act('Click the sign in button') await page.act('Fill in the email field with john@example.com') await page.act('Select United States from the country dropdown') ``` ### Type-Safe Data Extraction The [`page.extract()`](./extract) method extracts structured data from pages using AI, with optional Zod schema validation for type safety: ```typescript theme={null} const userSchema = z.object({ name: z.string(), email: z.string().email(), posts: z.array(z.object({ title: z.string(), date: z.string() })) }) const result = await page.extract( 'Extract user profile information including their recent posts', userSchema ) // result.data is fully typed! ``` ## Supported Providers * **Anthropic** - Claude models (fully supported) * **OpenAI** - GPT models (coming soon) ## How It Works When you call `page.act()` or `page.extract()`: 1. The AI agent captures a snapshot of the current page state 2. Your instruction is sent to the AI provider along with page context 3. The AI analyzes the page and determines what actions to take 4. For `act()`: Actions are executed automatically on the page 5. For `extract()`: Data is extracted and validated against your schema ## Next Steps Configure AI agents with Anthropic or OpenAI Perform browser actions with natural language Extract structured data with type safety Learn effective patterns for AI automation # AI Agent Setup Source: https://docs.uplink.build/api-reference/ai/setup Configure AI agents for browser automation with Anthropic or OpenAI To use AI features in Uplink, you need to create an AI agent and configure it with your preferred provider. ## Installation First, install the AI package: ```bash theme={null} npm install @uplink-code/ai ``` ## Creating an Agent Use the `ai.createAgent()` factory function to create an agent: ```typescript theme={null} import ai from '@uplink-code/ai' const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) ``` ## Provider Configuration ### Anthropic (Claude) Anthropic's Claude models are fully supported and recommended for browser automation. ```typescript theme={null} import ai from '@uplink-code/ai' const agent = ai.createAgent({ provider: 'anthropic', model: 'claude-sonnet-4-5-20250929', // optional, this is the default maxTokens: 4096, // optional options: { apiKey: process.env.ANTHROPIC_API_KEY, // Additional Anthropic SDK options baseURL: 'https://api.anthropic.com', // optional defaultHeaders: {} // optional } }) ``` **Get an API key**: Sign up at [console.anthropic.com](https://console.anthropic.com) and create an API key. **Available models**: * `claude-sonnet-4-5-20250929` (default) - Best balance of speed and intelligence * `claude-opus-4-5-20251101` - Most capable, slower and more expensive * `claude-haiku-4-20241024` - Fastest and cheapest, less capable ### OpenAI (GPT) OpenAI support is coming soon and not yet available. ```typescript theme={null} // Coming soon const agent = ai.createAgent({ provider: 'openai', model: 'gpt-4', maxTokens: 4096, options: { apiKey: process.env.OPENAI_API_KEY } }) ``` ## Using the Agent You can set the agent at two levels: ### Client-Level Agent (Recommended) Set the agent when connecting to apply it to all browsers and pages: ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) // Agent applies to all browsers and pages const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) const browser = await client.launch() const page = await browser.newPage() // AI methods now available await page.act('Click the login button') ``` ### Page-Level Agent Override the agent for a specific page: ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) const browser = await client.launch() const page = await browser.newPage() // Set agent on this page only const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) page.setAgent(agent) // AI methods now available on this page await page.act('Fill in the form') ``` ## Environment Variables Store API keys in environment variables for security: ```bash theme={null} # .env ANTHROPIC_API_KEY=sk-ant-api03-... ``` Access them in your code: ```typescript theme={null} const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) ``` Never commit API keys to version control. Use environment variables or a secrets manager. ## Complete Setup Example Here's a complete example with environment variables and error handling: ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' import 'dotenv/config' // Load .env file // Validate API key if (!process.env.ANTHROPIC_API_KEY) { throw new Error('ANTHROPIC_API_KEY environment variable is required') } // Create agent const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) // Connect with agent const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) try { const browser = await client.launch() const page = await browser.newPage() await page.goto('https://example.com') await page.act('Click the get started button') await page.close() await browser.close() } finally { await client.close() } ``` ## Cost Considerations AI automation makes API calls to your chosen provider, which incurs costs: * **Anthropic Claude**: \~\$3-15 per 1M input tokens depending on model * **OpenAI GPT**: \~\$2.50-30 per 1M input tokens depending on model Each `act()` or `extract()` call sends page context (usually 1-5K tokens) plus your instruction. **Tips to reduce costs**: * Use Haiku model for simple tasks * Use `act()` sparingly for complex interactions only * Combine multiple actions into one instruction when possible * Cache results when extracting the same data repeatedly ## Next Steps Use page.act() to perform actions with AI Extract structured data with page.extract() # Android SDK Source: https://docs.uplink.build/api-reference/android Integrate the Uplink native SDK into your Android app The Uplink Android SDK embeds a worker inside your Android app, making the device controllable from server-side code written with the [JavaScript SDK](/api-reference/overview). Use it when you want Uplink automation to run inside your own branded app. **Key concepts:** * **Device**: A physical Android device or emulator running your app * **Worker**: A worker created by the Uplink SDK inside your app (one app can create multiple workers) * **Session**: An authenticated connection that pairs your worker with a JavaScript SDK client ## Requirements * Android API 21 (Android 5.0) or later * Kotlin 2.1 or later * Java 11 or later ## Installation The Android SDK is published to Maven Central as `build.uplink:core-android`. Ensure Maven Central is available to your build. In your root `settings.gradle.kts` (or `build.gradle.kts`): ```kotlin theme={null} dependencyResolutionManagement { repositories { mavenCentral() } } ``` In your app module's `build.gradle.kts`: ```kotlin theme={null} dependencies { implementation("build.uplink:core-android:0.0.0-alpha16") } ``` Check [Maven Central](https://central.sonatype.com/artifact/build.uplink/core-android) for the latest version. In `AndroidManifest.xml`: ```xml theme={null} ``` ## Quick example ```kotlin theme={null} import android.view.ViewGroup import androidx.activity.ComponentActivity import androidx.lifecycle.lifecycleScope import build.uplink.worker import kotlinx.coroutines.launch suspend fun startUplink(activity: ComponentActivity, container: ViewGroup, sessionUrl: String) { val worker = worker(activity, container) worker.connect(sessionUrl) activity.lifecycleScope.launch { worker.accept() } } ``` ## Initialization ### `worker(context, group)` Creates a new `Worker` and mounts the SDK's `WebView` into the provided `ViewGroup`. ```kotlin theme={null} fun worker(context: Context, group: ViewGroup): Worker ``` **Parameters:** * `context`: An Android `Context` — typically your `Activity` * `group`: The `ViewGroup` that will host the SDK's web view **Returns:** A new `Worker` ready to connect to a session ```kotlin theme={null} val worker = worker(activity, container) ``` The `ViewGroup` you pass in must remain in the view hierarchy for the lifetime of the worker. Tearing down the containing Activity or Fragment will close the worker. ## Obtaining a session Your backend creates a session using the JavaScript SDK's `uplink.session()` and delivers the resulting session URL to the app. How you deliver it — a backend API response, an app link, a QR code — is up to you. ```kotlin theme={null} val sessionUrl: String = // from your backend, deep link, or QR code ``` See [Sessions](/sessions) for how sessions are created on the server side. ## Connecting and accepting ### `worker.connect(session)` Connects the worker to an Uplink session. ```kotlin theme={null} suspend fun connect(session: String) ``` **Parameters:** * `session`: The session URL provided by your backend ```kotlin theme={null} worker.connect(sessionUrl) ``` ### `worker.accept()` Begins accepting commands from the JavaScript SDK client. `accept()` suspends until the worker closes, so launch it in a coroutine scope that matches your app's lifecycle. ```kotlin theme={null} suspend fun accept() ``` ```kotlin theme={null} lifecycleScope.launch { worker.accept() } ``` ## Cleanup ### `worker.close()` Closes the worker and releases its resources. ```kotlin theme={null} suspend fun close() ``` ```kotlin theme={null} worker.close() ``` Closing a worker ends its session. Any browsers the JavaScript SDK has launched on the worker will be terminated. ## Complete example A minimal `Activity` that starts an Uplink worker and tears it down on destroy: ```kotlin theme={null} import android.os.Bundle import android.view.ViewGroup import android.widget.FrameLayout import androidx.activity.ComponentActivity import androidx.lifecycle.lifecycleScope import build.uplink.worker import build.uplink.worker.Worker import kotlinx.coroutines.launch class UplinkActivity : ComponentActivity() { private var worker: Worker? = null override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) val container = FrameLayout(this) setContentView(container, ViewGroup.LayoutParams(MATCH_PARENT, MATCH_PARENT)) val sessionUrl = intent.getStringExtra("sessionUrl") ?: return val worker = worker(this, container).also { this.worker = it } lifecycleScope.launch { try { worker.connect(sessionUrl) worker.accept() } catch (e: Exception) { println("Uplink worker ended: $e") } } } override fun onDestroy() { super.onDestroy() lifecycleScope.launch { worker?.close() } } companion object { private const val MATCH_PARENT = ViewGroup.LayoutParams.MATCH_PARENT } } ``` ## Related Integrate Uplink into your iOS app Control your worker from server-side code How sessions are created and secured Architecture overview # Browser Source: https://docs.uplink.build/api-reference/browser API reference for the Browser class The `Browser` class represents a browser instance running on a mobile device. It provides methods for managing pages (tabs) within the browser. ## Page operations ### `browser.newPage()` Creates a new page (tab) in the browser. ```typescript theme={null} browser.newPage(): Promise ``` **Returns:** `Promise` - New page instance **Example:** ```typescript theme={null} const browser = await client.launch() const page = await browser.newPage() await page.goto('https://example.com') ``` ### `browser.pages()` Lists all pages currently open in the browser. ```typescript theme={null} browser.pages(): Promise ``` **Returns:** `Promise` - Array of page instances **Example:** ```typescript theme={null} const pages = await browser.pages() console.log(`${pages.length} pages open`) for (const page of pages) { const url = await page.url() console.log('Page URL:', url) } ``` ## Browser management ### `browser.close()` Closes the browser and all its pages. ```typescript theme={null} browser.close(): Promise ``` **Returns:** `Promise` **Example:** ```typescript theme={null} await browser.close() console.log('Browser closed') ``` Always close browsers when done to free up device resources. Closing a browser automatically closes all its pages. ## Properties ### `browser.handle` The unique handle identifier for this browser instance. Use this handle to reconnect to the browser later. ```typescript theme={null} browser.handle: string ``` **Example:** ```typescript theme={null} const browser = await client.launch() console.log('Browser handle:', browser.handle) // Save handle for later reconnection const handle = browser.handle // Later, reconnect to the same browser const sameBrowser = await client.connect(handle) ``` ## Complete examples ### Basic browser usage ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) const browser = await client.launch() try { // Create a page and navigate const page = await browser.newPage() await page.goto('https://example.com') console.log('Page loaded successfully') await page.close() } finally { // Always close the browser await browser.close() } ``` ### Multiple pages ```typescript theme={null} const browser = await client.launch() // Open multiple pages const page1 = await browser.newPage() const page2 = await browser.newPage() const page3 = await browser.newPage() // Navigate in parallel await Promise.all([ page1.goto('https://example.com'), page2.goto('https://google.com'), page3.goto('https://github.com') ]) // Get page titles const titles = await Promise.all([ page1.evaluate(() => document.title), page2.evaluate(() => document.title), page3.evaluate(() => document.title) ]) console.log('Page titles:', titles) // Close all pages const pages = await browser.pages() await Promise.all(pages.map(p => p.close())) await browser.close() ``` ### Switching between pages ```typescript theme={null} const browser = await client.launch() const page1 = await browser.newPage() const page2 = await browser.newPage() await page1.goto('https://example.com') await page2.goto('https://google.com') // Show page1 (bring to foreground) await page1.show() console.log('Showing page 1') // Wait a bit await new Promise(resolve => setTimeout(resolve, 2000)) await page1.hide() // Switch to page2 await page2.show() console.log('Showing page 2') await browser.close() ``` ### Browser reconnection ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) // Launch and save handle const browser = await client.launch() const handle = browser.handle console.log('Browser handle:', handle) const page = await browser.newPage() await page.goto('https://example.com') // Later, reconnect to the same browser const reconnectedBrowser = await client.connect(handle) // Browser state is preserved const pages = await reconnectedBrowser.pages() console.log(`Reconnected browser has ${pages.length} page(s)`) const existingPage = pages[0] const url = await existingPage.url() console.log('Page still on:', url) await reconnectedBrowser.close() ``` ## Related Page automation methods Launching browsers Device-level browser operations Architecture overview # Client Source: https://docs.uplink.build/api-reference/client API reference for the Uplink Client class The `Client` class is your main entry point for Uplink automation. It manages the WebSocket connection to the Uplink relay server and provides methods for worker and browser management. **Key concepts:** * **Device**: A physical iOS or Android device * **Worker**: A worker created using the native Uplink SDK (a device can create multiple workers) * **Address**: A hex-encoded identifier for workers ## Connection ### `uplink.session()` Creates an Uplink session using your project credentials. ```typescript theme={null} uplink.session( auth: Auth, options?: SessionOptions ): Promise ``` **Parameters:** * `auth`: Your OAuth client credentials from Uplink Console — `{ clientId, clientSecret }`. The session is automatically scoped to the project the client belongs to. See [Authentication](/fundamentals/authentication), or the [Auth](#auth) type below for the accepted shapes. * `options` (optional): Session configuration * `include`: Cryptographic key options * `ecdsa`: Boolean, include ECDSA keys (default: false) * `ecdh`: Boolean, include ECDH keys (default: false) * `restrict`: `{ verifier: string }` — a secret value that binds the session to your code, so that holding the session's URL is not enough to drive it. Sessions are restricted either way: if you omit this, a verifier is generated for you. Supply your own only when you need to reproduce it in another runtime. See [Session restriction](/sessions#session-restriction). **Returns:** `Promise` - A [Session](#session), with the shape `{ sessionId, sessionUrl, qrUrl, credential, keys? }`. `credential` is the verifier — generated here if you did not supply one. Pass the object directly to [`uplink.client.fromSession()`](#uplinkclientfromsession) to connect. **Example:** ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) ``` **Example — choosing your own verifier:** ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true }, restrict: { verifier: process.env.UPLINK_SESSION_VERIFIER } } ) ``` Uplink stores only a challenge derived from the verifier, never the verifier itself. It cannot be recovered from the API — so if you plan to reconnect from a different runtime, either persist `session.credential` or supply a verifier you can read again. ### `uplink.getSession()` Looks up an existing session by ID and returns the same `Session` shape that `uplink.session()` returns — including a fresh `sessionUrl`. Useful when you already have a `sessionId` (for example, persisted across runs) and want to keep using the original session instead of creating a new one, or recover the keys that were generated when it was created. ```typescript theme={null} uplink.getSession( auth: Auth, sessionId: string, options?: GetSessionOptions ): Promise ``` **Parameters:** * `auth`: Your OAuth client credentials from Uplink Console — `{ clientId, clientSecret }` * `sessionId`: The ID of an existing session belonging to the client's project * `options` (optional): * `include`: Cryptographic key options * `ecdsa`: Boolean, include the project ECDSA key pair (default: false) * `ecdh`: Boolean, include the session ECDH key pair if one was created at session-create time (default: false) **Returns:** `Promise` - Same shape as `uplink.session()`, but **never** with a `credential`. Uplink does not store the verifier, so it cannot return one. Merge yours back in before passing the session to `fromSession()`. You only need to worry about merging `credential` when you're picking a session up later through `getSession()`. If you call `fromSession()` directly with the object returned by `uplink.session()`, the credential is already attached for you. See [Reconnecting from another runtime](/sessions#reconnecting-from-another-runtime). The `include.ecdh` / `include.ecdsa` flags only re-export keys that **Uplink generated** for the session. If you brought your own key pair instead, `getSession()` won't return anything in `keys` — you're expected to re-attach your own. **Example:** ```typescript theme={null} import uplink from '@uplink-code/uplink' // The verifier for this session: either the `credential` you persisted when you // created it, or the one you passed as `restrict.verifier` at create time. const credential = process.env.UPLINK_SESSION_VERIFIER const session = await uplink.getSession( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, '', { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession({ ...session, credential }) ``` Returns `404` if the session does not exist or does not belong to the client's project. ### `uplink.sessionDetails()` Fetches a richer view of an existing session for display in dashboards or admin views — including paired devices, total bytes streamed, total connection duration, tags, and the parent project and organization. Use this when you need to inspect a session, not to reconnect to it. ```typescript theme={null} uplink.sessionDetails( auth: Auth, sessionId: string, options?: { host?: string } ): Promise ``` **Parameters:** * `auth`: Your OAuth client credentials from Uplink Console — `{ clientId, clientSecret }` * `sessionId`: The ID of an existing session belonging to the client's project **Returns:** `Promise` - Session metadata. See the [SessionDetails](#sessiondetails) type below. **Example:** ```typescript theme={null} import uplink from '@uplink-code/uplink' const details = await uplink.sessionDetails( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, '' ) console.log('Devices paired:', details.devices.length) console.log('Bytes streamed:', details.total_bytes) console.log('Connection time (s):', details.total_connection_duration) ``` ### `uplink.token()` Exchanges your OAuth client credentials for an access token. You don't normally need this — passing credentials directly to the methods above lets the SDK obtain and renew tokens for you. Reach for it when you want to hold the token yourself, for example to share one across processes. ```typescript theme={null} uplink.token( auth: Auth, options?: { host?: string; signal?: AbortSignal } ): Promise ``` **Parameters:** * `auth`: Your OAuth client credentials from Uplink Console — `{ clientId, clientSecret }` * `options` (optional): * `host`: Override the Uplink API host * `signal`: An `AbortSignal` to cancel the exchange **Returns:** `Promise` - `{ token, expiresAt }`. Pass it anywhere an `Auth` is accepted. **Example:** ```typescript theme={null} import uplink from '@uplink-code/uplink' const token = await uplink.token({ clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }) const session = await uplink.session(token, { include: { ecdsa: true, ecdh: true } }) const details = await uplink.sessionDetails(token, session.sessionId) ``` A token you obtained yourself is **not** renewed for you — only credentials the SDK holds are. Once `expiresAt` passes, calls made with the token fail; call `uplink.token()` again with your client credentials to get a new one. **When to reach for which method:** * `uplink.session()` — create a brand-new session. * `uplink.getSession()` — refetch an existing session in the same shape as `session()` so you can pass it to `fromSession()`. * `uplink.sessionDetails()` — inspect a session's metadata (devices, usage, tags). Not for reconnecting. * `uplink.token()` — mint an access token to hold and reuse yourself, instead of letting the SDK manage one. ### `uplink.client.fromSession()` Creates a client from an Uplink session. ```typescript theme={null} uplink.client.fromSession( session: Session, options?: ClientOptions ): Promise ``` **Parameters:** * `session`: A [Session](#session) from `uplink.session()`, or one from `uplink.getSession()` with its `credential` merged back in. `session.credential` is sent to the relay as the connection's proof that it owns the session; `session.keys` are re-imported for you. * `options` (optional): Connection options * `agent`: Optional AI agent for natural language automation (requires `@uplink-code/ai`) **Returns:** `Promise` - Connected client instance **Complete example:** ```typescript theme={null} import uplink from '@uplink-code/uplink' const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) console.log('Connected to Uplink') ``` **With AI agent:** ```typescript theme={null} import uplink from '@uplink-code/uplink' import ai from '@uplink-code/ai' const agent = ai.createAgent({ provider: 'anthropic', options: { apiKey: process.env.ANTHROPIC_API_KEY } }) const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session, { agent }) // All pages created from this client will have AI capabilities const browser = await client.launch() const page = await browser.newPage() await page.goto('https://example.com') await page.act('Click the sign in button') // AI-powered action ``` ### `uplink.client.connect()` (Alternative) Connects directly to an Uplink session via WebSocket URL. This is an alternative to using `uplink.session()` + `fromSession()`. ```typescript theme={null} uplink.client.connect( url: string | URL, options?: ClientOptions ): Promise ``` **Parameters:** * `url`: WebSocket URL in the format `wss://relay.uplink.build/session/` * `options` (optional): Connection options * `agent`: Optional AI agent for natural language automation (requires `@uplink-code/ai`) **Returns:** `Promise` - Connected client instance **Example:** ```typescript theme={null} import uplink from '@uplink-code/uplink' const client = await uplink.client.connect( 'wss://relay.uplink.build/session/' ) console.log('Connected to Uplink') ``` **Recommended approach**: Use `uplink.session()` + `fromSession()` for better credential management and project organization. Use `connect()` when you need to work with pre-generated session URLs. ## Browser operations ### `client.launch()` Launches a new browser on a worker. If no worker address is provided, uses the first available worker. ```typescript theme={null} client.launch(address?: Address): Promise ``` **Parameters:** * `address` (optional): Worker address to launch browser on **Returns:** `Promise` - New browser instance **Example:** ```typescript theme={null} // Launch on first available worker const browser = await client.launch() // Launch on specific worker using its address const workers = await client.workers() const browser = await client.launch(workers[0].address) ``` ### `client.connect()` Connects to an existing browser by its handle. ```typescript theme={null} client.connect(handle: string, address?: Address): Promise ``` **Parameters:** * `handle`: Browser handle identifier * `address` (optional): Worker address where browser is running **Returns:** `Promise` - Connected browser instance **Example:** ```typescript theme={null} // Connect to browser by handle (searches all workers) const workers = await client.workers() const browsers = await workers[0].browsers() const existingBrowser = await client.connect(browsers[0].handle) const page = await existingBrowser.newPage() ``` ### `client.browsers()` Lists all browsers on a worker. ```typescript theme={null} client.browsers(address?: Address): Promise ``` **Parameters:** * `address` (optional): Worker address to query. If not provided, queries first available worker. **Returns:** `Promise` - Array of browser instances **Example:** ```typescript theme={null} // List browsers on first available worker const browsers = await client.browsers() console.log(`${browsers.length} browsers running`) for (const browser of browsers) { console.log('Browser:', browser.handle) } ``` ## Worker operations ### `client.workers()` Returns list of currently connected workers. ```typescript theme={null} client.workers(): ClientWorker[] ``` **Returns:** `ClientWorker[]` - Array of connected workers **Example:** ```typescript theme={null} const workers = client.workers() console.log(`${workers.length} workers connected`) workers.forEach(worker => { console.log('Worker address:', worker.address) }) ``` ### `client.terminate()` Terminates a worker connection. ```typescript theme={null} client.terminate(address?: Address): Promise ``` **Parameters:** * `address` (optional): Worker address to terminate. If not provided, terminates first available worker. **Returns:** `Promise` **Example:** ```typescript theme={null} // Terminate first available worker await client.terminate() // Terminate specific worker await client.terminate(worker.address) ``` Terminating a worker closes all browsers running on that worker and disconnects the device from the session. ## Connection management ### `client.close()` Closes the client connection and cleans up resources. ```typescript theme={null} client.close(): Promise ``` **Returns:** `Promise` **Example:** ```typescript theme={null} await client.close() ``` Always call `close()` when done to properly clean up WebSocket connections and resources. Connection time is determined by how long clients and workers are connected to a session. ## Events The client emits events for worker connection lifecycle. ### `worker-connected` Emitted when a new worker connects to the session (i.e., when a device running the Uplink SDK joins). ```typescript theme={null} client.on(event: 'worker-connected', handler: (worker: ClientWorker) => void) ``` **Example:** ```typescript theme={null} client.on('worker-connected', (worker) => { console.log('Worker connected:', worker.address) }) ``` ### `worker-disconnected` Emitted when a worker disconnects from the session (i.e., when a device leaves or loses connection). ```typescript theme={null} client.on(event: 'worker-disconnected', handler: (worker: ClientWorker) => void) ``` **Example:** ```typescript theme={null} client.on('worker-disconnected', (worker) => { console.log('Worker disconnected:', worker.address) }) ``` ## Types ### Session Returned by `uplink.session()` and `uplink.getSession()`. Plain data — pass it to [`uplink.client.fromSession()`](#uplinkclientfromsession). ```typescript theme={null} interface Session { sessionId: string sessionUrl: string // Internal relay WebSocket URL the SDK connects to qrUrl: string // The pairing link you show the user credential?: string // The session's verifier. Always set by uplink.session(), // never by uplink.getSession() — see Session restriction keys?: { ecdh?: { public: string; private: string } ecdsa?: { public: string; private: string } } } ``` `credential` is optional on the type because `getSession()` returns a `Session` without one. It is always present on the result of `uplink.session()`. ### Auth Accepted by `uplink.session()`, `uplink.getSession()`, `uplink.sessionDetails()`, and `uplink.token()`. Either your client credentials, or an access token you already hold. ```typescript theme={null} type Auth = ClientCredentials | Token ``` ### ClientCredentials ```typescript theme={null} interface ClientCredentials { clientId: string clientSecret: string scope?: string // Request fewer permissions than the client was granted } ``` ### Token Returned by `uplink.token()`. ```typescript theme={null} interface Token { token: string expiresAt?: number // Epoch ms; absent when the lifetime is unknown } ``` ### ClientOptions ```typescript theme={null} interface ClientOptions { agent?: Agent // Optional AI agent from @uplink-code/ai } ``` ### Address Worker identifier - a hex-encoded address: ```typescript theme={null} type Address = string // Hex-encoded worker address ``` ### SessionDetails Returned by `uplink.sessionDetails()`. Aggregates session-level metadata across the API, devices, billing, and tags. ```typescript theme={null} interface SessionDetails { id: string project: { id: string name: string description: string | null } organization: { id: string name: string avatar_url: string | null } created_at: Date updated_at: Date devices: Array<{ id: string device_id: string device_type: string device_model: string | null platform: string | null platform_version: string | null }> total_bytes: number total_connection_duration: number // seconds tags: Record sessionUrl: string } ``` ## Complete example ```typescript theme={null} import uplink from '@uplink-code/uplink' async function main() { // Connect to session that will be used for multiple workers doing the same script const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) // Listen for worker events client.on('worker-connected', async (worker) => { console.log('Worker joined:', worker.address) const browser = await worker.launch() const page = await browser.newPage() await page.goto('https://example.com') await page.close() await browser.close() await client.close() }) client.on('worker-disconnected', (worker) => { console.log('Worker left:', worker.address) }) } main().catch(console.error) ``` ## Related Device-specific operations Browser management Architecture overview Session management # ClientWorker Source: https://docs.uplink.build/api-reference/client-worker API reference for the ClientWorker class representing a worker The `ClientWorker` class represents a worker (the Uplink SDK running on a physical device) connected to your Uplink session. It provides methods for managing browsers on the worker and querying device information. A **worker** is an instance of the Uplink SDK running on a physical iOS or Android device. A device can create multiple workers when connecting to sessions. ## Browser operations ### `worker.launch()` Launches a new browser on this worker. Used when listening for `worker-connected` events from the `Client` class. ```typescript theme={null} worker.launch(): Promise ``` **Returns:** `Promise` - New browser instance **Example:** ```typescript theme={null} client.on('worker-connected', async (worker) => { const browser = await worker.launch() const page = await browser.newPage() await page.goto('https://example.com') await page.close() await browser.close() await client.close() }) ``` ### `worker.browsers()` Lists all browsers currently running on this worker. ```typescript theme={null} worker.browsers(): Promise ``` **Returns:** `Promise` - Array of browser instances **Example:** ```typescript theme={null} const browsers = await worker.browsers() console.log(`${browsers.length} browsers on this worker`) for (const browser of browsers) { const pages = await browser.pages() console.log(`Browser ${browser.handle} has ${pages.length} pages`) } ``` ## Device information ### `worker.getDeviceInfo()` Gets device information including model, platform, and OS version. ```typescript theme={null} worker.getDeviceInfo(): Promise ``` **Returns:** `Promise` - Device information object **Example:** ```typescript theme={null} const info = await worker.getDeviceInfo() console.log('Device model:', info.deviceModel) // "iPhone 14 Pro" console.log('Platform:', info.platform) // "iOS" console.log('OS version:', info.platformVersion) // "17.2" console.log('Type:', info.deviceType) // "phone" ``` ### WorkerInfo type ```typescript theme={null} interface WorkerInfo { deviceModel: string // Device model name platform: string // "iOS" or "Android" platformVersion: string // OS version number deviceType: string // "phone" or "tablet" } ``` ## Worker management ### `worker.terminate()` Terminates this worker's connection to the session. ```typescript theme={null} worker.terminate(): Promise ``` **Returns:** `Promise` **Example:** ```typescript theme={null} await worker.terminate() console.log('Worker terminated') ``` Terminating a worker closes all browsers running on it and disconnects the device from the session. This action cannot be undone. ## Properties ### `worker.address` The unique hex-encoded address identifier for this worker. ```typescript theme={null} worker.address: Address // Hex-encoded worker address ``` ## Related Session and connection management Browser operations Device management patterns Architecture overview # iOS SDK Source: https://docs.uplink.build/api-reference/ios Integrate the Uplink native SDK into your iOS app The Uplink iOS SDK embeds a worker inside your iOS app, making the device controllable from server-side code written with the [JavaScript SDK](/api-reference/overview). Use it when you want Uplink automation to run inside your own branded app. **Key concepts:** * **Device**: A physical iOS device running your app * **Worker**: A worker created by the Uplink SDK inside your app (one app can create multiple workers) * **Session**: An authenticated connection that pairs your worker with a JavaScript SDK client ## Requirements * iOS 15.0 or later * Xcode 26.1 or later ## Installation The iOS SDK is distributed via Swift Package Manager. In Xcode, go to **File → Add Package Dependencies…** and enter: ``` https://github.com/uplink-code/uplink-ios-public ``` Choose either the static `Uplink` library product or the `Uplink-Dynamic` library product and add it to your app target. Link only one variant. ```swift theme={null} import Uplink ``` If you manage dependencies with a `Package.swift` file: ```swift theme={null} dependencies: [ .package(url: "https://github.com/uplink-code/uplink-ios-public", from: "1.0.0") ], targets: [ .target( name: "YourApp", dependencies: [ .product(name: "Uplink", package: "uplink-ios-public") ] ) ] ``` `Uplink` is static by default. If your app requires dynamic linking, depend on `.product(name: "Uplink-Dynamic", package: "uplink-ios-public")` instead. Do not link both variants. ## Quick example ```swift theme={null} import UIKit import Uplink @MainActor func startUplink(in viewController: UIViewController, session sessionUrl: String) async throws { let worker = await Worker(controller: viewController) try await worker.connect(session: sessionUrl) Task { // accept() blocks until the worker finishes. await worker.accept() } } ``` ## Initialization Create a `Worker` with the `UIViewController` that will present the browser, usually the view controller currently on screen. Retain each active worker for as long as its connection is active. ```swift theme={null} let worker = await Worker(controller: viewController) ``` ### `Worker(controller:)` Creates a new `Worker` associated with a `UIViewController`. The SDK uses that controller to host its `WKWebView`. ```swift theme={null} Worker(controller: UIViewController) async ``` **Parameters:** * `controller`: The `UIViewController` that will host the SDK's web view **Returns:** A new `Worker` ready to connect to a session ```swift theme={null} let worker = await Worker(controller: self) ``` Keep a strong reference to the `Worker` for as long as the connection is active. If the worker is deallocated, the session will close. ## Obtaining a session Your backend creates a session using the JavaScript SDK's `uplink.session()` and delivers the resulting session URL to the app. How you deliver it — a backend API response, a universal link, a QR code — is up to you. ```swift theme={null} let sessionUrl: String = // from your backend, deep link, or QR code ``` See [Sessions](/sessions) for how sessions are created on the server side. ## Connecting and accepting ### `worker.connect(session:)` Connects the worker to an Uplink session. ```swift theme={null} func connect(session: String) async throws ``` **Parameters:** * `session`: The session URL provided by your backend ```swift theme={null} try await worker.connect(session: sessionUrl) ``` ### `worker.accept()` Begins accepting commands from the JavaScript SDK client. `accept()` suspends until the worker closes, so call it from a `Task` to avoid blocking the rest of your app. ```swift theme={null} func accept() async ``` ```swift theme={null} Task { await worker.accept() } ``` ### `worker.watch(_:)` Registers a callback for worker events. Callbacks are not delivered on the `MainActor`. If a callback updates UIKit or other main-actor-isolated state, explicitly hop to the main actor. ```swift theme={null} await worker.watch { [weak self] event in switch event { case WorkerEvent.terminated: Task { @MainActor in // Update UI. } // Optionally handle other Page or Worker events. default: break } } ``` ## Cleanup ### `worker.close()` Closes the worker and releases its resources. ```swift theme={null} func close() async ``` ```swift theme={null} await worker.close() ``` Closing a worker ends its session. Any browsers the JavaScript SDK has launched on the worker will be terminated. ## Complete example A minimal `UIViewController` that starts an Uplink worker when it appears: For a complete working app, see the [Connect example](https://github.com/uplink-code/uplink-ios-public/tree/main/example). ```swift theme={null} import UIKit import Uplink final class UplinkViewController: UIViewController { private var worker: Worker? func start(sessionUrl: String) { Task { let worker = await Worker(controller: self) self.worker = worker do { try await worker.connect(session: sessionUrl) await worker.accept() } catch { print("Uplink worker ended: \(error)") } } } override func viewWillDisappear(_ animated: Bool) { super.viewWillDisappear(animated) Task { [worker] in await worker?.close() } } } ``` ## Related Integrate Uplink into your Android app Control your worker from server-side code How sessions are created and secured Architecture overview # JavaScript SDK Overview Source: https://docs.uplink.build/api-reference/overview Overview of the Uplink JavaScript SDK for mobile browser automation The Uplink JavaScript SDK provides a Puppeteer/Playwright-like API for controlling mobile browsers remotely. This reference covers all available classes, methods, and types. **Key terminology:** * **Device**: A physical iOS or Android device * **Worker**: A worker created using the native Uplink SDK (a device can create multiple workers) * **Address**: A hex-encoded identifier used to reference specific workers ## Installation ```bash theme={null} npm install @uplink-code/uplink ``` ### Optional: AI Automation For AI-powered browser automation with natural language instructions: ```bash theme={null} npm install @uplink-code/ai ``` See the [AI Automation](/api-reference/ai/overview) section for details. ## Quick example ```typescript theme={null} import uplink from '@uplink-code/uplink' const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) const browser = await client.launch() const page = await browser.newPage() await page.goto('https://example.com') const title = await page.evaluate(() => document.title) await browser.close() await client.close() ``` ## API structure The SDK is organized into four main classes, plus an optional AI automation package: Main entry point for connecting to Uplink and managing workers Represents a worker (Uplink SDK running on a physical device) Browser instance running on a device Browser tab/window with automation methods Optional: Natural language actions and data extraction ## TypeScript support The SDK is written in TypeScript and includes full type definitions: ```typescript theme={null} import type { Page, Cookie, Request } from '@uplink-code/uplink/page/types' import type { Client } from '@uplink-code/uplink/client/types' import type { Browser } from '@uplink-code/uplink/browser/types' import type { Agent, ActResult, ExtractResult } from '@uplink-code/ai' ``` ## Error handling All async methods can throw errors. Always use try-catch for robust automation: ```typescript theme={null} try { await page.goto('https://example.com') } catch (error) { console.error('Navigation failed:', error) } ``` ## Common patterns ### Connection and cleanup ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) try { const browser = await client.launch() const page = await browser.newPage() // Your automation here } finally { // Always clean up await page.close() await browser.close() await client.close() } ``` ### Waiting for workers ```typescript theme={null} const session = await uplink.session( { clientId: process.env.UPLINK_CLIENT_ID, clientSecret: process.env.UPLINK_CLIENT_SECRET }, { include: { ecdsa: true, ecdh: true } } ) const client = await uplink.client.fromSession(session) // Check currently connected workers const workers = client.workers() console.log(`${workers.length} workers connected`) // Or handle workers as they connect client.on('worker-connected', async (worker) => { console.log('Worker connected:', worker.address) const browser = await worker.launch() // ... automation }) ``` ### Multi-page automation ```typescript theme={null} const browser = await client.launch() const page1 = await browser.newPage() const page2 = await browser.newPage() await Promise.all([ page1.goto('https://example.com'), page2.goto('https://google.com') ]) ``` ## Next steps Connection and device management Browser automation methods # Authentication Source: https://docs.uplink.build/api-reference/page/authentication Handle authentication flows on mobile devices Uplink provides specialized methods for waiting for users to complete authentication on their devices. Since real users log themselves in on real mobile devices, authentication flows work naturally. **The Uplink Advantage**: Users authenticate themselves on their own devices. No credential management needed. Simply wait for the user to complete authentication, then automate the rest. ## Methods ### `page.authenticate()` Navigates to a URL and waits for authentication to complete. ```typescript theme={null} page.authenticate( startURL: string, authEvaluator: AuthEvaluator, options?: AuthenticateOptions ): Promise ``` **Parameters:** * `startURL`: URL to navigate to for authentication * `authEvaluator`: Function that checks if authentication is complete * `options` (optional): Authentication options **AuthEvaluator:** ```typescript theme={null} type AuthEvaluator = (page: Page) => boolean | Promise ``` **Options:** ```typescript theme={null} interface AuthenticateOptions { timeout?: number // Timeout in milliseconds (default: 90000) } ``` **Returns:** `Promise` - Either `'authenticated'` or `'aborted'` **Examples:** ```typescript theme={null} // Wait for session cookie const status = await page.authenticate( 'https://example.com/login', async (page) => { const cookies = await page.cookies('https://example.com') return cookies.some(c => c.name === 'session_token') }, { timeout: 90000 } ) if (status === 'authenticated') { console.log('Authentication successful') } // Wait for URL change const status = await page.authenticate( 'https://example.com/oauth/authorize', async (page) => { const url = await page.url() return url.includes('/dashboard') } ) ``` ### `page.waitForAuthentication()` Waits for authentication to complete without navigating. ```typescript theme={null} page.waitForAuthentication( authEvaluator: AuthEvaluator, options?: AuthenticateOptions ): Promise ``` **Parameters:** * `authEvaluator`: Function that checks if authentication is complete * `options` (optional): Authentication options **Returns:** `Promise` - Either `'authenticated'` or `'aborted'` **Example:** ```typescript theme={null} // User manually logs in on the device await page.goto('https://example.com/login') // Wait for them to complete authentication const status = await page.waitForAuthentication( async (page) => { const url = await page.url() return url.includes('/dashboard') }, { timeout: 120000 } // 2 minutes ) if (status === 'authenticated') { console.log('User logged in successfully') } ``` ## Complete examples ### OAuth flow ```typescript theme={null} async function handleOAuthLogin(page) { const status = await page.authenticate( 'https://app.example.com/oauth/google', async (page) => { // Check if we're back at the app with an auth token const url = await page.url() if (!url.includes('app.example.com')) return false // Verify we have the auth cookie const cookies = await page.cookies('https://app.example.com') return cookies.some(c => c.name === 'oauth_token') }, { timeout: 120000 } // Give user 2 minutes to log in ) if (status === 'authenticated') { console.log('OAuth login successful') return true } else { console.log('OAuth login aborted or timed out') return false } } const success = await handleOAuthLogin(page) if (success) { // Continue with authenticated session await page.goto('https://app.example.com/dashboard') } ``` ### Social login ```typescript theme={null} async function socialLogin(page, provider) { const status = await page.authenticate( `https://example.com/login/${provider}`, async (page) => { // User completes login on social provider's site // Check for success indicator const url = await page.url() if (url.includes('example.com/welcome')) { return true // Redirected to welcome page } // Or check for authentication cookie const cookies = await page.cookies('https://example.com') return cookies.some(c => c.name === 'auth_session') }, { timeout: 180000 } // 3 minutes for social login ) return status === 'authenticated' } // Try Google login const success = await socialLogin(page, 'google') console.log('Google login:', success ? 'success' : 'failed') ``` ### Two-factor authentication ```typescript theme={null} async function waitForTwoFactorLogin(page) { // Navigate to login page await page.goto('https://example.com/login') // User enters their credentials and 2FA code on the device console.log('Waiting for user to complete login with 2FA...') const status = await page.waitForAuthentication( async (page) => { const url = await page.url() return url.includes('/dashboard') }, { timeout: 180000 } // Give user 3 minutes to enter credentials + 2FA ) if (status === 'authenticated') { console.log('User successfully completed 2FA login') return true } return false } await waitForTwoFactorLogin(page) ``` ### Session-based authentication ```typescript theme={null} async function checkSession(page) { await page.goto('https://example.com') const status = await page.waitForAuthentication( async (page) => { const cookies = await page.cookies('https://example.com') const sessionCookie = cookies.find(c => c.name === 'session_id') if (!sessionCookie) return false // Optionally validate the session cookie const isValid = await page.evaluate(() => { return document.body.classList.contains('logged-in') }) return isValid }, { timeout: 10000 } ) return status === 'authenticated' } const isAuthenticated = await checkSession(page) if (!isAuthenticated) { console.log('No valid session, need to log in') // Proceed with login flow } ``` ### Mobile app authentication ```typescript theme={null} async function appAuthentication(page) { // Navigate to the authentication start page const status = await page.authenticate( 'https://example.com/app/auth', async (page) => { // Check for mobile-specific auth indicator const authToken = await page.evaluate(() => { return localStorage.getItem('auth_token') }) if (!authToken) return false // Verify token is valid const url = await page.url() return url.includes('/app/home') }, { timeout: 180000 } ) if (status === 'authenticated') { // Get the auth token for API calls const token = await page.evaluate(() => { return localStorage.getItem('auth_token') }) console.log('Authenticated, token:', token) return token } return null } ``` ## Best practices With Uplink, users authenticate themselves on their own devices: ```typescript theme={null} // Navigate to login page and wait - user handles the rest await page.authenticate( 'https://example.com/login', async (page) => { const cookies = await page.cookies('https://example.com') return cookies.some(c => c.name === 'session') } ) // Now automate authenticated actions ``` Authentication often requires user interaction, so use longer timeouts: ```typescript theme={null} const status = await page.authenticate( url, evaluator, { timeout: 180000 } // 3 minutes for complex auth ) ``` Verify authentication using multiple methods: ```typescript theme={null} async (page) => { const url = await page.url() const cookies = await page.cookies('https://example.com') return url.includes('/dashboard') && cookies.some(c => c.name === 'session_token') } ``` Always handle the aborted case: ```typescript theme={null} const status = await page.authenticate(url, evaluator) if (status === 'aborted') { console.log('User closed authentication or it timed out') // Handle gracefully } ``` Extract and store authentication tokens for later use: ```typescript theme={null} if (status === 'authenticated') { const token = await page.evaluate(() => { return localStorage.getItem('token') }) // Store token for API calls } ``` ## Related Manage authentication cookies Wait for authentication indicators Navigate to auth pages Back to Page API overview # Cookies Source: https://docs.uplink.build/api-reference/page/cookies Manage browser cookies for authentication and session handling Read and write browser cookies to manage authentication, sessions, and user preferences. ## Methods ### `page.cookies()` Gets all cookies visible to a given URL. ```typescript theme={null} page.cookies(url: string): Promise ``` **Parameters:** * `url`: URL to get cookies for (cookies are domain-scoped) **Returns:** `Promise` - Array of cookie objects **Example:** ```typescript theme={null} const cookies = await page.cookies('https://example.com') console.log(`Found ${cookies.length} cookies`) cookies.forEach(cookie => { console.log(`${cookie.name}: ${cookie.value}`) }) ``` ### `page.setCookie()` Sets one or more cookies. ```typescript theme={null} page.setCookie(...cookies: Cookie[]): Promise ``` **Parameters:** * `cookies`: One or more cookie objects to set **Returns:** `Promise` **Example:** ```typescript theme={null} await page.setCookie({ name: 'session', value: 'abc123xyz', domain: 'example.com', path: '/', secure: true, httpOnly: true }) // Set multiple cookies await page.setCookie( { name: 'user_id', value: '12345', domain: 'example.com' }, { name: 'theme', value: 'dark', domain: 'example.com' } ) ``` ## Cookie type ```typescript theme={null} interface Cookie { name: string // Cookie name value: string // Cookie value domain?: string // Domain (e.g., "example.com") path?: string // Path (e.g., "/") expires?: number // Expiration timestamp (Unix time) size?: number // Cookie size in bytes httpOnly?: boolean // HTTP-only flag secure?: boolean // Secure flag (HTTPS only) } ``` ## Complete examples ### Check authentication status ```typescript theme={null} async function isLoggedIn(page) { const cookies = await page.cookies('https://example.com') const sessionCookie = cookies.find(c => c.name === 'session_token') if (!sessionCookie) { return false } // Check if cookie is expired if (sessionCookie.expires && sessionCookie.expires < Date.now() / 1000) { console.log('Session cookie expired') return false } return true } await page.goto('https://example.com') const authenticated = await isLoggedIn(page) console.log('Authenticated:', authenticated) ``` ### Transfer session between browsers ```typescript theme={null} // Get cookies from first browser const browser1 = await client.launch() const page1 = await browser1.newPage() await page1.goto('https://example.com/login') // ... perform login ... const cookies = await page1.cookies('https://example.com') console.log('Captured session cookies') // Transfer to second browser const browser2 = await client.launch() const page2 = await browser2.newPage() await page2.setCookie(...cookies) await page2.goto('https://example.com/dashboard') // page2 is now logged in with the same session console.log('Session transferred to second browser') ``` ### Inject authentication cookies ```typescript theme={null} async function loginWithCookies(page, authToken) { // Set authentication cookie before navigating await page.setCookie({ name: 'auth_token', value: authToken, domain: 'example.com', path: '/', secure: true, httpOnly: true, expires: Math.floor(Date.now() / 1000) + 86400 // 24 hours }) // Navigate - now authenticated await page.goto('https://example.com/dashboard') // Verify authentication worked const pageTitle = await page.evaluate(() => document.title) console.log('Logged in, page title:', pageTitle) } await loginWithCookies(page, 'my-auth-token-xyz') ``` ### Save and restore session ```typescript theme={null} import fs from 'fs/promises' // Save session async function saveSession(page, filename) { const cookies = await page.cookies('https://example.com') await fs.writeFile(filename, JSON.stringify(cookies, null, 2)) console.log('Session saved to', filename) } // Restore session async function restoreSession(page, filename) { const cookiesJson = await fs.readFile(filename, 'utf-8') const cookies = JSON.parse(cookiesJson) await page.setCookie(...cookies) console.log('Session restored from', filename) } // Usage await page.goto('https://example.com/login') // ... perform login ... await saveSession(page, 'session.json') // Later, in a new session await restoreSession(page, 'session.json') await page.goto('https://example.com/dashboard') ``` ### Clear cookies ```typescript theme={null} async function clearCookies(page, domain) { const cookies = await page.cookies(`https://${domain}`) // Delete each cookie by setting expired date const expiredCookies = cookies.map(cookie => ({ ...cookie, expires: 0 // Expire immediately })) await page.setCookie(...expiredCookies) console.log(`Cleared ${cookies.length} cookies for ${domain}`) } await clearCookies(page, 'example.com') // Verify cookies are cleared const remaining = await page.cookies('https://example.com') console.log('Remaining cookies:', remaining.length) ``` ### Cookie-based feature flags ```typescript theme={null} async function enableFeature(page, featureName) { await page.setCookie({ name: `feature_${featureName}`, value: 'enabled', domain: 'example.com', path: '/' }) console.log(`Feature "${featureName}" enabled`) } async function isFeatureEnabled(page, featureName) { const cookies = await page.cookies('https://example.com') const featureCookie = cookies.find(c => c.name === `feature_${featureName}`) return featureCookie?.value === 'enabled' } // Enable dark mode await enableFeature(page, 'dark_mode') // Check if enabled const darkModeEnabled = await isFeatureEnabled(page, 'dark_mode') console.log('Dark mode:', darkModeEnabled ? 'ON' : 'OFF') ``` ### Manage user preferences ```typescript theme={null} async function setUserPreferences(page, prefs) { await page.setCookie({ name: 'user_prefs', value: JSON.stringify(prefs), domain: 'example.com', path: '/', expires: Math.floor(Date.now() / 1000) + (365 * 86400) // 1 year }) } async function getUserPreferences(page) { const cookies = await page.cookies('https://example.com') const prefsCookie = cookies.find(c => c.name === 'user_prefs') if (prefsCookie) { return JSON.parse(prefsCookie.value) } return null } // Set preferences await setUserPreferences(page, { theme: 'dark', language: 'en', notifications: true }) // Read preferences const prefs = await getUserPreferences(page) console.log('User preferences:', prefs) ``` ### Debug cookie issues ```typescript theme={null} async function debugCookies(page, url) { const cookies = await page.cookies(url) console.log(`\n=== Cookies for ${url} ===`) console.log(`Total cookies: ${cookies.length}\n`) cookies.forEach(cookie => { console.log(`Name: ${cookie.name}`) console.log(`Value: ${cookie.value}`) console.log(`Domain: ${cookie.domain}`) console.log(`Path: ${cookie.path}`) console.log(`Secure: ${cookie.secure}`) console.log(`HttpOnly: ${cookie.httpOnly}`) if (cookie.expires) { const expiryDate = new Date(cookie.expires * 1000) const isExpired = expiryDate < new Date() console.log(`Expires: ${expiryDate.toISOString()} ${isExpired ? '(EXPIRED)' : ''}`) } else { console.log('Expires: Session') } console.log('---') }) } await page.goto('https://example.com') await debugCookies(page, 'https://example.com') ``` ### A/B testing with cookies ```typescript theme={null} async function assignABTestVariant(page) { const variant = Math.random() < 0.5 ? 'A' : 'B' await page.setCookie({ name: 'ab_test_variant', value: variant, domain: 'example.com', path: '/', expires: Math.floor(Date.now() / 1000) + (30 * 86400) // 30 days }) console.log('Assigned A/B test variant:', variant) return variant } async function getABTestVariant(page) { const cookies = await page.cookies('https://example.com') const variantCookie = cookies.find(c => c.name === 'ab_test_variant') return variantCookie?.value || null } // Assign variant if not already assigned let variant = await getABTestVariant(page) if (!variant) { variant = await assignABTestVariant(page) } await page.goto('https://example.com') console.log('Testing with variant:', variant) ``` ## Best practices Ensure the URL domain matches the cookie domain: ```typescript theme={null} // Cookie domain: example.com const cookies = await page.cookies('https://example.com') // ✓ Correct const cookies = await page.cookies('https://other.com') // ✗ Won't find it ``` Use the `secure` flag for cookies on HTTPS sites: ```typescript theme={null} await page.setCookie({ name: 'auth_token', value: 'secret', domain: 'example.com', secure: true, // Only sent over HTTPS httpOnly: true // Not accessible via JavaScript }) ``` Verify cookies haven't expired before using them: ```typescript theme={null} const sessionCookie = cookies.find(c => c.name === 'session') if (sessionCookie.expires && sessionCookie.expires < Date.now() / 1000) { console.log('Cookie expired, need to re-authenticate') } ``` Always check if cookies exist before accessing: ```typescript theme={null} const cookies = await page.cookies('https://example.com') const authCookie = cookies.find(c => c.name === 'auth') if (!authCookie) { console.log('Not authenticated') // Handle unauthenticated state } ``` ## Related Handle authentication flows Navigate with cookies set Access cookies via JavaScript Back to Page API overview # Events Source: https://docs.uplink.build/api-reference/page/events Listen to page events and lifecycle hooks Monitor page events including HTTP requests, navigation, DOM changes, and custom events emitted by the page. ## Methods ### `page.on()` Listens for page events. Returns an AbortController for cleanup. ```typescript theme={null} page.on(event: string, handler: EventListener): Promise ``` **Parameters:** * `event`: Event type to listen for * `handler`: Function to handle the event **Returns:** `Promise` - Controller for stopping the listener **Example:** ```typescript theme={null} const controller = await page.on('xhr', async (event) => { console.log('HTTP Request:', event.data.url) }) // Later, stop listening await page.off(controller) ``` ### `page.once()` Listens for a page event once, then automatically removes the listener. ```typescript theme={null} page.once(event: string, handler: EventListener): Promise ``` **Parameters:** * `event`: Event type to listen for * `handler`: Function to handle the event **Returns:** `Promise` - Controller (listener auto-removed after first event) **Example:** ```typescript theme={null} await page.once('finished', async (event) => { console.log('Page finished loading') }) ``` ### `page.off()` Stops listening to events by providing the AbortController. ```typescript theme={null} page.off(controller: AbortController): void ``` **Parameters:** * `controller`: AbortController returned from `on()` or `once()` **Example:** ```typescript theme={null} const controller = await page.on('domchange', handler) await page.off(controller) ``` ## Available events ### `xhr` Emitted when an HTTP request (fetch/XHR) is made. **Event data:** ```typescript theme={null} { type: string // e.g., "fetch", "xhr" url: string // Request URL method: string // HTTP method headers: Record // Request headers data: Record // Request body response?: { status: string // Response status code data: Record // Response body headers: Record // Response headers } } ``` **Example:** ```typescript theme={null} await page.on('xhr', async (event) => { const request = event.data console.log(`${request.method} ${request.url}`) if (request.response) { console.log(`Response status: ${request.response.status}`) } }) ``` ### `started` Emitted when page load starts. **Example:** ```typescript theme={null} await page.on('started', async (event) => { console.log('Page load started') }) ``` ### `finished` Emitted when page load finishes. **Example:** ```typescript theme={null} await page.on('finished', async (event) => { console.log('Page load finished') }) ``` ### `locationchange` Emitted when the page URL changes. **Example:** ```typescript theme={null} await page.on('locationchange', async (event) => { const newUrl = await page.url() console.log('URL changed to:', newUrl) }) ``` ### `domchange` Emitted when the DOM is mutated. **Example:** ```typescript theme={null} await page.on('domchange', async (event) => { console.log('DOM changed') }) ``` ### `dispatch` Emitted for custom events dispatched by the page. **Example:** ```typescript theme={null} await page.on('dispatch', async (event) => { console.log('Custom event:', event.data) }) ``` ### `close` Emitted when the browser requests to close (e.g., user clicks close). **Example:** ```typescript theme={null} await page.on('close', async (event) => { console.log('Browser close requested') // Cleanup or save state }) ``` ### `closed` Emitted when the page is actually closed. **Example:** ```typescript theme={null} await page.on('closed', async (event) => { console.log('Page closed') }) ``` ### `hostblocked` Emitted when a host is blocked by an allowlist. **Example:** ```typescript theme={null} await page.on('hostblocked', async (event) => { console.log('Blocked host:', event.data) }) ``` ## Complete examples ### Monitor all HTTP requests ```typescript theme={null} const controller = await page.on('xhr', async (event) => { const req = event.data console.log(`[${req.method}] ${req.url}`) if (req.response) { console.log(` Status: ${req.response.status}`) console.log(` Headers:`, req.response.headers) } }) await page.goto('https://example.com') // Stop monitoring after some time await new Promise(resolve => setTimeout(resolve, 10000)) await page.off(controller) ``` ### Filter API requests ```typescript theme={null} await page.on('xhr', async (event) => { const req = event.data // Only log API calls if (req.url.includes('/api/')) { console.log('API Request:', req.method, req.url) if (req.response) { console.log('API Response:', req.response.status) // Log response data for specific endpoints if (req.url.includes('/api/users')) { console.log('Users data:', req.response.data) } } } }) await page.goto('https://example.com/app') ``` ### Track page lifecycle ```typescript theme={null} await page.on('started', async () => { console.log('[Lifecycle] Page load started') }) await page.on('finished', async () => { console.log('[Lifecycle] Page load finished') }) await page.on('locationchange', async () => { const url = await page.url() console.log('[Lifecycle] URL changed:', url) }) await page.goto('https://example.com') await page.click('#navigate-link') ``` ### Count DOM changes ```typescript theme={null} let domChangeCount = 0 const controller = await page.on('domchange', async () => { domChangeCount++ }) await page.goto('https://example.com/dynamic-page') // Wait for page to settle await new Promise(resolve => setTimeout(resolve, 5000)) await page.off(controller) console.log(`Total DOM changes: ${domChangeCount}`) ``` ### Wait for specific request ```typescript theme={null} async function waitForAPICall(page, apiPath) { return new Promise((resolve) => { page.on('xhr', async (event) => { const req = event.data if (req.url.includes(apiPath) && req.response) { resolve(req.response) } }) }) } // Click button that triggers API call await page.click('#load-data') // Wait for the API call to complete const response = await waitForAPICall(page, '/api/data') console.log('API call completed:', response.status) ``` ### Detect failed requests ```typescript theme={null} const failedRequests = [] await page.on('xhr', async (event) => { const req = event.data if (req.response && parseInt(req.response.status) >= 400) { failedRequests.push({ url: req.url, method: req.method, status: req.response.status }) console.error(`Failed request: ${req.method} ${req.url} - ${req.response.status}`) } }) await page.goto('https://example.com') console.log(`Total failed requests: ${failedRequests.length}`) ``` ### Monitor page navigation ```typescript theme={null} const navigationHistory = [] await page.on('locationchange', async () => { const url = await page.url() navigationHistory.push({ url, timestamp: Date.now() }) console.log('Navigated to:', url) }) await page.goto('https://example.com') await page.click('#link1') await page.click('#link2') await page.click('#back-button') console.log('Navigation history:') navigationHistory.forEach((nav, i) => { console.log(`${i + 1}. ${nav.url} at ${new Date(nav.timestamp).toISOString()}`) }) ``` ### Measure request performance ```typescript theme={null} const requestTimes = new Map() await page.on('xhr', async (event) => { const req = event.data if (!requestTimes.has(req.url)) { requestTimes.set(req.url, { startTime: Date.now(), url: req.url }) } if (req.response) { const startTime = requestTimes.get(req.url)?.startTime if (startTime) { const duration = Date.now() - startTime console.log(`${req.url} took ${duration}ms`) } } }) await page.goto('https://example.com') ``` ### Handle custom events ```typescript theme={null} // Add user script to dispatch custom events await page.addUserScript(` window.addEventListener('load', () => { // Dispatch custom event window.dispatchEvent(new CustomEvent('app-ready', { detail: { version: '1.0' } })) }) `) // Listen for custom events await page.on('dispatch', async (event) => { console.log('Custom event received:', event.type, event.data) }) await page.goto('https://example.com') ``` ### Cleanup on page close ```typescript theme={null} let eventListeners = [] async function setupListeners(page) { const xhrController = await page.on('xhr', xhrHandler) const domController = await page.on('domchange', domHandler) eventListeners.push(xhrController, domController) // Cleanup when page closes await page.on('closed', async () => { console.log('Page closed, cleaning up listeners') for (const controller of eventListeners) { await page.off(controller) } eventListeners = [] }) } await setupListeners(page) ``` ### Debug mode logging ```typescript theme={null} async function enableDebugMode(page) { await page.on('xhr', async (event) => { const req = event.data console.log('[XHR]', req.method, req.url) }) await page.on('started', async () => { console.log('[NAV] Page load started') }) await page.on('finished', async () => { console.log('[NAV] Page load finished') }) await page.on('locationchange', async () => { const url = await page.url() console.log('[NAV] Location changed:', url) }) await page.on('domchange', async () => { console.log('[DOM] DOM mutated') }) console.log('Debug mode enabled') } await enableDebugMode(page) await page.goto('https://example.com') ``` ## Best practices Always remove event listeners when done: ```typescript theme={null} const controller = await page.on('xhr', handler) // Do work await page.off(controller) ``` Use `once()` for events you only need to handle once: ```typescript theme={null} await page.once('finished', async () => { console.log('Page loaded') }) ``` Filter events in your handler to avoid processing irrelevant data: ```typescript theme={null} await page.on('xhr', async (event) => { const req = event.data if (!req.url.includes('/api/')) return // Skip non-API requests // Process API request }) ``` Event handlers can be async, but they don't block the page: ```typescript theme={null} await page.on('xhr', async (event) => { // This is async but doesn't block page execution await processRequest(event.data) }) ``` ## Related Wait for specific requests Make HTTP requests Execute JavaScript Back to Page API overview # Interaction Source: https://docs.uplink.build/api-reference/page/interaction Methods for interacting with page elements The Page interaction API provides methods for clicking elements, typing input, and interacting with forms. ## Methods ### `page.click()` Clicks an element matching the CSS selector. ```typescript theme={null} page.click(selector: string, options?: ClickOptions): Promise ``` **Parameters:** * `selector`: CSS selector for the element to click * `options` (optional): Click options **Options:** ```typescript theme={null} interface ClickOptions { simulateFramework?: boolean // Simulate framework-specific click events (default: false) } ``` **Returns:** `Promise` - `true` if click succeeded **Examples:** ```typescript theme={null} // Basic click await page.click('#submit-button') // Click with framework simulation (for React, Vue, etc.) await page.click('.link', { simulateFramework: true }) // Wait for element before clicking await page.waitForSelector('#button') await page.click('#button') ``` Use `simulateFramework: true` when clicking elements in React, Vue, or Angular applications that rely on framework-specific event handlers. ### `page.input()` Types text into an input element. ```typescript theme={null} page.input(selector: string, text: string, options?: InputOptions): Promise ``` **Parameters:** * `selector`: CSS selector for the input element * `text`: Text to type * `options` (optional): Input options **Options:** ```typescript theme={null} interface InputOptions { simulateFramework?: boolean // Simulate framework-specific events (default: false) simulateTyping?: boolean // Simulate human typing with delays (default: true) triggerEvents?: boolean // Trigger DOM events (default: true) } ``` **Returns:** `Promise` **Examples:** ```typescript theme={null} // Basic input await page.input('#username', 'myuser') // Input with options await page.input('#password', 'secret123', { simulateFramework: true, // For React/Vue simulateTyping: true, // Realistic typing speed triggerEvents: true // Fire input/change events }) // Fast input without typing simulation await page.input('#email', 'user@example.com', { simulateTyping: false }) ``` ### `page.select()` Selects an option in a `