AI Model Wiki

    Skip to main content
    25 min
    Lesson 3 of 5

    Web Automation with Computer Use

    From the course Anthropic Computer Use Agent

    Learn to automate web-based tasks using Claude's Computer Use capabilities, from simple navigation to complex multi-step workflows.

    Basic Web Navigation

    Opening and Controlling Browsers

    def automate_web_task(task_description):
        screenshot, size = capture_screen()
        
        response = client.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=1024,
            tools=[{
                "type": "computer_20241022",
                "name": "computer",
                "display_width_px": size[0],
                "display_height_px": size[1],
            }],
            messages=[{
                "role": "user",
                "content": [
                    {"type": "text", "text": task_description},
                    {
                        "type": "image",
                        "source": {
                            "type": "base64",
                            "media_type": "image/png",
                            "data": screenshot
                        }
                    }
                ]
            }]
        )
        
        # Execute the suggested actions
        for content_block in response.content:
            if content_block.type == "tool_use":
                execute_computer_action(content_block.input)
        
        return response
    
    # Example usage
    automate_web_task("Open a new browser tab and navigate to example.com")
    

    Form Automation Examples

    Contact Form Filling

    def fill_contact_form(contact_data):
        task = f"""
        Fill out the contact form on this page with the following information:
        Name: {contact_data['name']}
        Email: {contact_data['email']}
        Subject: {contact_data['subject']}
        Message: {contact_data['message']}
        Then click the submit button.
        """
        
        return automate_web_task(task)
    
    # Usage
    contact_info = {
        'name': 'John Doe',
        'email': '[email protected]',
        'subject': 'Product Inquiry',
        'message': 'I am interested in learning more about your services.'
    }
    
    fill_contact_form(contact_info)
    

    E-commerce Automation

    def search_and_compare_products(search_query):
        task = f"""
        1. Search for '{search_query}' on this e-commerce site
        2. Look at the first 5 results
        3. For each product, note the name, price, and rating
        4. Create a comparison of the options
        """
        
        return automate_web_task(task)
    
    # Example
    search_and_compare_products("wireless headphones under $100")
    

    Advanced Web Workflows

    Multi-Tab Research

    def research_topic(topic, sites):
        task = f"""
        Research '{topic}' by:
        1. Opening separate tabs for each site: {', '.join(sites)}
        2. Searching for the topic on each site
        3. Collecting key information from the top 3 results on each
        4. Summarizing the findings across all sources
        """
        
        return automate_web_task(task)
    
    # Usage
    research_sites = [
        "reddit.com", 
        "stackoverflow.com", 
        "medium.com"
    ]
    research_topic("computer use automation", research_sites)
    

    Data Extraction Pipeline

    def extract_table_data(url, table_selector=None):
        task = f"""
        1. Navigate to {url}
        2. Find the main data table on the page
        3. Extract all rows and columns of data
        4. Format the data as CSV or JSON
        5. Copy the formatted data to clipboard
        """
        
        if table_selector:
            task += f"\nLook specifically for the table: {table_selector}"
        
        return automate_web_task(task)
    
    # Example
    extract_table_data("https://example.com/data-page", "#main-table")
    

    Error Handling and Reliability

    Robust Task Execution

    def execute_with_retry(task, max_attempts=3):
        for attempt in range(max_attempts):
            try:
                result = automate_web_task(task)
                
                # Check if task completed successfully
                if "error" not in result.content[0].text.lower():
                    return result
                
            except Exception as e:
                print(f"Attempt {attempt + 1} failed: {e}")
                
            if attempt < max_attempts - 1:
                time.sleep(2)  # Wait before retry
        
        raise Exception("Task failed after all retry attempts")
    

    Best Practices

    Task Description Guidelines

    • Be specific about the desired outcome
    • Break complex tasks into clear steps
    • Include context about what success looks like
    • Specify data formats for extraction tasks

    Performance Optimization

    • Take screenshots only when necessary
    • Use specific element descriptions to reduce processing time
    • Batch related actions when possible
    • Implement proper waiting mechanisms for page loads

    Key Takeaways

    • •Computer Use excels at web automation from simple navigation to complex workflows
    • •Form filling and data extraction can be automated with natural language instructions
    • •Multi-tab research and comparison tasks leverage Claude's ability to understand context
    • •Robust error handling and retry logic improve reliability of automated tasks
    0 of 5 lessons marked complete

    Progress is kept in this browser only.