AI Model Wiki

    Skip to main content
    40 min
    Lesson 4 of 6

    Vision & Multimodal Inputs

    From the course Anthropic Claude API Deep Dive

    Vision Capabilities

    Claude can analyze images alongside text, enabling powerful multimodal applications.

    Sending Images

    const response = await client.messages.create({
      model: "claude-sonnet-4-20250514",
      max_tokens: 1024,
      messages: [{
        role: "user",
        content: [
          {
            type: "image",
            source: { type: "base64", media_type: "image/png", data: base64String }
          },
          { type: "text", text: "Describe what you see in this image." }
        ]
      }],
    });
    

    URL-Based Images

    {
      type: "image",
      source: { type: "url", url: "https://example.com/chart.png" }
    }
    

    Vision Use Cases

    • Document analysis: Extract data from invoices, receipts, forms
    • Chart interpretation: Analyze graphs and data visualizations
    • UI/UX review: Get feedback on interface designs and screenshots
    • Code screenshots: Convert code images to editable text
    • Diagram understanding: Parse flowcharts, architecture diagrams, wireframes

    Image Best Practices

    • Resize images to reduce token costs (Claude auto-processes at optimal resolution)
    • Use PNG for screenshots and diagrams, JPEG for photos
    • Multiple images per message are supported (useful for comparisons)
    • Combine vision with tool use for automated document processing pipelines

    Key Takeaways

    • •Claude accepts images via base64 encoding or URL references
    • •Vision excels at document analysis, charts, UI review, and diagrams
    • •Multiple images per message enable comparison workflows
    • •Resize images to reduce token costs in production
    0 of 6 lessons marked complete

    Progress is kept in this browser only.