End-to-End Vision App Architecture

Image Upload Pipeline

async function processImage(file: File) {
  // 1. Validate
  if (!["image/jpeg", "image/png", "image/webp"].includes(file.type)) {
    throw new Error("Unsupported format");
  }
  if (file.size > 10 * 1024 * 1024) throw new Error("File too large");

  // 2. Optimize
  const optimized = await resizeImage(file, { maxWidth: 2048 });

  // 3. Upload to storage
  const url = await uploadToStorage(optimized);

  // 4. Analyze with AI
  const analysis = await analyzeImage(url);

  // 5. Store results
  await saveAnalysis(analysis);

  return analysis;
}

Batch Processing

For processing many images efficiently:

• Queue images for async processing
• Use batch APIs where available
• Process in parallel with concurrency limits
• Cache results to avoid reprocessing

Cost Optimization

• Resize images before sending to APIs (most accept up to 2048px)
• Use lower-cost models for simple classification
• Cache analysis results for repeated images
• Use local models for high-volume, simple tasks

Performance Monitoring

• Track analysis latency per image size
• Monitor API error rates and retry patterns
• Measure accuracy with periodic human review
• Set up alerts for unusual processing patterns or costs

Common Pitfall: Analyzing Every Frame of Video with Vision LLMs

Sending video frame-by-frame to GPT-4o or Gemini is extremely expensive. A 60-second video at 30fps = 1,800 frames. At GPT-4o's vision pricing this could cost $5–$15 per minute of video. For video analysis, use purpose-built video APIs (Google Video Intelligence, AWS Rekognition Video) or sample intelligently — extract keyframes every 2–5 seconds or detect scene changes and only process frames where the scene changes. Use a cheap image classifier to pre-filter frames before sending relevant ones to the expensive LLM.

Best Practice: Generate Alt Text for All Vision-Analyzed Images

Any image you analyze with an AI vision API should have AI-generated alt text stored alongside it. This improves accessibility, SEO, and provides a natural fallback when images fail to load. The marginal cost is minimal since you are already paying for the vision analysis:

async function analyzeWithAltText(imageUrl: string) {
  const response = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [{
      role: "user",
      content: [
        { type: "image_url", image_url: { url: imageUrl } },
        { type: "text", text: 'Return JSON: { "analysis": "<your main analysis>", "alt_text": "<concise descriptive alt text for screen readers, under 125 chars>" }' },
      ],
    }],
    response_format: { type: "json_object" },
  });
  return JSON.parse(response.choices[0].message.content!);
}

Named Product Reference: Roboflow

Roboflow (roboflow.com) is the most developer-friendly platform for training and deploying custom object detection models. You upload labeled images, choose a detection architecture (YOLOv8, RT-DETR), train in the cloud, and deploy to a hosted endpoint or export to ONNX for edge deployment. For domain-specific detection tasks — defect detection on a production line, counting items on shelves, identifying specific plant species — Roboflow's custom training typically outperforms general-purpose LLM vision at 10–100x lower inference cost per image.