Modern AI models can process text, images, audio, and video simultaneously. Mastering multimodal prompting unlocks powerful capabilities that text-only prompting can't match.

Image + Text Prompting

Most frontier models (GPT-5, Gemini, Claude) accept images alongside text:

Analysis Prompts:

[Upload image] "Analyze this UI screenshot. Identify: accessibility issues, visual hierarchy problems, and inconsistent spacing. Return findings in a table with severity ratings."

[Upload chart] "Extract the data from this bar chart into a CSV format. Then identify the top 3 trends."

Comparison Prompts:

[Upload 2 images] "Compare these two product designs. Which is more likely to convert? Analyze layout, CTA placement, color psychology, and information hierarchy."

Creative Prompts:

[Upload image] "Describe this image in enough detail that a different AI model could recreate it. Include composition, colors, lighting, style, and mood."

Image-to-Code:

[Upload wireframe] "Convert this wireframe into a responsive React component using Tailwind CSS. Match the layout, spacing, and element hierarchy exactly."

Audio Prompting

With models like Gemini that accept audio:

[Upload audio clip] "Transcribe this audio, then: 1) Summarize the key points, 2) List action items, 3) Identify the speaker's tone and sentiment."

[Upload song] "Analyze this song's structure (verse, chorus, bridge), key, tempo, instrumentation, and lyrical themes."

Video Prompting

Gemini can process entire videos:

[Upload video] "Watch this product demo video and create: 1) A 30-second script summarizing the key features, 2) Timestamps for each feature shown, 3) Three social media posts promoting it."

[Upload tutorial video] "Create a step-by-step written tutorial from this video. Include screenshots descriptions at key moments."

Cross-Modal Workflows

The real power is combining modalities:

1. Image → Text → Image: Upload a sketch → get a detailed description → use the description to generate a polished image in Midjourney
2. Audio → Text → Action: Transcribe a meeting → extract action items → create Jira tickets
3. Video → Text → Content: Analyze a competitor's video → generate a comparison blog post
4. Document Image → Structured Data: Upload a form/receipt → extract data into JSON

Prompting Tips for Multimodal

• Reference the image/audio/video explicitly: "In the uploaded image..."
• Be specific about what to focus on: "Focus on the navigation bar in this screenshot"
• Combine modalities: "Compare the audio tone with the visual mood of this video"
• Use images as examples: upload an example of your desired output format
• For multiple images, label them: "Image 1 is the current design. Image 2 is the proposed redesign."