18 min
Lesson 1 of 4
AI Threat Landscape
From the course Enterprise AI Security
Enterprise AI systems face a unique threat landscape that extends beyond traditional cybersecurity into AI-specific attack vectors.
AI-Specific Threats
Training Phase Attacks
- Data Poisoning: Injecting malicious training data to corrupt model behavior
- Backdoor Attacks: Hidden triggers that cause misclassification
- Model Stealing: Extracting model behavior through API queries
- Training Data Extraction: Recovering private training data from models
Inference Phase Attacks
- Adversarial Examples: Carefully crafted inputs that fool models
- Prompt Injection: Manipulating LLM behavior through malicious inputs
- Jailbreaking: Bypassing safety filters in LLMs
- Model Inversion: Reconstructing training data from model outputs
- Membership Inference: Determining if specific data was in training set
Example Threats by AI System Type
LLMs:
- Prompt injection via user-controlled content
- Indirect injection through retrieved documents (RAG)
- Jailbreaking safety filters
- System prompt extraction
Computer Vision:
- Adversarial patches on physical objects
- Digital perturbations undetectable to humans
- Physical-world attacks (fooling autonomous vehicles)
Recommendation Systems:
- Data poisoning to promote specific items
- Profile injection attacks
- Shilling attacks by competing merchants
OWASP Top 10 for LLMs
- Prompt Injection
- Insecure Output Handling
- Training Data Poisoning
- Model Denial of Service
- Supply Chain Vulnerabilities
- Sensitive Information Disclosure
- Insecure Plugin Design
- Excessive Agency
- Overreliance
- Model Theft
Key Takeaways
- •AI threats span both training (poisoning, backdoors) and inference (adversarial, injection) phases
- •LLMs face unique threats including prompt injection and indirect injection through RAG
- •OWASP Top 10 for LLMs provides a comprehensive threat taxonomy
- •Model theft via API queries can extract proprietary model behavior without access to weights
0 of 4 lessons marked complete
Progress is kept in this browser only.