AI systems face a fundamentally different threat model than traditional software. Instead of exploiting code vulnerabilities, attackers manipulate the AI's reasoning through carefully crafted inputs.

Why AI Security Is Different

Traditional software security relies on clear boundaries between code and data. LLMs blur this boundary — user input (data) directly influences behavior (code). This creates a new attack surface with no direct equivalent in classical security.

OWASP Top 10 for LLM Applications

The OWASP Foundation identified the top security risks for LLM applications:

1. Prompt Injection: Manipulating model behavior through crafted inputs
2. Insecure Output Handling: Trusting LLM outputs without validation
3. Training Data Poisoning: Corrupting training data to influence model behavior
4. Model Denial of Service: Crafting inputs that consume excessive resources
5. Supply Chain Vulnerabilities: Compromised models, plugins, or dependencies
6. Sensitive Information Disclosure: Model revealing training data or system details
7. Insecure Plugin Design: Plugins with excessive permissions or poor input validation
8. Excessive Agency: Giving AI too much autonomous action capability
9. Overreliance: Trusting AI outputs without human verification
10. Model Theft: Extracting model weights or capabilities through API probing

Attack Motivations

The Fundamental Challenge

There is currently no provably secure defense against prompt injection. Every defense is a heuristic that can potentially be bypassed with sufficient effort. Security is therefore about raising the cost of attack and detecting/responding to breaches, not eliminating all risk.

This mirrors traditional security: no system is unhackable, but well-defended systems aren't worth the effort for most attackers.