Edge AI refers to the practice of running artificial intelligence inference and, in some cases, training directly on end-user devices — smartphones, IoT sensors, wearables, autonomous vehicles, or industrial edge servers — rather than sending data to a centralised cloud service. The model executes where the data originates, which eliminates the round-trip latency of a remote API call and allows the system to keep functioning when network connectivity is unavailable. Privacy benefits are substantial because raw sensor data, audio, or video never leaves the device.

Think of it like having a translator travel with you rather than phoning a translation bureau every time you need a word. The answers come instantly, you aren't dependent on a phone signal, and your private conversation stays between you and the translator.

A practical example is on-device wake-word detection. The tiny neural network that listens for "Hey Siri" runs entirely on the phone's neural engine. No audio is streamed to Apple's servers until the wake word is confirmed. The model is typically an aggressively compressed network produced through Quantization and Knowledge Distillation to fit within a few megabytes and a tight power budget. Inference is measured in milliseconds.

Edge AI overlaps with topics like Quantization (compressing weights to int8 or int4 to fit constrained memory), Knowledge Distillation (training a small student model from a larger teacher), and Foundation Model deployment strategies. The Neural Network architectures chosen for edge deployment are often Convolutional Neural Networks or small Transformer variants specifically benchmarked against mobile hardware accelerators.

You will encounter Edge AI when reading about mobile ML frameworks (Core ML, TensorFlow Lite, ONNX Runtime Mobile), embedded vision systems, smart speakers, and privacy-preserving healthcare wearables where sending raw biometric data to the cloud is either impractical or legally restricted.