Neural Processing Unit

A neural processing unit (NPU) is a specialised processor designed to accelerate the neural-network computations — above all the matrix arithmetic behind deep learning — that power artificial-intelligence features on smartphones, personal computers and other devices. NPUs are tuned for efficient, always-on inference at low power, and their performance is typically expressed in TOPS, or trillions of operations per second; since 2024 the term has entered mainstream computing through the AI PC category and Microsoft's Copilot+ standard, which requires an NPU of at least 40 TOPS alongside 16 GB of memory and 256 GB of storage.^[1][2]

Background

Apple pioneered the consumer NPU with the Neural Engine in its A11 Bionic chip in 2017, rated at roughly 600 billion operations per second; the design scaled across iPhone generations and Apple silicon, reaching about 38 TOPS in the M4 processor.^[1][7] Smartphone chipmakers followed with dedicated AI blocks, and the PC industry arrived later still: AMD introduced its first x86 processors with integrated NPUs in the Ryzen 7040 series in 2023, Intel added a neural engine to its Core Ultra chips later that year, and Qualcomm's Snapdragon X series powered the first Copilot+ PCs after Microsoft defined the standard in mid-2024.^[2][3] Generational jumps followed quickly — Intel's Lunar Lake at 48 TOPS and Panther Lake at 50 TOPS from early 2026, AMD's Ryzen AI 300 at up to 50 TOPS and Ryzen AI 400 at 60 TOPS, and Qualcomm's Snapdragon X2 Elite, rated at 80 TOPS for 2026 flagship devices.^[3][4]

Architecture and performance

NPUs execute the multiply-and-accumulate operations that dominate neural networks — convolutions and matrix multiplications — in fixed-function, low-precision hardware, trading the flexibility of CPUs and GPUs for energy efficiency and low, always-on power draw.^[1] TOPS ratings are the industry's headline metric, but memory capacity and bandwidth often matter more for large language models running locally, and software support — through runtimes such as ONNX and OpenVINO and vendor toolkits — determines what actually runs on a given device.^[2][3]

Applications

The first wave of NPU workloads is consumer-facing: live captions and translation, background blur in video calls, photo and video processing, face recognition, and local inference of language models that keeps personal data on the device.^[1][3] The market has grown in step: analysts valued the global NPU market at between US$2.5 billion and US$8.6 billion in 2024, projected to reach US$15.7 billion to US$25.9 billion by 2033, and IDC projects that 93 per cent of personal computers will ship with NPUs by 2028.^[1]

>See Also

• Tensor Processing Unit
• AI PC
• Edge AI
• Quantization
• ONNX