Neural Processing Unit

A neural processing unit (NPU), also known as an AI accelerator or deep learning processor, is a specialised microprocessor designed to accelerate artificial intelligence and machine learning workloads, in particular neural networks.^[1][2] Unlike general-purpose CPUs and graphics-oriented GPUs, an NPU is built around arrays of multiply-accumulate (MAC) units and on-chip memory optimised for the data-parallel, weight-stationary patterns of deep learning inference, enabling AI features to run continuously at low power.^[1][3]

History

The concept of a dedicated AI processor dates to research on neural network hardware in the 1980s, but commercial NPUs became mainstream in the late 2010s as smartphone vendors added on-device AI.^[1][2] Apple introduced the Neural Engine in its A-series chips, and Qualcomm integrated the Hexagon NPU into its Snapdragon platforms; Huawei and Samsung followed with their own designs.^[1][2] Google's Tensor mobile processors also integrate NPU-class hardware.^[2]

In the mid-2020s, NPUs became the defining component of the AI PC: Intel's Lunar Lake processors include an NPU tile rated at 48 TOPS (tera operations per second), AMD's Ryzen AI 300 series pushes its NPU to 50 TOPS, and Qualcomm's Snapdragon X Elite reaches 45 TOPS with an 18% reduction in cost per TOPS reported for the Snapdragon X2 Elite generation.^[3][4] Apple's A18 Pro Neural Engine delivers 35 TOPS on mobile.^[1]

Key Concepts

An NPU accelerates inference — running a trained model — rather than training. Its design differs from a GPU: instead of shader cores for parallel graphics, an NPU uses a dataflow architecture with MAC arrays, on-chip SRAM and sparsity-exploitation hardware aimed at deep learning's repeated multiply-accumulate operations.^[1] Key concepts include:

• TOPS (tera operations per second): the common performance metric for NPUs; AI PC procurement increasingly tracks NPU throughput alongside CPU-GPU specifications.^[3][4]
• On-device AI: because the NPU runs models locally, features such as background blur, real-time translation, keyboard prediction and small language models operate without cloud round-trips, improving privacy and responsiveness.^[1][3][5]
• Power efficiency: mobile NPUs sustain AI workloads at under 5 watts, which is what makes always-on assistants and camera AI practical in phones.^[1]
• Heterogeneous computing: NPUs work alongside CPUs, GPUs and dedicated sensing hardware within a system-on-chip, with each unit handling the workloads it does best.^[5]

Applications and Impact

NPUs are used for image processing and photography, voice recognition, real-time translation, video background removal, and increasingly for running small and medium language models on-device.^[1][3] The integration of 40+ TOPS NPUs into laptops and desktops has made local AI development and deployment feasible for enterprises, with business deployments accounting for about half of 2025 AI PC shipments.^[3][4] NPUs also reduce the load on GPUs in graphics pipelines, allowing AI-assisted upscaling and frame generation in real time.^[2]

>See Also

• Tensor Processing Unit
• AI PC
• Edge AI
• GPU Cluster
• Small Language Models

References