MLX

MLX is an open-source array framework for machine learning developed by Apple's machine learning research team. Released in December 2023 under the MIT licence, it is designed from the ground up for Apple silicon, exploiting the unified memory architecture of M-series chips so the CPU and GPU operate on the same data without copying it between memory pools.^[1][2]

History

MLX was released in December 2023 by Apple researchers Awni Hannun, Jagrit Digani, Angelos Katharopoulos and Ronan Collobert. Its design drew on numerical and machine-learning frameworks including NumPy, PyTorch and JAX: the Python interface mirrors NumPy, the neural-network package mlx.nn follows PyTorch conventions, and its composable function transformations echo JAX.^[1][2] The framework matured through 2024 and 2025, reaching version 0.30+ with support for the neural accelerators in Apple's M5 chips announced at WWDC 2025.^[3]

Key Concepts and Technology

Four ideas define MLX.^[2][4]

• Unified memory. Arrays are allocated once in shared memory visible to CPU and GPU. Instead of moving tensors with .to("cuda"), the programmer specifies the device when an operation runs — mx.add(a, b, stream=mx.cpu) or stream=mx.gpu — eliminating host-to-device copies and an entire class of placement bugs. A Mac with 128 GB of unified memory can therefore hold models far larger than any consumer discrete GPU's VRAM.
• Lazy evaluation. Operations record nodes in a computation graph that is only evaluated when results are needed (mx.eval), letting MLX fuse and schedule work before execution — which also means a forgotten eval in a loop can silently grow memory.
• Composable transformations. grad, vmap and compile nest arbitrarily; a gradient of a compiled function is a valid program in one line.
• Metal GPU backend. Kernels are hand-written Metal shaders running through Apple's GPU stack, with bindings for Python, Swift, C and C++ so the same model runs on macOS, iOS, iPadOS and visionOS.^[1][4]

The companion package mlx-lm provides text generation, quantised model loading, fine-tuning and an OpenAI-compatible local server. Because token generation is bound by memory bandwidth rather than raw compute, bandwidth — not chip name — is the practical ceiling on local LLM speed.^[4]

Applications and Impact

MLX is used mainly for local LLM inference, experimentation and prototyping on MacBooks and Mac Studios, plus image generation and speech recognition on Apple devices. It lets a single laptop run quantised 7B–70B models privately, with no cloud round-trip, and is the natural on-device path for researchers who want CUDA-free workflows. Compared with PyTorch + CUDA it remains weaker for large-scale production training, and there is no fallback into the CUDA kernel ecosystem — FlashAttention, bitsandbytes and Triton kernels must be reimplemented or replaced.^[3][4] Apple positions MLX as the research and power-user framework, with Core ML and the Foundation Models framework serving shipping applications.^[4]

>See Also

• PyTorch
• JAX
• Ollama
• llama.cpp
• Quantization