PyTorch

PyTorch is an open-source machine learning framework used for building, training, and deploying deep neural networks. First released in 2016 by Facebook's AI Research lab (now Meta AI), it has become the dominant framework in academic research and increasingly in industry. Since 2022 it has been governed by the vendor-neutral PyTorch Foundation under the Linux Foundation, with contributions from Meta, Microsoft, NVIDIA, Amazon, and others.

PyTorch is built around the tensor, a multi-dimensional array similar to a NumPy array but with the ability to run on graphics processing units (GPUs) and other accelerators. Its defining characteristic is the dynamic computational graph, in which the structure of the network is defined as code runs rather than compiled ahead of time. This "define-by-run" approach makes models easier to debug and modify, which contributed strongly to its early adoption among researchers.

Core components

The framework is organised into several building blocks. The torch package provides the core tensor library and mathematical operations. The torch.autograd engine performs automatic differentiation, computing the gradients required for training through backpropagation. The torch.nn module supplies pre-built layers, activation functions, and loss functions, while torch.optim implements optimisation algorithms such as stochastic gradient descent and Adam. Data handling is managed through torch.utils.data, which provides datasets and loaders for efficient batching.

A typical training loop in PyTorch is written explicitly: the developer passes a batch of data through the model, computes a loss, calls a method to calculate gradients, and updates the model weights. This transparency is valued for teaching and experimentation.

Performance and compilation

With the release of PyTorch 2.0 in 2023, the framework introduced torch.compile, a feature that compiles Python model code into optimised kernels for faster execution while retaining the flexibility of eager mode. Subsequent releases have continued this trajectory. PyTorch 2.9, released in October 2025, added a stable application binary interface for third-party C++ and CUDA extensions, symmetric memory for programming multi-GPU kernels across NVLink and RDMA networks, and broader hardware support including AMD ROCm, Intel XPU, and CUDA 13. Releases through the 2.10 to 2.12 series in early 2026 extended these optimisations further.

Ecosystem and adoption

PyTorch underpins a large ecosystem. The Hugging Face Transformers library, the most widely used hub for pre-trained models, is built primarily on PyTorch. Domain libraries such as torchvision, torchaudio, and torchtext provide datasets and models for vision, audio, and language tasks. PyTorch Lightning and similar wrappers reduce boilerplate for large-scale training.

By 2025, PyTorch was used in roughly 85 percent of published deep-learning research papers and held a majority share of production model development, while TensorFlow retained strength in certain enterprise deployment pipelines. The two frameworks have converged in capability, and many organisations use PyTorch for research before porting stable models to production runtimes.

Comparison with other frameworks

| Framework | Origin | Graph style | Primary strength | |-----------|--------|-------------|------------------| | PyTorch | Meta AI | Dynamic (define-by-run) | Research, flexibility | | TensorFlow | Google | Static and eager | Production deployment | | JAX | Google | Functional, compiled | High-performance research |

References

1. PyTorch Foundation. (2025). PyTorch 2.9 Release Blog. pytorch.org/blog/pytorch-2-9.
2. Paszke, A., et al. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. NeurIPS.
3. PyTorch Foundation. (2026). PyTorch Documentation and Release Notes. pytorch.org.
4. Wikipedia. (2026). PyTorch. en.wikipedia.org/wiki/PyTorch.