Triton (compiler)

Triton is an open-source programming language and compiler for writing GPU kernels in Python. It sits between high-level machine-learning frameworks, which abstract away the hardware, and low-level CUDA C++, which exposes every detail of memory layout and thread scheduling. Triton lets a developer express an algorithm in terms of tiles and blocks of computation while the compiler handles instruction selection, memory coalescing and shared-memory management.^[1]

History

Triton was created at OpenAI by Philippe Tillet and released publicly in 2021 alongside the original FlashAttention work, where it was used to express attention kernels that outperformed hand-written CUDA for the relevant shapes. The project was open-sourced under the MIT licence and hosted at openai/triton before moving to the triton-lang organisation as contributions from AMD, Google, Meta, Microsoft and others grew.^[1][2]

Its significance lies in what it replaced. Writing efficient GPU kernels in CUDA requires expert-level control over warp scheduling, shared memory tiling and occupancy; Triton's compiler performs much of that work automatically from a much simpler program description. PyTorch integrated Triton as the backend for torch.compile generated kernels, and the library became the implementation substrate for FlashAttention-2/3, grouped-query attention kernels and the fused optimiser used in large-model training.^[3]

How It Works

A Triton program is written in Python but does not run in Python: the decorator @triton.jit marks a kernel whose body is captured by the compiler's abstract syntax tree and translated into a target-specific program. The programmer reasons about blocks — tiles of the computation assigned to groups of threads — rather than individual threads.^[2]

The compiler pipeline lowers this representation through MLIR, applies optimisations including vectorisation, pipelining of memory operations and layout assignment, and emits PTX (for NVIDIA) or AMDGCN (for AMD). The result is code that can approach hand-tuned CUDA performance while being an order of magnitude shorter: a fused attention or matrix-multiplication kernel that would run to hundreds of lines of CUDA can be written in a few dozen lines of Triton.^[4]

Because kernels are cached and compiled just in time, Triton programs are portable across GPU generations without source changes in most cases. The project now targets NVIDIA, AMD and Intel accelerators, and the same design principles have been extended by third-party libraries such as Liger (LinkedIn's fused kernels for language-model training) and by research on using language models to synthesise Triton kernels directly.^[4]

Applications and Impact

Triton is used mainly where framework-level operators are too slow or where memory traffic dominates: fused activations, rotary position embeddings, attention variants, optimiser updates and quantised-inference kernels. Serving stacks including vLLM and SGLang rely on Triton-generated kernels for paged attention, and major training frameworks use them to reduce memory bandwidth pressure.^[3]

The compiler has also become a testbed for AI-assisted kernel generation: because Triton programs are short and declarative, language models can produce candidate kernels that are validated by compilation and benchmarking — an approach explored in research on compilers that synthesise kernels without the Triton toolchain itself.^[4]

>Key Takeaways

• Triton lets developers write GPU kernels in Python while the compiler handles memory and thread-level optimisation.
• It is the implementation layer behind FlashAttention, torch.compile kernels and many inference-server operators.
• MLIR-based lowering makes the same kernel portable across NVIDIA, AMD and Intel hardware.
• For resource-constrained markets such as Malaysia, it converts scarce kernel-engineering effort into measurable throughput gains.

See Also

• CUDA
• Flash Attention
• Inference
• vLLM
• NVIDIA H100
• National Semiconductor Strategy