Search Results
3 results for “attention”
Foundations
Attention Mechanism
A neural network technique that enables models to dynamically weight the relevance of different parts of an input sequence when producing each output element, forming the core of transformer architectures.
6 min readUpdated May 2026
Foundations
Context Window
The maximum number of tokens — including the prompt, prior conversation, retrieved documents, and the model's own output — that a large language model can process in a single forward pass.
5 min readUpdated May 2026
Foundations
Transformer Architecture
A neural network architecture introduced in 2017 that uses self-attention mechanisms to process sequential data in parallel, forming the foundation of modern large language models and multimodal AI systems.
7 min readUpdated May 2026