GPT

GPT (Generative Pre-trained Transformer) is a series of large language models developed by OpenAI that generate text by predicting the next token in a sequence. First introduced in 2018, the GPT line demonstrated that a single neural architecture trained on web-scale text could be adapted to almost any language task, a pattern that became the template for the modern AI industry and the foundation of ChatGPT.^[1]

History

The first GPT model was described in OpenAI's 2018 paper "Improving Language Understanding by Generative Pre-Training". It combined an unsupervised pre-training phase on the BookCorpus dataset with supervised fine-tuning per task — the recipe the family is named after. GPT-2 (2019), with 1.5 billion parameters, showed that larger models could perform zero-shot task completion, and its staged, cautious release became an early reference point for responsible disclosure of language-model capabilities. GPT-3 (2020), at 175 billion parameters, established few-shot learning via prompts alone and motivated the shift away from task-specific model pipelines.^[2]

GPT-4 (March 2023) added multimodal input and markedly higher performance on professional and academic benchmarks. GPT-4o (2024) unified text, audio and image processing in a single real-time model, and GPT-5 (August 2025) introduced dynamic routing between fast-response and extended-reasoning modes. The GPT-6 generation arrived in September 2026: GPT-6 Astra for frontier reasoning and computer use, followed by the cheaper GPT-6 Sol and GPT-6 Luna tiers, and GPT-6.1 Sol on 29 September 2026.^[3][4]

Key Concepts and Technology

GPT models are decoder-only Transformers trained with a self-supervised next-token prediction objective. Because the training signal comes from the text itself, no labelled datasets are required, allowing the use of extremely large and diverse corpora. The generative pre-training stage produces general linguistic competence; instruction-following behaviour is then added through fine-tuning and reinforcement learning from human feedback (RLHF).^[2]

Scaling is the family's defining trend: each generation has grown in parameters, training data and compute, accompanied by improvements such as sparse mixture-of-experts routing, grouped-query attention for efficient inference, and long context windows reaching millions of tokens. Since the GPT-4 era, OpenAI has described model behaviour in terms of adjustable reasoning effort rather than a single fixed configuration, letting callers trade latency and cost against depth of reasoning.^[4]

Applications and Impact

GPT models power ChatGPT — the consumer application that reached 100 million users within two months of its November 2022 launch — as well as the OpenAI API used for coding assistants, document drafting, customer support, translation, tutoring and data extraction. The acronym "GPT" also entered general usage as a shorthand for conversational AI systems of any make. Economically, the GPT line made OpenAI one of the most valuable private companies in the world and triggered parallel model families at every major technology firm.^[3]

>See Also

• ChatGPT
• OpenAI
• GPT-5
• GPT-6 coverage — see GPT-5
• Transformer Architecture
• Large Language Models
• RLHF