Generative pre-trained transformer

Generative pre-training (GP) was a long-established concept in machine learning applications,[14][15] but the transformer architecture was not available until 2017 when it was invented by Google.[16] That development led to the emergence of large language models like BERT in 2018[17] and XLNet in 2019,[18] which were pre-trained transformers (PT) but not designed to be generative (they were "encoder-only").[19] Also around that time, in 2018, OpenAI published its article entitled "Improving Language Understanding by Generative Pre-Training," in which it introduced the first generative pretrained transformer (GPT) system.[20]

Prior to transformer-based architectures, the best-performing neural NLP (natural language processing) models commonly employed supervised learning from large amounts of manually-labeled data. The reliance on supervised learning limited their use on datasets that were not well-annotated, and also made it prohibitively expensive and time-consuming to train extremely large language models.[20]

The semi-supervised approach OpenAI employed to make a large scale generative system—and was first to do with a transformer model—involved two stages: an unsupervised generative "pre-training" stage to set initial parameters using a language modeling objective, and a supervised discriminative "fine-tuning" stage to adapt these parameters to a target task.[20]

A foundational model is an AI model trained on broad data at scale such that it can be adapted to a wide range of downstream tasks.[21]

Thus far, the most notable GPT foundation models have been from OpenAI's GPT-n series. The most recent from that is GPT-4, for which OpenAI declined to publish the size or training details (citing "the competitive landscape and the safety implications of large-scale models").[22]

Other such models include Google's PaLM, a broad foundation model that has been compared to GPT-3 and has recently been made available to developers via an API,[27][28] and Together's GPT-JT, which has been reported as the closest-performing open-source alternative to GPT-3 (and is derived from earlier open-source GPTs).[29]

Foundational GPTs can also employ modalities other than text, for input and/or output. GPT-4 is a multi-modal LLM that is capable of processing text and image input (though its output is limited to text).[30] Regarding multimodal output, some generative transformer-based models are used for text-to-image technologies such as diffusion[31] and parallel decoding[32]. Such kinds of models can serve as visual foundation models (VFMs) for developing downstream systems that can work with images.[33]

A foundational GPT model can be further adapted to produce more targeted systems directed to specific tasks and/or subject-matter domains. Methods for such adaptation can include additional fine-tuning (beyond that done for the foundation model) as well as certain forms of prompt engineering.[34]

An important example of this is fine-tuning models to follow instructions. In January 2022, OpenAI introduced "InstructGPT"—a series of models which were fine-tuned to follow instructions using a combination of supervised training and reinforcement learning from human feedback (RLHF) on base GPT-3 language models.[35][36] Advantages this had over the bare foundational models included higher accuracy, less negative/toxic sentiment, and generally better alignment with user needs. Hence, OpenAI began using this as the basis for its API service offerings.[37] Other instruction-tuned models have been released by others, including a fully open version.[38][39]

Another (related) kind of task-specific models are chatbots, which engage in human-like conversation. In November 2022, OpenAI launched ChatGPT—an online chat interface powered by an instruction-tuned language model trained in a similar fashion to InstructGPT.[40] They trained this model using RLHF, with human AI trainers providing conversations in which they played both the user and the AI, and mixed this new dialogue dataset with the InstructGPT dataset for a conversational format suitable for a chatbot. Other major chatbots currently include Microsoft's Bing Chat, which uses OpenAI's GPT-4 (as part of a broader close collaboration between OpenAI and Microsoft),[41] and Google's competing chatbot Bard (initially based on their LaMDA family of conversation-trained language models, with plans to switch to PaLM).[42]

Yet another kind of task that a GPT can be used for is the meta-task of generating its own instructions, like developing a series of prompts for 'itself' to be able to effectuate a more general goal given by a human user.[43] This is known as an AI agent, and more specifically a recursive one because it uses results from its previous self-instructions to help it form its subsequent prompts; the first major example of this was Auto-GPT (which uses OpenAI's GPT models), and others have since been developed as well.[44]

This section lists the main official publications from OpenAI and Microsoft on their GPT models.

GPT-1: report,[6] GitHub release.[57]

GPT-2: blog announcement,[58] report on its decision of "staged release",[59] GitHub release.[60]

GPT-3: report.[26] No GitHub or any other form of code release thenceforth.

InstructGPT: blog announcement,[35] report.[36]

ChatGPT: blog announcement (no report).[40]

GPT-4: blog announcement,[61] reports,[62][63] model card.[64]