What is a Large Language Model?
A large language model (LLM) is a language model with billions of parameters, trained on a very large body of text, that can follow instructions and produce fluent text on almost any topic. Well-known families include OpenAI's GPT, Anthropic's Claude, Google's Gemini and Meta's Llama.
How LLMs Work
- Tokens: text is split into tokens, pieces of words; in English a token is roughly three quarters of a word.
- Transformer architecture: a deep learning design whose attention mechanism lets the model weigh every token in the input when predicting the next one.
- Parameters: the numbers the model adjusts while learning. GPT-3 (2020) had 175 billion.
- Context window: the amount of text the model can consider at once, from a few pages to whole books in recent models.
How LLMs Are Trained
- Pretraining: the model learns to predict the next token across huge amounts of text (its training data).
- Fine-tuning: it is trained further on examples of good answers to instructions.
- Learning from feedback: people rate answers and the model is tuned towards the preferred ones, a method called reinforcement learning from human feedback (RLHF).
LLM vs AI vs GPT
- AI is the whole field; an LLM is one kind of AI, and part of generative AI.
- GPT (generative pre-trained transformer) is OpenAI's family of LLMs.
- ChatGPT is an app built on GPT models. So yes, ChatGPT runs on a large language model, and so does Claude.
Open and Closed Models
Some models, such as Llama and some of Mistral's, publish their weights so anyone can run them; others, such as GPT and Claude, are available only through the provider's apps and API.
LLMs in Academic Writing
LLMs are strong at rephrasing, summarising and explaining, and weak at facts: they can invent references (see AI hallucination). Academic tools often connect the model to real sources instead. In Papero, for example, the Claims check finds sentences that need a citation and offers papers from OpenAlex to cite.