What is a Language Model?
A language model is a system that estimates how likely a sequence of words is, which lets it predict the next word given the words before it. In simple terms: it has seen so much text that it can guess, very well, what usually comes next. That one ability, repeated word after word, is how it produces text.
A Simple Example
Given "The results were statistically", a language model rates "significant" as far more likely than "purple". Your phone keyboard's next-word suggestions and the autocomplete in writing tools work on this principle.
From Counting Words to Neural Networks
- N-gram models: the first language models counted how often short word sequences appear. Claude Shannon used this idea to approximate English in 1948.
- Neural language models: from the 2000s, networks learned word meanings as numbers, so similar words got similar predictions.
- Transformers: since 2017, models can weigh every word in a long passage at once, which led to large language models.
Types of Language Model
- Large language models: billions of parameters, general purpose, such as the models behind ChatGPT, Claude and Gemini.
- Small language models: compact enough to run on a laptop or phone, often trained for narrower tasks.
- Vision-language models: work with images as well as text.
Applications
- Text generation, chat assistants and summarisation.
- Grammar checking and translation.
- Speech recognition, where the model picks the most likely words from similar sounds.
- Search, matching queries to documents by meaning; part of natural language processing.
What Language Models Cannot Do
A language model predicts plausible text; it does not check facts. Its knowledge comes from its training data, which has a cut-off date and gaps, so it can produce confident errors and invented sources (see AI hallucination). Check anything you plan to rely on.