LLMs (Large Language Models)

A large language model (LLM) is a neural network trained on a very large amount of text to predict language, and used to read and generate it. Google’s machine learning glossary defines the term loosely: “At a minimum, a language model having a very high number of parameters”, and more informally any Transformer-based language model. There is no agreed size at which a model becomes large; the term covers a family of large language models with different architectures and training objectives.

How one works

Text is split into tokens, and an autoregressive decoder computes probabilities for the next token from its input and generated prefix. To generate, one token is selected, appended to the input, and the process repeats until a stop condition. The selection step, decoding, may take the most probable token or sample among likely ones, which is why the same question can get different answers. The comparison with a powerful autocomplete is fair as far as the mechanism goes; it undersells what training at this scale produces, and it is a reminder that the model continues text rather than consulting facts.

Most current LLMs use the Transformer architecture, whose attention mechanism combines information from positions allowed by its attention mask; causal decoders cannot attend to future tokens, while bidirectional encoders can use both sides of an unmasked position. The architecture is explained step by step in Transformers from Scratch. Transformers come in three arrangements with different uses. Decoder models generate text left to right and are what chat assistants and most current LLMs are. Encoder models, such as BERT, are trained to fill in masked words and produce representations of text for classification and search, rather than serving as ordinary left-to-right generators. Their masked-token predictions use a different objective. Encoder-decoder models read an input and write an output, a design first used for translation.

How one is built

Most assistants are made in stages. Pretraining fits the model to predict tokens across a very large, broad body of text, often billions to trillions of tokens, and produces a base or foundation model: general and capable, but inclined to continue text rather than follow requests. Objectives, data, and the scaling behavior of this stage are covered in LLM Pretraining. Instruction tuning then trains it on examples of instructions paired with good responses, so that it answers requests. Preference alignment adjusts it further using judgments of which responses are better, to make it more helpful and to reduce harmful output. These later stages are described in Fine-Tuning and Alignment.

An application can adapt a model without training it, by writing instructions and examples into the prompt, or can fine-tune it on task data. Google’s course material describes a foundation model as a platform rather than a finished solution for tasks such as regression or classification: something to build on, not a replacement for a system designed for the task.

What it does well, and what needs checking

LLMs are strong at transforming text they are given: summarizing, extracting, classifying, translating, rewriting, and drafting, as well as writing and explaining code. They handle many such tasks from instructions alone, though quality varies by task and language and has to be measured on examples of the actual work.

The limits follow from the same mechanism. Training leaves information and patterns in model parameters, but a reported cutoff does not promise complete or correct knowledge before that date. A model call has no independent fact lookup unless the application or service supplies one. The same course states the consequence plainly: “LLMs hallucinate, meaning their predictions often contain mistakes.” A hallucination reads as fluently as a correct answer. Training text carries social and factual biases into outputs. The reasons behind a particular answer are hard to inspect. And both training and serving consume substantial computation, so cost and latency are design constraints, not afterthoughts. A useful rule for deciding where to rely on a model is whether the answer is in the input it receives or must come from somewhere else; supplied evidence can support checking, while absent facts need an authorized source. Exact arithmetic and counting can still fail even when all operands are present, so use deterministic computation where exactness matters.

How these properties translate into system design, including tokens, context limits, structured output, latency, cost, and choosing a model, is worked through in Valid JSON, Wrong Date.

References:

• Google for Developers, Machine Learning Glossary (Creative Commons Attribution 4.0) — large language model, foundation model, instruction tuning.
• Google for Developers, Machine Learning Crash Course: LLMs: What’s a large language model? (Creative Commons Attribution 4.0).
• Google for Developers, Machine Learning Crash Course: Fine-tuning, distillation, and prompt engineering (Creative Commons Attribution 4.0).
• Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding