Large Language Model
A neural network trained to predict text, used as a general-purpose instruction-following system.
A large language model is a neural network trained on very large text corpora to predict continuations of a sequence. The training objective is narrow — what comes next — but at sufficient scale the resulting system can follow instructions, write and analyze code, summarize, translate, and hold a conversation.
Three properties drive nearly all practical design around them.
They are probabilistic. The same input can produce different outputs. This is not a defect to be engineered away but a property to be designed for; see Nondeterminism.
They have a bounded working memory. Everything the model can consider must fit in its Context Window, measured in Tokens. Deciding what goes in there is most of the engineering.
They produce fluent text regardless of whether they know the answer. A confident, well-formed, wrong answer costs the same to generate as a right one; see Hallucination.
Models are typically offered in tiers trading capability against cost and latency, which makes Model Routing a real architectural decision. Around the model sit the mechanisms that make it useful in a system: a System Prompt establishing role and rules, Tool Use for reaching the outside world, Structured Output for machine-readable results, and an Evaluation Harness for knowing whether a change helped.
See also4
Prompt
The text supplied to a language model to elicit a response.
Agents & Language Models13 connections
Agentic Loop
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
Agents & Language Models13 connections
Token Budget
The finite allowance of model usage a task may consume, and the design decisions it forces.
Agents & Language Models15 connections
Guardrail
A rule constraining what a model may do, enforced by prompt, schema, or surrounding code.
Agents & Language Models14 connections
Linked from32
- Agentic LoopAgents & Language Models
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
- Anchoring EffectMethod
A stated number pulls subsequent estimates toward it, regardless of relevance.
- Automatic Speech RecognitionAgents & Language Models
Turning audio into text, and the reasons real-world accuracy is set by the signal more than by the model.
- Chain-of-Thought PromptingAgents & Language Models
Prompting a model to write out intermediate reasoning steps before its final answer, which measurably improves accuracy on multi-step problems.
- Constrained DecodingAgents & Language Models
Restricting a model's next-token choices to those a formal grammar permits.
- Context WindowAgents & Language Models
The bounded span of tokens a model can attend to in a single request.
- EmbeddingAgents & Language Models
A numeric vector representing a piece of content such that semantic similarity between items becomes geometric distance between vectors.
- Evaluation HarnessAgents & Language Models
A repeatable test suite for model behavior, since prompts and tool descriptions cannot be type-checked.
- Fine-TuningAgents & Language Models
Continuing to train an already-trained model on a narrower dataset to specialize its behavior, instead of prompting the base model.
- FingerprintAgents & Language Models
A hash over everything that determined a result, used to detect when the result has gone stale.
- GuardrailAgents & Language Models
A rule constraining what a model may do, enforced by prompt, schema, or surrounding code.
- HallucinationAgents & Language Models
Fluent, confident output that is not grounded in anything real.
- JSON SchemaAgents & Language Models
A vocabulary for describing the shape of JSON data, and the lingua franca of model tool interfaces.
- Markov ChainAgents & Language Models
A process whose next state depends only on the present state, and the basis of pre-neural text generation.
- Mechanistic InterpretabilityAgents & Language Models
Reverse-engineering the computation a trained network learned into human-legible features and circuits.
- Model RoutingAgents & Language Models
Choosing which model handles which step, trading capability against cost and latency.
- NondeterminismAgents & Language Models
The property that identical inputs may produce different outputs, and what it costs to test around.
- PromptAgents & Language Models
The text supplied to a language model to elicit a response.
- Prompt EngineeringAgents & Language Models
Shaping a model's input to make the desired behavior reliable rather than occasional.
- Prompt InjectionAgents & Language Models
Untrusted content that reaches a model's context and is treated as instruction.
- QuantizationAgents & Language Models
Storing a model's weights at lower numeric precision to shrink memory and speed inference, at some cost to accuracy.
- Record and Replay TestingAgents & Language Models
Capturing a real interaction once and replaying it deterministically in later test runs.
- Retrieval-Augmented GenerationAgents & Language Models
Fetching relevant documents at query time and feeding them into the prompt, so a model answers from retrieved text rather than memory.
- Schema ValidationAgents & Language Models
Checking data against a declared shape at runtime, and using the result as the typed value.
- Structured OutputAgents & Language Models
Constraining a model to emit data conforming to a declared schema rather than free text.
- System PromptAgents & Language Models
The standing instructions that frame every turn of a conversation with a model.
- Temperature SamplingAgents & Language Models
A parameter that scales how sharply a model favors its most likely next token, trading determinism for variety.
- Test DoubleTesting & Verification
A stand-in for a real dependency during a test — and the ways a stand-in can lie.
- TokenAgents & Language Models
The sub-word unit a language model actually reads and writes; the unit of cost and of context.
- Token BudgetAgents & Language Models
The finite allowance of model usage a task may consume, and the design decisions it forces.
- Tool UseAgents & Language Models
Giving a model a set of callable functions so it can act on the world rather than only describe it.
- Vector DatabaseAgents & Language Models
A datastore built to find the nearest neighbors of a high-dimensional vector quickly, at a scale exact search can't reach.