Large Language Model
A neural network trained to predict text, used as a general-purpose instruction-following system.
A large language model is a neural network trained on very large text corpora to predict continuations of a sequence. The training objective is narrow — what comes next — but at sufficient scale the resulting system can follow instructions, write and analyse code, summarise, translate, and hold a conversation.
Three properties drive nearly all practical design around them.
They are probabilistic. The same input can produce different outputs. This is not a defect to be engineered away but a property to be designed for; see Nondeterminism.
They have a bounded working memory. Everything the model can consider must fit in its Context Window, measured in Tokens. Deciding what goes in there is most of the engineering.
They produce fluent text regardless of whether they know the answer. A confident, well-formed, wrong answer costs the same to generate as a right one; see Hallucination.
Models are typically offered in tiers trading capability against cost and latency, which makes Model Routing a real architectural decision. Around the model sit the mechanisms that make it useful in a system: a System Prompt establishing role and rules, Tool Use for reaching the outside world, Structured Output for machine-readable results, and an Evaluation Harness for knowing whether a change helped.
See also4
Hand-picked in the note itself — the neighbours worth reading next.
Prompt
The text supplied to a language model to elicit a response.
Agents & Language Models12 connections
Agentic Loop
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
Agents & Language Models12 connections
Token Budget
The finite allowance of model usage a task may consume, and the design decisions it forces.
Agents & Language Models11 connections
Guardrail
A rule constraining what a model may do, enforced by prompt, schema, or surrounding code.
Agents & Language Models13 connections
Linked from22
Notes elsewhere in the wiki that reach for this one.
- Agentic LoopAgents & Language Models
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
- Anchoring EffectMethod
A stated number pulls subsequent estimates toward it, regardless of relevance.
- Constrained DecodingAgents & Language Models
Restricting a model's next-token choices to those a formal grammar permits.
- Context WindowAgents & Language Models
The bounded span of tokens a model can attend to in a single request.
- Evaluation HarnessAgents & Language Models
A repeatable test suite for model behaviour, since prompts and tool descriptions cannot be type-checked.
- FingerprintAgents & Language Models
A hash over everything that determined a result, used to detect when the result has gone stale.
- GuardrailAgents & Language Models
A rule constraining what a model may do, enforced by prompt, schema, or surrounding code.
- HallucinationAgents & Language Models
Fluent, confident output that is not grounded in anything real.
- JSON SchemaAgents & Language Models
A vocabulary for describing the shape of JSON data, and the lingua franca of model tool interfaces.
- Model RoutingAgents & Language Models
Choosing which model handles which step, trading capability against cost and latency.
- NondeterminismAgents & Language Models
The property that identical inputs may produce different outputs, and what it costs to test around.
- PromptAgents & Language Models
The text supplied to a language model to elicit a response.
- Prompt EngineeringAgents & Language Models
Shaping a model's input to make the desired behaviour reliable rather than occasional.
- Prompt InjectionAgents & Language Models
Untrusted content that reaches a model's context and is treated as instruction.
- Record and Replay TestingAgents & Language Models
Capturing a real interaction once and replaying it deterministically in later test runs.
- Schema ValidationAgents & Language Models
Checking data against a declared shape at runtime, and using the result as the typed value.
- Structured OutputAgents & Language Models
Constraining a model to emit data conforming to a declared schema rather than free text.
- System PromptAgents & Language Models
The standing instructions that frame every turn of a conversation with a model.
- Test DoubleTesting & Verification
A stand-in for a real dependency during a test — and the ways a stand-in can lie.
- TokenAgents & Language Models
The sub-word unit a language model actually reads and writes; the unit of cost and of context.
- Token BudgetAgents & Language Models
The finite allowance of model usage a task may consume, and the design decisions it forces.
- Tool UseAgents & Language Models
Giving a model a set of callable functions so it can act on the world rather than only describe it.