Context Window
The bounded span of tokens a model can attend to in a single request.
The context window is the maximum number of Tokens a Large Language Model can consider at once. Everything the model knows about the immediate situation — instructions, conversation history, tool definitions, retrieved files, tool results — must fit inside it.
Its boundedness is the central constraint of applied language-model work. As windows have grown from a few thousand tokens to a million, the constraint has changed character rather than disappeared: the question is now less "will it fit" and more "does including this help". Filling a window with marginally relevant material measurably degrades attention to the material that mattered.
This makes context engineering the real discipline: choosing what to load, when, and in what form. The recurring techniques are to summarize rather than paste, to store artifacts outside the window and pass handles to them, to retrieve on demand rather than pre-loading, and to prune instructions describing capabilities that no longer exist.
Long-running work eventually exceeds any window. The usual answers are summarizing older turns into a compact form, delegating self-contained work to a Subagent with its own window, and persisting durable facts to Agent Memory.
See also4
Token Budget
The finite allowance of model usage a task may consume, and the design decisions it forces.
Agents & Language Models20 connections
Prompt
The text supplied to a language model to elicit a response.
Agents & Language Models14 connections
Agentic Loop
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
Agents & Language Models16 connections
Multi-Agent Orchestration
Coordinating several model instances on one body of work, each with its own context.
Agents & Language Models11 connections
Related2
Nearby in the graph rather than deliberately chosen. Looser, sometimes surprising.
Linked from25
- Agent MemoryAgents & Language Models
Durable facts stored outside the context window and recalled into later sessions.
- Agentic LoopAgents & Language Models
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
- Byte-Pair EncodingAgents & Language Models
The compression trick that splits text into the tokens a language model reads, by repeatedly merging the most frequent pair of symbols into a new one.
- EmbeddingAgents & Language Models
A numeric vector representing a piece of content such that semantic similarity between items becomes geometric distance between vectors.
- Fan-Out and Fan-InAgents & Language Models
Splitting work across parallel workers and recombining their results.
- Fine-TuningAgents & Language Models
Continuing to train an already-trained model on a narrower dataset to specialize its behavior, instead of prompting the base model.
- Human in the LoopAgents & Language Models
Requiring a person's judgment at chosen points in an otherwise automated process.
- In-Context LearningAgents & Language Models
A language model's ability to pick up a task from examples or instructions in its prompt, without any change to its weights, and the quirks of how it does so.
- KV CacheAgents & Language Models
The stored attention keys and values for every token already processed, which lets a model generate each new token without re-reading the whole sequence.
- Large Language ModelAgents & Language Models
A neural network trained to predict text, used as a general-purpose instruction-following system.
- Markov ChainAgents & Language Models
A process whose next state depends only on the present state, and the basis of pre-neural text generation.
- Mechanistic InterpretabilityAgents & Language Models
Reverse-engineering the computation a trained network learned into human-legible features and circuits.
- Model Context ProtocolAgents & Language Models
An open protocol for exposing tools, data, and prompts to AI applications through a uniform interface.
- Multi-Agent OrchestrationAgents & Language Models
Coordinating several model instances on one body of work, each with its own context.
- PromptAgents & Language Models
The text supplied to a language model to elicit a response.
- Prompt CachingAgents & Language Models
Reusing the computed state of a prompt's unchanged prefix across requests, which only works if that prefix stays byte-identical.
- Prompt InjectionAgents & Language Models
Untrusted content that reaches a model's context and is treated as instruction.
- Reasoning ModelAgents & Language Models
Language models trained to think at length before answering, trading tokens and latency for accuracy on hard problems, and the second axis of scaling after model size.
- Recurrent Neural NetworkAgents & Language Models
Networks that read a sequence one step at a time while carrying a hidden state forward, the architecture transformers displaced and that state space models are reviving.
- RerankingAgents & Language Models
Retrieving many candidates cheaply and then reordering a short list with a slower, more accurate model, the standard way search and RAG systems get relevant results into a limited context.
- Retrieval-Augmented GenerationAgents & Language Models
Fetching relevant documents at query time and feeding them into the prompt, so a model answers from retrieved text rather than memory.
- SubagentAgents & Language Models
A model instance spawned by another to handle a scoped task with its own context.
- TokenAgents & Language Models
The sub-word unit a language model actually reads and writes; the unit of cost and of context.
- Token BudgetAgents & Language Models
The finite allowance of model usage a task may consume, and the design decisions it forces.
- Vector DatabaseAgents & Language Models
A datastore built to find the nearest neighbors of a high-dimensional vector quickly, at a scale exact search can't reach.