Token Budget
The finite allowance of model usage a task may consume, and the design decisions it forces.
A token budget is a cap on how much model usage a piece of work may consume, whether denominated in money, in Tokens, or in a provider's rate limit. It is a real engineering constraint rather than an accounting detail, because the cheapest way to exceed it is to do something that looks reasonable.
Parallelism is the main multiplier. Running several model workers concurrently multiplies spend by their number, and a wide fan-out repeated twice can exhaust a monthly allowance mid-task, killing work in flight. Narrow, staggered batches cost the same in total and fail more gracefully. See Fan-Out and Fan-In.
The second multiplier is context. Every turn resends the accumulated conversation, so a long session's cost grows superlinearly. Storing artifacts outside the window and passing handles, summarizing rather than pasting, and delegating self-contained work to a Subagent are all budget techniques as much as clarity techniques.
Budgeting also shapes what is worth automating. A recording run that costs a few dollars is worth doing without ceremony; one that costs orders of magnitude more deserves a decision first. And an expensive regeneration launched against a working copy someone else is mid-edit produces a stale result at full price — check before spending. See Model Routing.
See also5
Context Window
The bounded span of tokens a model can attend to in a single request.
Agents & Language Models20 connections
Model Routing
Choosing which model handles which step, trading capability against cost and latency.
Agents & Language Models9 connections
Multi-Agent Orchestration
Coordinating several model instances on one body of work, each with its own context.
Agents & Language Models10 connections
Resource Starvation
Failures caused by competition for a finite shared resource rather than by any defect.
Testing & Verification12 connections
Large Language Model
A neural network trained to predict text, used as a general-purpose instruction-following system.
Agents & Language Models32 connections
Related2
Nearby in the graph rather than deliberately chosen. Looser, sometimes surprising.
Linked from15
- Agentic LoopAgents & Language Models
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
- Context WindowAgents & Language Models
The bounded span of tokens a model can attend to in a single request.
- Fan-Out and Fan-InAgents & Language Models
Splitting work across parallel workers and recombining their results.
- Git WorktreeVersion Control & Delivery
An additional working directory attached to one repository, each on its own branch.
- Large Language ModelAgents & Language Models
A neural network trained to predict text, used as a general-purpose instruction-following system.
- Model RoutingAgents & Language Models
Choosing which model handles which step, trading capability against cost and latency.
- Multi-Agent OrchestrationAgents & Language Models
Coordinating several model instances on one body of work, each with its own context.
- QuantizationAgents & Language Models
Storing a model's weights at lower numeric precision to shrink memory and speed inference, at some cost to accuracy.
- Race ConditionTesting & Verification
A defect whose occurrence depends on the relative timing of concurrent operations.
- Resource StarvationTesting & Verification
Failures caused by competition for a finite shared resource rather than by any defect.
- Retrieval-Augmented GenerationAgents & Language Models
Fetching relevant documents at query time and feeding them into the prompt, so a model answers from retrieved text rather than memory.
- SubagentAgents & Language Models
A model instance spawned by another to handle a scoped task with its own context.
- Time as ResourcePlay & Games
A design in which the scarce resource is the schedule itself, so every activity is paid for with another left undone.
- TokenAgents & Language Models
The sub-word unit a language model actually reads and writes; the unit of cost and of context.
- Vector DatabaseAgents & Language Models
A datastore built to find the nearest neighbors of a high-dimensional vector quickly, at a scale exact search can't reach.