Guardrail
A rule constraining what a model may do, enforced by prompt, schema, or surrounding code.
A guardrail is a constraint on model behavior: a category of action it must refuse, a confirmation it must obtain, a shape its output must take. Guardrails live at three different strengths, and conflating them is the usual source of trouble.
Prompt-level rules are advisory. They shape behavior reliably enough to be useful and are not enforcement — a rule stated in prose is followed most of the time.
Schema-level rules are enforced by Constrained Decoding: the model cannot emit what the grammar disallows. This is the strongest lever and the reason "make it required" beats "ask nicely".
Code-level rules are enforced by the host: a permission check, a scope restriction, an approval gate. This is the only level that holds against Prompt Injection, because it does not depend on the model cooperating.
Two failure modes are worth naming. A guardrail can be too strong for its context — a policy against unsupported claims will cause a model to decline a task that merely resembles the prohibited one, which turns an evaluation into a coin flip. And an asymmetric permission (writes constrained to one target, reads unconstrained) leaves a gap where the model aims a write at something it read; closing it may come down to wording an error message so it names the likely mistake rather than offering a bare list of valid choices.
See also11
Human in the Loop
Requiring a person's judgment at chosen points in an otherwise automated process.
Agents & Language Models12 connections
Prompt Engineering
Shaping a model's input to make the desired behavior reliable rather than occasional.
Agents & Language Models22 connections
System Prompt
The standing instructions that frame every turn of a conversation with a model.
Agents & Language Models16 connections
Agentic Loop
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
Agents & Language Models16 connections
Tool Use
Giving a model a set of callable functions so it can act on the world rather than only describe it.
Agents & Language Models27 connections
Least Privilege
Granting only the permissions actually required, so a compromise has a small ceiling.
Systems & Tooling38 connections
Evaluation Harness
A repeatable test suite for model behavior, since prompts and tool descriptions cannot be type-checked.
Agents & Language Models18 connections
Large Language Model
A neural network trained to predict text, used as a general-purpose instruction-following system.
Agents & Language Models38 connections
Model Card
A short standard document published alongside a machine learning model describing what it's for, how it was trained and evaluated, where it performs worse, and what it shouldn't be used for, now routine on model hubs and for frontier releases.
Agents & Language Models5 connections
Agent Sandbox
Confining an agent's commands to a limited environment, so that a mistaken or hijacked action can only damage what the sandbox allows it to reach.
Agents & Language Models14 connections
Precision and Recall
The two measures of how well a classifier or search finds what it should, how often its hits are right and how many of the real cases it catches, and why improving one usually costs the other.
Agents & Language Models14 connections
Linked from16
- Agent SandboxAgents & Language Models
Confining an agent's commands to a limited environment, so that a mistaken or hijacked action can only damage what the sandbox allows it to reach.
- Agentic LoopAgents & Language Models
The cycle of model turn, tool call, observation, repeat — the basic mechanism of an AI agent.
- Constrained DecodingAgents & Language Models
Restricting a model's next-token choices to those a formal grammar permits.
- Evaluation HarnessAgents & Language Models
A repeatable test suite for model behavior, since prompts and tool descriptions cannot be type-checked.
- Fair-Play MysteryMeaning & Society
The convention that a detective story must give the audience every clue it needs to solve the case.
- HallucinationAgents & Language Models
Fluent, confident output that is not grounded in anything real.
- Human in the LoopAgents & Language Models
Requiring a person's judgment at chosen points in an otherwise automated process.
- Large Language ModelAgents & Language Models
A neural network trained to predict text, used as a general-purpose instruction-following system.
- Least PrivilegeSystems & Tooling
Granting only the permissions actually required, so a compromise has a small ceiling.
- Model CardAgents & Language Models
A short standard document published alongside a machine learning model describing what it's for, how it was trained and evaluated, where it performs worse, and what it shouldn't be used for, now routine on model hubs and for frontier releases.
- Model Context ProtocolAgents & Language Models
An open protocol for exposing tools, data, and prompts to AI applications through a uniform interface.
- Precision and RecallAgents & Language Models
The two measures of how well a classifier or search finds what it should, how often its hits are right and how many of the real cases it catches, and why improving one usually costs the other.
- PromptAgents & Language Models
The text supplied to a language model to elicit a response.
- Prompt InjectionAgents & Language Models
Untrusted content that reaches a model's context and is treated as instruction.
- System PromptAgents & Language Models
The standing instructions that frame every turn of a conversation with a model.
- Tool UseAgents & Language Models
Giving a model a set of callable functions so it can act on the world rather than only describe it.