Kelly Mears

Human in the Loop

Requiring a person's judgement at chosen points in an otherwise automated process.

Agents & Language Models2 min read206 words9 out · 7 in
also calledApproval gateHITL

Human in the loop describes a system that pauses for a person's decision at defined points rather than running to completion unattended. The design question is never whether to have one but where — a gate on every step destroys the value of automation, and a gate on nothing eventually does something irreversible.

The placement heuristic that holds up: gate on actions that are hard to reverse or outward-facing. Publishing, sending, deleting, spending, and anything that leaves the machine are worth confirming. Local, reversible, inspectable work is not — version control makes most of it undoable, and asking about it trains people to approve without reading, which is worse than not asking.

Two subtleties. Authorisation does not generalise across contexts. Approval to do a thing once, or in one place, is not approval to do it everywhere. And the gate must receive what it needs to judge with. A reviewer who reads changes on a hosting platform cannot see evidence that lives only in a local scratch directory; if the evidence is not where the decision is made, the gate is decorative.

The complementary practice is honest reporting: a gate is only as good as the account it is given. See Fail Fast and Code Review.

See also6

Hand-picked in the note itself — the neighbours worth reading next.

Related2

Nearby in the graph rather than deliberately chosen. Looser, sometimes surprising.

Linked from7

Notes elsewhere in the wiki that reach for this one.