Kelly Mears

Ground Truth

The measurement that arbitrates, as opposed to the artifact that merely reports.

Method2 min read207 words12 out · 98 in
also calledSource of truth

Ground truth is the observation that settles a question, as distinct from the many artifacts that describe it. Most debugging confusion comes from treating a report as a measurement.

Examples of the distinction are everywhere. A build tool's summary table is a report; the actual script tags in the served HTML are ground truth. A hosting platform's "mergeable" flag is a report; performing the merge locally is ground truth. A green exit code is a report; the count of tests that actually ran is ground truth — a filter that matched nothing exits zero. A comment claiming an invariant is a report; the type system is ground truth.

The habit worth building is to ask, for any claim you are about to act on, what would I have to look at to be wrong about this? — then look at that. It is usually cheaper than the reasoning it replaces.

Ground truth also has a staleness dimension. A measurement is only about the thing you measured, at the moment you measured it. Verifying against a server compiled before your change, or a cached artifact from a prior state, produces a real number about the wrong world. Restart, rebuild, refetch before every claim you intend to stand behind.

See also12

Reproducible Case

The smallest set of steps that reliably produces a fault, and the unit of real debugging.

Method27 connections

Instrumentation

Adding measurement to code so its execution can be observed, and the distortions that introduces.

Testing & Verification19 connections

Vacuous Truth

A statement that holds only because its subject set is empty — a green result that measured nothing.

Method26 connections

Exhaustive Claim

A statement of the form "the last one" or "nothing else does this" — load-bearing, and only as good as the search behind it.

Method23 connections

Truncation Bias

Reading a truncated result as if it were the whole result, so every counterexample is invisible.

Method28 connections

Falsifiability

A claim is only worth something if you know what observation would refute it.

Method47 connections

Observability

How much of a system's internal state can be inferred from what it emits.

Method32 connections

Determinism

The property that identical inputs produce identical outputs, and the foundation of caching and verification.

Systems & Tooling61 connections

Silent Failure

A failure that produces no signal, so the absence of an error reads as success.

Method73 connections

Root Cause Analysis

Escalating past the visible symptom until you find the layer that actually produced it.

Method49 connections

Test Oracle

Whatever decides whether a test's output is correct, and the often-overlooked hard part of testing when nobody knows the right answer in advance.

Testing & Verification11 connections

Precision and Recall

The two measures of how well a classifier or search finds what it should, how often its hits are right and how many of the real cases it catches, and why improving one usually costs the other.

Agents & Language Models14 connections

Linked from98