Regression
A previously working behaviour that a change has broken.
A regression is a behaviour that used to work and no longer does. The word carries an implicit accusation — something we changed caused this — which makes it the most useful and most misapplied label in a bug tracker.
Distinguishing a real regression from its impostors is most of the skill. An intermittent failure that clears on rerun is a Flaky Test. A failure that appears only on one machine is usually environmental — a different port, a different model tier, a different disk. A failure in a file the change never touched may be a Semantic Conflict with someone else's change, invisible to both diffs. Treating any of these as a regression sends you looking for a defect that is not there.
The tool that separates them is bisection: find the first revision at which the behaviour differs. It converts an argument into an interval. Bisection depends on each revision being independently sound, which is one of the arguments for the Atomic Commit.
Guarding against regressions is what a test suite is for, and the guard only exists if it has been seen to fail — see Falsifiability. A subtler case: a change can delete the only assertion that covered a property without deleting any code, so the property silently stops being checked. Nothing goes red; the net simply has a hole.
See also5
Hand-picked in the note itself — the neighbours worth reading next.
Reproducible Case
The smallest set of steps that reliably produces a fault, and the unit of real debugging.
Method17 connections
Code Coverage
The proportion of code executed by a test suite — a map of what is untested, not a measure of quality.
Testing & Verification12 connections
Version Control
A system recording the history of a project's files, enabling branching, merging, and recovery.
Version Control & Delivery13 connections
Root Cause Analysis
Escalating past the visible symptom until you find the layer that actually produced it.
Method21 connections
Visual Regression Testing
Comparing rendered output against a stored baseline to catch unintended visual change.
Testing & Verification10 connections
Linked from15
Notes elsewhere in the wiki that reach for this one.
- AssertionTesting & Verification
The statement in a test that must hold, and the only part that can actually fail.
- Atomic CommitVersion Control & Delivery
A commit containing exactly one logical change, complete and independently sound.
- Code CoverageTesting & Verification
The proportion of code executed by a test suite — a map of what is untested, not a measure of quality.
- CommitVersion Control & Delivery
A recorded snapshot of a project with a message explaining it.
- DeprecationMethod
Marking something as superseded and scheduled for removal, without removing it yet.
- Flaky TestTesting & Verification
A test that passes and fails on identical input, and the most corrosive thing a suite can contain.
- RebaseVersion Control & Delivery
Replaying a branch's commits onto a new base, producing new commits with the same changes.
- Reproducible CaseMethod
The smallest set of steps that reliably produces a fault, and the unit of real debugging.
- Root Cause AnalysisMethod
Escalating past the visible symptom until you find the layer that actually produced it.
- Semantic ConflictVersion Control & Delivery
Two changes that merge cleanly and are incompatible in meaning.
- Test-Driven DevelopmentTesting & Verification
Writing a failing test first, making it pass, then improving the code with the test as a net.
- Three-Way MergeVersion Control & Delivery
Combining two lines of work by comparing both against their common ancestor.
- Unit TestTesting & Verification
A fast, isolated test of one unit of behaviour with its collaborators replaced.
- Version ControlVersion Control & Delivery
A system recording the history of a project's files, enabling branching, merging, and recovery.
- Visual Regression TestingTesting & Verification
Comparing rendered output against a stored baseline to catch unintended visual change.