Write-Ahead Logging
Recording the intent of a change durably before applying it, so a crash can be recovered.
Write-ahead logging (WAL) appends a description of a change to a sequential log and forces it to durable storage before the change is applied to the data itself. If the process dies mid-update, recovery replays the records belonging to committed transactions and discards the incomplete ones, so the store is never observed half-written.
The ordering constraint is the whole idea, and it is also fast: appending is sequential where updating in place is scattered, so the log absorbs the writes and the slow rewriting of pages is deferred and batched.
Durability rests on fsync. An ordinary write reaches only the operating system's page cache; an explicit flush forces the device and reports back. Flushing once per transaction bounds loss to nothing and costs a device round trip, so systems offer group commit, and looser modes that trade a window of recent commits for latency. Devices with volatile write caches have historically reported completion early, so a durability guarantee nobody has tested by cutting power is a datasheet claim rather than evidence.
Checkpointing bounds recovery. The log grows without limit, so the system periodically flushes dirty state and writes a checkpoint to replay from. Replay must be safe to repeat, making Idempotence a requirement — usually enforced by stamping each page with the sequence number last applied to it, so older records are skipped. Each record carries a checksum from a Hash Function so a torn write at the tail is discarded rather than replayed, and a record saying "set the column to the current time" breaks Determinism and must store the computed value instead.
The log doubles as a Provenance trail for every mutation, which is why it feeds replication out of a Relational Database and why followers replaying a leader's log reach Eventual Consistency.
See also5
Hash Table
A key-value structure that uses a digest of the key to index an array of buckets.
Computation & Algorithms23 connections
Seed Data
Initial content created programmatically to make an empty system usable or demonstrable.
Data & Content14 connections
Record and Replay Testing
Capturing a real interaction once and replaying it deterministically in later test runs.
Agents & Language Models14 connections
Checksum
A small value computed from data so that accidental corruption can be detected later, from parity bits and credit card check digits to the CRCs in every Ethernet frame, and useless against deliberate tampering.
Networks & Distribution6 connections
SQLite
A complete SQL database packaged as a small library that stores everything in one file, public domain since 2000, tested to an extreme standard, and probably the most widely deployed database in the world.
Data & Content9 connections
Linked from14
- ACIDData & Content
The four guarantees a transactional database makes so concurrent, interrupted work never leaves data half-done.
- Atomic File WriteSystems & Tooling
Replacing a file so that readers and crashes see either the complete old version or the complete new one, never a half-written mix, by writing a temporary file and renaming it over the original.
- B-TreeComputation & Algorithms
A self-balancing tree with many children per node, designed to minimize the number of disk reads a lookup costs.
- Change Data CaptureData & Content
Turning a database's own internal write log into a stream other systems can subscribe to, without touching the application.
- ChecksumNetworks & Distribution
A small value computed from data so that accidental corruption can be detected later, from parity bits and credit card check digits to the CRCs in every Ethernet frame, and useless against deliberate tampering.
- Database MigrationData & Content
A versioned, ordered change to a database schema, applied once and recorded.
- Database ReplicationData & Content
Copying the same data to multiple database nodes for redundancy and read scale, at the cost of a lag window.
- Event SourcingData & Content
Storing every change that ever happened, instead of only the current state, so the current state becomes a derived view.
- Eventual ConsistencyNetworks & Distribution
A guarantee that replicas converge given no further updates, without guaranteeing when.
- Hash FunctionSystems & Tooling
A function mapping arbitrary input to a fixed-size digest, used for identity, integrity, and addressing.
- Page CacheSystems & Tooling
The operating system's use of spare memory to hold recently read and written file data, which makes repeated disk access fast and makes "free memory" look alarmingly low.
- Seed DataData & Content
Initial content created programmatically to make an empty system usable or demonstrable.
- SQLiteData & Content
A complete SQL database packaged as a small library that stores everything in one file, public domain since 2000, tested to an extreme standard, and probably the most widely deployed database in the world.
- Write-Behind CacheData & Content
A cache that acknowledges a write immediately and persists it to the real store later, trading durability for write latency.