Memoization
Caching a function's result against its input so a repeat call with the same input skips recomputation.
Memoization caches a Pure Function's return value keyed by its arguments, so a second call with the same arguments returns the cached result instead of recomputing it. It's only sound for pure functions — a memoized wrapper around a function whose output can change for the same input (reads the clock, reads a mutable global) will happily serve a stale answer with no way to know it's wrong, since the cache has no way to detect the function's real dependency changed.
The classic demonstration is recursive Fibonacci: computed naively, fib(n) makes exponentially many redundant calls, because fib(n-2) gets recomputed from scratch by both fib(n-1) and the direct call from fib(n). Memoizing each fib(k) the first time it's computed collapses that to linear work, with no change to the recursive structure of the code itself — the technique is a pure add-on, not a rewrite.
This is the exact overlap point with Dynamic Programming: memoization applied systematically to a recursive definition, so that every overlapping subproblem is computed once, is the top-down form of dynamic programming. The bottom-up form builds the same table by iterating from the base cases forward instead of recursing down and caching on the way back up; both compute the same values, and the choice between them is usually about which is easier to write correctly for a given recurrence, not about performance.
The failure modes are a cache with no eviction policy — a memoized function called with unboundedly many distinct arguments over a long-running process leaks memory exactly like any unbounded cache, and needs a Least Recently Used Cache or similar bound — and, more subtly, memoizing something that looks pure but has a hidden dependency: memoizing a function of a mutable object's reference rather than its contents will serve a stale result the moment the object is mutated in place, since the cache key never changed even though the meaningful input did.
See also4
Dynamic Programming
Solving a problem by solving each distinct overlapping subproblem exactly once and reusing the results.
Computation & Algorithms4 connections
Pure Function
A function whose output depends only on its input and which causes no observable effect outside itself.
Computation & Algorithms13 connections
Recursion
A function defined in terms of itself, solving a problem by solving smaller instances of the same problem.
Computation & Algorithms10 connections
KV Cache
The stored attention keys and values for every token already processed, which lets a model generate each new token without re-reading the whole sequence.
Agents & Language Models10 connections
Linked from10
- Amortized AnalysisComputation & Algorithms
Averaging cost over a sequence of operations so occasional expensive ones don't distort the per-operation bound.
- Dynamic ProgrammingComputation & Algorithms
Solving a problem by solving each distinct overlapping subproblem exactly once and reusing the results.
- KV CacheAgents & Language Models
The stored attention keys and values for every token already processed, which lets a model generate each new token without re-reading the whole sequence.
- Lazy EvaluationComputation & Algorithms
Deferring a computation until its result is actually needed, and then caching it so it never runs twice.
- Materialized ViewData & Content
The stored result of a query, kept as a table so that expensive reads become cheap, at the cost of keeping that result up to date.
- Prompt CachingAgents & Language Models
Reusing the computed state of a prompt's unchanged prefix across requests, which only works if that prefix stays byte-identical.
- Pure FunctionComputation & Algorithms
A function whose output depends only on its input and which causes no observable effect outside itself.
- RecursionComputation & Algorithms
A function defined in terms of itself, solving a problem by solving smaller instances of the same problem.
- Shader Compilation StutterGraphics & Rendering
Frame hitches caused by compiling GPU programs the first time they are needed, and the caching schemes that try to move that work elsewhere.
- Test IsolationTesting & Verification
The property that each test passes or fails on its own, regardless of which other tests run, in what order, or in parallel.