Beginner6 min readlive prototype

Random

Throw out a random entry. Embarrassingly close to LRU on real workloads.

Overview

What this concept solves

Random replacement is exactly what it sounds like: when you need to evict, pick a random entry and drop it. There is no tracking of recency, frequency, or age. There is no list to maintain. The only state is the set of cached entries.

It's the policy people assume must be terrible — and then they measure it. On many realistic workloads, random eviction lands within a few percentage points of LRU. CPU caches have used pseudo-random policies for decades for precisely this reason: the implementation cost is zero, and the hit-rate loss is small.

Mechanics

How it works

There is no algorithm

State: a set or hash map of entries. On insert, if the cache is over capacity, pick a uniformly random entry and remove it. That's the whole policy.

  1. Get(key) — look up in the map. Return value or miss. No bookkeeping.
  2. Put(key, value) — insert. If size > capacity, pick a random victim and delete.

Pseudo-random is usually enough

CPU caches use 'pseudo-LRU' or true-random bits derived from a lightweight LFSR — anything that isn't easily predictable. The randomness doesn't need to be cryptographic; it just needs to be uncorrelated with the access pattern.

Why it isn't a disaster

If 80% of your cache holds the hot working set, then a random eviction has an 80% chance of evicting a hot key — bad! But here's the rescue: most workloads have re-references. The next time the evicted hot key is asked for, it'll be reloaded, and now it's protected from the very next random eviction (which is statistically more likely to land on cold data again). The error self-corrects.

Interactive prototype

Run it. Break it. Tune it.

Sandboxed simulation embedded right in the page. No setup, no install.

About this simulation

Capacity 4. When the cache is full and a miss arrives, a spinner cycles through the slots before landing on a random victim. No bookkeeping, no list shuffling — just luck. Hit Run demo twice: you'll get different evictions each time, and surprisingly similar hit rates.

Hands-on

Try these on your own

Open the prototype above, run each experiment, predict the answer, then verify.

try 01

Watch the spinner pick a loser

Click A, B, C, D to fill the cache. Now click E — the spinner lights up each slot in sequence, slows down, and lands on a random one. The evicted letter is whoever was unlucky.

try 02

Run the demo twice and compare

Hit Run demo, note the hit count and which keys ended up in the cache. Reset and run it again. Different victims, different final cache state, and yet a similar hit rate — that's the surprising part.

try 03

A hot key can still be evicted

Click A five times — it's now your hottest key. Click B, C, D, then trigger a miss with E. There's still a 25% chance A is the victim. Random doesn't protect hot keys; the workload has to do that by re-asking for them.

In practice

When to use it — and what you give up

When to reach for it

  • Embedded systems and CPU caches — when even one extra metadata byte per line is unacceptable.
  • High-contention paths — no per-entry state means no contended writes.
  • Memcached's sampling fallback — Memcached samples a few random items and evicts the oldest among them. With sample size 1, that's pure random; with larger samples, it approaches LRU.
  • As a sanity baseline — if your fancy policy isn't measurably beating random, your fancy policy isn't pulling its weight.

Real-world example

Many x86 CPUs use pseudo-LRU, but ARM and several POWER chips use pure random replacement in their L2/L3 caches. Redis can also approximate LRU by random sampling — the maxmemory-samples 5 knob means 'pick 5 at random, evict the oldest'.

Pros

  • Zero bookkeeping — no list, no counter, no timestamp.
  • Trivially thread-safe — no shared mutable state on reads.
  • Surprisingly competitive on realistic workloads, often within 5-10% of LRU's hit rate.
  • Hardware-friendly — pseudo-random victim selection is a single LFSR step.

Cons

  • No protection for hot keys — they're evicted at the same rate as cold ones.
  • Variance: two runs of the same trace give different hit rates. Painful for reproducible benchmarks.
  • Easy to beat for the cost of a tiny bit of metadata. If you can afford LRU's overhead, LRU wins outright.

Reference

Code & further reading

A minimal reference implementation and pointers worth bookmarking.

random.ts
// True random replacement. Stupidly simple, surprisingly effective.
class RandomCache<K, V> {
  private store = new Map<K, V>();
  constructor(private capacity: number) {}

  get(key: K): V | undefined {
    return this.store.get(key);
  }

  set(key: K, value: V): void {
    if (!this.store.has(key) && this.store.size >= this.capacity) {
      // Pick a random key and evict it.
      const keys = Array.from(this.store.keys());
      const victim = keys[Math.floor(Math.random() * keys.length)];
      this.store.delete(victim);
    }
    this.store.set(key, value);
  }
}

// Redis-style "sample N, evict oldest of N" — random as N -> 1,
// approaches LRU as N -> cache size.
class SampledLRU<K, V> {
  private store = new Map<K, { value: V; lastUsed: number }>();
  private clock = 0;
  constructor(private capacity: number, private samples = 5) {}

  set(key: K, value: V): void {
    if (!this.store.has(key) && this.store.size >= this.capacity) {
      const keys = Array.from(this.store.keys());
      let victim = keys[Math.floor(Math.random() * keys.length)];
      for (let i = 1; i < this.samples; i++) {
        const candidate = keys[Math.floor(Math.random() * keys.length)];
        if (this.store.get(candidate)!.lastUsed < this.store.get(victim)!.lastUsed) {
          victim = candidate;
        }
      }
      this.store.delete(victim);
    }
    this.store.set(key, { value, lastUsed: ++this.clock });
  }
}

References & further reading

3 sources

Knowledge check

Did the prototype land?

Quick questions, answers revealed on submit. No scoring saved.

question 01 / 03

Why is random replacement competitive with LRU on many realistic workloads?

question 02 / 03

Redis sets `maxmemory-samples 5` by default for its LRU eviction. What does this control?

question 03 / 03

What is the single biggest downside of random eviction in a benchmarking context?

0/3 answered

Was this concept helpful?

Tell us what worked, or what to improve. We read every note.