Advanced12 min readlive prototype

ARC — Adaptive Replacement Cache

Self-tunes between recency and frequency using 'ghost' lists of recently-evicted keys.

Overview

What this concept solves

ARC — Adaptive Replacement Cache — is what you build when you can't decide between LRU and LFU. Invented at IBM in 2003 by Megiddo and Modha, it keeps two LRU-ish lists in parallel: one for entries seen once recently, and one for entries seen more than once. The brilliant trick is that it also remembers the keys it just evicted — and uses that history to retune the boundary between the two lists.

The result is a cache that behaves like LRU when the workload is recency-dominated and like LFU when frequency dominates — without any tuning parameter. It famously powers Sun ZFS's adaptive cache and inspired the algorithms behind PostgreSQL's clock-sweep and IBM DB2.

Mechanics

How it works

Four lists, one moving target

ARC maintains four lists for a cache of size c:

  • T1 — entries seen exactly once recently (the LRU half).
  • T2 — entries seen at least twice (the frequent half).
  • B1 — ghost entries: keys recently evicted from T1. Only keys are stored, not values.
  • B2 — ghost entries: keys recently evicted from T2.

T1 and T2 together hold the cached values; B1 and B2 are pure metadata that fit on the side. ARC tracks a target size p for T1 (so T2 targets c - p). The algorithm:

  1. New key, not anywhere → insert at the head of T1. Evict from whichever list is over its target.
  2. Hit in T1 or T2 → move to the head of T2 (it's been seen twice now).
  3. Hit in B1 (ghost) → 'I evicted that too soon; recency matters more.' Increase p. Bring the key back into T2.
  4. Hit in B2 (ghost) → 'I evicted that too soon; frequency matters more.' Decrease p. Bring the key back into T2.

The ghost lists are the magic

Without B1 and B2, ARC would have no signal about which axis is hurting it. The ghosts cost almost nothing — just a key, no value — but they let ARC see the cost of its recent eviction decisions and adapt.

Interactive prototype

Run it. Break it. Tune it.

Sandboxed simulation embedded right in the page. No setup, no install.

About this simulation

Four lists plus a tunable target p. T1 holds keys seen once; T2 keys seen twice or more (together they hold the real cache, capped at 4). B1 and B2 are ghost lists — keys recently evicted from T1 / T2 with no data attached. A ghost-hit in B1 bumps p up (favor recency); a ghost-hit in B2 bumps p down (favor frequency). Watch the p value at the top retune itself in real time.

Hands-on

Try these on your own

Open the prototype above, run each experiment, predict the answer, then verify.

try 01

Promote from T1 to T2

Click A. It enters T1 (top-left). Click A again — it moves to T2 (top-right) because ARC now considers it 'frequent'. Two accesses is all it takes.

try 02

Watch a ghost hit retune p

Click A, B, C, D, E — fill the cache. By now some keys have spilled into B1 (dashed ghosts, bottom-left). Click one of those ghost keys. The status line flags it as GHOST B1, and the p value at the top jiggles up. ARC just learned 'I evicted that too soon — favor recency next time'.

try 03

Send a scan, protect T2

Click Reset. Click A, A to plant A in T2. Now scan: C, D, E, F. Watch T1 churn through them while T2 (and A) stay untouched — T2 only accepts keys seen at least twice, so a single-pass scan can't pollute it.

In practice

When to use it — and what you give up

When to reach for it

  • Mixed workloads where the recency/frequency balance varies over time — backup jobs, then OLTP, then a scan, then OLTP again.
  • You don't want a tuning parameter — ARC chooses p for you, continuously.
  • Buffer pools for storage engines and filesystems, where the workload is genuinely unpredictable.

Real-world example

Solaris ZFS uses ARC for its in-RAM block cache — that's where most engineers first encounter the name. PostgreSQL adopted a clock-based approximation after considering ARC. Worth noting: ARC is patented by IBM (US 6,996,676) — which is why some projects, including the Linux kernel, deliberately avoid it.

Pros

  • Self-tuning — adapts between recency and frequency without a knob.
  • Scan-resistant — a one-shot scan only churns T1 while T2 (the proven-popular entries) is protected.
  • Consistently better hit rate than LRU and LFU across a wide range of workload mixes.

Cons

  • Significantly more complex to implement and reason about than LRU.
  • Higher memory overhead — four lists plus ghost metadata.
  • IBM patent (US 6,996,676) — some projects can't use it for legal reasons. CAR and CAR-T are patent-free variants designed for the same purpose.

Reference

Code & further reading

A minimal reference implementation and pointers worth bookmarking.

arc.ts
// Sketch only — full ARC needs careful list manipulation.
// T1, T2: hold real values; B1, B2: hold keys only (ghosts).
class ARC<K, V> {
  private t1 = new Map<K, V>(); // recent, seen once
  private t2 = new Map<K, V>(); // frequent, seen 2+ times
  private b1 = new Set<K>();    // ghost of T1
  private b2 = new Set<K>();    // ghost of T2
  private p = 0;                // target size of T1, learned

  constructor(private c: number) {}

  get(key: K): V | undefined {
    // Hit in T1 or T2 -> promote to T2 (head)
    if (this.t1.has(key)) {
      const v = this.t1.get(key)!;
      this.t1.delete(key);
      this.t2.set(key, v);
      return v;
    }
    if (this.t2.has(key)) {
      const v = this.t2.get(key)!;
      this.t2.delete(key); this.t2.set(key, v); // touch
      return v;
    }
    return undefined;
  }

  set(key: K, value: V): void {
    // Ghost hits adapt p: B1 -> favor recency; B2 -> favor frequency
    if (this.b1.has(key)) {
      this.p = Math.min(this.c, this.p + Math.max(1, this.b2.size / this.b1.size));
      this.replace(key);
      this.b1.delete(key);
      this.t2.set(key, value);
      return;
    }
    if (this.b2.has(key)) {
      this.p = Math.max(0, this.p - Math.max(1, this.b1.size / this.b2.size));
      this.replace(key);
      this.b2.delete(key);
      this.t2.set(key, value);
      return;
    }
    // Brand new key
    if (this.t1.size + this.t2.size >= this.c) this.replace(key);
    this.t1.set(key, value);
  }

  private replace(_: K): void { /* evict from T1 or T2 based on p */ }
}

References & further reading

3 sources

Knowledge check

Did the prototype land?

Quick questions, answers revealed on submit. No scoring saved.

question 01 / 03

What is the purpose of ARC's 'ghost' lists B1 and B2?

question 02 / 03

Why is ARC scan-resistant in a way that LRU is not?

question 03 / 03

ARC has an active patent. What's the most common workaround in patent-sensitive codebases?

0/3 answered

Was this concept helpful?

Tell us what worked, or what to improve. We read every note.