Caching: remember it instead of redoing it

Why does a site you have already opened show up almost instantly the second time? Thanks to the cache.

The idea: keep a copy within reach

Imagine asking the same thing a hundred times from someone who has to walk to the other end of the building each time. Smarter: they bring back a copy, keep it on their desk, and answer you right away.

A cache is that desk. A temporary copy of something that was already fetched or computed, stored somewhere closer or faster, so it can be reused instead of redone.

Two possible cases

• In the cache (a "hit"): the answer is already there and goes out immediately.
• Not in the cache (a "miss"): it is fetched from the source, which is slower, and a copy is kept for next time.

Where does caching happen?

Almost everywhere, often several times at once:

• Your browser keeps the images, styles and scripts of a site you already visited.
• A content delivery network (CDN) holds copies of a site on servers around the world, close to each visitor.
• The application server remembers the result of an expensive operation, for example in Redis.
• The database keeps frequently requested data in memory.

The price to pay: stale data

A copy can become wrong. If a product's price changes while the cache still holds the old one, some visitors see an outdated price.

There are two ways to deal with it:

• a time to live (TTL): the copy expires after a few seconds, minutes or hours, then it is rebuilt;
• an invalidation: when the data changes, the copy is deleted right away.

Knowing when to throw a copy away is famously hard: that is the tricky part of caching.

Watch out for personal data

A shared cache must never keep a page that belongs to one user (their account, their basket) and serve it to someone else. In that case you do not cache, or you cache per user.

Remember

A cache trades freshness for speed. Use it on what is slow to get and rarely changes, with a clear time to live, and never on shared personal data.