devShakib

Where I Put the Cache Decides Everything Else

CDN and edge caching is architecture, not config. How cache placement, stale while revalidate, stale if error, surrogate keys, and cache invalidation decide consistency and failure…

A client once asked me to "just add a CDN" the week before a launch. Traffic was going to spike, the origin was a single modest server, and someone had read that Cloudflare makes things fast. I did add the CDN. Then I spent three days deciding what to cache, where to cache it, and how it would be wrong when it was wrong. The CDN took ten minutes. The other three days were the actual job.

That project taught me something I now treat as a rule: caching is not a performance feature you bolt on at the end. Where you put the cache is an architectural decision, and it silently sets your consistency guarantees, your invalidation cost, and your failure modes. You don't get to pick those separately afterwards. The moment you decide a response lives at the edge for 60 seconds, you've decided that some users see a 60-second-old world, that a bad deploy is visible until it expires, and that a purge is now a distributed operation across dozens of POPs. Nobody framed it that way when they told me to "add a CDN."

So I stopped treating edge caching as a config panel and started treating it as a distributed-systems problem I happen to be configuring through a dashboard. That reframing changes every decision that follows, and it's the whole subject of this post. If you take one thing away before the rest: decide where the truth lives before you decide anything else, because every other caching choice inherits from that one.

The cache hierarchy is a chain of small databases

The first thing to internalize: you don't have "a cache." You have a stack of them, each with its own copy of the truth and its own idea of how stale that truth is allowed to be.

For a typical web request the HTTP caching layers look like this:

Every layer is a small distributed database holding a replica of data it doesn't own. That's the mental model. And the classic hard problems of replicated data — staleness, invalidation, split-brain reads — show up at every single layer. The browser cache is a replica you can't reach. The edge is a replica across a hundred machines that don't coordinate. Treating these as "just caches" is how you get the class of bug where one user swears the site is broken and you can't reproduce it, because they're reading a different replica than you are.

There's a vocabulary worth being precise about here, because CDNs use it in their headers and dashboards:

Once you see the hierarchy as replicas, the design question stops being "should I cache this" and becomes: at which replica does the truth need to be fresh, and what am I willing to pay for that freshness? Everything below is me answering that question in different situations.

stale-while-revalidate is a promise about consistency, not speed

Most people meet stale-while-revalidate as a speed trick, and it is a wonderful one. But read what it actually says:

Cache-Control: public, max-age=60, stale-while-revalidate=600

This is a consistency contract. It says: for 60 seconds serve the cached copy as fresh; for the next 600 seconds serve the stale copy instantly while you go refresh it in the background. Translated into distributed-systems language, you've chosen availability and low latency over consistency. You have explicitly told the system: I would rather return a wrong answer fast than a correct answer slowly.

That is often the right call. On a marketing page, a docs site, or a product listing that changes hourly, serving a ten-minute-old copy while you revalidate is invisible to users and saves your origin. But it's a choice, and it has a blast radius. The moment you put stale-while-revalidate on a response, ask:

That last point is worth dwelling on. Revalidation is cheapest when you give the cache a validator: an ETag or Last-Modified header on the response. Then the background revalidation is a conditional request (If-None-Match / If-Modified-Since), and a healthy origin answers 304 with an empty body instead of re-rendering and re-sending the whole page. Without validators, every "revalidate" is a full regeneration. If you run stale-while-revalidate on expensive pages, ship ETags too — otherwise you've optimized the read path and left the refresh path as heavy as it ever was.

The related header is stale-if-error:

Cache-Control: max-age=60, stale-if-error=86400

That one is pure failure-mode engineering. It says: if the origin returns a 5xx or times out, keep serving the last good copy for up to a day. I'll come back to this, because it's the single most underused directive in the whole spec and it's the reason some of my sites don't go down when the origin does.

Cache keys, Vary hell, and the personalization tax

A cache is a hash map. The key is everything. Get the cache key wrong and you either cache too little (useless) or serve one user's data to another (a resume-generator disaster where person A downloads person B's file). The default key is method + host + path, and the second you need to vary on anything else, you're in negotiation with the Vary header.

Vary: Accept-Encoding is fine — a handful of encodings, a handful of copies. Vary: Cookie is a trap. Cookies are effectively unique per user, so you've just told the edge to store a separate copy per user, which means your hit rate collapses to roughly zero and the cache is doing nothing but burning storage. I've reviewed setups that had Vary: Cookie on the homepage and wondered why the CDN "wasn't working." It was working perfectly. It was caching a million unique keys.

This is the personalization tax, and it's the central tension of edge caching: the more a response is tailored to one person, the less it can be shared, and sharing is the entire point of a cache. The way out is not to fight it but to split the response along the personalization seam:

On a storefront I worked on, the product pages were nominally "personalized" because they showed a cart badge. That one badge was forcing the entire HTML document to be uncacheable. We pulled the badge into a tiny client-side fetch, made the document body a shared cache key, and the edge hit rate on product pages went from near-zero to about 96%. Same page. We just stopped letting one dynamic pixel poison the cacheability of everything around it.

Normalize your keys, too. Strip marketing query params (utm_source, utm_campaign, fbclid, gclid) from the cache key, or every ad campaign fragments your cache into thousands of copies of the same page. Sort the query args you do keep so ?a=1&b=2 and ?b=2&a=1 hash to one entry. Lowercase the host. Most CDNs let you define exactly which query args, cookies, and headers participate in the key — and increasingly you can do it in code (a Worker) rather than opaque dashboard toggles, which means you can test it. Spend time here. **The cache key is your cache design.**

Cache invalidation: TTL is a confession of ignorance

There are famously two hard problems in computing, and cache invalidation is one and a half of them. In practice you have three tools, roughly in order of increasing intelligence.

1. TTL (time-based expiry)

You set max-age/s-maxage and hope. TTL is what you use when you don't actually know when the data changed, so you guess an interval and accept being wrong for up to that long. I say this with affection: a TTL is a confession that you don't have an invalidation signal. It's a fallback, not a strategy. It's fine for things that genuinely change on a clock, or where staleness is harmless. It's a poor fit for "publish this article now."

2. Purge by URL

You explicitly tell the CDN to drop a path. Precise, but brittle: you have to know every URL that contains the changed data. Change a product's price and you might need to purge the product page, three category pages, the homepage, the search results, and the API endpoint that feeds the mobile app. Miss one and it's stale. This maps badly onto reality because one piece of data lives in many URLs, and a URL-purge forces you to enumerate that fan-out by hand every time.

3. Surrogate keys (cache tags)

This is the grown-up option and where I spend my effort. You tag each response at the origin with the entities it depends on:

Surrogate-Key: product-812 category-shoes brand-nike homepage

Then when product 812 changes, you fire one purge:

curl -X POST "https://api.cdn.example/purge" \  -H "Authorization: Bearer $TOKEN" \  -d '{"surrogate_keys": ["product-812"]}'

Every cached response tagged product-812 — page, category, API, search — is invalidated in one shot, no matter how many URLs that is. This is the difference between "I hope I purged everything" and "I invalidated the entity." It turns invalidation from an enumeration problem into a graph problem the CDN solves for you, and it's the thing that makes aggressive edge caching safe rather than terrifying. (Fastly ships this as Surrogate-Key; Cloudflare Enterprise calls it Cache Tags; the concept is the same — tag on write, purge by tag on change.)

There's a subtlety worth calling out: purge-by-tag isn't instant everywhere. A purge fans out across POPs and takes a bounded but non-zero time to reach them all. For most content that's fine. For anything where two users disagreeing for a few seconds is unacceptable, don't rely on the purge race — don't edge-cache that field at all (see the split-brain section below).

The rule I follow: cache with a long TTL, and rely on surrogate-key purges for freshness. Long TTL protects the origin; event-driven purges keep the data correct. TTL becomes the safety net for the rare case your purge misses, not the primary mechanism.

Edge compute: move the logic, not the data

For years the only knob was "cache this response or don't." Edge functions — Cloudflare Workers, Fastly Compute, Vercel Edge, AWS Lambda@Edge — changed the shape of the problem. Now you can run logic at the cache, which lets you cache things that used to be uncacheable.

The reframing: instead of moving data all the way from the origin to the user on every request, you move a little logic out to the edge and keep the data cached there. Concrete patterns I actually use:

Here's a minimal Cloudflare Worker that does request collapsing-adjacent caching and stale-if-error by hand, which also demonstrates the shape of edge logic:

export default {  async fetch(request, env, ctx) {    const cache = caches.default;    const cacheKey = new Request(new URL(request.url).toString(), request);    let response = await cache.match(cacheKey);    if (response) return response; // edge hit    // Miss: fetch origin, but tolerate origin failure    try {      response = await fetch(request, { cf: { cacheTtl: 300 } });      if (response.ok) {        // Store a clone; serve the original        ctx.waitUntil(cache.put(cacheKey, response.clone()));      }      return response;    } catch (err) {      // Origin is down: serve last good copy if we have one      const stale = await cache.match(cacheKey, { ignoreMethod: true });      return stale || new Response("upstream unavailable", { status: 503 });    }  },};

The important part isn't the code, it's the shift: the edge is no longer a dumb reverse proxy. It's a programmable layer where you get to decide consistency and failure behavior per-route. That's a lot of power, and like all power at the edge it runs in a hundred places at once with no shared memory. So keep the logic small, stateless, and idempotent — anything that needs coordination (counters, locks, uniqueness) doesn't belong in a stateless edge function; push it to a durable store built for it.

The thundering herd and why request collapsing matters

Here's the failure that catches teams who cache naively. A popular page's cache entry expires. In the same second, ten thousand users request it. All ten thousand miss the cache. All ten thousand requests hit the origin simultaneously to regenerate the exact same page. Your origin, sized for the cache-hit load, falls over. Now nothing is cached, every request misses, and you're in a full outage — caused, ironically, by the cache expiring.

This is the thundering herd (also called a cache stampede or dogpile), and it's a distributed-systems problem, not a config toggle. The defenses:

Origin protection is the real job of a CDN. Latency is the marketing; shielding the origin so it survives its own popularity is the engineering. If you only tune one thing for a traffic spike, make it request collapsing plus a shield.

A caching topology I run on a $0 budget

I ship almost everything on free tiers on purpose, and caching is where that discipline pays off, because a good cache means the expensive origin barely runs. Here's a real topology I use for content-style sites — a blog, a docs site, a marketing front:

Cache-Control: public, s-maxage=3600, stale-while-revalidate=86400, stale-if-error=86400Surrogate-Key: post-hello-world section-web homepage feed

The result: origin egress is a rounding error, the site serves from the edge in single-digit milliseconds worldwide, and a fresh publish is live in the time it takes one purge API call to fan out. All of it inside free limits, because I decided where the truth lives before I decided anything else. On one project this took a function that was creeping toward paid-tier invocation counts and dropped it back to a few thousand origin hits a day — the edge was answering everything else. The lesson generalizes past hobby budgets: on any project, edge hit rate is the number that decides your origin bill.

Failure modes: what a stale edge serves when the origin dies

This is the part that separates people who "set up a CDN" from people who designed their caching. The most important question about any cache is: what does it do when the thing behind it is broken?

Three scenarios I plan for explicitly:

Origin is down (5xx or timeout). With stale-if-error, the edge keeps serving the last good copy for as long as you allowed. Your origin can be on fire, mid-deploy, or simply asleep, and users see a slightly stale but fully working site. Without it, they see the origin's 502 the instant a cache entry expires. I've had origins go down for twenty minutes during a bad migration and gotten zero user reports, purely because the edge was serving stale-if-error copies the whole time. That header is cheap insurance and almost nobody sets it.

Bad deploy poisons the cache. You ship a broken page and the edge dutifully caches the broken version for its full TTL. Now the bug is pinned at the edge and outlives the rollback of your origin — you rolled back the code and the site is still broken, which is a genuinely confusing incident to be in. This is why a fast, reliable purge path is a safety tool, not just a freshness tool: your rollback has to include a cache purge or the rollback is a lie. Test your purge like you test your deploy, and wire it into the rollback runbook.

Split-brain staleness. Different POPs hold different versions because a purge only reached some of them, or edges expired at different times. One user sees the new price, another sees the old, and both are adamant they're right. If your data model can't tolerate two users disagreeing for a bounded window, that data should not be edge-cached at all — push it to a client-side fetch against a consistent origin (or a low-TTL, no-shared-cache API route) and stop pretending the edge is a source of truth. Prices at checkout, inventory counts, permission checks: these are the fields to keep off the edge.

The throughline: a cache changes what "down" means. Designed well, the edge is a shock absorber that keeps you up while the origin recovers. Designed carelessly, it's an amplifier that pins your worst bug in front of every user across the planet and hands you a distributed purge problem in the middle of an incident.

How I'd approach a new cache decision today

When I add caching to a route now, I walk the same short checklist, and it falls straight out of everything above:

Five questions, and none of them are answered in the CDN dashboard's "on/off" toggle. That's the whole point.

Key takeaways