A cache keeps a reusable copy of data or results to avoid repeating more expensive work. It can reduce queries, computation, and latency, but adds a decision: how long that copy can be used and what happens when it is no longer available or up to date.
The design starts with the read contract. A catalog may tolerate some delay in reflecting a change; a decision about inventory or a balance needs to protect its rule in the authoritative system. The same data can have uses with different requirements.
Location and access cost
The cache can reside within the application process, in a shared service such as Redis, in the browser, or in a CDN. Being close to the consumer reduces part of the journey, but proximity and speed must be measured along the real path.
A local cache avoids a network hop and has capacity limited per instance. Each process retains its own copy. External Redis allows entries to be shared between instances, at the cost of connections, network traffic, and another dependency. Not every cache resides exclusively in RAM: some caches use disk and other storage layers.
A claim such as 10,000 times faster needs a concrete comparison. The difference depends on the query avoided, load, result size, network, and serialization or deserialization. A simple indexed query and an expensive computation do not produce the same savings; a remote hit can also wait for capacity.
The key must represent the result that can be reused: entity, parameters, and, when they affect the response, tenant or permissions. The value can include a version and information about its freshness. The design must define which reads accept a copy and which require querying the source.
Reading with cache-aside
In cache-aside, the application manages the cache alongside database access. To read data:
- Look up the key in the cache.
- If a usable entry is found, respond with it: a hit.
- If it is not found, query the database: a miss.
- Attempt to store a copy with the corresponding TTL and respond with the data obtained.
--- config: look: handDrawn --- flowchart TD client["Client"] --> lookup["Application<br/>Look up key"] lookup --> cache[("Shared Redis")] cache --> hit{"Usable hit?"} hit -->|"Yes"| response["Return data"] hit -->|"No: miss"| query["Application<br/>Query source"] query --> database[("Authoritative database")] database --> fill["Application<br/>Attempt to store copy with TTL"] fill --> cache fill --> response response --> client
The application performs the query and fill; Redis does not automatically query the database in this pattern. The figure represents a read with the cache available. If storing the copy fails after reading the source, that result can still be returned. A Redis outage also requires limiting fallback to protect the database.
The TTL defines how long the entry lives from its insertion or update. A short period increases misses and fill work; a long one allows more reuse, but requires resolving what happens when the original data changes.
Read and write architectures
Read patterns decide who resolves a miss. Write patterns decide when the database is updated and what accepting an operation means. They can be combined: read-through can accompany write-through or write-behind. These layers require an adapter; standard Redis does not automatically integrate an external database.
| Pattern | Who coordinates | When the operation completes | Reads | Cost or risk |
|---|---|---|---|---|
| Cache-aside | Application. | Read: when the data is obtained. | Miss resolved on demand by the application. | Query and fill on each miss. |
| Write-through | Write layer. | Write: when DB and cache report success. | According to the chosen read pattern. | Synchronous latency and possible partial failures. |
| Write-behind / write-back | Layer with a write buffer. | Write: upon acceptance by the layer; DB later. | Must account for pending writes. | Buffer durability and persistence delay. |
| Read-through | Read layer. | Read: when the data is returned. | The layer queries the source and fills the miss. | Additional integration; retains fill costs and failures. |
Cache-aside is often a practical starting point when the application already controls the source. It loads what is requested, but does not guarantee a small cache: without TTL, eviction, and a capacity budget, it can also grow or retain data with little use.
Write-through updates the source and cache synchronously before reporting success. It adds waiting time and may load values that are never read. It does not provide atomicity or strong consistency by itself: one update may complete and the other fail. Concurrency, reads through different paths, and direct database writes still require coordination and invalidation according to the contract.
Write-behind accepts changes before persisting them in the DB. It can reduce write waiting time, but its pending buffer is no longer a copy that can be rebuilt from that database. Before acknowledging a write whose loss is unacceptable, its durability, replication, and the failures covered by the acknowledgement must be established. Accepting it only in memory can lose it if the process crashes before persisting it.
Deferred persistence needs ordering, idempotent retries, and protection against a stale write arriving late. It also requires observing backlog and limiting admission when it cannot drain. Pending changes must not be evicted as disposable copies; filling the buffer or losing contact with the DB requires an explicit policy.
Read-through moves source access and filling into the layer, simplifying that application path. It defines reads; the write strategy is chosen separately. Moving the code does not eliminate freshness races, stampedes, or the need to limit fallback.
Consistency and invalidation
In cache-aside, a write can commit the change in the database and then invalidate the corresponding entry. Doing so after the commit avoids treating a change that could still fail as final. The application must account for invalidation that does not arrive, is delayed, or repeats.
The transactional outbox can record a pending invalidation event together with the local change. Publication and consumption need recovery and idempotency. That mechanism helps preserve pending work; it does not eliminate the interval during which a cache can serve the previous copy.
A race between readers and writers also exists:
- A reader has a miss and obtains version 7 from the source.
- A writer commits version 8 and invalidates the cache.
- The earlier reader stores version 7 with a new TTL.
The cache is repopulated with old data even though invalidation occurred. The TTL does not guarantee strong freshness or, by itself, a maximum age relative to the source’s latest change. It limits the lifetime of the entry stored; that entry may have originated from an old read.
The design can accept that window if the product tolerates stale data. When a stricter contract is needed, it can version the data and condition filling on the current version. That coordination must work even when the entry is absent: comparing only against a value that has just been deleted does not necessarily reject an old fill. A key that includes a version also requires resolving how the current version is obtained.
Invalidation events must respect that contract despite delays and reordering. If a decision requires current state, it is queried or validated at the authority that protects the rule; a hit does not replace that validation. The case in SD: Ticketing System separates the fast inventory view from the transactional decision about a seat.
Capacity and eviction
A cache’s memory and storage space are finite. The budget includes values, keys, metadata, and the headroom the service needs. Storing more entries than fit requires evicting them or rejecting new fills, depending on the policy.
LRU, or Least Recently Used, prioritizes evicting data that has gone longest without being used. Redis offers approximate LRU policies that select candidates without maintaining an exact ordering of all entries. The chosen policy also determines whether all keys or only those with expiration can be evicted; a policy without eviction may reject writes upon reaching its limit.
Expiration and eviction serve different purposes. The TTL limits an entry’s lifetime; eviction recovers capacity and can remove it earlier. Both produce misses. Choosing the policy requires observing access distribution and the cost of rebuilding values, rather than assuming LRU suits any workload.
In Kubernetes, resource requests and limits take that budget into workload operations. Limiting container memory and choosing an eviction policy are related decisions with different mechanisms.
A cache of copies must be rebuildable from its source. If Redis also retains state that cannot be lost, that state needs its own persistence and recovery contract; it must not automatically be treated as evictable material.
Stampedes and fill coordination
A cache stampede occurs when many requests need to rebuild the same data at the same time. It can appear when a TTL expires, after eviction, when the service restarts, or with a cold cache. One logical miss ends up generating many source queries.
Singleflight, or request coalescing, allows a process to share one fill result among concurrent requests. It reduces work within that instance. With multiple instances, each can start its own fill; a shared lease can reduce that concurrency between processes under its coordination contract.
The lease has a bounded duration. If it expires while the first worker is still working, another can start. A failure or primary change can also affect coordination. Release must check the owner’s identity to avoid deleting someone else’s lease. That check does not turn a Redis mutex into durable exclusion against every failure, nor does it by itself prevent publishing an old result.
Jitter in TTLs spreads expiration across different keys and reduces synchronized expiry. It does not by itself prevent concurrency on a hot key. With stale-while-revalidate, previous copies can be served while being refreshed, provided the product accepts that staleness and defines how long they can be used.
Fills must have concurrency and waiting limits. If coordination fails, the policy chooses between serving an acceptable copy, waiting for a bounded period, or rejecting work. Source protection and version validation remain necessary even when duplicate fills are reduced.
Hot keys and load distribution
A hot key concentrates many reads or updates. Sharding distributes keys among nodes, but a single key belongs to one shard: adding shards does not automatically divide its load.
A local L1 cache in front of Redis can absorb repeated reads of that key. The benefit depends on which instances receive traffic and how much the copy can be reused. Each L1 needs its TTL, memory limit, and coherence policy; distributing invalidations among many instances adds fanout and recovery work.
Request coalescing can share concurrent reads or refreshes when the contract permits it. It must not delay requests indefinitely or group results that depend on different identities.
Replicas can distribute certain reads, at the cost of accepting the permitted replication lag. They do not automatically resolve the primary’s write load and can return earlier data. The solution must start with actual traffic distribution, value size, and required freshness.
Failures and verification
If Redis stops responding, sending all misses and timeouts directly to the database can move the spike the cache was absorbing there. Fallback needs timeouts, concurrency limits, and backpressure. Depending on the contract, it can also serve acceptable older data or limit admission. Source capacity must be verified with a cold cache and during an outage.
With write-behind, reading the database as fallback can omit writes the layer has already accepted but has not yet persisted; that read is valid only if the contract accepts that result.
Verification must make an old fill compete with a write and invalidation, duplicate events, and delay them. It must also expire a lease during a query, force expirations and evictions, and concentrate traffic on a single key. The expected result must state which data can be served and how much work is admitted at the source.
Useful signals include:
- Hit rate by query type and weighted by the cost or load avoided: many cheap hits can hide expensive misses.
- p95 and p99 latency for the complete read, on both hits and misses, including network and deserialization.
- Data age and invalidation delay, using a reference from the source when freshness must be measured.
- Memory, expirations, evictions, and rejected fills.
- Database load and latency, concurrent fills, and traffic concentrated on hot keys.
A cache is justified by the work it avoids and the behavior it preserves when the copy ages or disappears. Savings must be measured alongside the read contract and recovery capacity.